Skip to content

Adding JS To all Opa Resources: Use Case Google Analytics

I decided I wanted to add Google Analytics to OpaDo but had no idea how to easily tell each page to include the necessary Javascript. I asked on the Opa mailing list and got a quick and simple response. Frederic Ye pointed me to Resource.register_external_js

It couldn’t have been any easier. You simply place your google_analytics.js file in your project and use the Resource.register_external_js function to modify the default customization of all Resources. See code below or on the github repo.

package opado.main

import opado.user
import opado.admin
import opado.todo

urls : Parser.general_parser(http_request -> resource) =
  parser
  | {Rule.debug_parse_string(s -> Log.notice("URL",s))} Rule.fail -> error("")
  | "/todos" result={Todo.resource} -> result
  | "/user" result={User.resource} -> result
  | "/login" result={User.resource} -> result
  | "/admin" result={Admin.resource} -> result
  | (.*) result={Todo.resource} -> result

do Resource.register_external_js("/resources/js/google_analytics.js")
server = Server.of_bundle([@static_resource_directory("resources")])
server = Server.make(urls)

For a longer article/tutorial on dealing with external resources check out this blog post from the Opa team Dealing with External Resources.

Opa Database Migrations

Nicolas Glondu posted a comment on an earlier post detailing ways for doing database migrations with Opa. I thought it was useful enough that I should put up a post around it:

If you have complex changes in database structures in a OPA program, you have two choices :

1 – Keep both structure in your database and create a function which populates the new empty field from the other fields. Launch your program once with –db-force-upgrade and launch this function once (adding fields is safe but the new fields are empty). After this, clean removed fields and the function and re-run your program with –db-force-upgrade once more (removing fields is also safe). After this, you can keep using your program normally. A bit long but convenient for changes in small applications (with one instance running). You don’t have to relaunch your program immediately and you can spread ou migration on two program updates.

2 – Create another OPA program with both database definitions but with root nodes named differently [1]. Your program should take data from the original node and put it in the new node with your changes. That way you can have a standalone opa application for your migration. Which is more convenient if your program is used by other users. Once your program compiled, you can use command line options (–db-local) to use your true program’s database as the input root node an to put the output root node in a safe place. However, I don’t know if it works if your program has no specific root node defined.

[1] You can define a root node with `database my_root = @meta` and use it with `db /my_root/toto : stringmap(int)`. You can have multiple root nodes in one program.

OpaDo: Personal ToDo Lists

This is a continuation of two past posts (one, two) on my first application with Opa called OpaDo. You can try the live demo here and check out the full source code on Github

Updating OpaDo to add user accounts the project structure has been changed a bit and modularized. Below is the new project layout.

opado/
├── Makefile
├── README.md
├── dotcloud.yml
├── resources
│   ├── destroy.png
│   └── todos.css
└── src
    ├── main.opa
    ├── todo.opa
    └── user.opa

Now there is a main, todo and user module. The main module is the entry point for the app and looks like:

package opado.main

import opado.user
import opado.todo

urls : Parser.general_parser(http_request -> resource) =
  parser
  | {Rule.debug_parse_string(s -> Log.notice("URL",s))} Rule.fail -> error("")
  | "/todos" result={Todo.resource} -> result
  | "/user" result={User.resource} -> result
  | "/login" result={User.resource} -> result
  | (.*) result={Todo.resource} -> result

server = Server.of_bundle([@static_resource_directory("resources")])
server = Server.make(urls)

Here we define the name of this package and import the user and todo modules. Next is the url matching code. urls is a parser that takes an HTTP request and returns a resource. The matching is pretty straight forward. For example:

  | "/todos" result={Todo.resource} -> result

Here we are matching on URLs that begin with /todos but could have anything after that. What is contained after /todos is passed to the Todo.resource which the variable result is set to. And finally that result is returned.

The last two lines simple define the reource directory for the server and pass in the matching function for the HTTP requests.

The todo resource isn’t important to us in this post since its hardly changed. But there are a two important changes:

db /todo_items : stringmap(stringmap(todo_item))
db /todo_items[_][_]/done = false

Here we see that the /todo_items database is not longer simply a stringmap of todo_item‘s but a stringmap of a that. This is so we can reference the items by a user identifier. For example a user identified by the string “user01″ who has a todo item identified by “aaa” would be read from the data base as /todo["user01"]["aaa"].

There are a few other changes to the todo module so that items are properly inserted for the logged in user and deleting must be done in the second stringmap. But we’ll move on to the user module now.

Much of the user module was taken from Matthieu Guffroy’s OpaCMS code on github. But I’ve made a number of modification for my needs.

@abstract type User.password = string
@abstract type User.ref = string

type User.t =
  {
    username : string
    fullname : string
    password : User.password
  }

type User.status = { logged : User.ref } / { unlogged }
type User.info = UserContext.t(User.status)
type User.map('a) = ordered_map(User.ref, 'a, String.order)

db /users : User.map(User.t)

User_data = {{
  mk_ref( login : string ) : User.ref =
    String.to_lower(login)

  ref_to_string( login : User.ref ) : string =
    login

  save( ref : User.ref, user : User.t ) : void =
    /users[ref] <- user

  get( ref : User.ref ) : option(User.t) =
    ?/users[ref]
}}

Above we have the data, types and database definitions necessary to handle the users.

User.t provides the record for storing necessary user data. Next, we have types for checking the user status of if they are logged in or not.

UserContext is a module provided by Opa for dealing with associating the user values with the client — via cookies. And the data for that user can only accessed by the user that owns it.

User_data object provides functions for accessing and manipulating users.

Now we can look at the User module.

User = {{

  @private state = UserContext.make({ unlogged } : User.status)

  create(username, password) =
    do match ?/users[username] with
      | {none} ->
          user : User.t =
            { username=username ;
              fullname="" ;
              password = Crypto.Hash.sha2(password) }
          /users[username] <- user

      | _ -> void
    Client.goto("/login")

At the beginning of the User object we declare a UserContext and a function for creating new users. The function simply checks if the user exists already with the match statement and if not creates a new User.t record and inserts it to the users database.

If we wish to login we must also modify the UserContext

  login(login, password) =
    useref = User_data.mk_ref(login)
    user = User_data.get(useref)
    do match user with
     | {some = u} -> if u.password == Crypto.Hash.sha2(password) then
                       UserContext.change(( _ -> { logged = User_data.mk_ref(login) }), state)
     | _ -> void
    Client.goto("/todos")

The function attempts to read the user from the database and checks if the passwords match. If so, it will set the UserContext to logged in. The function then tells the client to go to /todos. If the login was unsuccessful, it doesn’t matter and will just redirect to the sign up page.

Obviously, better error handling and notification is the next step for the application.

The last interesting part for this I think is the request matching. The rest of the code is mostly just HTML and piecing together the functions I already described.

  resource : Parser.general_parser(http_request -> resource) =
    parser
    | "/new" ->
      _req -> Resource.styled_page("New User", ["/resources/todos.css"], new())
    | "/edit" ->
      _req -> edit()
    | "/view/" login=(.*) ->
      _req -> view(Text.to_string(login))
    | .* ->
      _req -> start()

The key match to look at is:

    | "/view/" login=(.*) ->
      _req -> view(Text.to_string(login))

This shows the request matching /view, which in this case comes after the main module matches ‘/user’ and routes to the User module resource. But then we have login=(.*), this is matching the variable login to the rest of the url. This variable login can then be used in view(Text.to_string(login)) to pass to the view function so it knows what user is being asked to be displayed.

There’ll be more to come. Next, I need to add some validation, an admin page and then the ability for users to have categories to organize their todo items under.

And let me know anything else people would like to see!

Announcing ErlangDC: An Epic One-Day Erlang Conference in the Washington, DC Area

We are happy to announce ErlangDC: An Epic One-Day Erlang Conference in the Washington, DC area.

New to Erlang? Learn the basics — and find out why Erlang should be in your programmer’s toolkit — during the morning bootcamp. Meet fellow DC-area Erlang enthusiasts at lunch. Learn advanced Erlang techniques in the afternoon tech talks. Swap Erlang war stories and make lifelong friends over pints at the post- conference Happy Hour.

It will be reliably awesome. Just like Erlang.

ErlangDC is organized by the local DC Erlang Meetup Group, with help from Erlang Solutions and Erlang Factory. The event will be hosted at the AOL Headquarters in Dulles, VA.

Sign up when you get a chance. The early-bird tickets have been sold out fast! Tickets are only $40.

OpaDo Data Storage

OpaDo (a port of the TodoMVC app to Opa) now persists todo items to the Opa database. The new version is up on dotcloud, http://opado-tristan.sloughter.dotcloud.com/

I’ve added a todo_item type which stores the item’s value and two other attributes we won’t use until the next post when we have user accounts for their own todo_item stores.

type todo_item = { user_id : string
                 ; value : string
                 ; created_at : string
                 }

To tell Opa where to store the records we’ll create, we provide a path to the Opa db function and set its type. For our todo items we use a stringmap since currently the id’s are randomly generated strings (I know, I know, but its just an example!). We can then reference a record in the database with the path /todo_item[some_id_string].

db /todo_items : stringmap(todo_item)

Now we can insert todo_item‘s to this db path as so:

/todo_items[id] <- { value=x user_id="" created_at="" }

For now user_id and created_at are empty, but I’ll be updating that when I add user accounts.

Since we are storing each item, we need to populate the list on page load with whats already stored:

add_todos() =
  items = /todo_items
  StringMap.iter((x, y -> add_todo_to_page(x, y.value)), items)

The first line of the function sets the variable items to all the todo_item records in the database. We use StringMap.iter to take each todo_item and add it to the page. The first argument to the anonymous function is the id the item is stored in the database with (the id we will use in the HTML as well) and the second is the actual todo_item, so we take its value field and pass that to the add_todo_to_page function along with the id.

To have the add_todos function when the list element is ready we add an on_ready attribute that will call add_todos:

<ul id=#todo_list onready={_ -> add_todos() } ></ul>

Lastly, we want to be able to delete a todo_item from the database:

remove_item(id: string) =
  do Dom.remove(Dom.select_parent_one(#{id}))
  do Db.remove(@/todo_items[id])
  update_counts()

remove_all_done() =
  Dom.iter(x -> remove_item(Dom.get_id(x)), Dom.select_class("done"))

The main piece to notice here is @/todo_items[id] in Db.remove(). The @ is saying that we are passing the path itself to remove() and not the value at that path.

Nice and easy! No database to setup or deploy, just Opa. Next time we’ll add user accounts, so we don’t have to all share the same todo list.

TodoMVC in Opa

Edit: I just learned that dotcloud supports Opa! So I’ve pushed OpaDo and you can see a demo here http://opado-tristan.sloughter.dotcloud.com/

I wanted something quick and simple to do in Opa to give it a try so I decided to implement the TodoMVC example that has been redone in almost all Javascript frameworks, https://github.com/addyosmani/todomvc.

The code can be found on GitHub here: https://github.com/tsloughter/OpaDo

Opa is unique in that it is not only a new language but also a new web server and database. While Opa’s page pushes the idea that its for the cloud and its easy distribution, I found the nicest part being the static typing and no need for Javascript.

The functions below handle interactions with the Todo items. It somewhat reminds me of Lift but taken even farther.

/** * {1 User interface} */
update_counts() =
  num_done = Dom.length(Dom.select_class("done"))
  total = Dom.length(Dom.select_class("todo"))
  do Dom.set_text(#number_done, Int.to_string(num_done))
  Dom.set_text(#number_left, Int.to_string(total - num_done))

make_done(id: string) =
  do if Dom.is_checked(Dom.select_inside(#{id}, Dom.select_raw("input"))) then Dom.add_class(#{id}, "done")
  else
    Dom.remove_class(#{id}, "done")

  update_counts()

remove_item(id: string) =
  do Dom.remove(#{id})
  update_counts()

remove_all_done() =
  do Dom.remove(Dom.select_parent_one(Dom.select_class("done")))
  update_counts()

add_todo(x: string) =
  id = Random.string(8)
  li_id = Random.string(8)
  line = <li id={ li_id }><div class="todo" id={ id }> <div class="display"> <input class="check" type="checkbox" onclick={_ -> make_done(id) } /> <div class="todo_content">{ x }</div> <span class="todo_destroy" onclick={_ -> remove_item(li_id) }></span> </div> <div class="edit"> <input class="todo-input" type="text" value="" /> </div> </div></li>
  do Dom.transform([#todo_list +<- line ])
  do Dom.scroll_to_bottom(#todo_list)
  do Dom.set_value(#new_todo, "")
  update_counts()

It is unique in combining the HTML into the language itself. Some have argued against this but when it works well it makes perfect sense. I don’t want to have to convert a designers HTML into some other representation! And being able to have type checked dynamic functionality within the HTML is a boon. Even with just this simple program, I found the usefulness of the type checker outstanding.

Next we have the main outline of the page and the entry part for the program.

start() =
  <div id="todoapp"> <div class="title"> <h1>Todos</h1> </div>
    <div class="content"> <div id=#create_todo> <input id=#new_todo placeholder="What needs to be done?" type="text" onnewline={_ -> add_todo(Dom.get_value(#new_todo)) } /> </div>
      <div id=#todos> <ul id=#todo_list></ul> </div>

      <div id="todo_stats"> <span class="todo_count"> <span id=#number_left class="number">0</span> <span class="word">items</span> left. </span> <span class="todo_clear"> <a href="#" onclick={_ -> remove_all_done() }> Clear <span id=#number_done class="number-done">0</span> completed <span class="word-done">items</span> </a> </span> </div>
    </div>
  </div>

/** * {1 Application} */

/** * Main entry point. */
server = Server.one_page_bundle("Todo",
       [@static_resource_directory("resources")],
       ["resources/todos.css"], start)

It won’t be able to replace my use of Erlang for the backend and Coffeescript for the frontend, but it looks very promising.

I’ll be extending this example to include persistence, sessions and users and will add posts as I complete those.

Centos6 : Chef Node Creation

I thought I’d share the scripts I use to take a fresh Centos6 install and have it configured to work with a Chef server. Maybe its not as easy as when running in a virtualized environment, but it saves plenty of time.

On the new node I run the setup_client.sh script which calls in the end client_gen.sh on the Chef server once everything is installed on the node. I left the version numbers for Ruby and Chef in the script so you know what versions I’ve tested this with.

setup_client.sh

#!/bin/bash
CHEF_IP=XXX.XXX.XXX.XXX
CHEF=http://$CHEF_IP:4000
CHEF_USER=XXXXX
NODE=XXXXXXXX
RUBY_VSN=1.3.7
CHEF_VSN=0.9.16

sudo rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm

sudo yum update

sudo yum install ruby ruby-shadow ruby-ri ruby-rdoc gcc gcc-c++ ruby-devel ruby-static

cd /tmp
wget http://production.cf.rubygems.org/rubygems/rubygems-$RUBY_VSN.tgz
tar zxf rubygems-$RUBY_VSN.tgz
cd rubygems-$RUBY_VSN
sudo ruby setup.rb --no-format-executable

sudo gem install chef -v $CHEF_VSN 

mkdir ~/.chef

cat > ~/.chef/knife.rb <<EOF log_level :info log_location STDOUT node_name '$NODE' client_key '/home/$USER/.chef/$NODE.pem' validation_client_name 'chef-validator' validation_key '/etc/chef/validation.pem' chef_server_url '$CHEF' cache_type 'BasicFile' cache_options( :path => '/home/$USER/.chef/checksums' ) EOF

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub $CHEF_IP

ssh $CHEF_IP "yes | knife client delete $NODE"
ssh $CHEF_IP "yes | /home/$CHEF_USER/client_gen.sh $NODE"
scp $CHEF_IP:/tmp/$NODE ~/.chef/$NODE.pem

client_gen.sh

#!/bin/bash
knife client create $1 -n -a -f /tmp/$1
knife node create $1 --no-editor
Follow

Get every new post delivered to your Inbox.