cloud

Opa Plugin Development with a PubNub Example

This will be a two part series of posts on writing plugins for Opa. One of Opa's greatest features is you write everything in the functional, staticly typed Opa language. This even includes the frontend code you'd usually do in Javascript. This means your code is less error-prone which significantly reduces the amount of time spent on debugging code.

But how do you use Javascript library X within your Opa code?

While you can include Javascript code in a resource, like we do here for Google Analytics:

do Resource.register_external_js("/resources/js/google_analytics.js")

This isn't what we want to do if we want to call external Javascript functions from Opa code. For this we create an Opa plugin. In this post I'll be showing how I built the Opa client side bindings for the PubNub Javascript API, fork it on github and see all the other language bindings. PubNub is a cloud hosted real-time publish and subscribe interface for all types of applications.

Opa allows binding to external functions through its Binding System Library (BSL). In order to use this to create a plugin for some external Javascript, we create a file pubnub.js that we will compile to a plugin using opa-plugin-builder. Below is an example of one of the external functions that needs to be defined to use PubNub's publish.

##register publish : string, string -> void  
##args(c, m)  
{  
    PUBNUB.publish({  
                       channel : c, message : m  
                   });  

    return js_void;  
}

The first line registers the function publish and specifies its arguments to be two strings and its return value to be void. The next line names the arguments. You can think of ##args like Javascript's function keyword. In the function, we can use the PUBNUB library just like we would normally. Here we pass to publish the channel and the message to be published. Note that the message can be any Javascript object, even though its a string to the plugin. We simply must serialize the JSON before passing to publish, which we'll see shortly.

Now after defining external functions for all PubNub api calls in the pubnub.js file we can compile it:

$ opa-plugin-builder src/plugin/pubnub.js -o pubnub

This creates the plugin pubnub.opp.

To make using the plugin easier, we create a package in pubnub.opa that will contain a module PubNub. Below is the contents of pubnub.opa for the publish function that will show how to register the external functions and create a module named PubNub around them.

package pubnub  

pubnub_publish = %%pubnub.publish%%  

@client publish_client = pubnub_publish  

PubNub = {{  
  publish(channel: string, j : RPC.Json.json) =  
    publish_client(channel, Json.to_string(j))  
}}  

do Resource.register_external_js("http://cdn.pubnub.com/pubnub-3.1.min.js")

The first thing to notice is setting the pubnubpublish_ variable. The %% syntax is specifying that pubnub.publish is the key of an external primitive. pubnub is the name of the plugin (pubnub.opp) and publish is the function we registered in the plugin.

After this we create another variable publishclient_ that specifies that that this function (variables can contain functions) is to be run on the client side. Lastly, we define a module PubNub and a publish function inside the module to be used by our Opa programs.

At the end we make sure that the needed PubNub Javascript file is includes in all resources by registering it and giving the URL to the PubNub CDN.

Now we can compile it all together with the opa-plugin-builder command from above and use the plugin.

import pubnub  

broadcast(author, msg) =  
   username : RPC.Json.json = {String = author}  
   message : RPC.Json.json = {String = msg}  
   record_json : RPC.Json.json = {Record = [("username", username), ("message", message)]}  
   do PubNub.publish("chat", record_json)  
   Dom.clear_value(#entry)

The broadcast function is part of pubnubchat.opa. _broadcast takes 2 arguments, the username of the user sending the message and the message itself. We then construct three variables of Opa's json type, username, message and recordjson. _username and message are simply Json strings and they are then combined into a Json Record type that is a list of string/Json pairs. Since our PubNub.publish function converts the Opa Json to a string before sending to the Javascript binding we can simply pass the Opa Json variable recordjson_.

Lastly, lets take a quick look at the subscribe function. Defining it in pubnub.js look like:

##register subscribe : string, (string -> void) -> void

Since it takes a callback function you'll notice that the second argument's type looks different from in publish. (string -> void) is defining that the function takes a variable that is a function which takes a single argument of type string and has a void return type.

Now in pubnub.opa we have a subscribe function that is hardly different from the publish function we described above.

subscribe(channel: string, callback) =  
  subscribe_client(channel, callback)

But how do we define a function we can pass to subscribe? That is the great thing, it can be any Opa function!

PubNub.subscribe("chat", (x -> user_update(x)))

Above we are subscribing to the channel chat and passing an anonymous function that takes a single argument and sends it to the function userupdate_.

user_update(x) =  
  match Json.deserialize(x) with  
         | {some={Record = [("username", username), ("message", message)]}} ->  
                    line = <div class="line"> <div class="user">{Json.to_string(username)}:</>  
                             <div class="message">{Json.to_string(message)}</>  
                            </>  
                    do Dom.transform([#conversation +<- line ])  
                    Dom.scroll_to_bottom(#conversation)  
         | _ -> void

Now every time a message is posted to the chat channel, our callback, the anonymous function, will be run and call the function userupdate_ which we see deserializes the Json string to get an Opa Json Record. The rest is basic Opa DOM manipulation to add the new message to the messages in the conversation element.

In my next post on Opa and PubNub, I'll be describing an Opa server side API for PubNub.

OpaDo Data Storage

OpaDo (a port of the TodoMVC app to Opa) now persists todo items to the Opa database. The new version is up on dotcloud, http://opado-tristan.sloughter.dotcloud.com/

I've added a todoitem_ type which stores the item's value and two other attributes we won't use until the next post when we have user accounts for their own todoitem_ stores.

type todo_item = { user_id : string  
                 ; value : string  
                 ; created_at : string  
                 }

To tell Opa where to store the records we'll create, we provide a path to the Opa db function and set its type. For our todo items we use a stringmap since currently the id's are randomly generated strings (I know, I know, but its just an example!). We can then reference a record in the database with the path /todoitem[someid_string].

db /todo_items : stringmap(todo_item)

Now we can insert todoitem_'s to this db path as so:

/todo_items[id] <- { value=x user_id="" created_at="" }

For now userid_ and createdat_ are empty, but I'll be updating that when I add user accounts.

Since we are storing each item, we need to populate the list on page load with whats already stored:

add_todos() =  
  items = /todo_items  
  StringMap.iter((x, y -> add_todo_to_page(x, y.value)), items)

The first line of the function sets the variable items to all the todoitem_ records in the database. We use StringMap.iter to take each todoitem_ and add it to the page. The first argument to the anonymous function is the id the item is stored in the database with (the id we will use in the HTML as well) and the second is the actual todoitem, so we take its value field and pass that to the _addtodotopage_ function along with the id.

To have the addtodos_ function when the list element is ready we add an onready_ attribute that will call add_todos:

<ul id=#todo_list onready={_ -> add_todos() } ></ul>

Lastly, we want to be able to delete a todoitem_ from the database:

remove_item(id: string) =  
  do Dom.remove(Dom.select_parent_one(#{id}))  
  do Db.remove(@/todo_items[id])  
  update_counts()  

remove_all_done() =  
  Dom.iter(x -> remove_item(Dom.get_id(x)), Dom.select_class("done"))

The main piece to notice here is @/todoitems[id]_ in Db.remove(). The @ is saying that we are passing the path itself to remove() and not the value at that path.

Nice and easy! No database to setup or deploy, just Opa. Next time we'll add user accounts, so we don't have to all share the same todo list.