Hello Meteor


Javascript/AJAX

(modified from http://asynchrotron.com/blog/2012/05/22/hello-meteor/)

This post is for the benefit of all those who are feeling like they might be missing a step with frameworks like Backbone, Ember, (and now Meteor), et. al. Read on for recommendations on what to read up on to understand more about the problems they’re solving.

First a brief note about MVC

The Model-View-Controller application architecture (MVC) is used in most modern applications to separate an application’s data from its business rules and user interface. Briefly, Models represent an application’s data and functions to access it. Views represent information presented to the user. Controllers represent intermediary resources required to generate Views. The MVC pattern isn’t new (it can be traced back to Smalltalk), but it was popularized, along with the REST style of client-server API design by server-side web frameworks such as Rails and Django.

So what’s all the fuss about JavaScript MVC frameworks (Backbone, Ember, et. al.)?

If you think of the screens – or pages – in your application as different “states” in your application, server-side frameworks require a round trip back to the server when the user goes from one application state to another. This is fine for content-heavy apps or websites. Or if you’re accessing the pages via an ethernet cable or 802.11. But when you move to high latency connections and along the continuum — and I believe it really is a continuum — from a web site to a web app, you can deliver a much better user experience if you avoid the server round trip for each state change, and instead switch from state to state on the client, within a single browser page load cycle.

This saves you all of the overhead of destroying and recreating the DOM tree, flushing browser context (JavaScript, CSS, etc.) on each new page load. You also are transmitting less data on the wire (or ether) since HTML is generated on the client, and only JSON passed over the network. Both of these factors (page refresh time and network traffic optimization) are significant when you consider the fact that more and more internet usage is taking place via mobiles and tablets.

You would still need a server to authenticate users and perform other privileged operations, and to store the “official” version of the data that different clients synchronize with, but instead of sending dynamically created HTML to the client (a la aforementioned server-side frameworks) the server would send as much data as needed (typically as JSON) to the client, and the client would create all of the different application states dynamically. The problem with this approach — and I’ve been writing apps this way since 2008, without the benefit of any client side frameworks — is that they quickly turn into a mass of spaghetti code.

This is where the likes of Backbone, Ember, etc. come in. They are a re-imagination of the MVC application architecture for JavaScript application development in the client, where all (or nearly all) application states are handled in the client, in JavaScript. They provide a way to structure your JavaScript applications in a more organized MVC style.

Addy Osmani’s articles, and especially his Backbone Fundamentals book are a great place to start reading more about this. Although it’s focused on one specific framework (Backbone), it has a good overview of MVC and how the pattern has moved from server side frameworks (like Rails, Django) to the client side (Backbone, et. al.).

I also wholeheartedly recommend his recent talks on providing structure to JavaScript apps.

And this is a useful comparison of various front-end frameworks, although the conclusions should be taken with a grain of salt (it’s one persons’s perspective, and things are not quite as quantifiable as the article suggests):
So, all of these are client side frameworks, which solve the problems of a) separating client side code into MVC, and b) optionally, allowing views to “observe” client side models, so that views get updated automatically as the data in the models is updated or as new data arrives from the server.

This gets you pretty far, but all of these applications have something else in common which the previously mentioned MVC frameworks don’t address: They need to send data back to the server and check in if data has changed on the server. In other words, they need to synchronize client side models with the server side models.

Enter Meteor.

Meteor is a full stack framework for end-to-end JavaScript applications. The beauty of coding it all in JavaScript is that you can easily implement the observer pattern (views watching for changes to the models) all the way to the server side. Meteor
apps can subscribe to models (or subsets of models) on the server, and meteor takes care of all of the plumbing. This really is the bleeding edge of JS today.

For more on the problems Meteor solves, read the Meteor docs from the intro section through to as many of the detailed concepts you’re interested in. Or come to my talk at JS.Chi() and we will work through it together. Here are the slides from the talk.

Have fun!

Pallavi

Using YUI DataTable with Rails


Javascript/AJAXRailsTips and Tricks

I am currently working on an Rails app that integrates the YUI dataTable, and in going through the tutorials I noticed they are all assume a PHP back-end. I also saw a number of people asking how to get this to work with a Rails controller, so I thought I’d write up my experience in the hopes that it helps someone else. For basic info about setting up the dataTable, refer to the YUI site, linked above. I’m also going to try to clarify a few things that I found a bit obscure.

To create a dataTable you have to define a few basic ingredients:

  1. A dataSource. This defines where the info in the table comes from, and what format it is returned in. I’m using JSON.
  2. A schema that you define. This is a part of the dataSource and is essentially a map to your data. The schema tells the table where to find the field values.
  3. An array of column definitions. You supply the name and “key” of each column and any additional information, such as whether it is editable (and if so, which editor to use) and how to format the data inside the column if you don’t want to just spit it out directly.

Let’s start with a dataSource. In this example, we’re making a table of tasks, so I want to hit my tasks controller to return the data. Standard Rails controller action. In the code snippet below,

  1. in line 1, I supply a URL (note that the url ends in .json, and in my controller I have a responds_to block which constructs my json response).
  2. In line 2, I’m telling the dataSource — hey, you’re gonna get some JSON.
  3. And line 3, is where we define the responseSchema, which has two critical parts: the resultsList and the fields.
  var dataSource = new YAHOO.util.XHRDataSource("/projects/13/elements/21/tasks.json");
  dataSource.responseType = YAHOO.util.XHRDataSource.TYPE_JSON;
  dataSource.responseSchema = {
   resultsList: "Resources.data",
   fields: [
      {key:"task.id"},
      {key:"task.status"},
      {key:"task.percent_complete"},
      {key:"task.description"},
      {key:"task.due_date", parser:date}
      ]
  }

Let me be clear since I got hung up on this a bit. You as the application developer are responsible for two things. One, constructing your response — in other words, you can build your JSON response any way you want, with any keys, values and hierarchical structure that make sense (though it will help you to think it through and standardize it, Satyam has some good info on that here). And two, telling your dataSource how to navigate your response: i.e. — where do find what the data you’re looking for (the needle) in the response (the haystack of JSON)? This is the schema, the map of your data.
Controller code that creates my custom JSON:

# tasks_controller.rb
def index
 respond_to do |format|
      format.json{
         tasks = []
         @tasks.each do |task|
           task_container = {}
           task_container  = task
           task_container['editable_by_user'] = permission.edit? # some metadata I use
           task_container['deletable_by_user'] = permission.delete?
           task_container['resource_name'] = "task"
           tasks << task_container
        end
        data = {"Resources" => {"data" => tasks}}
        render :json => data.to_json
      }
  end
end

And just for fun, here’s a look at the JSON my controller gives me back when I hit this url ‘/projects/13/elements/21/tasks.json’.
The JSON:

{"Resources": {"data": [{"task": {"status": "not_started", "started_on": null, "updated_at": "2009-02-07T22
:03:18Z", "project_id": 13, "percent_complete": null, "high_priority": null, "element_id": 32, "deletable_by_user"
: true, "completed_on": null, "editable_by_user": true, "element_title": "looking good", "id": 50, "created_by_id"
: 7, "resource_name": "task", "description": "Pass the stimulus bill", "assignments": [], "due_date"
: "", "users": [], "resource_url": "/projects/13/elements/32/tasks/50", "due": null, "created_at": "2009-02-07T22
:03:18Z"}}, {"task": {"status": "not_started", "started_on": null, "updated_at": "2009-02-07T22:03:39Z"
, "project_id": 13, "percent_complete": null, "high_priority": null, "element_id": 32, "deletable_by_user"
: true, "completed_on": null, "editable_by_user": true, "element_title": "looking good", "id": 51, "created_by_id"
: 7, "resource_name": "task", "description": "Negotiate without pre-conditions", "assignments": [], "due_date"
: "", "users": [], "resource_url": "/projects/13/elements/32/tasks/51", "due": null, "created_at": "2009-02-07T22
:03:39Z"}}]}}

So that’s it! I’ve defined this JSON array and built it, then returned it from the controller when I get a .json request.

Now that you’ve seen the response, take another look at the responseSchema (you created above) and the two properties that you set:

  1. ResultsList. Notice it is set to “Resources.data”, which are the JSON keys I used. It uses dot-syntax to point to array of tasks in my json.
  2. Fields. Again using dot syntax, and having the ResultsList array to navigate through, it can pull the specific values it wants from the list; so ‘task.status’, ‘task.started_on’, etc., will retrieve those values from the response.

Make sense?

You are now one step away from being able to see your table. So, create your column definitions, an array of data about each column in the table. This is also where you can specify formatting and editing information (not covered in this article).

var columnDefinitions = [
  {key:"task.status",formatter:"formatPriority", sortable:true},
  {key:"task.percent_complete", label:"Percent Complete", sortable:true},
  {key:"task.description", label:"Description"},
  {key:"task.due_date", label:"Due Date", editable:true,sortable: "true",formatter:YAHOO.widget.DataTable.formatDate, editor: new YAHOO.widget.DateCellEditor({resource:'task', updateParams:"task[due]"})}
  ];

Notice that each column definition has a key: this key must be accessible in your JSON, AND it must be defined as a field.

And then you create your table. The first argument is the id of a div on the page to which the table will be attached, the last is an optional configuration hash (pagination, anyone?).

var dataTable = new YAHOO.widget.DataTable("project_tasks", columnDefinitions, dataSource, optionalConfigurationHash);

I hope this is helpful to get you started with the YUI dataTable on Rails. I may do further posts on XHR editing of individual cells in a dataTable using Rails.

Getting started with YUI’s Connection Manager in Rails and PHP; or "All Happy Families Are Not Alike"


Javascript/AJAXPHPRuby

This post is geared towards folks who haven’t done the A part of AJAX before (And I mean the first A, as in Asychronous); are new to Yahoo’s implementation of the XMLHttpRequest object (The Yahoo! Connection Manager) and would like added information on how that works; or bothThis is not meant to supplant the excellent yui! tutorials which you should read in detail for thorough explanations and examples. What I am adding here are a few examples of using this in the Rails framework and some thoughts on the callback object and scope.

Your “AJAX” goals are simple: you want to communicate with your server, get a response back that you can use (or not), do something with that response (or not), and move on. As this is asynchronous, you want to do this without reloading your web page. Or, as a client once said to me, referring to certain animated gifs in the upper right-hand corner of certain browsers, without “making the world spin”. To this end, Yahoo! has supplied us one line of code:

var transaction =
YAHOO.util.Connect.asyncRequest(
method, uri, callback, postData);

or, same line with some data plugged in:

var transaction =
YAHOO.util.Connect.asyncRequest(
'POST',   'php/post.php', callback,"id=1&old_id=2");

When I was making the switch from synchronous to a-, it helped me to visualize a standard web form to see how form elements and attributes are translated to an AJAX request. It’s pretty obvious, but if you need an “aha!” moment, the above line is akin to the html form printed below (though unless you’re one of those people whose definition of interactivity is “The Monologue”, do refrain from creating forms with 2 hard-coded hidden inputs and nothing else! :) ).

<form method="post" action="php/post.php">
<input type="hidden" name="id" value="1" />
<input type="hidden" name="old_id" value="2" />
<input type="submit">
</form>

So, excluding a reference to the callback for the moment (which is not addressed in this example), that form maps to the Connection Manager call quite simply: method, action (uri), data. Let’s look at the arguments required:

method: the method of the server request (POST, GET and others also available).

uri: the uri that’s receiving and processing the data you send (in our example, “php/post.php”). YUI’s examples use php, but, if you’re using the Connection Manager in a Rails app, it’s easy to adapt: your argument uri might read “/projects/update which would pass the data to the update method in projects_controller.rb, which would then be able to access the data through the params array, like so:

def update
@project = Project.find(params[:id])
end

In php you’d probably do some type of db query [assume input cleanup and some type of database abstraction layer, such as PEAR/DB_DataObject, here]

$project = DB_DataObject::factory('Projects');
$project->get($_POST['id']);

Callback: a reference to the callback object you are supplying. This is how everything is handled. More on that in a minute.

postData: the data itself in standard query-string format (“new=1&old=2″). NOTE: if you’re doing a GET transaction, your 4th argument would be false and your second argument would include the url and query string, like so:

“php/post.php?new=1&old=2".

So, what’s this callback?. In a synchronous transaction, you have the luxury of redrawing the page to process your data (and yes, nothing says luxury like a nice, slow, page reload…). In an asynchronous transaction you need to essentially “sneak” your data back into the page without reloading it. This is where your callback object comes in. It helps you get your data “in the door”, so to speak, so your page or application can change in a way that feels seamless to the user but often returns a visible result (changing a div, displaying some text, etc.) and if not a visible result at least a meaningful one (setting the value of a hidden form element, for example). Your callback is responsible for executing actions based on the data retrieved (or the failure to retrieve data) from the uri. In a standard synchronous form this action might be “generate an HTML table that displays your database results”. Or, “Print a message saying there are no results”. Of course, you can do anything you want with your data, that’s just an example of a fairly common scenario.

Once you’ve sent your data to the uri for processing, you need to wait for your response — without, of course, appearing to wait (save for the ubiquitous web 2.0 spinner you know you’re dying to try!). And, of course you want to know if your transaction failed. If you don’t watch for these things — “success” and “failure” in technical terms — you’re not going to be able to make an appropriate decision about what to do next in your app. So you feed your AJAX request a callback object: an object that defines functions for what to do in the cases of success and failure. In simplest terms, we’ve got

var callback = {
success: handleSuccess,
failure: handleFailure
};

where “handleSuccess” and “handleFailure” are user-defined functions that take the http response object and do stuff with it.

handleSuccess = function(o){
// cheer! (or process data returned from the server)
}

handleFailure = function(o){
// cry, vow to try again! (or display failure message)
}

There’s also the ability to pass scope, timeouts, and additional arguments to the callback object. To do so you’d read the great tutorials at the links above and add the lines below, of course changing the values to values meaningful in your application.

scope: Ajaxobject,
timeout: 5000,
args: ['arg1', 'arg2']

The handlers. handleSuccess and handleFailure both take an object o, which is the http response object. There’s a detailed list of all the properties of o on the yui page (Not to be confused with the Story of O, which I will not link to as it is beyond the scope of this article, you dirty rascals, you…). The property you’ll likely use most often is o.responseText, which is the server’s response as a string. This is what you pass back from good old ‘php/post.php’, and getting it is simple: echo. What? echo. What? echo..o..o… ok, sorry, moving on. For instance, if we wanted to capture the update_date in our successHandler to print to our page and we’re using php, we’d write something like this:

echo $project->update_date;

and if we’re in Rails? something like this:

render_text @project.update_date

If you need more data than a string — an array or collection of objects passed back from the server, you’ll find that’s simple, too: call the ruby method to_json() on your array instead. This essentially serializes your object so it can survive the journey to the Client. Once there, you can access the data using JavaScript’s magic wand: eval(). It’s great. So if you had an array of users connected to a project (and your database relationships are set up correctly), you could write

render_text @project.users.to_json

in php, assume you’ve got your $users array, and use print_r

print_r($users);

The in your JavaScript successHandler use eval(), like so:

var users =  eval(o.responseText)

and bingo: in two lines you’re happily in your JavaScript parsing your users array like you would any other JavaScript array. You have connected your server to your DOM and no one is the wiser for it.

All this is great. We have our AJAX call and our callback object and are ready to go. But, suppose you don’t want to rewrite the AJAX call all over your app? The Yahoo folks have a great example of a ‘“hypothetical ajax object” (mysteriously named “AjaxObject”) that encapsulates success, failure, and process methods and calls a callback object that defines AjaxObject as it’s scope. Encapsulating your AJAX request so you can call it from wherever you want in your scripts in a DRY fashion makes your code cleaner and easier to manage. Yahoo! does this well in their example: in my usage I changed it up a little bit to meet my needs.

To quote the great Chicago writer Leo Tolstoy from his famous novel Anna Karenina Does Lake Michigan, ‘“Happy families are all alike; every unhappy family is unhappy in its own way”. I’ve learned that when working on an app, the opposite is true: success cases call for a range of actions: failures can more easily handled (log, display error, abort). Based on this, I’ve adapted the yui AjaxObject example to accept a postAction, successHandler, and object (used to define scope). This allows you to call AjaxObject from other objects, pass specific success handlers, and pass this (a reference to your current object) so you can access it in your successHandler from within the calling object. The AjaxObject builds the callback using those arguments. Like so:

var AjaxObject = {

handleFailure:function(o){
// Fail gracefully
},

/**
* Wrapper for AJAX calls using YUI connector
*
* @param postAction {String} URL to post to
* @param callBackSuccess {String} Success handler
* @param postData {String} Data to post
* @param obj {Object} Object that handler has scope in
*
*/
startRequest:function(postAction, callBackSuccess, postData, obj) {

var callback = {
success:callBackSuccess,
failure:this.handleFailure,
scope:obj
}
// ASSUME you've shortened your yui connection mgr to $C
$C('POST', postAction, callback, postData);
}

};

Then you can call AjaxObject from within a class, like so, and pass it a class method as it’s success-handler:

var Project = function Project(){
// initialize project however you like
this.foo = "bar";
...
// CREATE in db and return id
AjaxObject.startRequest('/project/create',
this._generateDbId, postData, this);

}   // Success handler  Project.prototype._generateDbId = function(o){
if(o.responseText !== undefined){
this._setDbid(Number(o.responseText));
// DO other stuff..
}
}

This way your AJAX calls are in one place, you can use them in the scope of the calling object and define as many success handlers as success cases (or pass false); and fail in one standard way (gracefully, of course). Of course, this can be adapted to pass failure cases in too, or however you like. This was a way I found helpful in my work, and I hope it’s helpful to you as well. And thanks again to the folks at Yahoo! for providing so much great stuff to work with in the first place.

A brief look at Yahoo! Event Utility [addListener]


Javascript/AJAXLanguages

I’ve been getting familiar with the Event Utility in the Yahoo library , and it’s a great way to handle and manage javascript events in a clean centralized way. It also smooths out the differences between browser event handling so you no longer need lines like this in your event functions:

e  = e ? window.event;

If you’re unfamiliar with events, it might help to read the article on quirksmode, as well as the yui page.

This library has lots of great methods for event listening, creating custom events, passing objects to event handler functions (a great feature), stopping default event behavior, and attaching listeners to all elements of a particular class rather than cluttering up your code with hundreds of individual listener calls. It’s all encapsulated in neat, well-documented code and a clear API.

One current simple example. I’m building a project where there’s a text input field on stage that serves as a quick “editor”: the user types a number into the field that corresponds with an object on the stage, and then they hit ENTER. [Result: they land in the object in an open editor, saving them mouse-action on the stage and allowing for easy keyboard access to dispersed elements]. We’re not actually submitting a form and since the text area is embedded in another form we just want to listen for the ENTER key but disable form submission, which automatically happens on enter. We also need to make sure that if the user double-clicks within the text area, the usual dblclick behavior (which creates a new object) is disabled. So we want this editor to listen for two events: keydown and dblclick; as well as stop the default behavior of the enter key (submitting the form).

Instead of something like this [pseudo-but-close code]:

< input type="text" id="editor"
onkeydown="javascript:edit();return false"
ondblclick="javascript:return false;" />

We can simply write

< input type="text" id="editor" />

And then in script tags in a centralized location,

$E = YAHOO.util.Event;
$E.addListener('editor', 'keydown', edit);
$E.addListener('editor', 'dblclick', interceptAction);

The first argument is name of the id of the element in question (or an element object itself); the second is the javascript event to listen for; and the third is the name of your function. There’s an optional 4th argument, obj, which is an arbitrary object you can pass to your handler. If you do this, define your handler with two arguments: (e,obj).

Then define your functions:

/* handler to open your editor */
var edit = function(e){
// GRAB enter key
if( e.keyCode == 13){
// PREVENT form from submitting on enter
//[the default behavior of the enter key]
$E.preventDefault(e);
// DO things here
openInlineEditor();
}
}
/* handler to prevent default actions */
var interceptAction= function(e) {
// STOP the event from propagating to other listeners
$E.stopPropagation(e);
}

This is just the tip of the iceberg. I recommend reading up on this and looking at their examples and related articles linked on the page. It’s made my event handling easier and more fun.

CSS Debugging and Editing with Firebug


Javascript/AJAXLanguagesReviewsSoftware

Occasionally a new tool comes along that literally changes my life. Prototype, del.iciou.us, and most recently, Firebug. Using it, especially when evaluating CSS and javascript (especially when it’s all over!), has increased my productivity and saved me from many migraines. In this post, I plan to focus on some of the helpful features for debugging CSS. In a later post, I plan to cover how wonderful it is to debug and test javascript in firebug,

Especially on websites that have a lot of nested styles, finding where that extra padding is coming from can be a headache and waste a lot of time that could be spent being productive. With firebug, you can quickly see where each element is getting its styles and the full cascade of styles affecting it. From there, it is easy to alter the CSS and html live to figure out what needs to be done for the desired effect. This feature can be used for a world of uses. For example, say I wanted to override some of the CSS on my orkut.com profile. First, I open up my profile and open firebug:

The green check mark in the bottom right indicates that there are no errors (you can choose to display any or all of the following: javascript Errors, javascript warnings, CSS errors, or xml Errors) used on the page is valid. The dropdown list displays where styles are being defined.

Select “inspect” from the menu then hover to select the element to edit. On the right hand column, the styles are defined specifically for the element, but also what it inherits (font from panel table) and what is over-written (font from main table class). Dreamweaver 8 has a similar feature for showing the cascade of elements. As an added benefit, each of the rules listed has a hyperlink back to the file where it is defined. Also in this screen shot, notice that firebug displays the html hierarchy as a breadcrumb like chooser for the currently selected element. This breadcrumb is clickable and allows for quick access to the nearby html elements.

Here you can see how easy it is to work with or turn off styles. One great feature displayed in this screenshot is that when hovering over a defined color, a pop-up displays the color. The same is true for any images that are used in a style or in html.

And, viola! In only a few short minutes, I am able to find and define what style classes I want to override and see what the page will look like live as I made the adjustments: