node recap

Figured I would whip up a quick blog post during lunch to mention some recent additions / releases to node projects.

Express Mongoose

Aaron recently released express-mongoose which is a fantastic plugin for any of you using Mongoose allowing you to pass query promises directly to res.render(), res.partial(), and `res.send() calls, deferring rendering until complete.

For example you may currently be nesting as shown below:

var News = db.model('News');
app.get('/dashboard', function (req, res, next) {
  req.user.getLikes(function(err, likes){
    if (err) return next(err);
    News.getLatest(function(err, news){
      if (err) return next(err);
      res.render('dashboard', { likes: likes, latestNews: news });
    });
  });
});

well now you can simply pass the promises, next()ing exceptions. Much cleaner!

var News = db.model('News');
app.get('/dashboard', function (req, res) {
  res.render('dashboard', {
      likes: req.user.getLikes()
    , latestNews: News.getLatest()
  });
});

Asset

asset is a tiny but helpful utility written with node to manage assets such a JavaScript or css libraries. You could think of this as the homebrew or bundler of assets.

Installing assets is extremely simple, and by default will be installed to ./public. For example we can install g.raphael and g.pie to ./public/javascripts which will install the raphael dependency as well:

$ asset g.raphael g.pie -o public/javascripts

   install : raphael@1.4.7
   install : g.raphael@0.4.1
   install : g.pie@0.4.1
  download : raphael@1.4.7
  complete : raphael@1.4.7 public/javascripts/raphael.js
  download : g.raphael@0.4.1
  complete : g.raphael@0.4.1 public/javascripts/g.raphael.js
  download : g.pie@0.4.1
  complete : g.pie@0.4.1 public/javascripts/g.pie.js

Last night I added a tiny patch which adds support for assets.json, which you can add to your project to list dependencies. For example this file might contain something like below

{
     "g.raphael": "0.4.1"
   , "jquery": "1.5.2"
   , "modernizr": "1.7"
 }

Which we can then install with a single command:

$ asset

    install : g.raphael@0.4.1
 dependency : raphael@1.4.7
    install : raphael@1.4.7
    install : jquery@1.5.2
    install : modernizr@1.7
   download : jquery@1.5.2
   complete : jquery@1.5.2 public/jquery.js
   download : raphael@1.4.7
   complete : raphael@1.4.7 public/raphael.js
   download : g.raphael@0.4.1
   complete : g.raphael@0.4.1 public/g.raphael.js
   download : modernizr@1.7
   complete : modernizr@1.7 public/modernizr.js

Cluster updates

Cluster is a multi-process server manager for node. Recent additions now allow you to run cluster without a server, which is great for processing job queues! below is an example of how to utilize cluster for such a task:

var cluster = require('../');

var proc = cluster()
  .set('workers', 4)
  .use(cluster.debug())
  .start();

if (proc.isWorker) {
  var id = process.env.CLUSTER_WORKER;
  console.log('  worker #%d started', id);
  setInterval(function(){
    console.log('  processing job from worker #%d', id);
  }, 1000);
} else {
  setTimeout(function(){
    proc.close();
  }, 10000);
}

Express Resource

For those of you who have been looking for resourceful routing for Express, you may have heard or express-resource already, and if not express-resource utilizes regular Express HTTP verb/pathname routing dressed up in an API better suited for resources.

Installation:

$ npm install express-resource

Below is an example use-case providing actions for forums and their associated threads. By calling forum.add(threads) routes such as GET /forums/:forum/threads/:thread and GET /forums/:forum/threads are mapped to action callbacks. This library also supports auto-loading of resources, providing them as req.forum, req.thread etc instead of manually doing this using the values of req.params.forum or req.params.thread.

  var forum = app.resource('forum', actionsHere)
    , threads = app.resource('thread', actionsHere)

  forum.add(threads);

For more details check out the documentation and tests in the repo.

RedisKit

Week or so ago I started a small high-level redis library called rediskit. I started creating the lower level structures, and will be building up to composite structures and higher level abstractions.

Install with:

$ npm install rediskit

Example usage:

var list = new List('pets')
  , client = list.client
  , tobi = new Hash('pet:tobi')
  , loki = new Hash('pet:loki')
  , jane = new Hash('pet:jane');

list.rpush('tobi');
list.rpush('jane');
list.rpush('loki');

tobi.set('age', 1);
loki.set('age', 0.5);
jane.set('age', 3);

list.sort.by('pet:*->age').get('#').get('pet:*->age').end(function(err, res){
  res.should.eql(['loki', '0.5', 'tobi', '1', 'jane', '3']);
});

RedBack

I also just saw a tweet about a similar library redback which is worth checking out if you are a redis fan.

Installation:

$ npm install redback

Usage:

var redback = require('redback').createClient();

//redback.create<Structure>(key)

var user1 = redback.createHash('user1');
user.set({username:'chris', password:'redisisawesome'}, callback);

var log = redback.createCappedList('log', 1000);
log.push('Log message ...');

var user3 = redback.createSocialGraph(3);
user3.follow(1, callback);

Redis Implemented With Node

Nedis is a (partial) redis implementation written with node. Primarily for fun, however as our team grows larger, and as we add more non-technical team members over at LearnBoost I figured it would be nice help prevent the need for compiling development dependencies.

Nedis is an old side project I had going, and is no where near complete, but it does work, so I figured I would open-source it. Currently Nedis implements the unified Redis protocol which is an brilliantly simple binary-safe protocol that is human and machine friendly.

Using Existing Tools

Currently we use Redis for sessions in our app, so having a drop-in replacement is a great way to get session support for your app without booting up redis-server. For example the nodejs module connect-redis can utilize Matt Ranney’s fantastic redis client without change.

Another neat side-effect is that you can use existing redis tools such as redis-cli to interact with Nedis. First let’s start Nedis with nedis-server:

 $ nedis-server

Now we can play with the cli, interacting with node

 $ redis-cli 

 redis> hmset users:tj email tj@vision-media.ca age 23
 OK
 redis> hgetall users:tj
 1) "email"
 2) "tj@vision-media.ca"
 3) "age"
 4) "23"
 redis> keys users:*
 1) "users:tj"

Note that nedis-server basically consists of no more than the line of js below, so it’s easy to boot from within your process if desired.

 nedis.createServer(options).listen(port);

Supported Commands

Below is a list of the commands currently supported by Nedis

  • PING
  • ECHO
  • QUIT
  • SELECT
  • HLEN
  • HVALS
  • HKEYS
  • HSET
  • HMSET
  • HGET
  • HGETALL
  • HEXISTS
  • TYPE
  • EXISTS
  • RANDOMKEY
  • DEL
  • RENAME
  • KEYS
  • FLUSHDB
  • FLUSHALL
  • DBSIZE
  • INFO
  • BGREWRITEAOF
  • GET
  • GETSET
  • GET
  • SETNX
  • INCR
  • INCRBY
  • DECR
  • DECRBY
  • STRLEN
  • APPEND
  • SETRANGE
  • GETRANGE
  • MGET
  • MSET

  • MSETNX

    I have yet to do any kind of profiling, heavy optimization, or stress testing. If nothing more hopefully Nedis will help you guys explore Redis, or how you can prototype basic databases with node. Head over to the GitHub repo for installation instructions etc.

Stylus 0.4.0 released

Stylus for those who are not familiar with it, is a dynamic language which compiles down to css, written with JavaScript for node.js.

CSS Syntax Support

Previously Stylus only allowed the use of our indented grammar, which may deter designers that are not comfortable learning or using a new syntax. For this reason and to aid in copy/pasting of css stylesheets I have added support for optional semi-colons, colons, commas, and braces, this means that most css is valid.

Example

The example below is completely valid, showing the use of mixins within both our indentation-style usage, and our css-style usage.

  vendor(prop, args)
    -webkit-{prop} args
    -moz-{prop} args
    {prop} args

  border-radius()
    vendor('border-radius', arguments)

  button
  a.button
  input[type=submit]
  input[type=button]
    border-radius 3px 5px
    color black
    &:hover
      background black
      color white

  button,
  a.button,
  input[type=submit],
  input[type=button] {
    border-radius: 3px 5px;
    color: black;
    &:hover {
      background: black;
      color: white;
    }
  }

yielding:

button,
a.button,
input[type=submit],
input[type=button] {
  color: #000;
  -webkit-border-radius: 3px 5px;
  -moz-border-radius: 3px 5px;
  border-radius: 3px 5px;
}
button:hover,
a.button:hover,
input[type=submit]:hover,
input[type=button]:hover {
  background: #000;
  color: #fff;
}
button,
a.button,
input[type=submit],
input[type=button] {
  color: #000;
  -webkit-border-radius: 3px 5px;
  -moz-border-radius: 3px 5px;
  border-radius: 3px 5px;
}
button:hover,
a.button:hover,
input[type=submit]:hover,
input[type=button]:hover {
  background: #000;
  color: #fff;
}

This of course applies to use within mixins as well, using the indentation-style we may define the mixin as:

  fade-to(to = .5)
    &:hover
      opacity to

  button
    fade-to(.1)

or with the css-style as:

  fade-to(to = .5)
    &:hover {
      opacity: to;
    }

  button {
    fade-to(.1);
  }

both yielding:

  button:hover {
    opacity: 0.1;
  }

Hope you enjoy the new feature, be sure to watch the project on GitHub to keep up to date with improvements.

Introducing Tobi

A few days ago we released another small test related project named Tobi for nodejs. Tobi is similar to Ruby tools such as Capybara or Webrat in fulfilling the need for headless acceptance testing. Tobi utilizes jQuery and node’s jsdom to give you an expressive foundation for testing your application in a familiar way.

Motivation

For those of you who follow our work at LearnBoost we have a Selenium / Sauce Labs based acceptance testing tool named Soda, which is fantastic to work with and helps us test our massive app before pushing to production. However, large soda suites typically take several minutes to run, even when running parallel in our targeted Sauce Lab browers. This is of course a very important task to perform, but is far to slow for constant execution during development, so we created Tobi.

Examples

Tobi is test-framework agnostic, meaning it will generally work with any test framework out there for node. Below is an example which runs stand-alone.

Paired with the should.js library expressive assertions can be made on both the response as well as the DOM, for example:

res.should.have.header('Content-Type', 'text/html');
$('ul.messages').should.have.one('li', 'Successfully authenticated');

Locators

Tobi uses “locators” which are similar to those found with Selenium, to essentially expand on css selectors making your code that much more readable. For example rather than selecting a form input via css selector such as

$('[name=username]').val('tj');

we can use the browser.* methods to help us

browser.type('username', 'tj');

Zombies?

Oddly enough a similar project yet-again sprung up a few days before we open sourced ours, named zombie.js. It seems to have similar intentions, leaning more towards a “real” browser implementation. Although interesting acceptance tests should be run in real browsers, which is why I like to consider Tobi more of an integration testing framework, or a pseudo-acceptance sanity check for developers, soda still takes care of deployment.

So, check them both out, see which works for you, but dont forget to apply functional tests, I highly recommend giving Sauce Labs a look. There is no replacement for real browsers! get on it!

Tobi?

Tobi is my ferret, hes a node hacker tobi

More Information

For the full usage documentation head over to the github repo.

node-canvas open sourced

Today we open sourced our latest LearnBoost project, node-canvas, a canvas implementation for nodejs. The project is certainly a work in progress but we implement a large portion of the api, as well as node-specific additions such as Canvas#toBuffer() and Canvas#createPNGStream().

Examples

Shown below is the test suite running side by side with the browser implementation. node-canvas renders the left, and the browser (chrome) renders the canvas on the right using the same code.

node canvas test suite

Below is an example of chrome rendering flot on the left, and node-canvas on the right.

flot running on nodejs