Connect 2.0

Connect 2.0 is here with new core middleware, miscellaneous improvements, and some new docs.

var app = connect()
  .use(connect.logger('dev'))
  .use(connect.static('public'))
  .use(function(req, res){
    res.end('hello world\n');
  })

 app.listen(3000);

HTTP and HTTPS

Previously connect.Server inherited from Node’s core net.Server, this made it difficult to provide both HTTP and HTTPS for your application. The result of connect() (formerly connect.createServer()) is now simply a JavaScript Function. This means that you may omit the call to app.listen(), and simply pass app to a Node net.Server as shown here:

var connect = require('connect')
  , http = require('http')
  , https = require('https');

var app = connect()
  .use(connect.logger('dev'))
  .use(connect.static('public'))
  .use(function(req, res){
    res.end('hello world\n');
  })

http.createServer(app).listen(80);
https.createServer(tlsOptions, app).listen(443);

Body parsers

The bodyParser() middleware is now nothing but a short-hand for adding the json(), multipart(), and urlencoded() middleware. Each of these populate req.body with an object containing the parsed values, multipart() provides req.body and req.files for uploads.

Core compression

As of Node 0.6.0 fast, native compression capabilities are available, so now we have the compress() middleware supporting deflate and gzip.

Cookie parser

The cookieParser() middleware now supports signed cookies, and accepts a secret. This replaces the need to pass session({ secret: string }) to the session() middleware. Signed cookies are available via req.signedCookies, and unsigned as req.cookies.

Error delegation

Previously a few of the core middleware would respond to error situations directly, these have been changed to simply next(err)-them along. This change allows you to specify customized behaviour by adding an error-handling middleware:

app.use(function(err, req, res, next){
  if (4 == err.status / 100) {
    // render a client-error page
  } else {
    // render a server-error page
  }
});

Session

As mentioned session() no longer requires a secret. The cookie .maxAge has been defaulted to null, meaning that it will be a browser-session cookie, expiring once the visitor closes their browser.

Third-party middleware

Third-party middleware should remain perfectly functional. This release of Connect is not compatible with Express 2.x, Express 3.0 is coming soon.

Changelog

The following significant changes were made, as well as several others that do not impact public API, such as a full rewrite of the tests using Mocha.

  • Added cookieSession() middleware for cookie-only sessions
  • Added compress() middleware for gzip / deflate support
  • Added session() “proxy” setting to trust X-Forwarded-Proto
  • Added json() middleware to parse “application/json”
  • Added urlencoded() middleware to parse “application/x-www-form-urlencoded”
  • Added multipart() middleware to parse “multipart/form-data”
  • Added cookieParser(secret) support so anything using this middleware may access signed cookies
  • Added signed cookie support to cookieParser()
  • Added support for JSON-serialized cookies to cookieParser()
  • Added err.status support in Connect’s default end-point
  • Added X-Cache MISS / HIT to staticCache()
  • Added public res.headerSent checking nodes res._headerSent until node does
  • Changed basicAuth() req.remoteUser to req.user
  • Changed: default session() to a browser-session cookie. Closes #475
  • Changed: no longer lowercase cookie names
  • Changed bodyParser() to use json(), urlencoded(), and multipart()
  • Changed: errorHandler() is now a development-only middleware
  • Changed middleware to next() errors when possible so applications can unify logging / handling
  • Removed http[s].Server inheritance, now just a function, making it easy to have an app providing both http and https
  • Removed .createServer() (use connect())
  • Removed secret option from session(), use cookieParser(secret)
  • Removed connect.session.ignore array support
  • Removed router() middleware. Closes #262
  • Fixed: set-cookie only once for browser-session cookies
  • Fixed FQDN support. dont add leading “/”
  • Fixed 404 XSS attack vector. Closes #473
  • Fixed HEAD support for 404s and 500s generated by Connect’s end-point

Connect 1.8.0 - multipart support

Connect 1.8.0 is a tiny but substantial release because it adds multipart/form-data support via Felix Geisendörfer’s fantastic multipart parser formidable, it’s great and chances are you’re already using it!

The bodyParser() middleware now unifies multipart, json, and x-www-form-urlencoded parsing, providing the req.body object containing the parsed data. For security purposes the files are placed in a separate object, req.files, however just as accessible. A constant struggle that I’ve seen in the community is the concept of “missing” request data events so this will prevent further confusion. The downside to this is that if you wish to report upload progress, or access files and fields as the request is streamed, you will have to use formidable directly.

Before this addition your use of formidable might look something like the following (with Express):

  app.post('/someform', function(req, res, next){
    var form = new formidable.IncomingForm;
    form.parse(req, function(err, fields, files){
      if (err) return next(err);
      // do something with files.image etc
    });
  });

With the new bodyParser() all you need is:

  app.use(express.bodyParser());

  app.post('/some-form', function(req, res){
    // do something with req.files.image
  })

The middleware takes the same options that are mentioned on formidable’s GitHub repo page, so you can specify things like size limits, retaining extensions, the upload directory etc.

  app.use(express.bodyParser({
    uploadDir: '/tmp/uploads'
  }));

That’s it! but I think this will cover well above the 80% use-case and prevent a lot of headaches that people have with the previous the inconsistencies.

Connect 1.7.0 - fast static file memory cache and more

Be sure to update Connect to the latest release 1.7.0 for the following goodies!

staticCache() middleware

1.7.0 now provides a new middleware named staticCache(), acting as a memory cache on top of the regular static() middleware. Older versions of Connect used to provide memory caching, however it was baked right into static(), bloating the middleware more than necessary.

Benchmarks

The test file for these benchmarks is a small 4kb readme:

$ du -h Readme.md 
4.0K    Readme.md

and the following ApacheBench command:

$ ab -n 5000 -c 50 -k http://local/Readme.md

First up we have Connect static(), which does not perform any caching, performing disk I/O each request, serving ~2400rps.

connect()
  .use(connect.static(__dirname))
  .listen(3000)

Next up we have node-static one of the other popular node solutions, serving ~3800rps

var static = require('node-static')
  , file = new static.Server(__dirname);

http.createServer(function(req, res){
  file.serve(req, res);
}).listen(3000);

Then we have the new staticCache() middleware paired with static() serving ~5000rps, a 24% increase over node-static (which also performs memory caching), and ~52% over static() alone.

connect()
  .use(connect.staticCache())
  .use(connect.static(__dirname))
  .listen(3000)

staticCache() is configurable, allowing you to specify the maximum number of cache objects to store, and the maximum size allowed, so you can cap resources appropriately. The middleware implements a Least Recently Used algorithm to prioritize popular files, knocking less popular objects out of the cache.

res.headerSent

Node core has a private flag named res._headerSent, allowing you to check if the header has been written or not, this is extremely useful in some cases, so Connect has publicized this property as res.headerSent until Node core does (if ever), using a getter to reference the private property.

logger() immediate option

The logger() middleware now provides { immediate: true }, as by default this middleware will log the response of a request, not when the request is first received, allowing logging of response-times etc.

Connect vs JSGI

Last week Kris Zyp posted JSGI vs Connect, claiming that Connect is slower than JSGI.

Another claim is that JSGI is far more intuitive, which I consider false. First of all the library does not follow common nodejs idioms, and the configuration also causes the “boomerang effect”, although in the end only the community will decide what they want.

Krisbenchmarks generally consisted of testing the Connect middleware loop which invokeshandle()` on each of the layers. Even with the typical “Hello World” benchmark these claims can be quickly disproven, since a simple for loop measures in the microseconds, and has no effect in comparison to ANY real-world application routine.

Comparing Connect / JSGI / Node

The following gist contains the test files used. As you can see Connect middleware can easily be used stand-alone and in my opinion feels more natural to implement, where as the JSGI middle is comparably awkward.

Below are the results of ab -n 8000 -c 50 -k http://dev:8080/ per each server. Funny enough, JSGI actually placed last in the “Hello World” game:

connect: 8859 rps
jsgi: 8357 rps
node: 9184 rps

OMG JSGI IS SO SLOW.. well no, it is just fine, but so is Connect. We are talking about milliseconds here! ANY call to the database ANY “real” operation will be far more expensive.

Response Curve

In my Connect Introduction article the following graph illustrates that Connect has nearly no overhead, as the response curve is almost identical to that of a plain node server.

Hello World

Closing Thoughts

The truth is, be it Connect, JSGI, fab, or plain old node, none of these will have a profound effect on performance, and can certainly not be considered a bottleneck.

Use what you like! Connect is backed by the all mighty Ext JS team, is licensed MIT until the end of time, and is in development by several node rockstars, it will adapt as the community sees fit, but is already gaining speed.

Happy nodeing!

Connect - Middleware For NodeJS

Yesterday myself and Tim Caswell open-sourced the first Ext JS nodejs project, Connect. Connect is an abstraction layer, providing node developers with an effective, high performance solution for rapid development using interchangable components called “middleware”.

Middleware

Middleware provides node developers with simple “plug and play” modules, which may be stacked in any order desired, aiding in rapid development. Connect middlware are regular node modules exporting the handle() method, however conceptually they fall into two categories, filters, and providers.

A “filter” conceptually sits arbitrarily within the middlware stack, processing incoming and outgoing traffic, but typically does not respond to a request. An example of this is the log filter provided by Connect, it does not respond to any request, it simply proxies function calls in order to log request data.

A “provider” differs conceptually in the fact that it provides an end-point in the stack. However this may not always be the case. For example a json-rpc provider may choose not to process or respond to a request if the Content-Type header is not application/json. A classical example of a provider is the static provider which serves static files.

Executable

Connect is a dual purpose library, rapid development is not the only goal in mind. Also provided is the connect executable which can be used to daemonize Connect servers (and regular node servers) using an extremely simple command-line interface.

For more information on the connect executable install Connect and run:

$ man connect

Performance

Performance is a top priority for Connect, and we have the benchmarks to prove it. The following benchmarks were performed with ApacheBench, node 0.1.97, thin 1.2.7, sinatra 1.0, and rack 1.1.0.

The benchmarks bundled with Connect consistently showed that the library has nearly no overhead compared to a regular node server when responding with the typical “Hello World” response.

Hello World

Next up we have node, sinatra (thin), and Connect serving jquery.js, a roughly ~57kb.

Static jQuery

If you wish to run the benchmarks on your own machine, first execute:

$ make benchmark

Then generate the graphs with gnuplot:

$ make graphs

Once complete:

$ open results/graphs

Conclusion

Connect is awesome, what are you waiting for! head over to the Github repo and fork away.