Roughly a year ago Salvatore Sanfilippo the author of Redis wrote a blog post discussing the inclusion of Lua as a scripting language. I finally decided to try this out, and let’s just say it’s pretty badass.
Lua is a great fit for Redis, they have similar philosophies, being simple, small, and fast. Suppose for example you have 200,000 jobs, each represented in Redis as a hash, and you want to map/reduce the job duration, the new scripting capabilities make this really easy!
Here’s the node setup script to generate these jobs:
var redis = require('redis')
, db = redis.createClient();
var n = 500000
, pending = n
, ms;
while (n--) {
ms = Math.random() * 200 | 0;
db.hset('job:' + n, 'duration', ms, function(){
--pending || process.exit();
})
}
Next here is what you might consider scripting in your host language without the new Redis scripting feature, manually reducing the value:
var redis = require('redis')
, db = redis.createClient();
var n = 200000
, start = new Date
, pending = n
, ms = 0;
while (n--) {
db.hget('job:' + n, 'duration', function(err, n){
if (err) throw err;
ms += ~~n;
--pending || (function(){
console.log('%d minutes spent processing jobs', ms / (1000 * 60) | 0);
console.log('took %ds', (new Date - start) / 1000 | 0);
process.exit();
})();
})
}
On my Air this took roughly 7s, not too great, keep in mind that there is no throttling here I’m just plastering it with 200k commands. Now let’s try it with Lua! The
following script is ad-hoc, but it’ll do the trick. To signal an error all you
have to do is return a table with the err slot. redis.call() is effectively
the public Redis API exposed to your Redis script, so you can use it just like you
would your host language Redis bindings or redis-cli(1).
local sum = 0
for i = 0, 200000, 1 do
local key = "job:" .. i
local ms = tonumber(redis.call("hget", key, "duration"))
if ms == nil then return { err = key .. " is not an integer" } end
sum = sum + ms
end
return sum
Here I’ve embedded it in the JS script, but you could of course generate these, load them from files etc (beware of redis-injection?).
var redis = require('redis')
, db = redis.createClient();
var script = '\
local sum = 0 \
for i = 0, 200000, 1 do \
local key = "job:" .. i \
local ms = tonumber(redis.call("hget", key, "duration")) \
if ms == nil then return { err = key .. " is not an integer" } end \
sum = sum + ms \
end \
return sum';
var start = new Date;
db.eval(script, 0, function(err, ms){
if (err) throw err;
console.log('%d minutes spent processing jobs', ms / (1000 * 60) | 0);
console.log('took %dms', new Date - start | 0);
process.exit();
});
After running the script with the new EVAL command what previously took several seconds dropped to ~850ms, much better. EVAL and EVALSHA are actually a lot more flexible than I’ve explained here, accepting keys and arbitrary arguments.
Check out antirez.com/post/an-update-on-redis-and-lua.html for more.
Mocha 0.14.0 adds a single feature - string diffs! This is a small but very handy feature for some. When the strings are small, Mocha will use a character diff, when consisting of several lines a line-numbered “gutter” is added and a word diff is used as shown in the following image:
This is very useful when authoring things like template engines, transpilers, and other string-based libraries. For example the Stylus test suite is comprised of nothing but acceptance tests, the input file is compiled, and the resulting CSS is checked using actual.trim().should.equal(css);. In combination with Mocha’s BDD interface I can now simply iterate through the files and define test-cases as shown here:
var stylus = require('../')
, fs = require('fs');
// test cases
var cases = fs.readdirSync('test/cases').filter(function(file){
return ~file.indexOf('.styl');
}).map(function(file){
return file.replace('.styl', '');
});
describe('integration', function(){
cases.forEach(function(test){
var name = test.replace(/[-.]/g, ' ');
it(name, function(){
var path = 'test/cases/' + test + '.styl';
var styl = fs.readFileSync(path, 'utf8');
var css = fs.readFileSync('test/cases/' + test + '.css', 'utf8');
var style = stylus(styl)
.set('filename', path)
.include(__dirname + '/images')
.include(__dirname + '/cases/import.basic')
.define('url', stylus.url());
if (~test.indexOf('compress')) style.set('compress', true);
style.render(function(err, actual){
if (err) throw err;
actual.trim().should.equal(css);
});
})
});
})
Now if something were go to wrong, you get a nice diff!
To support this feature all you have to do is populate err.expected and err.actual with their respective values, Mocha will take care of the presentation.
NOTE: I just toggled the colors, green is now the expected color
Connect 2.0 is here with new core middleware, miscellaneous improvements, and some new docs.
var app = connect()
.use(connect.logger('dev'))
.use(connect.static('public'))
.use(function(req, res){
res.end('hello world\n');
})
app.listen(3000);
Previously connect.Server inherited from Node’s core net.Server, this made it difficult to provide both HTTP and HTTPS for your application. The result of connect() (formerly connect.createServer()) is now simply a JavaScript Function. This means that you may omit the call to app.listen(), and simply pass app to a Node net.Server as shown here:
var connect = require('connect')
, http = require('http')
, https = require('https');
var app = connect()
.use(connect.logger('dev'))
.use(connect.static('public'))
.use(function(req, res){
res.end('hello world\n');
})
http.createServer(app).listen(80);
https.createServer(tlsOptions, app).listen(443);
The bodyParser() middleware is now nothing but a short-hand for adding the json(), multipart(), and urlencoded() middleware. Each of these populate req.body with an object containing the parsed values, multipart() provides req.body and req.files for uploads.
As of Node 0.6.0 fast, native compression capabilities are available, so now we have the compress() middleware supporting deflate and gzip.
The cookieParser() middleware now supports signed cookies, and accepts a secret. This replaces the need to pass session({ secret: string }) to the session() middleware. Signed cookies are available via req.signedCookies, and unsigned as req.cookies.
Previously a few of the core middleware would respond to error situations directly, these have been changed to simply next(err)-them along. This change allows you to specify customized behaviour by adding an error-handling middleware:
app.use(function(err, req, res, next){
if (4 == err.status / 100) {
// render a client-error page
} else {
// render a server-error page
}
});
As mentioned session() no longer requires a secret. The cookie .maxAge has been defaulted to null, meaning that it will be a browser-session cookie, expiring once the visitor closes their browser.
Third-party middleware should remain perfectly functional. This release of Connect is not compatible with Express 2.x, Express 3.0 is coming soon.
The following significant changes were made, as well as several others that do not impact public API, such as a full rewrite of the tests using Mocha.
cookieSession() middleware for cookie-only sessionscompress() middleware for gzip / deflate supportsession() “proxy” setting to trust X-Forwarded-Protojson() middleware to parse “application/json”urlencoded() middleware to parse “application/x-www-form-urlencoded”multipart() middleware to parse “multipart/form-data”cookieParser(secret) support so anything using this middleware may access signed cookiescookieParser()cookieParser()err.status support in Connect’s default end-pointstaticCache()res.headerSent checking nodes res._headerSent until node doesbasicAuth() req.remoteUser to req.usersession() to a browser-session cookie. Closes #475bodyParser() to use json(), urlencoded(), and multipart()errorHandler() is now a development-only middlewarenext() errors when possible so applications can unify logging / handlinghttp[s].Server inheritance, now just a function, making it easy to have an app providing both http and https.createServer() (use connect())secret option from session(), use cookieParser(secret)connect.session.ignore array supportrouter() middleware. Closes #262
The latest release of Mocha adds two new reporters, now totaling 13 packaged with the library. The two new ones are “json-cov” and “html-cov”, the latter inherits from the former in order to produce a single-page test coverage report.
If you’re curious what a live example looks like, check out this page.
With Node’s require.paths array out of the picture there’s a bit of boiler-plate involved in setting up test coverage for your project, but it’s pretty minimal. First you’ll want to install the jscoverage executable, this program parses your source code and spits out an instrumented version, effectively a bunch of lines that look like this:
$_jscoverage[filename][line]++;
This enables libraries like Mocha to execute the code as they normally would, however we can then extract this coverage information and generate fancy reports. jscoverage takes the input directory, and an output directory, I usually calls this “lib-cov”:
$ jscoverage lib lib-cov
Typically when testing your library you’ll use relative require() calls that look like this: var express = require('../'), and the library entry point is ./index.js. Normally this file might look something like below, or you may not have an ./index.js file at all.
module.exports = require('./lib/express');
By using ./index.js we can then alter this statement to conditionally export the instrumented version when the EXPRESS_COV environment variable is present:
module.exports = process.env.EXPRESS_COV
? require('./lib-cov/express')
: require('./lib/express');
Then all you need to do is invoke mocha with --reporter html-cov and redirect the stdout to a file such as coverage.html. Line 25 the test-cov target of the Express Makefile illustrates how I typically handle this but depending on your setup it may vary.
That’s it! remember to add “coverage.html” to your .npmignore and .gitignore files ;)
@extend is a great little feature which originated in SASS, and now finds a home in Stylus. If you’ve ever written CSS like this, you know it can become quite the pain to maintain large lists of selectors:
.message,
.warning,
.error {
font-size: 14px;
padding: 5px 10px;
border: 1px solid #eee;
border-radius: 5px;
}
.warning {
color: yellow;
}
.error {
color: red;
}
@extend makes this process flow naturally, and even supports inheritance. In Stylus you may use @extend or @extends, I prefer the latter personally but that’s up to you! Using this feature you would define .message on it’s own, and simply __@extend_ it from within the other rulesets:
.message {
font-size: 14px;
padding: 5px 10px;
border: 1px solid #eee;
border-radius: 5px;
}
.warning {
@extends .message;
color: yellow;
}
.error {
@extends .message;
color: red;
}
Taking this even further with inheritance:
.fatal-error {
@extends .error;
font-weight: bold
}
Would yield the following CSS:
.message,
.warning,
.error,
.fatal-error {
font-size: 14px;
padding: 5px 10px;
border: 1px solid #eee;
border-radius: 5px;
}
.warning {
color: #ff0;
}
.error,
.fatal-error {
color: #f00;
}
.fatal-error {
font-weight: bold;
}
Here’s another example:
form
button
padding: 10px
border: 1px solid #eee
border-radius: 5px
ul
li a
@extends form button
Yielding:
form button,
ul li a {
padding: 10px;
border: 1px solid #eee;
border-radius: 5px;
}
And of course you may utilize the alternative Stylus syntax if you prefer:
.message
font-size: 14px
padding: 5px 10px
border: 1px solid #eee
border-radius: 5px
.warning
@extends .message
color: yellow
.error
@extends .message
color: red
.fatal-error
@extends .error
font-weight: bold
Grab Stylus 0.22.4 for these goodies!
hahaha awesome