IT IS HERE! Get Smashing Node.JS on Amazon Kindle today!
Show Posts
← Back to homepage

I’m writing this blog post to explain why I no longer consider assertion testing through projects like should.js, expect.js or chai ideal.

For the uninitiated, these modules allow you to write assertions in a very clear english-like way:

// my test file
expect(number).to.be.below(5);
someAssertion().should.be.ok;

But the most attractive quality is that they provide helper functions to make a lot of the common testing procedures like type checking straightforward.

Consider the situation where you want to test that a returned value is an object. With expect.js you write:

expect(ret).to.be.an('object');

Whereas, as we all know, in JavaScript you would have to write something much longer, because you need to make sure that typeof returns 'object' but also that the value is not null because we all know typeof null == 'object'

assert(ret && 'object' == typeof ret);

Forgetting to check that the value is not null before making such a test is a fairly common error, which can be very costly to make while writing tests. Even if you’re disciplined or just forego the usage of typeof altogether, you’ll find that Array type checking is just as complicated:

assert(Array.isArray(ret));

as the above line will definitely not work on browsers where Array.isArray is not supported.

Another common requirement is to test the “approximate” equality of two objects. We don’t actually want to test that the objects reference are equal, but to test that the objects are “equivalent”:

expect(obj).to.eql({ a: { b: { c: 'd' } });

If you want to see the usefulness of these utilities, I recommend you look at the tests I wrote for mongo-query. I implemented the entirety of the MongoDB query language for MyDB, and writing the tests for it was really easy and straightforward thanks to expect.js. The tests also need to run in all browsers and Node.JS, and that comes for free.

Beyond expect.js

The main problem with expect.js is that it acts as a framework. It’s a collection of loosely coupled utilities under a common namespace that happen to be useful when it comes to writing tests. This is fine in a world without a module system.

It’s rewarding to use expect.js because it makes things easy, but its design is not elegant nor simple. It’s easy in the way that jQuery makes it easy to do both AJAX and DOM by incorporating the same <script> tag, but that doesn’t mean it’s superior to combining modules that do each task separately, with their own code bases, tests and documentation.

The simple vs easy paradox has been expressed in this great presentation.

It’s not surprising expect.js was designed before the proliferation of excellent client-side module systems like browserify or component. A much better way to accomplish a similar outcome in a module-oriented way would be:

var assert = require('assert');
var type = require('type');
assert('object' == type(obj));
assert('array' == type(obj));

or if you needed to check for nested object equality:

var assert = require('assert');
var eql = require('deep-equal');
assert(eql(obj, { a: { b: { c: 'd' } } } }));

This has numerous advantages:

  • The learning curve is much lower.
    In the previous example it’s clear what the purpose of eql is. For expect, reading the documentation is mandatory, since like most frameworks it creates its own specific language. If someone is new to the codebase, they will only need to look into the modules that pertain a specific assertion they’re curious about, instead of an entire framework.
  • Smaller code footprint
    Tests will run faster, be faster to transport as a codebase to the cloud for automated browser testing.
  • Extensibility is controlled by the user. I receive numerous pull requests to add more functionality to expect.js all the time. The reason I’m writing this blog post is party to explain that it’s not that I should be merging more pull requests or adding new maintainers to expect.js, it’s that frameworks are not good for composing functionality.

The best assert

There’s one last advantage that expect-type modules have over simple old school assertions, in the context of JavaScript: they can provide more information about failures.

   ∴ ~ node
> require('assert')('undefined' != typeof window)
AssertionError: false == true
    at repl:1:19
    at REPLServer.self.eval (repl.js:110:21)
    at Interface. (repl.js:239:12)
    at Interface.EventEmitter.emit (events.js:95:17)
    at Interface._onLine (readline.js:202:10)
    at Interface._line (readline.js:531:8)
    at Interface._ttyWrite (readline.js:760:14)
    at ReadStream.onkeypress (readline.js:99:10)
    at ReadStream.EventEmitter.emit (events.js:98:17)
    at emitKey (readline.js:1095:12)

The error message AssertionError: false == true is not exactly useful. Luckily, there’s a way of solving this problem. In one of my previous articles I told you about the very useful v8 strack trace API.

Luckily, said API can be repurposed to get information about the line where the assertion failed, and produce a much better developer experience. TJ Holowaychuk (usual suspect) has created a little utility called better-assert to solve this problem. Update: another great module that was brought up to my attention that solves this as well is insist

$ test node index.js

/Users/guillermorauch/test/node_modules/better-assert/index.js:37
  throw err;
        ^
AssertionError: 'undefined' != typeof window
    at Object. (/Users/guillermorauch/test/index.js:4:1)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:902:3

There’s an elegant way to make this possible in all browsers however: we can rewrite the AST so that we automatically populate the second argument of assert calls (which corresponds to the message), with the content of the assertion itself.

Adam Reis is working on tackling this as part of our Open Academy mentorship program.

The hope is that when the V8 stack trace APIs are unavailable (such as when we’re performing automatic browser testing with Sauce Labs), code gets rewritten automatically.

assert('addEventListener' in window)

transparently becomes

assert('addEventListener' in window, "'addEventListener' in window")

through for example a browserify transform.

Conclusion

Thinking in a truly modular way for the JavaScript you write has an impact not on just your app’s main codebase, but also in the way you write tests, benchmarks and even documentation.

Whenever you catch yourself combining a lot of different utilities into a single module or namespace, consider if there’s a better alternative by making node_modules/ your namespace, and require() the way of accessing those utilities.

Your thoughts?

About Guillermo Rauch:

CTO and co-founder of LearnBoost / Cloudup (acquired by Automattic in 2013). Argentine living in SF.