Using Stubby in Javascript Integration Tests

A concern I often hear from people in the software world is that testing takes too long. Some of this is perception, as in some cases their test frameworks are being run in multiple threads and they do not realize their 1 minute build takes 10 CPU minutes to complete. In some cases they are correct, their tests do take longer than they could.

One idea I have been considering lately is separating the functional testing with both client and server and performing the majority of testing using integration frameworks for each. This makes the server side testing very simple as your framework merely needs to understand HTTP and many good frameworks like webtest in the Python world exist help you. Creating a client side framework is more difficult because stripping away the server side leaves you with a hole, something still needs to serve your application and provide it the data for your tests.

I have spent some time looking for a way to mock or stub complex servers for a while and even built a crude Python server for a set AngularJS tests that could serve mock responses back. I was looking for a few specific things in my mock server:

1. It needed it to also server my Javascript application, just like my real server
2. It had to be configurable during my test, so I can check my UI behavior with data and without
3. It had to be fast, very fast

I finally stumbled upon stubby one day. Its a web server designed to be given stub configurations to serve up in response to matching requests and is available for Java, NodeJS and .NET. Node has been very popular lately for build and package management of Javascript applications so this seemed like a nice fit. Stubby also supported templates in the response values which opens some interesting possibilities.

I had grunt installed already so getting started was as easy as installing the available grunt-stubby package:

npm install grunt-stubby

From their I knew I had two goals:

1. Stubby needed to serve my static application files
2. I needed to be able to start stubby in Grunt and then run a separate task

Out of the box stubby does not appear to support serving static files, at least I found no mention of it in their documentation. They do however support serving files as response bodies. With their templates and files as response bodies support we can setup a system to server any of our static files by creating a stub.

stubby-example/static.json

{
  "request": {"url": "^/static/(.+)"},
  "response": {"file": "./static/<% url[1] %>"}
}

Stubby allows you to write JSON or YAML files describing the requests to expect and the response values to return. Responses can be included as strings, JSON objects or files. You can also return response codes other than 200 to allow testing of error handling. This stub matches any files with URL paths starting with /static/, uses a regular expression to match any text after that and serves the file from disk with that path as the HTTP response. You can then feed this JSON file to stubby when its started, via its admin HTTP interface or its JS API.

The second problem to solve is starting stubby alongside my tests. Grunt comes in very handy here as creating a grunt task that starts stubby and runs my tests performs exactly as we desire.

In the example below I configured a simple JSON response as the contents of my static file:

{"value": true}

stubby-example/Gruntfile.js

module.exports = function(grunt) {
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    stubby: {
      stubsServer: {
        files: [{src: ['stubs/*.json']}],
        options: {
          callback: function (server, options) {
            server.get(1, function (err, endpoint) {
              if (!err) {
                console.log(endpoint);
              }
            });
          },
          stubs: 8882,
          admin: 8889,
          mute: false
        }
      }
    },
  });

  grunt.loadNpmTasks('grunt-stubby');

  grunt.registerTask('http', 'Make an HTTP request', function() {
    var done = this.async();

    grunt.log.writeln('Requesting data.json');
    var request = require('request');
    request('http://localhost:8882/static/data.json', function(error, resp, body) {
      if (!error && resp.statusCode == 200) {
        var data = JSON.parse(body);
        grunt.log.writeln('/static/data.json was server correctly: ' + data.value);
        grunt.log.writeln('body: ' + body);
      } else {
        grunt.log.writeln('/static/data.json was not served correctly');
        grunt.log.writeln('Response Code: ' + resp.statusCode.toString());
      }
      done();
    });
  });

  grunt.registerTask('run', 'Start a stubby server and make an HTTP request', function() {
    grunt.task.run(['stubby', 'http']);
  });
};

In this example I created a custom Grunt task to contact my stubby server and assert that the response was the expected value, but it could easily have been a set of tests running in karma or qunit. The key is the run task, which encapsulates both the stubby and http tasks, which causes stubby to continue running until all other tasks complete.

Overall stubby shows a lot of potential for stubbing out complex servers. With Angular code bases running e2e tests in karma it should be fairly easy to push stubs at the beginning of a testing using their admin HTTP interface and something like karma-fixture to make files available as JSON fixtures. For applications using intern and RequireJS or other AMD model structure the fixtures could be required directly and stubby potentially configured directly. In either case we now have a server that responds in 10s of milliseconds instead of seconds, making our UI testing that much faster.