Posts filled under: benchmark

Theoretical Node.js Real time Performance

Benchmarking Node.js seems to be hot topic lately, not week passes without a benchmark being posted on HackerNews All these benchmarks highlight different parts and usages of Node.js. On some of them Node.js performs well and on others it sucks, badly. Both types of benchmarks are useful as it shows other users what Node.js is capable off. But it also shows us, developers, what needs to be worked and could potentially become a bottleneck in our application stack. But either way, it’s a win/win situation.

Real time frameworks

When people start building real time applications they always want the fasted, best and the most bad-ass system there is available as it needs to handle a large amount of concurrent connected users. There isn’t much known about the performance of Node.js when handling real time connections. You can create a real time application on any language and stack you want but developers usually prefer to libraries and frameworks that do the hard lifting for them. The frameworks that they usually consider are Twisted or Tornado, a evented server written Python, Netty the Java powered asynchronous event framework or Node.js using Socket.IO.

Because I spend allot my free time working with and on Socket.IO I started wondering what the limits would be for Node.js based real time system. Can it handle 20k, 50k, 100k or even 500k of connected users? What is the limit and when will it break?

Felix Geisendörfer recently tweeted that Node.js can allocate 1.6 million concurrent HTTP server requests through a single socket before running out of memory. Resulting in a 1.53kb memory allocation per request. This was done by flushing a huge buffer of GET /\n\n in one HTTP connection. 1.6 million is quite impressive, but than again Node.js is known for it’s first class and high quality HTTP parser (Thanks Ry!).

Measuring connected sockets

It’s nice to know that your server can handle 1.6 million concurrent requests over one single connection, the possibility of this occurring in real life is 0.0%. A more realistic test would be to measure how many concurrent connected connections a single Node.js process could handle and much memory one single connection allocates. Creating a benchmark that would generate 1.6 million concurrent connected requests would require me to buy allot of IP addresses to distribute the requests as we are tied to 64k ephemeral ports per IP address. So instead of generating the actual load of 1.6 million sockets I decided to calculate the theoretical performance of a single node process.

I started coding up a small script that allows to create a bunch of real time connections:

/**
 * Dependencies
 */
var http = require('http')
  , agent = http.getAgent('127.0.0.1', 8080);

// Agent default to max sockets of 5, we need more
agent.maxSockets = Infinity;

// create more, agents un till we have enough
for(var i = 0; i < 4000; i++){
  http.get({
    agent: agent
  , path: '/'
  , port: 8080
  , host: '127.0.0.1'
  });

  console.log("Client connected: " + i);
}

While this doesn’t follow any best practices of doing a proper http benchmark it’s enough for this particular test as we are not testing processing performance of Node.js here but it’s HTTP / TCP socket limits. Now that the simple benchmark script ready I started building the script that handles the incoming the requests, to make it more realistic I made sure that both the request and the response object where stored in Node.js process memory so we could communicate with the connected sockets like you would normally do with a Comet / real time server. I decided to test of different storage backends for the request objects. First Array storage:

/**
 * Dependencies
 */
var http = require('http')
  , host = 'localhost'
  , port = '8080';

// the stuff
var connections = []
  , i = 0;

// process title so I can find it back more easily
process.title = "connection";
var server = http.createServer(function(req, res){
  connections[connections.length] = {req: req, res:res};

  res.writeHead(200);
  res.write('ping');

  console.log('established connections: ' + ++i);
});

/**
 * Send messages to the stored connections
 *
 * @api public
 */
function message(){

  var i = connections.length
    , connection;

  while(i--){
    connection = connections[i];
    connection.res.write('wut');
    console.log('pew');
  }
};

// spam each minute
setInterval(function(){
  message()
}, 1000 * 60 );

server.listen(port);
console.log('listening on 8080');

And another server instance that would use an Object to store the requests and responses.

/**
 * Dependencies
 */
var http = require('http')
  , host = 'localhost'
  , port = '8080';

// the stuff
var connections = {}
  , i = 0;

// process title so I can find it back more easily
process.title = "connection";
var server = http.createServer(function(req, res){
  connections[Object.keys(connections).length] = {req: req, res:res};

  res.writeHead(200);
  res.write('ping');

  console.log('established connections: ' + ++i);
});

/**
 * Send messages to the stored connections
 *
 * @api public
 */
function message(){

  var arr = Object.keys(connections)
    , i = arr.length
    , connection;

  while(i--){
    connection = connections[arr[i]];
    connection.res.write('wut');
    console.log('pew');
  }
};

// spam each minute
setInterval(function(){
  message();
}, 1000 * 60 );

server.listen(port);
console.log('listening on 8080');

Running the benchmark

So I stared testing out the Array based storage first as V8 is know for it’s high performing arrays. I started up the Node server, waited a while until the server was idle and fired off the simulation script. I got peak memory of 43.5mb while it was connecting all the sockets, after a few seconds (the garbage collected kicked in?) and the memory dropped back to 28.7mb RSS. Messaging the server gave it a small spike of memory but that was excepted. I re-ran the test 10x to confirm the findings and they produced the following averages:

Array:

  • Start: 12.6mb
  • Peak: 43.5mb
  • Idle: 28.7mb
  • messaging: 34.8mb / 11% cpu

Up next was the Object based store, surprisingly it came really close to Array based storage. It used a bit more memory, but for some use cases it would be worth to store the response/requests in a object because you can do easy key/client lookups. I re-ran the test 10x to confirm the findings and they produced the following averages:

Object:

  • Start: 12.6mb
  • Peak: 48.3mb
  • idle: 28.7mb
  • messaging: 35mb / 10.9% cpu

OMG what does it mean??

Now that we have these stats we can calculate the the theoretical limits of the one single process. We know that once single process is limited to a V8 heap of 1.7 gb. When you get near that limit your process already starts to die. We had a start up RSS memory of 12.6mb and the servers memory stabilized to 28.7mb so for the 4000 connections it spend 16.1mb thats 16.1mb / 4000 = 4.025kb for each connection. To fill up the server with stabilized connections it could reach a total of 1.7gb / 4.025kb = 422.000 connections. These findings come really close the 512000 concurrent Web Socket connections on groovy++ which took 2.5gb for the Java heap according the article.

I’m impressed. Node.js is the awesome sause that you should be eating when want to take a bite out of the real time web.

This article is translated to:

Top of Page