Posts filled under: IPC

This week in Node

This week in Node is a weekly blog post that focuses on the latest changes and updates in the Node.js repository and in NPM modules.

Forking and IPC

Commit 9e26dab150e and 337c48db5fe landed a new method in the child_process module. The fork method allows you to spawn a new Node.js process and pass messages between your current Node.js process and the newly created process using a Inter Process Communication (IPC) channel.

So why is this an important change? This is the first step to proper and hopefully first class support for multiple processing Node.js. When you are doing allot of CPU heavy processing it can take a while before your function is done and is ready to return to the event loop again. Ideally you would use a job server such as Gearman or Resque for heavy processing but that is not always and option. As your application is busy with processing data it can the amount of concurrent your application can handle, but you can now offload all the processing to a different Node.js process and be notified once the work is done.

The API is quite simple and straight forward:

var cp = require('child_process')
  , n = cp.fork(__dirname + '/process.js');

n.on('message', function(m) {
  console.log('PARENT got message:', m);
});

n.send({ hello: 'world' });

And inside process.js:

process.on('message', function(m) {
  // illustrative function ;) 
  hardcoreProcessing(m, function(err, res){
    process.send(res);
  })
});

Wait, what? No Web Worker API

That was the first thing that came to my mind, why don’t we just use the Web Worker API for it. It was designed for stuff like this and it’s already known to some developers even the Node.js website states:

But what about multiple-processor concurrency? Aren’t threads necessary to scale programs to multi-core computers? Processes are necessary to scale to multi-core computers, not memory-sharing threads. The fundamentals of scalable systems are fast networking and non-blocking design—the rest is message passing. In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design.

I don’t know how far in the future we currently are but it with the current release it’s actually quite easy to emulate with a few simple lines of code:

if (process.env.NODE_CHANNEL_FD){
  (function(){
    var onmessage;

    // create onmessage api
    Object.defineProperty(GLOBAL, 'onmessage', {
      get: function(){ return onmessage }
    , set: function(fn){
        onmessage = fn;
        process.on('message', function(msg){
          onmessage({data:msg});
        });
      }
    });

    // all we need to do is proxy the process.send
    Object.defineProperty(GLOBAL, 'postMessage', {
      get: function(){ return process.send }
    });
  })()
}

Adding those lines of code would allow you to use onmessage and postMessage as specified in the Web Worker API. But non the less this forking and IPC between different node instances is a more than welcome addition to the node core. See full .fork api here.

Libuv

Integration of Libuv (formerly known as liboio) in to Node.js has started. Libuv provides a high-concurrency and high-performance I/O on all operating systems. This would allow Node.js to be deployed on Windows without using Cygwin or MinGW Unix emulation layers. Because these are emulation layers, the performance has always been really poor and Libuv changes that by using native Windows APIs.

For more information about Libuv check out the github repository and Ry’s Nodeconf 2011 slides.

Top of Page