Posts filled under: Libuv

This week in Node

Welcome to another epic episode of This week in Node, where I blog about the latest news and updates in the Node.js community.

Node 0.4.8

A new Node stable has been released just one day after I posted This week in Node #20. This release brings us more performance for encrypted SSL connections. This is achieved by disabling the compressions in OpenSSL. This improves memory consumption and speed. If you want to apply compression it should now be done in “userland”.

Full list of fixes

  • #974 Properly report traceless errors (isaacs)
  • #983 Better JSON.parse error detection in REPL (isaacs)
  • #836 Agent socket errors bubble up to req only if req exists
  • #1041 Fix event listener leak check timing (koichik)
  • #1038 Fix dns.resolve() with ‘PTR’ throws Error: Unknown type “PTR” (koichik)
  • #1073 Share SSL context between server connections (Fedor Indutny)
  • Disable compression with OpenSSL. Improves memory perf.
  • Implement os.totalmem() and os.freemem() for SunOS (Alexandre Marangone)
  • Fix a special characters in URL regression (isaacs)
  • Fix idle timeouts in HTTPS (Felix Geisendörfer)
  • SlowBuffer.write() with ‘ucs2′ throws ReferenceError. (koichik)
  • http.ServerRequest ‘close’ sometimes gets an error argument (Felix Geisendörfer)
  • Doc improvements
  • cleartextstream.destroy() should close(2) the socket. Previously was being mapped to a shutdown(2) syscall.
  • No longer compile out asserts and debug statements in normal build.
  • Debugger improvements.
  • Upgrade V8 to 3.1.8.16.

Download the latest version from the Node.js site http://nodejs.org/dist/node-v0.4.8.tar.gz

New Core team members

Just announced today, 2 new core team members (Bert and Felix) have been added to Node. Bert Belder (@piscisaureus) has contributed countless bug fixes for the windows, not only fixing bugs but also implementing new features in windows. He has been working been working allot on Libuv the last couple of weeks / months and it’s being integrated in to node as we speak.

Felix (@felixge) has been working and patching allot of different I/O streams which are one the most important parts of Node.js. But not only did he work on Streams, he fixed and worked on allot of more important parts of Node.js.

So congratulations for the both of them thanks for all the bugs that you guys fixed, and will fix in the future!

This week in Node

This week in Node is a weekly blog post that focuses on the latest changes and updates in the Node.js repository and in NPM modules.

Forking and IPC

Commit 9e26dab150e and 337c48db5fe landed a new method in the child_process module. The fork method allows you to spawn a new Node.js process and pass messages between your current Node.js process and the newly created process using a Inter Process Communication (IPC) channel.

So why is this an important change? This is the first step to proper and hopefully first class support for multiple processing Node.js. When you are doing allot of CPU heavy processing it can take a while before your function is done and is ready to return to the event loop again. Ideally you would use a job server such as Gearman or Resque for heavy processing but that is not always and option. As your application is busy with processing data it can the amount of concurrent your application can handle, but you can now offload all the processing to a different Node.js process and be notified once the work is done.

The API is quite simple and straight forward:

var cp = require('child_process')
  , n = cp.fork(__dirname + '/process.js');

n.on('message', function(m) {
  console.log('PARENT got message:', m);
});

n.send({ hello: 'world' });

And inside process.js:

process.on('message', function(m) {
  // illustrative function ;) 
  hardcoreProcessing(m, function(err, res){
    process.send(res);
  })
});

Wait, what? No Web Worker API

That was the first thing that came to my mind, why don’t we just use the Web Worker API for it. It was designed for stuff like this and it’s already known to some developers even the Node.js website states:

But what about multiple-processor concurrency? Aren’t threads necessary to scale programs to multi-core computers? Processes are necessary to scale to multi-core computers, not memory-sharing threads. The fundamentals of scalable systems are fast networking and non-blocking design—the rest is message passing. In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design.

I don’t know how far in the future we currently are but it with the current release it’s actually quite easy to emulate with a few simple lines of code:

if (process.env.NODE_CHANNEL_FD){
  (function(){
    var onmessage;

    // create onmessage api
    Object.defineProperty(GLOBAL, 'onmessage', {
      get: function(){ return onmessage }
    , set: function(fn){
        onmessage = fn;
        process.on('message', function(msg){
          onmessage({data:msg});
        });
      }
    });

    // all we need to do is proxy the process.send
    Object.defineProperty(GLOBAL, 'postMessage', {
      get: function(){ return process.send }
    });
  })()
}

Adding those lines of code would allow you to use onmessage and postMessage as specified in the Web Worker API. But non the less this forking and IPC between different node instances is a more than welcome addition to the node core. See full .fork api here.

Libuv

Integration of Libuv (formerly known as liboio) in to Node.js has started. Libuv provides a high-concurrency and high-performance I/O on all operating systems. This would allow Node.js to be deployed on Windows without using Cygwin or MinGW Unix emulation layers. Because these are emulation layers, the performance has always been really poor and Libuv changes that by using native Windows APIs.

For more information about Libuv check out the github repository and Ry’s Nodeconf 2011 slides.

Top of Page