Posts filled under: This week in Node

This week in Node

Welcome to another episode of This week in Node where I write about the latest update in the Node.js repository and existing updates that happen in user land.

Socket.IO 0.6.18

This week we released Socket.IO 0.6.18, which will hopefully be our last maintenance release before the much anticipated 0.7 release. While I could dedicate a whole new post on the changes that where made in 0.6.18, I will touch the surface of the most noticeable changes.

JSDocs

There is no such thing as to much documentation, with Socket.IO this is definitely the case. The main documentation for Socket.IO which be found on the website and the Github’s README.md is sufficient for most users to get started with Socket.IO but it doesn’t cover the details on how it actually works. Some users stated they wanted a more detailed API documentation, so we started annotating the source code. We used the JSDoc syntax which is based on the more widely known JavaDoc syntax. This allows us to easily generated API documentation and annotated source code on the fly using a JSDoc compatible parser like dox. We haven’t pushed out any documentation yet but you can generate it your self with the following command:

dox --title "socket.io" socket.io.js > api.html

Reconnection support

We already announced this in the 0.6.17 release, but unfortunately we notice that we didn’t update the client library inside socket.io-node. But this time we are 100% sure that it’s available! Socket.IO is configured to automatically re-establish a the connection with the server when the current connection is disconnected by a transport. To prevent from creating a DOS attack on the server we have implemented a exponential back off algorithm, basically it multiplies the reconnection timeout by 2 every time it attempts to reconnect to the server. The reconnection is attempted with the same transport that was used before the connection got dropped. When the reconnection reaches the maximum allowed reconnection attempts (set to 10 by default) it will do one last attempt and try out all enabled transports.

Giving feedback to the user is always vital part of building real time application as they always expect it to work. We have implemented a couple of new events to help you with notifying your users of the reconnection process. When the reconnection has started we emit the reconnecting event and supply it with the reconnection duration for the next reconnection attempt.

// construct a socket.io instance
var socket = new io.Socket();
socket.connect();

// listen for the reconnecting event
socket.on('reconnecting', function(type, attempt){
  console.log('Attempting to reconnect to the server. This is attempt #' + attempt);
});

To know when the reconnection is successfully you can listen to the reconnect event. This event receives the transport type that was used to connect and how many attempts it took to reconnect successfully.

// handle successful reconnects
socket.on('reconnect', function(type, attempts){
  console.log('It only took ' + attempts ' attempts to reconnect using ' + type);
});

When the reconnection is failed we emit the reconnect_failed event. We could keep attempting to reconnect to the server, at this point there must be something seriously wrong.

// handle failures
socket.on('reconnection_failed', function(){
  console.error('Server fail');
  window.location.reload(); // use any custom logic here.
});

Annoying loading spinners

Webkit based browser suffered from an annoying issue, if you create a asynchronous long polling XHR request before the document’s resources are fully loaded it will keep triggering the browsers loading indicators. We solved this by doing a connect after the onload event has fired. This is the same trick that got applied to iPhone and Android devices, but it’s now extended to all Webkit based browsers. This will probably make allot users happy.

Flashsocket updated to the latest version

The flashsocket fallback has been updated to the latest build. This removes the FLABridge.js dependency as the ExternalInterface is being instead. This is great news as the FLABridge.js originates from the stone age, is leaking memory as a basket and is full of code that follows worst practises on the Internet. Because of this important change inside the flashsocket we decided to test it under Opera again to see if it’s more stable than the previous builds. Luckily this was the case and we enabled the flashsocket for Opera! The upgrade also solves allot of issues that users where having with cross port connections.

Removal of introduced globals

With commit 0a1a93b4846b0 allot of variables got exposed to the global scope. We have correctly identified this issue and have taken measurements to prevent this from happening. This is done with the introduction of a test suite that I have started to prototype which will hopefully be read for the 0.7 release, so stay tuned for more information about that.

For more information see the announcement on the Socket.IO google group.

node-http-proxy 5.3

More exiting new for those of us who build real time applications with the 5.3 update of Nodejitsu’s node-http-proxy we can finally proxy Web Socket request! This has been plague ever since Web Sockets where added to the browsers. Most HTTP proxies do not support the HTTP 1.1 protocol. HAProxy was the only proxy available to proxy successfully proxy Web Sockets. But this was only possible in TCP mode making HTTP header inspection etc impossible. The best thing about node-http-proxy is that it only requires one line of code to get started with proxying your requests:

// proxy all requests from port 80 to port 8080, including Web Socket requests.
require('node-http-proxy').createServer(8080, 'localhost').listen(80);

So thank the Nodejitsu’s for the hard work, by downloading the latest code now.

Buffer floats and doubles

Commit e505a1215c5 lands supports from reading and writing floats and doubles to the buffer object. These changes will make it allot easier to handle binary data inside Nodejs and JavaScript. JavaScript already started implementing typed Arrays but buffer objects are much useful for Node at this point as they add allot less to the total heap size of V8. And there for allow you allot more memory than the 1.7 GB restriction on 64 bit machines (this is a v8 restriction). But also Buffer objects are send to the sockets much faster because it’s easier for node to get the pointer to the raw data.

UPDATE: Thanks to the correction of @mraleph Nodes buffers are actually implemented on top the External Array API of V8. They have a different syntax, but they are basically the same thing.

Documentation

The is still allot of code in the Nodejs core that isn’t documented, but thankfully this is being worked on. Commit 68d840b47d1 adds documentation for the Readline module. This is module allows you to read use input from the stdin. This will allow even more command line based applications to be build on top of Node. The DNS module receive information on the resolveNs and the resolveCname methods with commit 56aa2fd4c3d.

This week in Node

This week in Node is a weekly blog post that focuses on the latest changes and updates in the Node.js repository and in NPM modules.

Forking and IPC

Commit 9e26dab150e and 337c48db5fe landed a new method in the child_process module. The fork method allows you to spawn a new Node.js process and pass messages between your current Node.js process and the newly created process using a Inter Process Communication (IPC) channel.

So why is this an important change? This is the first step to proper and hopefully first class support for multiple processing Node.js. When you are doing allot of CPU heavy processing it can take a while before your function is done and is ready to return to the event loop again. Ideally you would use a job server such as Gearman or Resque for heavy processing but that is not always and option. As your application is busy with processing data it can the amount of concurrent your application can handle, but you can now offload all the processing to a different Node.js process and be notified once the work is done.

The API is quite simple and straight forward:

var cp = require('child_process')
  , n = cp.fork(__dirname + '/process.js');

n.on('message', function(m) {
  console.log('PARENT got message:', m);
});

n.send({ hello: 'world' });

And inside process.js:

process.on('message', function(m) {
  // illustrative function ;) 
  hardcoreProcessing(m, function(err, res){
    process.send(res);
  })
});

Wait, what? No Web Worker API

That was the first thing that came to my mind, why don’t we just use the Web Worker API for it. It was designed for stuff like this and it’s already known to some developers even the Node.js website states:

But what about multiple-processor concurrency? Aren’t threads necessary to scale programs to multi-core computers? Processes are necessary to scale to multi-core computers, not memory-sharing threads. The fundamentals of scalable systems are fast networking and non-blocking design—the rest is message passing. In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design.

I don’t know how far in the future we currently are but it with the current release it’s actually quite easy to emulate with a few simple lines of code:

if (process.env.NODE_CHANNEL_FD){
  (function(){
    var onmessage;

    // create onmessage api
    Object.defineProperty(GLOBAL, 'onmessage', {
      get: function(){ return onmessage }
    , set: function(fn){
        onmessage = fn;
        process.on('message', function(msg){
          onmessage({data:msg});
        });
      }
    });

    // all we need to do is proxy the process.send
    Object.defineProperty(GLOBAL, 'postMessage', {
      get: function(){ return process.send }
    });
  })()
}

Adding those lines of code would allow you to use onmessage and postMessage as specified in the Web Worker API. But non the less this forking and IPC between different node instances is a more than welcome addition to the node core. See full .fork api here.

Libuv

Integration of Libuv (formerly known as liboio) in to Node.js has started. Libuv provides a high-concurrency and high-performance I/O on all operating systems. This would allow Node.js to be deployed on Windows without using Cygwin or MinGW Unix emulation layers. Because these are emulation layers, the performance has always been really poor and Libuv changes that by using native Windows APIs.

For more information about Libuv check out the github repository and Ry’s Nodeconf 2011 slides.

Top of Page