Lessons learned from entering Node Knock Out

As the second NKO is reaching it’s end I would like to share my experiences and lessons that I have learned from entering the contest in 2010 and 2011. I hope this would help some people in the future to prevent them or learn from the countless mistakes I have made.

Sleep is for the weak

Producing code for 48h is a really long run, it will feel as if your eye balls are shrinking to little peas, but you have a project to finish. So unless you are 100% positive that you have done enough work already to gain the privilege of sleep.. I wouldn’t close my eyes, at all. Yes 48h is long, but if you are 1 hour short at the end of the competition because you felt like you needed to take a power nap for 4 hours.. I will be more than happy to tell you, I told you so. Don’t forget that only ~50% of all the teams actually make it to the judging/voting round.

Select a database that matches the needs of your project

There are tons on MYSQL and NOSQL databases available but not every database is suitable for the need of your project. Do you have a lot of read or write request, or do want to maintain history of documents or should it just be really fucking fast. Deciding this up front allows you to fully focus on coding up your project during the competition.

Choose a database that can be hosted by a service

In 2010 I was ambitions enough to install MongoDB locally on my server. The installation was a pain, I was used to a different Linux distribution and was getting odd compilation errors all over the place. Once I finally managed to get it running smoothly, it completely crashed and killed the database file.

I had lost a total of 6 hours because I was to stubborn to use hosted service. This year I decided to use a hosted right away. Because I could really use those 6 hours.

Use your production database as development database

This is probably a more personal thing, but I noticed that when I deployed for production my site stopped working properly. I had forgotten to create indexes on the production database. I did add these on the development database. This costs to much time to debug. In a normal development process you would always use snapshot of the database but you only got 48 hours here, no time to debug these kind of silly issues.

Abstract your database connections if needed

Both years I decided to go with MongoDB because it suited my needs the best, but the amount of code that you actually need to write to fetch data from the database is immense. You could use a ORM like mongoose instead or create your own abstraction.

/**
 * Fetches a working mongodb connection.
 *
 * @param {String} collection
 * @param {Function} fn Callback
 * @api private
 */

exports.allocate = function (collection, fn) {
  /**
   * Fetches the correct collection.
   *
   * @param {Error} error from the connecting
   * @param {MongoDB} db Database
   * @api private
   */

  function collect (err, db) {
    db.collection(collection, function collection (err, col) {
      if (err) return fn.apply(fn, arguments);

      fn.call(col, err, db);
    });
  };

  // fast case
  var stream = exports.allocate.stream;
  if (stream) return collect.call(stream, null, stream);

  /**
   * Called once the database opens.
   *
   * @param {Error} err Connection error
   * @param {MongoDB} stream Mongodb connection stream
   * @api private
   */

  function open (err, stream) {
    if (err) return fn.call(fn, err, null);

    exports.allocate.stream = stream;

    // handle uncaught errors
    stream.on('error', function uncaughtError (err) {
      exports.allocate.stream.close();
      exports.allocate.stream = null;

      console.error('(mongodb) uncaught error', err.message, err.stack);
    });

    // start the whole collect thingy
    collect.apply(stream, arguments);
  }

  var mongodb = mongo.connect(conf.db, {
      auto_reconnect: true // auto reconnect broken connection
    , poolSize: 10         // connection pool
  }, open);
};

I use the that small allocate function to clean up the way I needed to connect with database. This way I could mock up database driven function really fast without creating a async mustache }}}}});}}});.

exports.exists = function (observer, fn) {
  exports.allocate('account', function (err, db) {
    if (err) return fn.call(exports, err);

    this.findOne({ _id: new objectid(observer)}, fn.bind(exports));
  });
};

Research the npm modules that you want to use and how to make them work

When I had the idea for my application, I started thinking about the technologies that I needed to make it work, using Socket.IO was no brainer for me because it’s simply awesome and I spend a lot of time patching bugs and making it super stable.

Socket.IO provides a great range of different features that makes it really easy to setup a real time application. To my surprise I was actually able to leverage a lot of this functionality, everything that I had to create last year like authorization, rooms is now baked in the core. So I suggest you start sniffing around in the modules you are about to use, to find hidden gems.

Pick your hosting before you are getting started

The hosting providers are usually announced a few days before the contest so you have enough time to look around and find out what they offer for your application. Pick a application that suites the needs of your project. Some hosts require more installation, but they will also provide a greater range of flexibility. If you want to build a real-time application make sure that your host is supporting Web Sockets. Yes, there are still hosts (Heroku) that doesn’t support Web Sockets. While other doesn’t allow you to deploy code on anything other than port 80. So pick wisely, you might time to swap during the contest but don’t place your bets on it.

When in doubt, choose the host you know

If you have no idea which one to pick, go with a host you are most comfortable with. For me this was Joyent for both years, the SmartOS is a really well designed Linux distribution with DTrace integration. Plus they have cloud analytics which allows you to put DTrace probes in your server and see it’s pain points in real time. Which is pretty sweet.

Expect hosting and deployment failures

Even if you already have experience with the hosting provider it doesn’t guarantee anything. I can’t tell you if you will be getting big or minor failures. But they are going to happen, so you better prepare for them.

The first time I entered I forgot that ISP usually do network and maintenance updates during the night because not a lot of people are surviving the web then. I was disconnected from the Internet for one hour, normally this doesn’t really matter but this was right before the end of the competition. I didn’t had the ability to tether my iPhones Internet connection so I could only continue working off-line and hope that my Internet connection got back eventually.

This year I made sure my iPhone was jail broken and able to tether it’s 3g connection. I experienced about 3 Internet drops, but none occurred during the competition. But it’s still nice to know you actually have a back up.

Have a back-up idea

If you are going to enter as a team make sure you have a back-up idea for when your team members get ill or find other excuses for not entering the competition with you. If you don’t have a back-up idea drop some less important features so that you project is still do-able in the 48h time frame.

Haters gonna hate

It doesn’t matter how good your idea or execution is people are going to dislike your entry. You should just ignore it, haters gonna hate.

Have fun

Remember it’s just contest, just have fun and enjoy your project. You have done good ;).

Last but not least, voting

While getting getting contestant and Judge votes is pretty awesome, don’t forget that public also counts for 20% of your total score.

Speaking of voting, If you haven’t done so already, voting for my solo application is open until 6 Sept. 2011.

I will try to keep this list as up to date as possible and add missing tips if I remember them.

Notes

  1. jamescharlesworth reblogged this from 3rdedenblog
  2. 3rdedenblog posted this
Top of Page