Combining Clouds: Accessing Azure Storage from Node.js Application in Cloud Foundry

I recently did a presentation (link here) on the topic of platform-as-a-service (PaaS) for my previous employer and thought that I’d share the application I built for the demonstration. While I’ve played with Node.js a bit before, I thought I’d keep digging in and see why @adron won’t shut up about it. I also figured that it’d be fun to put my application’s data in an entirely different cloud than my web application. So, let’s use Windows Azure for data storage and Cloud Foundry for application hosting. This simple application is a registry (i.e. CMDB) that an organization could use to track their active systems. This app (code) borrows heavily from the well-written tutorial on the Windows Azure Node.js Dev Center.

First, I made sure that I had a Windows Azure storage account ready to go.

Then it was time to build my Node.js application. After confirming that I had the latest version of Node (for Windows) and npm installed, I went ahead and installed the Express module with the following command:

This retrieved the necessary libraries, but I now wanted to create the web application scaffolding that Express provides.

I then updated the package.json file added references to the helpful azure module that makes it easy for Node apps to interact with many parts of the Azure platform.

{
"name": "application-name",
  "version": "0.0.1",
  "private": true,
  "scripts":
	{
    "start": "node app"
  },
    "dependencies":{
		    "express": "3.0.0rc2"	,
         "jade": "*"	,
         "azure": ">= 0.5.3",
         "node-uuid": ">= 1.3.3",
         "async": ">= 0.1.18"
	}
}

Then, simply issuing an npm install command will fetch those modules and make them available.

Express works in an MVC fashion, so I next created a “models” directory to define my “system” object. Within this directory I added a system.js file that had both a constructor and pair of prototypes for finding and adding items to Azure storage.

var azure = require('azure'), uuid = require('node-uuid')

module.exports = System;

function System(storageClient, tableName, partitionKey) {
	this.storageClient = storageClient;
	this.tableName = tableName;
	this.partitionKey = partitionKey;

	this.storageClient.createTableIfNotExists(tableName,
		function tableCreated(err){
			if(err) {
				throw error;
			}
		});
};

System.prototype = {
	find: function(query, callback) {
		self = this;
		self.storageClient.queryEntities(query,
			function entitiesQueried(err, entities) {
				if(err) {
					callback(err);
				} else {
					callback(null, entities);
				}
			});
	},
	addItem: function(item, callback) {
		self = this;
		item.RowKey = uuid();
		item.PartitionKey = self.partitionKey;
		self.storageClient.insertEntity(self.tableName, item,
			function entityInserted(error) {
				if(error) {
					callback(error);
				} else {
					callback(null);
				}
			});
	}
}

I next added a controller named systemlist.js to the Routes directory within the Express project. This controller used the model to query for systems that match the required criteria, or added entirely new records.

var azure = require('azure')
  , async = require('async');

module.exports = SystemList;

function SystemList(system) {
  this.system = system;
}

SystemList.prototype = {
  showSystems: function(req, res) {
    self = this;
    var query = azure.TableQuery
      .select()
      .from(self.system.tableName)
      .where('active eq ?', 'Yes');
    self.system.find(query, function itemsFound(err, items) {
      res.render('index',{title: 'Active Enterprise Systems ', systems: items});
    });
  },

   addSystem: function(req,res) {
    var self = this
    var item = req.body.item;
    self.system.addItem(item, function itemAdded(err) {
      if(err) {
        throw err;
      }
      res.redirect('/');
    });
  }
}

I then went and updated the app.js which is the main (startup) file for the application. This is what starts the Node web server and gets it ready to process requests. There are variables that hold the Windows Azure Storage credentials, and references to my custom model and controller.

/**
 * Module dependencies.
 */

var azure = require('azure');
var tableName = 'systems'
  , partitionKey = 'partition'
  , accountName = 'ACCOUNT'
  , accountKey = 'KEY';

var express = require('express')
  , routes = require('./routes')
  , http = require('http')
  , path = require('path');

var app = express();

app.configure(function(){
  app.set('port', process.env.PORT || 3000);
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(app.router);
  app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function(){
  app.use(express.errorHandler());
});

var SystemList = require('./routes/systemlist');
var System = require('./models/system.js');
var system = new System(
    azure.createTableService(accountName, accountKey)
    , tableName
    , partitionKey);
var systemList = new SystemList(system);

app.get('/', systemList.showSystems.bind(systemList));
    app.post('/addsystem', systemList.addSystem.bind(systemList));

app.listen(process.env.port || 1337);

To make sure the application didn’t look like a complete train wreck, I styled the index.jade file (which uses the Jade module and framework) and corresponding CSS. When I executed node app.js in the command prompt, the web server started up and I could then browse the application.

I added a new system record, and it immediately showed up in the UI.

I confirmed that this record was added to my Windows Azure Storage table by using the handy Azure Storage Explorer tool. Sure enough, the table was created (since it didn’t exist before) and a single row was entered.

Now this app is ready for the cloud. I had a little bit of a challenge deploying this app to a Cloud Foundry environment until Glenn Block helpfully pointed out that the Azure module for Node required a relatively recent version of Node. So, I made sure to explicitly choose the Node version upon deployment. But I’m getting ahead of myself. First, I had to make a tiny change to my Node app to make sure that it would run correctly. Specifically, I changed the app.js file so that the “listen” command used a Cloud Foundry environment variable (VCAP_APP_PORT) for the server port.

app.listen(process.env.VCAP_APP_PORT || 3000);

To deploy the application, I used vmc to target the CloudFoundry.com environment. Note that vmc works for any Cloud Foundry environment, including my company’s instance, called Web Fabric.

After targeting this environment, I authenticated using the vmc login command. After logging in, I confirmed that Cloud Foundry supported Node.

I also wanted to see which versions of Node were supported. The vmc runtimes command confirmed that CloudFoundry.com is running a recent Node version.

To push my app, all I had to do was execute the vmc push command from the directory holding the Node app.  I kept all the default options (e.g. single instance, 64 MB of RAM) and named my app SeroterNode. Within 15 seconds, I had my app deployed and publicly available.

With that, I had a Node.js app running in Cloud Foundry but getting its data from a Windows Azure storage table.

And because it’s Cloud Foundry, changing the resource profile of a given app is simple. With one command, I added a new instance of this application and the system took care of any load balancing, etc.

Node has an amazing ecosystem and its many modules make application mashups easy. I could choose to use the robust storage options of something AWS or Windows Azure while getting the powerful application hosting and portability offered by Cloud Foundry. Combining application services is a great way to build cool apps and Node makes that a pretty easy to do.