Using Multiple NoSQL Database Models with Orchestrate and Node.js

Databases aren’t a solved problem. Dozens of new options have sprouted up over the past five years as developers look for ways to effectively handle emerging application patterns. But how do you choose which one to use, especially when your data demands might span the competencies of an individual database engine?

NoSQL choices abound. Do you need something that stores key-value information for quick access? How about stashing JSON documents that represent fluid data structures? What should you do for Internet-of-Things scenarios or fast moving log data where “time” is a first class citizen? How should you handle dynamic relationships that require a social graph approach? Where should you store and use geo-spatial data? And don’t forget about the need to search all that data! If you need all of the aforementioned characteristics, then you typically need to stand up and independently manage multiple database technologies and weave them together at the app layer. Orchestrate.io does it differently, and yesterday, CenturyLink acquired them.

What does Orchestrate do? It’s a fast, hosted, managed, multi-model database fabric that is accessible through a single REST API. Developers only pay for API calls, and have access to a flexible key-value store that works with time-ordered events, geospatial data, and graph relationships. It runs an impressive tech stack under the covers, but all that the developers have to deal with is the web interface and API layer. In this blog post, I’ll walk through a simple sample app that I built in Node.js.

First off, go sign up for an Orchestrate account that comes with a nice free tier. Do it, now, I’ll wait.

Once in the Orchestrate management dashboard, I created an application. This is really just a container for data collections for a given deployment region. In my case, I chose one of the four brand new CenturyLink regions that we lit up last night. Psst, it’s faster on CenturyLink Cloud than on AWS.

2015.04.21orchestrate01

For a given app, I get an API key used to authenticate myself in code.  Let’s get to work (note that you can find my full project in this GitHub repo). I built a Node.js app, but you can use any one of their SDKs (Node, Ruby, Python, Java and Go with community-built options for .NET and PHP) or, their native API. To use the Node SDK, I added the package via npm.

2015.04.21orchestrate02

In this app, I’ll store basketball player profiles, relationships between players, and a time-ordered game log. I spun up a Node Express project (using Visual Studio, which I’m getting used to), and added some code to “warm up” my collection by adding some records, some event data, and some relationships. You can also query/add/update/delete via the Orchestrate management dashboard, but this was a more repeatable choice.

var express = require('express');
var router = express.Router();

//Orchestrate API token
var token = "<key>";
var orchestrate = require('orchestrate');
//location reference
orchestrate.ApiEndPoint = "<endpoint>";
var db = orchestrate(token);

/* Warmup the collection. */
router.get('/', function (req, res) {

    //create key/value records
    db.put('Players', "10001", {
        "name": "Blake Griffin",
        "team": "Los Angeles Clippers",
        "position": "power forward",
        "birthdate": "03/16/89",
        "college": "Oklahoma",
        "careerppg": "21.5",
        "careerrpg": "9.7"
    })
    .then(function (result1) {
        console.log("record added");
    });

    //create key/value records
    db.put('Players', "10002", {
        "name": "DeAndre Jordan",
        "team": "Los Angeles Clippers",
        "position": "center",
        "birthdate": "07/21/88",
        "college": "Texas A&M",
        "careerppg": "8.0",
        "careerrpg": "9.0"
    })
    .then(function (result2) {
        console.log("record added");
    });

    //create key/value records
    db.put('Players', "10003", {
        "name": "Matt Barnes",
        "team": "Los Angeles Clippers",
        "position": "strong forward",
        "birthdate": "03/09/80",
        "college": "UCLA",
        "careerppg": "8.1",
        "careerrpg": "4.5",
        "teams": [
            "Los Angeles Clippers",
            "Sacramento Kings",
            "Golden State Warriors"
        ]
    })
    .then(function (result3) {
        console.log("record added");
    });

    //create event
    db.newEventBuilder()
    .from('Players', '10001')
    .type('gamelog')
    .time(1429531200)
    .data({ "opponent": "San Antonio Spurs", "minutes": "43", "points": "26", "rebounds": "12" })
    .create()
    .then(function (result4) {
        console.log("event added");
    });

    //create event
    db.newEventBuilder()
    .from('Players', '10001')
    .type('gamelog')
    .time(1429012800)
    .data({ "opponent": "Phoenix Suns", "minutes": "29", "points": "20", "rebounds": "8" })
    .create()
    .then(function (result5) {
        console.log("event added");
    });

    //create graph relationship
    db.newGraphBuilder()
    .create()
    .from('Players', '10001')
    .related('currentteammate')
    .to('Players', '10002')
    .then(function (result6) {
        console.log("graph item added");
    });

    //create graph relationship
    db.newGraphBuilder()
    .create()
    .from('Players', '10001')
    .related('currentteammate')
    .to('Players', '10003')
    .then(function (result7) {
        console.log("graph item added");
    });

    res.send('warmed up');
});

module.exports = router;

After running the app and hitting the endpoint, I saw a new collection (“Players”) created in Orchestrate.

2015.04.21orchestrate03

It’s always nice to confirm that things worked, so I switched to the Orchestrate UI where I queried my key/value store for one of the keys I added above. Sure enough, the item for Los Angeles Clippers superstar Blake Griffin comes back. Success.

2015.04.21orchestrate04

Querying data items from code is super easy. Retrieve all the players in the collection? Couple lines of code.

/* GET all player listing. */
router.get('/', function (req, res) {

    db.list("Players")
    .then(function (result) {
        playerlist = result.body.results;
        //console.log(playerlist);

        res.render('players', { title: 'Players List', plist: playerlist });
    })
    .fail(function (err) {
        console.log(err);
    });
});

When the app runs, I get a list of players. Thanks to my Jade template, I’ve also got a hyperlink to the “details” page.

2015.04.21orchestrate05

The player details include their full record info, any relationships to other players, and a log of games played. As you can see in the code below, it’s extremely straightforward to pull each of these entity types.

/* GET one player profile. */
router.get('/:playerid', function (req, res) {

    console.log(req.params.playerid);

    //get player object
    db.get("Players", req.params.playerid)
    .then(function (result) {
        player = result.body;
        //console.log(player);

        //get graph of relationships
        db.newGraphReader()
        .get()
        .from('Players', req.params.playerid)
        .related('currentteammate')
        .then(function (relres) {
            playerlist = relres.body;
            //console.log(playerlist);

            //get time-series events
            db.newEventReader()
            .from('Players', req.params.playerid)
            .type('gamelog')
            .list()
            .then(function (evtres) {
                gamelog = evtres.body;
                //console.log(gamelog);

                res.render('player', { title: 'Player Profile', profile: player, plist: playerlist, elist: gamelog });
            });
        });
    })
    .fail(function (err) {
        console.log(err);
    });
});

2015.04.21orchestrate06

I didn’t bake in any true “search” capability into my app, but it’s one of the most powerful parts of the database service. Devs search through collections with a Lucene-powered engine with a robust set of operators. Do fuzzy search, deep full-text search of nested objects, grouping, proximity search, geo queries, aggregations, complex sorting, and more. Which player ever played for the Sacramento Kings? It’s easy to search nested objects.

2015.04.21orchestrate07

Summary

I don’t usually flog my own company’s technology on this blog, but this is fun, important stuff. Developers are faced with a growing set of technologies they need to know in order to build efficient mobile experiences, data-intensive systems, and scalable cloud apps. Something like Orchestrate flips the script by giving developers a quick on-ramp to a suite of database engines that solve multiple problems.

Take it for a spin, and give me feedback for where we should go with it next!



Categories: CenturyLink Cloud, Cloud, Node.js

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: