koajs – Your first application using the KAMN stack – part 1/2

koajs

Introduction

In my first tutorial on tastenjesus.de I want to show you how to write a basic webapplication with REST api using the koajs / AngularJS / mongoDB stack. Lets call it the KAMN stack. The N is for nodeJS.

Agenda

Part 1

  • The basics
  • The callback hell
  • Salvation came along: promises!
  • Generator functions (yield and function*() explained)
  • koajs
  • Whats the big deal about koajs?!
  • How to install
  • Hello koajs
  • Upstream & downstream
  • Routes & router
  • MongoDB
  • What is MongoDB?
  • Use MongoDB in NodeJS via mongoose
  • Use mongoose in koajs
  • Conclusion

Part 2 (to be written)

  • AngularJS
  • What is AngularJS?
  • Connect AngularJS to the koajs api
  • Conclusion

The basics

To write a webapplication via the next generation framework koajs you should be familiar with JavaScript and the principles of nodeJS. A good place to start for nodeJS basics is this tutorial by modulus.

The callback hell

When writing nodeJS applications you will often stumble upon asynchronous problems which often result in an ugly stack of callbacks:

var fs = require('fs');

var file1,
file2,
file3;

fs.readFile("myFile.txt","utf-8",function(err, data1) {
if(!err) {
fs.readFile("mySecondFile.txt","utf-8", function(err, data2) {
if(!err) {
fs.readFile("myThirdFile.txt","utf-8", function(err, data3) {
if(!err) {
console.log(data1,data2,data3)
} else {
console.error(err);
res.json(500, err);
}
})
} else {
console.error(err);
res.json(500, err);
}
})
} else {
console.error(err);
res.json(500, err);
}
})

Conclusion

There is a number of problems with the code posted above:

  1. It’s hard to read.
  2. Catching errors and debugging is going to be hard – not only because there is a lot of copy&paste going on.
  3. It’s just ugly.

Salvation came along: promises!

Not too long ago we were given native support for “promises” in JavaScript at html5rocks. The idea is that you always have an exact state of an action. These states are:

  • fulfilled – everything went fine.
  • rejected – an error happened
  • pending – still in the works, not rejected or fulfilled yet
  • settled – fulfilled or rejected

Let’s dive into some sample code, so you can see where this goes:

var http = require("http");
// use a lib like bluebird cause promises are not there _yet_ natively
var Promise = require("bluebird");

// Define a 'fetcher' function returning a promise
var getRemoteData = function(path, dataArray) {
// If dataArray is undefined when doing the first request, define it as an empty array
if(typeof dataArray == 'undefined' || typeof dataArray.push != 'function') {
dataArray = [];
}
return new Promise(function(resolve, reject) {
http.get(path, function(err, data) {
if(!err) {
// Push the data into the array
dataArray.push(data);
// Resolve the promise
resolve(dataArray);
} else {
// Reject the promise
reject(err);
}
});
})
}

// Use that promise to get a couple of urls
getRemoteData("http://tastenjesus.de/example.txt")
.then(function(result) {
return getRemoteData("http://anotherurl.com/example.txt", result);
})
.then(function(result) {
return getRemoteData("http://yetanotherurl.com/example.txt", result);
})
.then(function(result) {
// Woohoo. We're done
console.log(result);
}, function(err) {
// Handle the error
console.error(err);
});

Line 3 to 22

We first define a function “getRemoteData” which returns a promise. It takes a path from where to fetch the data and an “dataArray” argument which can contain an array on which the data has to be pushed. We then intialize a new Promise object and perform a regular http request. On success we push the http data onto the array and pass that along – the promise will be resolved. On error the promise will be rejected.

Line 24 to 39

We first call getRemote with the first path we want to fetch. Since the function returns a promise, we can use the then method which takes the following arguments:

  1. The function which should be called if the promise is resolved
  2. The function which should be called if the promise is rejected

One very important thing is to note here: we only have one error handling function. Since the rejection of a promise gets propagated you only have to deal with it in the last part of the chain.

Conclusion

Pros:
* Code is way more readable
* You can chain promises together
* Single error handling – no copy and paste functions
* Existing callback functions can be converted with ease

Cons:
* Passing along variables from one promise to another is a little bulky
* Doesn’t feel very “natural” to write – more like a temporary solution

Generator functions (yield and function*() explained)

The most recent way of getting rid of the callback trouble is to wirte your code with generators. A generator is a function which can be paused and later resumed. You can spot a generator function by looking out for an asterisk. You can even pass an argument when resuming the function. So how does this work? Lets start with a simple example:

Note: this is ES6 harmony stuff. It works in current Firefox and nodeJS 0.11.x with the harmony flag.

function* uselessLogger() {
var i = 0;
// make sure the function never stops
while(true) {
// Create a "pause"
yield console.log(i);
// Increment i after resuming
i++;
}
}

var test = uselessLogger();
test.next(); // Logs 0
test.next(); // Logs 1
test.next(); // Logs 2
test.next(); // Logs 3
test.next(); // Logs 4

The code above does not seem very useful but let’s go through it step by step.

function* uselessLogger() {
var i = 0;

Define a new generator function and initialize an initial value.

// make sure the function never stops
while(true) {

We create a while loop which never stops. Inside the loop we find the magic part:

// Create a "pause"
yield console.log(i);
// Increment i after resuming
i++;

We log the current value of i. yield marks the following expression to be executed with a following pause of the code execution. The codeparsing will be continued when next() is called. After that we increment i by one and start over.

To test our shiny new generator function we call it and jump through it with .next():

}
}

var test = uselessLogger();
test.next(); // Logs 0
test.next(); // Logs 1
test.next(); // Logs 2
test.next(); // Logs 3
test.next(); // Logs 4

This is a little hard to grasp at first so be sure to get a good understandig of it.

You can even put a generator inside a generator. If you put an asterisk after yield you can put another generator behind that: yield* anotherGenerator. The function will continue execution if the nested generator has finished executing.

This seems a little useless, so lets see a more practical use:

function* fileFetcher() {
try {
text1contents = yield readFile("text1.txt");
text2contents = yield readFile("text2.txt");
text3contents = yield readFile("text3.txt");
console.log(text1contents);
console.log(text2contents);
console.log(text3contents);
} catch(e) {
console.error(e);
this.body(e);
}
}

As you can see – readFile has to be compatible with our new generator function system. Don’t worry, you don’t have to write the function yourself. The piece of magic which makes this possible is called co. co (https://github.com/visionmedia/co) allows you to use “classic” callback functions and functions returning promises in generators with yield. Awesome, right? koajs uses co at its core.

koajs

koajs is a new framework created by the team behind expressJS, stylus, jade, mocha and many more. The summary from the http://koajs.com website describes best what the frameworks wants to achieve:

Koa is a new web framework designed by the team behind Express, which aims to be a smaller, more expressive, and more robust foundation for web applications and APIs. Through leveraging generators koa allows you to ditch callbacks and greatly increase error-handling. Koa does not bundle any middleware within core, and provides an elegant suite of methods that make writing servers fast and enjoyable.

Whats the big deal about koajs?!

As said before koajs is based on the co library. If you use libraries and functions which return promises or are thunks you can just go ahead and import your expressJS or plain nodeJS application into the generator world without callbacks, promises and other hassle. Write less, cleaner and more flexible code. Once you get a hold on those asterisk functions, yield and words like thunkyou will be absolutely amazed.

How to install

First of all it is important that you use nodeJS 0.11.x. To ensure that simply install n. Fire up your terminal:

sudo npm install -g n

followed by:

n 0.11.14

Now create a fresh project directory and cd into it

mkdir kamn
cd kamn

Initialize a new nodeJS project:

npm init

Install koajs:

npm install koa --save

Hello koajs

Let’s start with a simple hello world application:

Code:

// Require koa as a dependency
var koa = require("koa");

// Intialize the base application
var app = koa();

// Define our first and only middleware
app.use(function *() {
// Set the request body to 'Hello Koa!'
this.body = 'Hello Koa!';
});

// Listen on port 3000
app.listen(3000);

Explanation:

In the lines 1 to 5 we require koa as a library and initialize the application.

The interesting part resides in the lines 7 to 11: If you used expressJS before you maybe want to know where req and res are. All these functions and properties are combined in the context which is accessible via this. Neat, huh? We simply set this.bodyto 'Hello Koa!' and we are done.

We finally tell our app to listen on port 3000 in line 13 to 14.

Upstream & downstream

Another big advantage of koaJS is that the application runs through your middleware twice. You can prepare the data before and postprocess it after it ran through your controller, function or the next middleware. We could add a perfomance logger to our application as follows:

Code:

app.use(function*(next) {
// Before
var timestampBefore = new Date().getTime();

yield next;

// After
var timestampAfter = new Date().getTime();
var difference = timestampAfter - timestampBefore;
console.log("Processing took " + difference + "ms");
});

The complete code would then be:

// Require koa as a dependency
var koa = require("koa");

// Intialize the base application
var app = koa();

// Define the logger function
app.use(function*(next) {
// Before (Downstream)
var timestampBefore = new Date().getTime();

yield next;

// After (Upstream)
var timestampAfter = new Date().getTime();
var difference = timestampAfter - timestampBefore;
console.log("Processing took " + difference + "ms");
});

// Handle the request
app.use(function *() {
// Set the request body to 'Hello Koa!'
this.body = 'Hello koajs!';
});

// Listen on port 3000
app.listen(3000);

Using this mechanism you could for example check if a token is valid in the downstream and issue a new token in the upstream.

Routes & router

For a classical REST api we need a router. Fortunetaly the koajs community is very very alive at the moment and there already is module which we can use: koa-router.

To install it, simply fire up your terminal, cd to your projects directory and type npm install koa-router --save.

Now let’s modify our code to use some routing:

Code:

// Require koa as a dependency
var koa = require("koa"),
router = require("koa-router");

// Intialize the base application
var app = koa();

// Initialize koa-router
app.use(router(app));

// Define the logger function
app.use(function*(next) {
// Before (Downstream)
var timestampBefore = new Date().getTime();

yield next;

// After (Upstream)
var timestampAfter = new Date().getTime();
var difference = timestampAfter - timestampBefore;
console.log("Processing took " + difference + "ms");
});

// We no longer need to set body for all requests - this will now be handled by our routes
// Handle the request
// app.use(function *() {
// // Set the request body to 'Hello koajs!'
// this.body = 'Hello koajs!';
// });

// Define a 'http://myserver.tld:3000/users' route
app.all('/users', function*(next) {
// Figure out what to do, based on HTTP verbs
switch(this.request.method) {
case "GET":
var users = [];
this.body = users;
break;
case "POST":
this.body = "Not implemented yet";
}
});

// Listen on port 3000
app.listen(3000);

See where this is going? Pretty simple, huh? We register a router for /users in lines 30 to 41. We then figure out what to do by iterating over the HTTP verb with a switch statement. Note: You could register your routes directly via app.get and app.post but I prefer app.all and switch for better readability.

If it’s a GET request, we return an empty array since we have no database connection yet. If it’s a POST request, we return “Not implemented yet” because … well … it’s not implemented yet 😉 If you want to know more about routing (e.g. subrouters, multiple middlewares per router etc. check out the packages documentation: https://github.com/alexmingoia/koa-router/blob/master/README.md).

MongoDB

What is MongoDB?

MongoDB is a document based database. You don’t store your data in tables rows and columns as you would for example in MySQL, you store documents. In your JavaScript environment documents are represented as JSON structures. Imagine MongoDB as a filing cabinet where you put your written documents. You can store any document containing any information in your file cabinet.

By using MongoDB you gain much flexibility but you loose the benefits of a structured table based approach. To combine the best of both worlds mongoose was created.

Using MongoDB in NodeJS via mongoose

With mongoose you can define your data models plus their one-to-many, many-to-one or many-to-many relations to each other. You then can store, get or update your records without hassle.

Code:

// require mongoose
var mongoose = require("mongoose);

// define the person "schema"
var personSchema = new mongoose.Schema({
'firstname': {
type: string,
required: true
},
'lastname': {
type: string,
required: true
}
});

// initialize the mongoose person model out of the schema
var Person = mongoose.model("Person", personSchema);

// Create an example person
var bill = new Person({
firstname: 'Bill',
lastname: 'Gates'
});

// Save bill in the Person collection
bill.save(function(err) {
if(err) {
console.error(err);
} else {
console.log("success!")
}
});

// Get all persons
Person.find({}).exec(function(err, persons) {
if(err) {
console.error(err);
} else {
console.log(persons);
}
});

That only covers the very basics of mongooseJS. If you want to know more about the abilities of mongooseJS visit its documentation at: http://mongoosejs.com/docs/guide.html.

Lets go through it:

In lines 5 to 13 we define a simple person schema with a firstname and lastname, both required.

In lines 15 to 16 we initialize the mongoose model out of that schema.

In lines 18 to 22 we create a person with the firstname Bill and the lastname Gates.

In lines 24 to 31 we save that person in the database. It will be stored in the collection Person.

In lines 33 to 40 we query the Person collection to give us all stored documents. The output would be:

[
{

_id:
firstname: 'Bill'
lastname: 'Gates'
}
]

Use mongoose in koajs

In the example above we use mongoose with callback functions. As we have learned before koajs uses co which works with thunks (functions that take a callback function with the parameters errorand result) or promises. Luckily the newest (unstable) version of mongooseJS makes .exec() and .save() return a promise. Mongoose can therefore be used with koajs without any trouble.


Note: To use the following code examples, make sure you use the newest mongoose version inside your package.json. For example:

{
"name": "koa-js-example",
"version": "1.0.0",
"description": "KoaJS example with mongoose",
"main": "index.js",
"scripts": {
"start": "node index.js"
},
"dependencies": {
"koa": "^0.12.2",
"koa-router": "^3.4.0",
"mongoose": "git://github.com/LearnBoost/mongoose",
"nib": "^1.0.4"
},
"engines": {
"node": "0.11.14"
},
"keywords": [
"node",
"koa",
"express"
],
"license": "MIT"
}

Note that we specified the git repository not the version number of mongoose to get the latest version from GIT master.


Let’s modify our application code to:

  • Have a user model
  • Save a user via POST /users
  • Get a userlist via GET /users
// Require dependencies
var koa = require("koa"),
mongoose = require("mongoose"),
parse = require("co-body"),
router = require("koa-router");

// Define the user schema
var userSchema = new mongoose.Schema({
'firstname': {
type: string,
required: true
},
'lastname': {
type: string,
required: true
}
});

// Turn schema into a mongoose model
var User = mongoose.model("User", userSchema);

// Intialize the base application
var app = koa();

// Initialize koa-router
app.use(router(app));

// Define the logger function
app.use(function*(next) {
// Before (Downstream)
var timestampBefore = new Date().getTime();

yield next;

// After (Upstream)
var timestampAfter = new Date().getTime();
var difference = timestampAfter - timestampBefore;
console.log("Processing took " + difference + "ms");
});

// Define a 'http://myserver.tld:3000/users' route
app.all('/users', function*(next) {
// Figure out what to do, based on HTTP verbs
switch(this.request.method) {
case "GET":
// Try to query a userlist and catch any errors
try {
var users = yield User.find({}).exec();
this.body = users;
} catch(e) {
this.throw(500, e);
}
break;
case "POST":
// Parse the request body
var body = yield parse(this);

// Create the user which we want to save
var userToSave = new User({
firstname: body.firstname,
lastname: body.lastname
});

// Try to save it, catch errors
try {
yield userToSave.save();
this.body = "Success!";
} catch(e) {
this.throw(500, e);
}
break;
}
});

// Listen on port 3000
app.listen(3000);

Let’s go through it:

We first require a new library called co-body. It is required for parsing any POST data. We use it later in the code.

We define a userSchema with firstname and lastname in and intialize a mongoose model lines 7 to 20.

We find a few new code snippets inside the switch statement:

  • Inside the “GET” block we query for users. .exec() now returns a promise so it’s save to use with yield. We catch any errors with try-catch-.

  • Inside the “POST” block we parse the requiest body and create a new user. The user is then stored via mongoose. Any errors are again catched via a try-catch-block.

Conclusion

In this tutorial you learned the basics about generator functions, koajs and mongoose in conjunction with koajs. You can now explore the koajs api and the many third party libraries already out there and start writing nicer and cleaner code with proper error handling.

In part two I will show you how to connect your koajs application to a web frontend via AngularJS – so stay tuned!

If you like my tutorials or have any suggestions don’t hesitate to contact me:

Like what you read? – Donate

Trackbacks

  1. […] koajs – Your first application using the KAMN stack – part 1/2 Koa is a new web framework designed by the team behind Express, which aims to be a smaller, more expressive, and more robust foundation for web applications and APIs. Allow MICHEL ALBERS shows you how to write a basic web application with a REST api with this new framework. […]

Leave a Reply to Issue #22 – Oct 17, 2014 | Tales from the Front End Cancel reply

Your email address will not be published. Required fields are marked *