Dogfooding: Defining roles in an MVC architecture with internal APIs
Here’s a copy of the talk I did recently at Boston University discussing how to implement a clean MVC architecture for web apps, with a decoupled front end, using an internal API.
Abstract: The architectural design of an application often comes down to a single question: Where is the work done? Traditional client-server applications answer the question unequivocally: On the server. A new class of application, single-page (SPA), has blurred the separation of responsibility by moving data operations closer to the client. This talk discusses an approach that strictly segregates back-end models from front-end SPA views through use of an application-agnostic internal RESTful API, enhancing testability and re-use. While demonstration code will be in Javascript, the approach applies to most client-server application architectures.
Using Javascript Promises to synchronize asynchronous methods
The asynchronous, non-blocking Javascript runtime can be a real challenge for those of us who are used to writing in a synchronous style in languages such as Python of Java. Especially tough is when we need to do several inherently asynchronous things in a particular order...maybe a filter chain...in which the result of a preceding step is used in the next. The typical JS approach is to nest callbacks, but this leads to code that can be hard to maintain.
The following programs illustrate the problem and work toward a solution. In each, the leading comments describe the approach and any issues that it creates. The final solution can be used as a pattern to solve general synchronization problems in Javascript. The formatting options in the BU WordPress editor are a little limited, so you might want to cut and paste each example into your code editor for easier reading.
1. The problem
/* If you are used to writing procedural code in a language like Python, Java, or C++, you would expect this code to print step1, step2, step3, and so on. Because Javascript is non-blocking, this isn't what happens at all. The HTTP requests take time to execute, and so the JS runtime moves the call to a queue and just keeps going. Once all of the calls on the main portion of the call stack are complete, an event loop visits each of the completed request()s in the order they completed and executes their callbacks. Starting demo Finished demo step3: UHub step2: CNN step1: KidPub So what if we need to execute the requests in order, maybe to build up a result from each of them? */ var request = require('request'); var step1 = function (req1) { request(req1, function (err, resp) { console.log('step1: KidPub'); }); }; var step2 = function (req2) { request(req2, function (err, resp) { console.log('step2: CNN'); }); }; var step3 = function(req3) { request(req3, function (err, resp) { console.log('step3: UHub'); }); }; console.log('Starting demo'); step1('http://www.kidpub.com'); step2('http://www.cnn.com'); step3('http://universalhub.com'); console.log('Finished demo');
2. Callbacks work just fine, but...
/* This is the classic way to synchronize things in Javascript using callbacks. When each request completes, its callback is executed. The callback is still placed in the event queue, which is why this code prints Starting demo Finished demo step1: BU step2: CNN step3: UHub There's nothing inherently wrong with this approach, however it can lead to what is called 'callback hell' or the 'pyramid of doom' when the number of synchronized items grows too large. */ var request = require('request'); var req1 = 'http://www.bu.edu'; var req2 = 'http://www.cnn.com'; var req3 = 'http://universalhub.com'; var step1 = function () { request(req1, function (err, resp) { console.log('step1: BU'); request(req2, function (err, resp) { console.log('step2: CNN'); request(req3, function(err,resp) { console.log('step3: UHub'); }) }) }); }; console.log('Starting demo'); step1(); console.log('Finished demo');
3. Promises help solve the problem, but there's a gotcha to watch out for.
/* One way to avoid callback hell is to use thenables (pronounced THEN-ables), which essentially implement the callback in a separate function, as described in the Promise/A+ specification. A Javascript Promise is a value that can be returned that will be filled in or completed at some future time. Most libraries can be wrapped with a Promise interface, and many implement Promises natively. In the code here we're using the request-promise library, which wraps the standard HTTP request library with a Promise interface. The result is code that is much easier to read...an event happens, THEN another event happes, THEN another and so on. This might seem like a perfectly reasonable approach, chaining together calls to external APIs in order to build up a final result. The problem here is that a Promise is returned by the rp() call in each step...we are effectively nesting Promises. The code below appears to work, since it prints the steps on the console in the correct order. However, what's really happening is that each rp() does NOT complete before moving on to its console.log(). If we move the console.log() statements inside the callback for the rp() you'll see them complete out of order, as I've done in step 2. Uncomment the console.log() and you'll see how it starts to unravel. */ var rp = require('request-promise'); var Promise = require('bluebird'); var req1 = 'http://www.bu.edu'; var req2 = 'http://www.cnn.com'; var req3 = 'http://universalhub.com'; function doCalls(message) { return new Promise(function(resolve, reject) { resolve(1); }) .then(function (result) { rp(req1, function (err, resp) { }); console.log('step:', result, ' BU '); return ++result; }) .then(function (result) { rp(req2, function (err, resp) { // console.log('step:', result, ' CNN'); }); console.log('step:', result, ' CNN'); return ++result; } ) .then(function (result) { rp(req3, function (err, resp) { }); console.log('step:', result, ' UHub'); return ++result; }) .then(function (result) { console.log('Ending demo at step ', result); return; }) } doCalls('Starting calls') .then(function (resolve, reject) { console.log('Complete'); })
4. Using Promise.resolve to order nested asynchronous calls
/* Here's the final approach. In this code, each step returns a Promise to the next step, and the steps run in the expected order, since the promise isn't resolved until the rp() and its callback are complete. Here we're just manipulating a function variable, 'step', in each step but one can imagine a use case in which each of the calls would be building up a final result and perhaps storing intermediate data in a db. The result variable passed into each proceeding then is the result of the rp(), which would be the HTML page returned by the request. The advantages of this over the traditional callback method are that it results in code that's easier to read, and it also simplifies a stepwise process...each step is very cleanly about whatever that step is intended to do, and then you move on to the next step. */ var rp = require('request-promise'); var Promise = require('bluebird'); var req1 = 'http://www.bu.edu'; var req2 = 'http://www.cnn.com'; var req3 = 'http://universalhub.com'; function doCalls(message) { var step = 0; console.log(message); return new Promise(function(resolve, reject) { resolve(1); }) .then(function (result) { return Promise.resolve( rp(req1, function (err, resp) { console.log('step: ', ++step, ' BU ') }) ) }) .then(function (result) { return Promise.resolve( rp(req2, function (err, resp) { console.log('step: ', ++step, ' CNN'); }) ) } ) .then(function (result) { return Promise.resolve( rp(req3, function (err, resp) { console.log('step: ', ++step, ' UHub'); }) ) }) .then(function (result) { console.log('Ending demo '); return Promise.resolve(step); }) } doCalls('Starting calls') .then(function (steps) { console.log('Completed in step ', steps); });