Category: Promises
Composing the results of nested asynchronous calls in Javascript
Grokking asynchronicity
One of the toughest things to get your head around in Javascript is how to handle nested asynchronous calls, especially when a function depends on the result of a preceding one. I see this quite a bit in my software engineering course where teams are required to synthesize new information from two distinct third-party data sources. The pattern is to look something up that returns a collection, and then look something else up for each of the members in the collection.
There are multiple approaches to this problem, but most rely on Javascript Promises, either from a third-party library in ES5 and earlier or ES6’s built-in Promises. In this article we’ll use both by using ES6 native Promises and the async.js package. This isn’t a discussion of ES6 Promises per se, but it might help you get to the point of understanding them.
The file discussed here is intended to be mounted on a route in a Node/Express application:
//From app.js const rp = require('./routes/request-promises') app.use('/rp', rp)
[Unfortunately the WordPress implementation used here does a horrific job displaying code, so the code snippets below are images for the bulk of the discussion. The complete file is available at the bottom of the post.]
The route here uses two APIs:
https://weathers.co for current weather in 3 cities, and
https://api.github.com/search/repositories?q=<term> to search GitHub repos for the ‘hottest’ city.
In our example we’ll grab weather for 3 cities, pick the hottest one, then hit GitHub with a search for repos that have that city’s name. It’s a bit contrived but we just want to demonstrate using the results of one API call to populate a second when all of them are asynch.
There are two packages (apart from Express) used:
The first is a reduced-size packaging of the standard request-promise package; we don’t need all of the features of the larger library. request-promise itself is a wrapper that adds Promises to the standard request package used to make network calls, for example to an HTTP endpoint.
There’s only one route in this module, a GET on the top-level URL. In the snippet above, this router is mounted at ‘/rp’, and so the complete URL path is http://localhost:3000/rp (assuming the server is set to listen to port 3000).
On line 50 the async.waterfall method is used to create a chain of functions that will execute in turn. Each function returns a Promise, and so the next function in line waits until the preceding function’s Promise is resolved. We need this because each function provides data that the following function will use in some way. That data is provided to each function in turn along with a callback, which is the next function in the waterfall.
async.waterfall takes two parameters in this implementation: An array of functions to call in order, and a final function (renderTable on line 51) to execute last. A small side note: The final function can be anonymous, but when debugging a big application it can be tough to figure out exactly which anonymous function out of potentially hundreds is causing a problem. Naming those functions helps immensely when debugging!
getCityTemperatures
The first function in the waterfall, getCityTemperatures, calls the weathers.co API for each city in a hardcoded array. This is the first function in the waterfall and so it receives only one param, the callback (cb).
This is the most interesting function of the three because it has to do an API call for each city in the cities array, and they all are asynchronous calls. We can’t return from the function until all of the city weather has been collected.
In each API call, once the current temperature is known it is plugged into the city’s object in the cities array. (Note that weathers.co doesn’t always return the actual current temperature. Often it is a cached value.) The technique here is to create a Promise that encompasses the API calls on line 83. This Promise represents the entire function, and it won’t be resolved until all of the city weather has been collected.
A second Promise is set up in the local function getWeather on line 91, and that’s the one that handles each individual city’s API call. request.get() itself returns a Promise on line 92 (because
we are using the request-promise-lite package), and so we make the request.get() and follow it with a then() which will run when the API call returns. The resolve() at line 96 gets us out of this inner Promise and on to the next one.
Note that we don’t yet execute getWeather(), we’re just defining it here. Line 109 uses the Array.map method to return an array that has three functions in it. Each function has one of the city objects passed into it, and each returns a Promise (as defined on line 91). That cityPromises array looks like
[getWeather({name: 'Miami', temperature: null}), getWeather({name: 'Atlanta', temperature: null}), getWeather({name: 'Boston', temperature: null})]
after line 109 completes.
We still haven’t executed anything. The mapping in line 109 sets us up to use the Promise.all( ) method from the ES6 native Promise library in line 121, which only resolves when all of the Promises in its input array have resolved. It’s a bit like the async.waterfall( ) call from line 50, but in this case order doesn’t matter. If any of the Promises reject, the entire Promise.all( ) structure rejects at that moment, even if there are outstanding unresolved Promises.
Once all of the Promises have resolved (meaning that each city’s weather has been recorded, the then( ) in line 122 executes, and calls the passed-in callback with the now-complete array of cities and temperatures. The null value handed to the callback is interpreted as an error object.
Of interest here is that Promise.all( ) is, practically speaking, running the requests in parallel.
getHottestCity
The next function in the chain, getHottestCity, simply finds the largest temperature in the array of cities and returns that city’s object to the next function in the waterfall. This is synchronous code and so doesn’t require a Promise.
Line 140 creates a slice with just the temperatures from the cities array and calls Math.max( ) to determine the largest value. Line 143 then looks for the hottest temperature in the original cities array and returns the corresponding object. Both of these lines use the ES6 way of defining functions with fat-arrow notation (=>), and line 140 uses the new spread operator.
Once the ‘hot’ city has been determined it is passed to the next function in the waterfall in line 148. Here again, the null is in the place of an error object.
findGithubRepos
The next-to-last function in the waterfall searches Github for projects that include the name of the ‘hot’ city discovered earlier. The User-Agent header in line 167 is required by the Github search API — you can change it to any string that you like.
The API call in line 165 uses request-promise (from line 29) and so returns a Promise. Once the Promise is resolved, the then( ) function builds an array of objects from some of the data returned.
In line 180 the completed array of items is passed to the next function in the waterfall along with the original ‘hot’ city object.
The final function
The last function to execute simply takes the results of our waterfall and renders a Pug page on line 57.
Here’s a link to the complete file. If If you have questions or corrections, leave a comment!
