I use promises a lot in my development process. I find them extremely pliable for my needs. However, for whatever reason, I don’t use abstractions for most of my development, which is to say I use vanilla promises unencumbered by an additional API to learn.
Recently I needed to run a series of promises, but I had to also solve a race condition, so they had to run in sequence (i.e. a waterfall). I know libraries like [async.js](https://github.com/caolan/async) have this baked in, but I wanted to write up how to go about doing it without a library.
[](https://training.leftlogic.com/buy/terminal/cli2?coupon=BLOG\&utm_source=blog\&utm_medium=banner\&utm_campaign=remysharp-discount)
[READER DISCOUNTSave $50 on terminal.training](https://training.leftlogic.com/buy/terminal/cli2?coupon=BLOG\&utm_source=blog\&utm_medium=banner\&utm_campaign=remysharp-discount)
[I’ve published 38 videos for new developers, designers, UX, UI, product owners and anyone who needs to conquer the command line today.](https://training.leftlogic.com/buy/terminal/cli2?coupon=BLOG\&utm_source=blog\&utm_medium=banner\&utm_campaign=remysharp-discount)
The setup[](#the-setup)
My particular case I was running promises that took an undetermined amount of time to complete. In the interest of keeping things simple, let’s say the following is what the promise has to run, and it will return a sequence number:
var guid = 0;
function run() {
guid++;
var id = guid;
return new Promise(resolve => {
// resolve in a random amount of time
setTimeout(function () {
console.log(id);
resolve(id);
}, (Math.random() * 1.5 | 0) * 1000);
});
}First in parallel[](#first-in-parallel)
To get values 1..10 I would then call the run function via an array of promises through Promise.all (note that this is run in parallel and not in sequence at this point):
// make an array of 10 promises
var promises = Array.from({ length: 10 }, run);
Promise.all(promises).then(console.log);
// => [1,2,3,4,5,6,7,8,9,10]Note that the console logs in the bin below are out of order, but the final result is correct (which is what we’d expect with Promise.all):
Now as a waterfall[](#now-as-a-waterfall)
Promises naturally waterfall into each other, but for an undetermined length of an array, I need to recursively chain the promises together, passing the aggregate result from the previous promise into the next, and so on. A perfect match for [].reduce.
var promise = Array.from({ length: 10 }).reduce(function (acc) {
return acc.then(function (res) {
return run().then(function (result) {
res.push(result);
return res;
});
});
}, Promise.resolve([]));
promise.then(console.log);Remember that the Array.from({ length: 10 }) is simply generating an array from 0 to 9 for this demo purpose. Also note that run() is inside the reduce function, and the end result is no longer an array of promises, but in fact a single promise (and so I drop the use of Promise.all).
Now you can see from the JS Bin below that the promises run in sequence (I’ve added a console):
Worth noting[](#worth-noting)
Obviously the waterfall of promises is going to take longer to complete than a parallel run of the promises, but as I mentioned at the start of this post, I needed to solve a race condition, and this was the solution I needed.
Published 18-Dec 2015 under #code. [Edit this post](https://github.com/remy/remysharp.com/blob/main/public/blog/promise-waterfall.md)
Comments
Lock Thread
Login
Add Comment[M ↓ Markdown]()
[Upvotes]()[Newest]()[Oldest]()
Dror Ben-Gai
0 points
6 years ago
I’m using this pattern, but as a mechanism for skipping over promises that I don’t need to keep, based on previous promises.
Case in point: I have an array of sources from which to download a file, and I want to stop at the first source that actually delivers the file.
Using the waterfall pattern, I created a cascade of promises, where each would try to download the file only if the previous promise failed. If one of the promises succeeds, then all subsequent promises would just trickle the results down, without trying to download it again (I’m using 'request-promise' to get a promise out of an HTTP request):
const request = require('request-promise');\ function downloadFromMultipleSources(filename, sources) {\ return sources.reduceacc, source) ⇒ {\ return acc\ .then(contents ⇒ contents)\ .catch(() ⇒ request(source + '/' + filename;\ }, Promise.reject());\ }
Jeungmin Oh
0 points
6 years ago
Thanks for great posting.\ Is there any tip I can stop the waterfall process if any reject() called?
Dror Ben-Gai
0 points
6 years ago
See my comment above ^^ for skipping over rejected (or even resolved) promises.
Vítor Arjol
0 points
7 years ago
Remy, thank you so much! This example saved me today!
Diego Timido
0 points
7 years ago
Beautifully done. Thanks for the simple, yet elegant example. Other blogs I’ve read made this overly complicated.
Sébastien Ratez
0 points
7 years ago
Hi Remy,\ Your waterfall sample is elegant but I suppose the Array.from.reduce is actually not necessary and in fact probably not desirable in respect to the specific purpose of this example, is it? I played a bit in a plunker and I could achieve the same result by simply chaining the promises together, except that when guid reached 11 I would simply reject() it instead of resolving it.\ I’d like to know why you kept the relative complexity of Array.from.reduce: is it because it reflects a real world example you had in mind?\ Thank you, and thanks for the intellectual exercise for today :)\ Cheers,
rem
1 point
7 years ago
I’m not entirely sure I follow what you’re saying. Certainly the Array.from is to create the sample set, so that wouldn’t appear in the real code, but the reduce is necessary AFAIK. But feel free to share a bin with example code since I might find that easier to understand.
Sébastien Ratez
1 point
7 years ago
Of course, take a look at [https://plnkr.co/edit/MwuU0…;](https://plnkr.co/edit/MwuU0yInjncCNMY9gUHQ) in the script.js there’s a little piece I added. May you find a mistake, please share! Cheers
rem
1 point
7 years ago
plunkr doesn’t really work on mobile (which is all I have at the moment). [jsbin.com](http://jsbin.com) does work over mobile 😀 or I can take a look another day.
Sébastien Ratez
0 points
7 years ago
Oh wow, visiting your blog was definitely worth it today, I learned another thing :-) Here is the bin: [http://jsbin.com/vajuvitibi…;](http://jsbin.com/vajuvitibi/1/edit?js,console)
rem
0 points
7 years ago
Right, I see what you’re doing. That’s one way you could do it, and if you code allows for that, then you should use it if you find it easier.
The limitations I was working with are:
a) I can’t push onto a global (or in-scope) array like your code does, I had functional code which always returned the same value.\ b) I’m not keen on the reject when it’s not really a rejection (you’re using it as flow control in your example).
Although [].reduce is sometimes a bit more cognitive load on the reader, in this case, for me, it’s a good fit. I hope that helps explain the reasoning.
code\_monk
0 points
7 years ago
F*\*\*ing brilliant! brain exploding. (in a good way)
naayt
0 points
7 years ago
Super interesting, Remy. Thanks.
Gunar C. Gessner
0 points
8 years ago
The waterfall code is mesmerizing. It took me a while to understand that reduce function with only one argument, and Promise.resolve(\[]). I loved the challenge and that was brilliant.
You’re chaining the same function multiple times and run() needs no arguments. Although if it did, you’d have to be careful to match one function’s output with the next function’s input. This waterfall/chain can be thought of as a stream, like Highland.js does. I understand you like Vanilla code the most, but if I were you I’d at least take a look.
Thank you.
Chun
0 points
8 years ago
I would recommend separating your control flow logic from your business logic:
This way your promise abstractions don’t get in the way of understanding the important stuff your code has to do — your code is more about 'what you need to do', rather than 'how you need to do it' and it becomes a lot more functional as a result.
rem
1 point
8 years ago
Of course I do. This blog post was purely to illustrate how it can be solved without a library.
Dan L
0 points
8 years ago
Interesting solution, but it is a bit challenging to read the code (almost lisp-like). Thanks for sharing!
[Commento](https://commento.io)