Promises and concurrency

One helper

Snippets on this page use:

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

It settles via a timer — a task — just as a real network response does, which is why it stands in for one throughout.

The loop that waits

1.1 One at a time

You have written this. It is correct.

const results = [];
for (const url of urls) {
  results.push(await fetch(url));
}

Thirty requests, each answered in 100 ms:

How long

sequential await     3013 ms   peak concurrency  1

Measured against a stand-in server that answers every request after 100 ms.

Three seconds, and for nearly all of it your program is doing nothing. await returns to the caller and the rest of the loop waits for the response, so the second request is not sent until the first has come back.

The requests do not depend on each other. Nothing required them to be in a line. The for loop put them there.

1.2 All at once

The fix everyone reaches for:

const results = await Promise.all(urls.map((url) => fetch(url)));

How long

Promise.all           101 ms   peak concurrency 30

Thirty times faster. Now make it five hundred.

Five hundred requests at once. The other side sees a burst from one client and may slow you down, rate-limit you, or refuse you. Whether anything holds some of them back on your behalf depends on where the code runs: in a browser, usually something does; in Node, nothing does.
One failure loses everything. Promise.all rejects on the first rejection, and the results of every request that succeeded are thrown away.
No timeout. One stalled connection holds the whole thing open indefinitely.
No cancellation. The user leaves, and all five hundred carry on.
No retries. One dropped packet is a total failure.

Promise.all is not "the fast one". It is the unbounded one.

1.3 The gap

sequential      one at a time, however long that takes
Promise.all     all at once, however many that is
                ↑
                nothing here

The rest of the lecture fills that gap: run some of them at once, recover from the ones that fail, give up on the ones that hang, and stop the lot when nobody is waiting any more.

If you work in React, a data library probably does much of this for you already — TanStack Query, SWR or RTK Query. Build it anyway, for two reasons. Those libraries are built around components asking for data, and your project's library and server will need the same machinery with no components involved. And where you do use one, its defaults are decisions somebody else made about your application, and you can only judge them if you know what they are deciding.

But before any of that, one question decides which tool you need.

How the operations depend on each other

2.1 Ask this first

The one-at-a-time loop was slow because it treated independent operations as if each needed the one before. That is the mistake, and it has a general form.

Before choosing between a loop, Promise.all or anything else, ask: how do these operations depend on each other?

2.2 Four answers

They are…	Example	So you…
independent	fetch five hundred profiles	run them together, with a limit
dependent	page 2 of a list needs the `next` link from page 1	run them in order — the order is correct
apparently dependent	a component that waits for its parent before fetching	notice, then run them together
identical	five components all asking for the same user	send one, share the answer

Every example in this lecture is one of those four. When you meet a slow async function in your own code, working out which row it belongs to is usually most of the fix.

Independent work

3.1 A pool

What goes between one at a time and all at once:

async function mapLimit(items, limit, fn) {
  const results = new Array(items.length);
  let index = 0;

  async function worker() {
    while (index < items.length) {
      const i = index++;
      results[i] = await fn(items[i], i);
    }
  }

  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
  return results;
}

Eight jobs of varying length, three at a time:

let inFlight = 0;
let peak = 0;

async function job(n) {
  inFlight++;
  peak = Math.max(peak, inFlight);
  await sleep(n % 3 ? 20 : 60);
  inFlight--;
  return n * 2;
}

const results = await mapLimit([1, 2, 3, 4, 5, 6, 7, 8], 3, job);
console.log(results.join(","), "| peak:", peak); // ?

Output

2,4,6,8,10,12,14,16 | peak: 3

Read mapLimit twice, because there is no queue in it. limit workers share one cursor, and each takes the next index as soon as it finishes the last. A worker that draws a fast job takes another straight away, so the pool balances itself without any scheduling code.

Results are written to results[i], so input order is preserved even though the jobs finish in a different order.

3.2 When one of them fails

As written, mapLimit has the same weakness as Promise.all: one rejection and the whole thing rejects. Usually that is not what you want. "Fetch thirty things, show me the twenty-eight that worked, and tell me why the other two did not" is an ordinary requirement.

There are four combinators, and they differ exactly here:

Combinator	Settles when	If an input fails
`Promise.all`	all fulfil	rejects at the first failure
`Promise.allSettled`	all settle	never rejects — you get a status for each input
`Promise.race`	the first settles	adopts it, success or failure alike
`Promise.any`	the first fulfils	rejects with `AggregateError` only if all fail

const ok = () => sleep(10).then(() => "ok");
const bad = () =>
  sleep(10).then(() => {
    throw new Error("nope");
  });

console.log((await Promise.allSettled([ok(), bad()])).map((r) => r.status)); // ?
console.log(await Promise.any([bad(), ok()])); // ?

try {
  await Promise.any([bad(), bad()]);
} catch (e) {
  console.log(e.constructor.name, e.errors.length); // ?
}

Output

[ 'fulfilled', 'rejected' ]
ok
AggregateError 2

allSettled gives you a report instead of an exception. The pool can do the same, by catching each failure and recording it instead of letting it escape:

async function mapSettled(items, limit, fn) {
  const results = new Array(items.length);
  let index = 0;

  async function worker() {
    while (index < items.length) {
      const i = index++;
      try {
        results[i] = { status: "fulfilled", value: await fn(items[i], i) };
      } catch (reason) {
        results[i] = { status: "rejected", reason };
      }
    }
  }

  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
  return results;
}

Same shape as allSettled, bounded like mapLimit.

A rejection that nobody handles at all is a different problem — in Node it ends the process.

3.3 `all` does not stop the others

const slow = (n) => sleep(60).then(() => console.log(`  task ${n} finished anyway`));

try {
  await Promise.all([
    slow(1),
    slow(2),
    sleep(10).then(() => {
      throw new Error("fail fast");
    }),
  ]);
} catch (e) {
  console.log("caught:", e.message);
}
await sleep(120);

Output

caught: fail fast
  task 1 finished anyway
  task 2 finished anyway

Promise.all gave you an early answer. It did not stop anything. Both slow tasks ran to completion, held on to whatever they were holding, and had their results discarded.

A promise is a receipt for work that is already under way. The request started when you called fetch, not when you awaited it. Dropping the receipt does not stop the work. Stopping it takes a separate mechanism: a signal that the work itself listens for.

Two empty-array edges while we are here:

console.log(await Promise.all([])); // ?

let settled = false;
Promise.race([]).then(() => {
  settled = true;
});
await sleep(50);
console.log("race([]) settled:", settled); // ?

Output

[]
race([]) settled: false

Promise.all([]) fulfils immediately. Promise.race([]) never settles — no error, no warning, forever. If the array can be empty, guard it. Otherwise you have written a promise that never settles, and whatever awaits it waits forever, holding on to everything it references.

3.4 Identical work

Five components mount at once and all ask for the same user. That is not five independent operations. It is one, asked for five times.

const inFlight = new Map();

function once(key, fn) {
  if (!inFlight.has(key)) {
    inFlight.set(
      key,
      fn().finally(() => inFlight.delete(key))
    );
  }
  return inFlight.get(key);
}

let calls = 0;
const loadUser = () => {
  calls++;
  return sleep(30).then(() => ({ name: "Ada" }));
};

await Promise.all([once("user:1", loadUser), once("user:1", loadUser), once("user:1", loadUser)]);
console.log("three callers, underlying calls:", calls); // ?

await once("user:1", loadUser);
console.log("a later caller:", calls); // ?

Output

three callers, underlying calls: 1
a later caller: 2

This works because a promise settles exactly once and can be handed to any number of consumers: all three get the same answer from one request.

The finally is not optional. It passes the result through untouched and removes the entry however the request ended. Without it the map only grows: every key ever requested, and its answer, held for as long as the program runs.

Note what once deliberately does not do. It shares requests that overlap in time. It does not cache results — the caller who arrives after it settled gets a fresh request.

You have probably used this without writing it. TanStack Query tracks the in-flight promise for each query key, so three components asking for the same key at the same moment produce one request and three resolutions. SWR keeps the same map, but delays the delete: the entry stays for two seconds (dedupingInterval) after the request settles, so a caller who arrives just after the answer still shares it. That is once with a two-second cache attached.

Dependent work

4.1 Sometimes the order is the point

A paginated API. Each response contains the address of the next page. There is no way to ask for page two until page one has told you where it is.

Here the one-at-a-time loop is correct. The operations genuinely depend on each other, and running them together is not an option. The question is only how to make the consuming code pleasant — and that is what async generators are for.

4.2 A generator pauses

The minimum you need:

function* counter() {
  let n = 0;
  while (true) yield n++;
}

const g = counter();
console.log(g.next()); // ?
console.log(g.next()); // ?

Output

{ value: 0, done: false }
{ value: 1, done: false }

Calling a generator function runs no code. It returns an iterator. Each next() runs until the next yield, then freezes the function — local variables and all — until you ask again.

You will rarely write function*. You will use the asynchronous version, which is the same idea with await allowed inside.

4.3 Pagination as a loop

const pages = {
  "/items?page=1": { items: ["a", "b"], next: "/items?page=2" },
  "/items?page=2": { items: ["c"], next: null },
};

async function getPage(url) {
  await sleep(50);
  return pages[url];
}

async function* allItems(url) {
  while (url) {
    const page = await getPage(url);
    yield* page.items; // hands over each item in turn
    url = page.next;
  }
}

for await (const item of allItems("/items?page=1")) {
  console.log(item); // ?
}

Output

a
b
c

The consumer writes an ordinary for loop. No page counter, no while (hasMore), no array holding every item at once, and nothing fetched before it is asked for. In real code getPage is (await fetch(url)).json(); nothing else changes.

4.4 Breaking out cleans up

async function* withCleanup() {
  try {
    yield 1;
    yield 2;
    yield 3;
  } finally {
    console.log("generator cleanup ran");
  }
}

for await (const v of withCleanup()) {
  if (v === 2) break;
}

Output

generator cleanup ran

Leaving the loop early calls return() on the generator, which runs its finally. So a generator that holds a connection, a file handle or an AbortController releases it the moment the consumer stops caring — including when the consumer breaks, returns or throws.

False dependencies

5.1 Two awaits that did not need to be in a row

async function getUser(id) {
  await sleep(100);
  return { id, name: "Ada" };
}

async function getPosts(id) {
  await sleep(100);
  return [{ id: 1 }];
}

async function loadProfile(userId) {
  const user = await getUser(userId);
  const posts = await getPosts(userId); // never uses `user`
  return { user, posts };
}

getPosts does not use user. It needed only userId, which was known before either call started. The await on the line above made it wait anyway.

async function loadProfile(userId) {
  const [user, posts] = await Promise.all([getUser(userId), getPosts(userId)]);
  return { user, posts };
}

Timed

async function inSeries(userId) {
  const user = await getUser(userId);
  const posts = await getPosts(userId);
  return { user, posts };
}

async function together(userId) {
  const [user, posts] = await Promise.all([getUser(userId), getPosts(userId)]);
  return { user, posts };
}

for (const [label, load] of [
  ["two awaits in a row", inSeries],
  ["started together", together],
]) {
  const t0 = performance.now();
  await load(1);
  console.log(`${label}: ${Math.round(performance.now() - t0)} ms`);
}

two awaits in a row: 201 ms
started together: 101 ms

Compare a real dependency:

const post = await getPost(postId);
const author = await getUser(post.authorId); // genuinely needs `post`

Here the second call cannot be written until the first has answered. The test is not whether two awaits are adjacent. It is whether the second one uses something the first produced.

5.2 The same thing in a component tree

function Profile({ userId }) {
  const user = useUser(userId);
  if (!user) return <Spinner />;

  return (
    <>
      <h1>{user.name}</h1>
      <Posts authorId={userId} />
    </>
  );
}

Posts needs userId — a prop, known from the first render. But it does not mount until user arrives, so its request cannot start until then. Two round trips in series, and neither needed the other.

It is the same false dependency as two awaits in a row, wearing a component tree. The fix is the same: stop gating the second request behind the first.

function Profile({ userId }) {
  const user = useUser(userId);

  return (
    <>
      {user ? <h1>{user.name}</h1> : <Spinner />}
      <Posts authorId={userId} />
    </>
  );
}

Both components now mount on the first render and both requests start at once.

When the gating is spread across several levels of components, the fix usually moves up: fetch at the route level with a router loader, or start the requests where all the parameters are first known. What you cannot do is fix it inside the child — by the time the child exists, the time is already lost.

5.3 A list is an unbounded `Promise.all`

The opposite mistake, in the same tree:

{
  items.map((item) => (
    <ItemCard key={item.id} id={item.id} /> // each fetches in its own effect
  ));
}

Five hundred rows, five hundred requests, all started after the same render. Nobody wrote Promise.all, and the result is the same: every request in flight at once, with no limit.

Two fixes: fetch the list's data in one request at the parent, or route the per-item requests through a pool so only a handful are ever outstanding.

Work that fails or hangs

6.1 Retry, with backoff and jitter

function isRetryable(err) {
  if (err.name === "TimeoutError") return true;
  if (err.name === "TypeError") return true; // how fetch reports a network failure
  if (err.status === 429 || err.status >= 500) return true;
  return false; // a 404 or a 400 will fail the same way every time
}

async function retry(fn, { attempts = 3, baseMs = 200 } = {}) {
  let lastError;

  for (let attempt = 0; attempt < attempts; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (!isRetryable(err)) throw err;
      lastError = err;
      if (attempt === attempts - 1) break;
      await sleep(Math.random() * baseMs * 2 ** attempt);
    }
  }

  throw lastError;
}

Three decisions are packed into those lines.

What is worth retrying. A server error or a rate limit may clear up. So may a network failure — a connection reset, refused or not found — which fetch reports as a TypeError with no status at all. A 404 will not, and retrying it only delays the inevitable while adding load.

The TypeError line is broader than it looks. Browsers use the same error for a request that CORS refused, which will not clear up, and it is also what a bug in your own code throws. For a GET, retrying those costs a few wasted attempts, not correctness. For a POST it is not so simple: a connection that dropped after the request was sent leaves you not knowing whether the server acted on it.

Exponential backoff. Each wait is twice the last — 200 ms, then 400, then 800. A service that is struggling gets progressively more room.

Jitter. The Math.random(). This is the line that looks like noise and gets deleted in code review. Eight clients that failed at the same moment, each computing its second wait:

const waits = (withJitter) =>
  Array.from({ length: 8 }, () => {
    const backoff = 200 * 2 ** 1;
    return Math.round(withJitter ? Math.random() * backoff : backoff);
  });

console.log("without:", waits(false).join(", "));
console.log("with:   ", waits(true).join(", "));

Output (the second line differs every run)

without: 400, 400, 400, 400, 400, 400, 400, 400
with:    131, 242, 30, 35, 53, 151, 44, 313

Without jitter, everyone who failed together retries together, fails together, and retries together again — a thundering herd. The recovering service is hit by a synchronised wall of traffic every time it comes back up, and a brief outage becomes a long one.

Picking the wait uniformly between zero and the backoff, as here, is what Marc Brooker called full jitter in the analysis that made it the standard approach. His conclusion is worth keeping: exponential backoff is good, but on its own it is not enough.

6.2 A timeout that actually stops

The obvious timeout races the work against a timer. Here is what that does to a job that takes 200 ms, compared with passing the job a signal:

function slowJob(signal) {
  return new Promise((resolve, reject) => {
    const onAbort = () => {
      clearTimeout(timer);
      reject(signal.reason);
    };

    const timer = setTimeout(() => {
      signal?.removeEventListener("abort", onAbort); // finished: stop listening
      console.log("  the job finished its work");
      resolve("done");
    }, 200);

    signal?.addEventListener("abort", onAbort, { once: true });
  });
}

// racing it
try {
  await Promise.race([
    slowJob(),
    sleep(50).then(() => {
      throw new Error("Timeout");
    }),
  ]);
} catch (e) {
  console.log("race:", e.message);
}
await sleep(250);

// signalling it
try {
  await slowJob(AbortSignal.timeout(50));
} catch (e) {
  console.log("signal:", e.name);
}
await sleep(250);

Output

race: Timeout
  the job finished its work
signal: TimeoutError

Look at the middle line. After the race reported a timeout, the job carried on and finished its work anyway. The race gave you an early answer and stopped nothing — just as Promise.all did when one of its inputs failed.

With the signal, that line never appears. The job was told to stop, and it stopped.

const data = await fetch(url, { signal: AbortSignal.timeout(5000) });

AbortSignal.timeout(ms) is the tool to reach for whenever the operation accepts a signal. When you also have a signal from the caller, combine them — AbortSignal.any([callerSignal, AbortSignal.timeout(5000)]) aborts on whichever comes first.

Only if the work can be stopped

A signal is a flag and an event. fetch listens to it because the browser implements that. Code you wrote listens only if you wrote the listener — as slowJob does — and a loop you wrote only gets the chance to check if it yields with a task.

6.3 A timeout is not a failure

Put a timeout inside a retry, and a request that times out gets retried. For a GET, that is fine. For this, it is not:

await retry(() =>
  fetch("/payments", {
    method: "POST",
    body: JSON.stringify({ amount: 100 }),
    signal: AbortSignal.timeout(5000),
  })
);

A timeout tells you one thing: you stopped waiting. It does not tell you whether the server got the request, whether it charged the card, or whether it was one millisecond from sending the reply. A timeout is not a failure. It is an unknown — and retrying an unknown can do the thing twice.

The standard answer is an idempotency key: a unique value the client sends with the request, which the server uses to recognise a repeat and return the original answer instead of acting again.

const key = crypto.randomUUID(); // once per payment — NOT inside the retry

await retry(() =>
  fetch("/payments", {
    method: "POST",
    headers: { "Idempotency-Key": key },
    body: JSON.stringify({ amount: 100 }),
    signal: AbortSignal.timeout(5000),
  })
);

There are two ways to get this wrong, one on each side:

The experiment — a client that gives up after 50 ms and tries once more, against a server that takes 80 ms

function makeServer() {
  const charges = [];
  const seen = new Map();

  return {
    charges,
    pay({ amount, key }) {
      if (key && seen.has(key)) return seen.get(key); // in progress or done: same answer

      const work = (async () => {
        charges.push(amount); // the side effect happens here…
        await sleep(80); // …and the reply is slow
        return { ok: true, charge: charges.length };
      })();

      if (key) seen.set(key, work); // recorded when the work STARTS
      return work;
    },
  };
}

async function payWithRetry(server, makeRequest) {
  for (let attempt = 0; attempt < 2; attempt++) {
    const reply = server.pay(makeRequest());
    const winner = await Promise.race([reply, sleep(50).then(() => "timeout")]);
    if (winner !== "timeout") return winner;
  }
}

for (const [label, makeRequest] of [
  ["no key", () => ({ amount: 100 })],
  ["key made per attempt", () => ({ amount: 100, key: crypto.randomUUID() })],
  ["key made once", ((key) => () => ({ amount: 100, key }))(crypto.randomUUID())],
]) {
  const server = makeServer();
  await payWithRetry(server, makeRequest);
  await sleep(100);
  console.log(`${label}: charged ${server.charges.length} time(s)`);
}

no key: charged 2 time(s)
key made per attempt: charged 2 time(s)
key made once: charged 1 time(s)

On the client, the key has to be made once per logical operation, outside the retry. A key generated inside the retried function is a new key on every attempt, which is the same as having no key at all.

On the server, the key has to be recorded when the work starts, not when it finishes — and kept after it finishes. Record it on completion and the retry — which arrives precisely because the first attempt was slow — finds nothing yet, and charges again. Recording the in-flight work under the key is the same trick as sharing identical requests — a map from key to promise — applied to a side effect instead of a read. With one difference: the sharing map forgets a key the moment its request settles. An idempotency record has to outlive it, typically for hours, because the commonest repeat of all is a retry that arrives after the work finished and its reply was lost. The experiment above cannot show this, because its retry arrives while the first attempt is still running; makeServer gets it right by never deleting.

Putting it together

Now all of it together: many requests, a few at a time, each with a timeout, each retried if the failure is worth retrying, all of it stoppable, and a report at the end instead of an exception.

7.1 The layers

One signal, from the caller, has to reach every layer: the pool must stop starting new work, retry must stop retrying, every wait between attempts must end, and every request in flight must be aborted. Miss any one and cancelling is partial.

7.2 Every wait has to listen

The obvious way to make retry cancellable is to check the signal before each attempt:

async function retry(fn, { attempts = 3, baseMs = 200, signal } = {}) {
  let lastError;

  for (let attempt = 0; attempt < attempts; attempt++) {
    signal?.throwIfAborted(); // ← check before each attempt
    try {
      return await fn();
    } catch (err) {
      lastError = err;
      if (attempt === attempts - 1) break;
      await sleep(Math.random() * baseMs * 2 ** attempt);
    }
  }

  throw lastError;
}

It looks complete. A job that always fails, and a user who cancels 100 ms in:

const alwaysFails = () =>
  sleep(50).then(() => {
    throw new Error("503");
  });

const controller = new AbortController();
setTimeout(() => controller.abort(), 100);

const t0 = performance.now();
try {
  await retry(alwaysFails, { attempts: 5, baseMs: 1000, signal: controller.signal });
} catch {
  console.log(`gave up after ${Math.round(performance.now() - t0)} ms`);
}

Six runs

gave up after 779 ms
gave up after 490 ms
gave up after 923 ms
gave up after 973 ms
gave up after 317 ms
gave up after 566 ms

The user cancelled at 100 ms. retry noticed anywhere up to a second later, occasionally two — how long depends on the random backoff it happened to be sleeping through. (Two when the first backoff comes out under 50 ms: a second attempt runs, and the backoff after it can be up to 2 s.)

The check runs between attempts. When the cancel arrived, retry was inside sleep, waiting out its backoff — and sleep has no idea a signal exists. So retry slept until the timer ran out, then checked, then stopped.

Every await on the path is a place where the signal can go unheard. The fix is a sleep that listens:

function sleep(ms, { signal } = {}) {
  return new Promise((resolve, reject) => {
    if (signal?.aborted) return reject(signal.reason);

    const onAbort = () => {
      clearTimeout(timer);
      reject(signal.reason);
    };

    const timer = setTimeout(() => {
      signal?.removeEventListener("abort", onAbort);
      resolve();
    }, ms);

    signal?.addEventListener("abort", onAbort, { once: true });
  });
}

With sleep(…, { signal }) in retry, the same six runs:

Six runs

gave up after 99 ms
gave up after 100 ms
gave up after 100 ms
gave up after 100 ms
gave up after 99 ms
gave up after 99 ms

Almost always, anyway. Run it enough times and about one run in twenty still gives up late — by up to 50 ms. Those are the runs where the random backoff came out shorter than 50 ms, so the next attempt had already started when the cancel arrived. retry calls fn() without the signal, so that attempt cannot hear it, and retry waits for it to fail on its own.

The sleep was one deaf await. The attempt is another, and the fix has the same shape: retry hands the signal to each attempt — return await fn(signal); — and the attempt passes it on to whatever it waits for. alwaysFails takes no argument, so it has to change too:

const alwaysFails = (signal) =>
  sleep(50, { signal }).then(() => {
    throw new Error("503");
  });

With both waits listening, a thousand runs out of a thousand give up at the cancel. Every layer has to hear the signal, and fixing the loudest one is not the same as fixing all of them.

The removeEventListener in the timer's callback matters more than it looks. { once: true } only removes the listener if it fires. A sleep that finishes normally would otherwise leave its listener attached to the caller's signal — and a long-lived signal passed through hundreds of retries would collect hundreds of listeners, each holding its closure. That is a memory leak, hiding in a utility function.

Node has one built in

In Node, import { setTimeout } from "node:timers/promises" gives you await setTimeout(ms, undefined, { signal }), which already listens. The hand-written version above is for code that must also run in a browser.

One difference if you swap it in: Node's version rejects with a new AbortError that carries the signal's reason as its cause, not with the reason itself — even when the signal came from AbortSignal.timeout. retry does not mind: it never inspects the error a cancelled wait throws, it passes it straight up. Code that checks err.name === "TimeoutError" will.

7.3 The whole thing

function isRetryable(err) {
  if (err.name === "TimeoutError") return true;
  if (err.name === "TypeError") return true;
  if (err.status === 429 || err.status >= 500) return true;
  return false;
}

async function retry(fn, { attempts = 3, baseMs = 200, timeoutMs, signal } = {}) {
  let lastError;

  for (let attempt = 0; attempt < attempts; attempt++) {
    signal?.throwIfAborted();

    // each attempt gets its own deadline, and still hears the caller
    const attemptSignal = timeoutMs ? AbortSignal.any([signal, AbortSignal.timeout(timeoutMs)].filter(Boolean)) : signal;

    try {
      return await fn(attemptSignal);
    } catch (err) {
      if (signal?.aborted) throw err; // the caller gave up: stop, whatever the error says
      if (!isRetryable(err)) throw err;
      lastError = err;
      if (attempt === attempts - 1) break;
      await sleep(Math.random() * baseMs * 2 ** attempt, { signal });
    }
  }

  throw lastError;
}

async function mapSettled(items, limit, fn, { signal } = {}) {
  const results = new Array(items.length);
  let index = 0;

  async function worker() {
    while (index < items.length) {
      signal?.throwIfAborted(); // stop taking new work
      const i = index++;
      try {
        results[i] = { status: "fulfilled", value: await fn(items[i], i) };
      } catch (reason) {
        if (signal?.aborted) throw reason; // cancelled, not failed
        results[i] = { status: "rejected", reason };
      }
    }
  }

  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
  return results;
}

async function fetchJson(url, { signal } = {}) {
  const response = await fetch(url, { signal });
  if (!response.ok) {
    throw Object.assign(new Error(`HTTP ${response.status}`), {
      status: response.status,
    });
  }
  return response.json();
}

function fetchAll(urls, { limit = 6, attempts = 3, timeoutMs = 5000, signal } = {}) {
  return mapSettled(
    urls,
    limit,
    (url) =>
      retry((attemptSignal) => fetchJson(url, { signal: attemptSignal }), {
        attempts,
        timeoutMs,
        signal,
      }),
    { signal }
  );
}

One line is worth dwelling on:

if (signal?.aborted) throw err;

Why not decide from the error? Because the error cannot tell you who gave up. A per-attempt timeout arrives as a TimeoutError. So does the caller's own deadline, if the caller passed AbortSignal.timeout(…). Same name, opposite meanings: one is worth retrying, the other means nobody is listening any more. The signal knows who aborted. The error does not. So ask the signal.

The same line appears in mapSettled, for the same reason. A request that failed goes in the report. A request that was cancelled means the whole batch was cancelled, so the batch rejects instead of reporting thirty cancellations.

7.4 Running it

The demo runs exactly this code — copied from this page, not rewritten — with fetchJson swapped for a fake server so it needs no network, and a 300 ms per-attempt timeout so the hanging requests give up quickly. Thirty URLs, of which three hang forever, one returns 404, two return 503 on their first attempt and succeed on their second, and the rest succeed.

To completion, then cancelled 150 ms in

26 succeeded, 4 failed {"TimeoutError":3,"HTTP 404":1}; peak 6
cancelled at 150 ms → rejected with AbortError at 150 ms; requests in flight afterwards: 0

Read the first line against the list of what the fake server does.

The two 503s do not appear: retry recovered them, and the caller never saw the failures.
The 404 failed once and was not retried.
The three hangs each timed out, were retried twice more, and ended in the report as timeouts rather than holding the batch open.
Peak 6 — never more than the limit, however many URLs there are.

And the second line: cancelled at 150 ms, rejected at 150 ms, and nothing left running.

7.5 What the pool can hide

One more experiment. Take the finished fetchAll, put the old sleep back into retry, and cancel it.

fetchAll still rejects promptly. Any worker that happens to be mid-request sees the abort, rethrows it, and the pool's Promise.all rejects at once. From the caller's side, cancelling works.

But the process stays alive. In about three runs out of four, one worker was asleep in a backoff when the cancel arrived — the one holding URL 3, which got its 503 at 100 ms and drew a wait of up to 200 ms. It kept its timer running for up to another 150 ms, woke up, checked the signal, and only then stopped.

A prompt rejection is not proof that everything stopped. It proves that something noticed.

You already use this

If you write React, you have probably never needed a pool or a retry of your own, because a data library did it for you. Everything built so far is a set of decisions — how many at once, what to retry, how long to wait, when to stop — and each of those libraries has already made them — some the way this lecture did, some differently. Reading a library's defaults is how you find out what it is doing to your application.

8.1 Three libraries, three defaults

Library	Retries by default?	Delay between attempts
TanStack Query	queries: yes, 3 times (0 on the server) · mutations: no	doubles from 1 second, capped at 30
SWR	yes, on any error, with no limit	doubles from 5 seconds, times a random factor; levels off at 11–32 minutes
RTK Query	no — you wrap the base query in its `retry` utility	exponential, `maxRetries: 5` once you do

Three widely used libraries, three different answers to whether a failure is worth retrying. None of them is wrong. A default is a decision somebody made for the situation they had in mind, and it is right for you only if your situation matches theirs.

SWR's row is the one to read twice. Its errorRetryCount has no default, so a key that keeps failing is retried for as long as a component uses it and the tab is visible and online. The waits stop growing after eight retries, and from then on it carries on every 11 to 32 minutes. That is a reasonable choice for a dashboard left open all day. It is worth knowing about for an endpoint that will never recover.

One detail that catches people moving between them: TanStack's retry: 3 means three retries — four attempts in all. This lecture's attempts: 3 means three in total. Every library counts in whatever way it chose; check which.

8.2 Idempotency is written into the defaults

Look at the first row again. TanStack retries queries by default and mutations not at all. A query reads; a mutation changes something. Retrying a read is harmless; retrying a write may do it twice. That distinction is made for you.

RTK Query does not make it. Its retry wraps the base query, which queries and mutations share, and its default condition is simply "fewer than maxRetries attempts so far". A wrapped base query therefore retries a failed mutation five times — a 404 included — unless that endpoint sets extraOptions: { maxRetries: 0 }. If you use RTK, that line belongs on every mutation that is not idempotent.

On the Node side, fetch is built on a library called undici, and the undici package offers a retry agent you can opt into — Node's plain fetch retries nothing on its own. That agent makes the same call as TanStack, more explicitly. Its defaults for what to retry, written out as they are in undici 7 (the undici bundled with Node 22 has the same lists):

import { Agent, RetryAgent, fetch } from "undici";

const dispatcher = new RetryAgent(new Agent(), {
  methods: ["GET", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"],
  statusCodes: [500, 502, 503, 504, 429],
  errorCodes: [
    "ECONNRESET",
    "ECONNREFUSED",
    "ENOTFOUND",
    "ENETDOWN",
    "ENETUNREACH",
    "EHOSTDOWN",
    "EHOSTUNREACH",
    "EPIPE",
    "UND_ERR_SOCKET",
  ],
  retryAfter: true,
});

const response = await fetch(url, { dispatcher });

methods leaves out POST and PATCH. The six it lists are the methods the HTTP specification defines as idempotent — sending one twice has the same effect as sending it once. That is the idempotency rule, as a default. undici 8 adds a seventh, QUERY, a newer method that is also defined as safe and idempotent: same rule, longer list.
statusCodes is isRetryable, nearly line for line: rate limits and server errors, never a 404.
errorCodes is the TypeError line of isRetryable, spelled out: connections reset, refused or unreachable, and names that did not resolve. undici can list them one by one because it sees the socket. fetch hands you a TypeError, and in Node the code is on its cause.
retryAfter reads the Retry-After header, which a server sends with a 429 or a 503 to say how long to wait. The retry in this lecture ignores it and guesses with backoff. A server telling you when to come back is better information than any formula, so a client for a real API should read the header.

8.3 Who can tell a 404 from a 503

undici can check a status code because it is the HTTP client — it sees the response. The React libraries cannot. They call your function and learn only whether it threw.

That is why SWR's documentation, when it shows how to customise retries, begins with a rule never to retry a 404: by default it would. TanStack's default does the same, for the same reason.

So in a React application, deciding what is worth retrying is your job — and isRetryable and fetchJson drop straight in:

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      // the default's three retries, minus the ones not worth making
      retry: (failureCount, error) => failureCount < 3 && isRetryable(error),

      // TanStack's own curve, with full jitter added
      retryDelay: (attempt) => Math.random() * Math.min(1000 * 2 ** attempt, 30000),
    },
  },
});

function useUser(id) {
  return useQuery({
    queryKey: ["user", id],
    // TanStack passes the query function a signal; add a deadline to it
    queryFn: ({ signal }) =>
      fetchJson(`/api/users/${id}`, {
        signal: AbortSignal.any([signal, AbortSignal.timeout(5000)]),
      }),
  });
}

Three pieces of this lecture, inside a library you already use: the retry decision, full jitter, and a combined signal.

failureCount < 3 matches the default exactly. TanStack calls retry with a count that starts at 0 and is only incremented after the decision, so it answers for counts 0, 1 and 2 — three retries.

About retryDelay: TanStack's default doubles from one second with no randomness, where SWR's multiplies by a random factor. That is not a mistake — it is a default, and it can be changed. But it is exactly the kind of default worth knowing about. If many clients can fail against your server at the same moment, add the jitter yourself.

SWR's randomness is coarser than it sounds, too. It rounds the random factor down to a whole number before multiplying by five seconds, so every wait is a whole multiple of 5 s. At the first retry that means 5 s or 10 s, half the clients each, and nothing in between. That splits a herd in two, which is better than nothing and a long way from full jitter.

8.4 The pool, in Node

The browser libraries have no pool; they fetch whatever components ask for. In Node the usual choice is p-map, and its README is candid about the same weakness Promise.all has.

import pMap from "p-map";

const results = await pMap(urls, (url) => fetchJson(url), {
  concurrency: 6,
  stopOnError: false,
  signal: controller.signal,
});

concurrency is the pool's limit.
stopOnError: false waits for everything to settle, then rejects with an AggregateError holding every failure. Notice what that does not give you: the successful results. For a report of both, catch inside the mapper and return a status for each item — which is exactly what mapSettled does.
With the default, stopOnError: true, the README warns that mappers already running carry on after the first rejection, and points you to signal to stop them. That is "all does not stop the others", documented as a caveat.

The same package offers pMapIterable, which runs the mapper with a concurrency limit and hands you the results as an async iterable:

import { pMapIterable } from "p-map";

for await (const item of pMapIterable(urls, (url) => fetchJson(url), { concurrency: 6 })) {
  render(item);
}

That is a pool and for await, combined — independent work, consumed as it arrives.

What concurrency costs

9.1 Measured

Every timing so far came from a stand-in server that answered everything, however many requests arrived at once. Real servers do not. Here is the same thirty requests against a stand-in that refuses — with a fast 429 — anything beyond ten from one client at a time:

                    time    peak   failed
sequential       3013 ms      1     0 / 30
Promise.all       101 ms     30    20 / 30
mapLimit(4)       804 ms      4     0 / 30
mapLimit(6)       502 ms      6     0 / 30
mapLimit(8)       402 ms      8     0 / 30
mapLimit(10)      301 ms     10     0 / 30
mapLimit(12)      101 ms     12    20 / 30
mapLimit(16)      101 ms     16    20 / 30

Three things in that table.

Promise.all fails two thirds of its requests. That is the cost of a burst, made visible at thirty requests, not five hundred.

There is a cliff at the server's limit. Ten is the best result on the table. Twelve collapses completely. The right limit is not "as high as possible"; it is just below wherever the other side starts refusing.

The broken rows are the fastest. Promise.all, 12 and 16 all finish in about 100 ms — because a refusal comes back almost instantly. Measure only time, and you would choose the worst configuration here. Always count failures alongside duration.

Reproduce it yourself

The script is here. No server and no network:

node concurrency-cost.mjs

It prints two tables. The first uses a stand-in that answers everything; the second uses the refusing stand-in and reproduces the table above. Running both from one script is the point: same code, same requests, and the only change is whether the other side has a limit.

The stand-in settles through a timer — a task — exactly as a real response does, which is what makes the timing faithful. Each request increments a counter on the way in and decrements it on the way out; the peak column is the highest that counter reached.

A request arriving while ten are already outstanding is refused after 5 ms. That is the only difference between the two stand-ins, and it is the whole reason the table looks different: a limit exists to protect something, and until something is being protected, there is nothing for a limit to do.

9.2 Who pays

Most performance costs land on your own program: memory it holds, time it spends, a thread it blocks.

This cost is different in kind. It is paid by someone else — the server, its database, the other clients sharing it — and it comes back to you later, as refusals and slowdowns.

That is why the right limit is usually set by the other side rather than chosen by you:

Check whether it is published. Many APIs document a concurrency or rate limit. That is your answer, and no measurement is needed.
If not, measure — counting failures, not just time. Run the same batch at several limits against the real service, and stay below the point where refusals start.
Remember it is per process. Six per process across four processes is twenty-four arriving at the server together.

Where the limit lives

Whether anything limits you besides your own pool depends on where the code runs.

Where the code runs	What holds requests back
a browser, over HTTP/1.1	the browser — Chrome opens at most six connections per host, and queues the rest
a browser, over HTTP/2	the server's advertised stream limit on one shared connection — the specification recommends at least 100
Node, using `fetch`	nothing by default — each concurrent request to the same host gets its own socket

So in a browser, your pool is a second limit underneath one the platform already applies. It still matters — the platform's limit is often far above what the server tolerates, as the server that refused beyond ten showed.

In Node, your pool is the only limit there is. "Five hundred requests at once" there means five hundred sockets. That is the setting where an unbounded Promise.all turns from inefficient into an incident — and Node is where servers run.

See it in DevTools

Everything so far was measured in code. The Network panel shows the same three shapes directly, and it is the panel you will use most.

Open the demo page. Three buttons, each fetching the same small file thirty times with a cache-busting parameter: sequential, Promise.all, and a pool of six.

Open DevTools → Network. Tick Disable cache.
Click 1 · Sequential. The waterfall stair-steps — each bar starts where the one above ended. Thirty round trips in a line.
Click 2 · Promise.all. One block — all thirty bars start together. The site is served over HTTP/2, and thirty is well under the stream limit, so nothing is queued.
Click 3 · Pool of 6. Five distinct waves of six.

The log under the buttons prints the protocol each run used. If you serve the page yourself from a local dev server, it will probably say http/1.1, and step 3 looks different: the browser opens six connections and queues the rest, so run 2 comes out almost the same shape as run 3. That is the browser's six-connection limit, on your own machine. It is also why the log reports peak pending rather than concurrency: it counts the promises your code is waiting on, which is thirty in run 2 whatever the protocol, not the requests actually on the wire.

Two things to look at while you are there:

Hover a bar to see its timing breakdown: queueing, connection, waiting for the first byte, download. In run 1, nearly all of it is waiting. That wait is what the loop pays thirty times over.
Add the Connection ID column — right-click any column header. Over HTTP/2 all thirty share one connection. Against an HTTP/1.1 server you would see six, and genuine queueing in the timing breakdown.

Then record run 2 in the Performance panel. The main thread is almost entirely idle. All three runs are waiting on the network, not computing — which is why the answer is concurrency. Splitting the work into smaller chunks, the cure for a busy main thread, would not help at all.

Take these 5 things from here

Ask how the operations depend on each other. Independent, dependent, apparently dependent, or identical — the answer picks the tool.
Promise.all is the unbounded one. A pool is the same speed-up with a number you chose.
A promise is a receipt for work already under way. A race gives you an early answer; only a signal stops the work.
A timeout is an unknown, not a failure. Retrying it can do the thing twice, unless the request carries an idempotency key.
Every await on the path must hear the signal. One deaf sleep and cancelling is only partial.

You should now be able to

Classify a set of async operations as independent, dependent, apparently dependent or identical, and name the tool each calls for.
Write a concurrency-limited map, explain how it keeps input order without a queue, and turn it into one that reports failures instead of rejecting.
Choose between all, allSettled, race and any from what should happen when one input fails, and name the case where race never settles.
Explain why a rejected Promise.all leaves the other operations running.
Deduplicate simultaneous identical requests, and explain why the finally is required.
Consume a paginated API with for await, and say what runs when the consumer breaks out early.
Spot two awaits that do not need to be in sequence, in a function and in a component tree.
Write retry with exponential backoff and full jitter, and say which failures should not be retried.
Explain why racing a timeout does not stop the work, and use AbortSignal.timeout and AbortSignal.any so that it does.
Explain why a timed-out request may already have succeeded, and where an idempotency key must be created and recorded.
Thread one signal through a pool, a retry and a backoff, and find the await that is not listening.
Choose a concurrency limit from the other side's limit, measuring failures as well as time.
Say what limits concurrent requests in a browser over HTTP/1.1, over HTTP/2, and in Node.
Read a data library's retry defaults — whether it retries, what, how often and how long it waits — and adjust them with isRetryable, jitter and a combined signal.

Reading

Required

Andres Käver's base course lecture on promises and the event loop
MDN: Using promises
MDN: AbortSignal — timeout() and any() in particular

Recommended

Marc Brooker, "Exponential Backoff And Jitter" — AWS Architecture Blog. The simulations behind backoff with jitter, and where "full jitter" comes from
MDN: Iterators and generators
The READMEs of p-map and p-retry, read alongside mapLimit and retry — note what they handle that the versions here do not
TanStack Query: Important Defaults — TanStack's retry defaults, from the source
undici: RetryAgent — the retry defaults, with every option they sit alongside
The Promises/A+ specification — two pages, written by library authors before the language had promises, and adopted almost unchanged

Demos

Concurrency cost — time and failures at several limits, against a server with and without one
fetchAll — the finished pool with retry, timeouts and cancellation, runnable against a fake server
Network waterfall — the DevTools walkthrough

The loop that waits​

1.1 One at a time​

1.2 All at once​

1.3 The gap​

How the operations depend on each other​

2.1 Ask this first​

2.2 Four answers​

Independent work​

3.1 A pool​

3.2 When one of them fails​

3.3 all does not stop the others​

3.4 Identical work​

Dependent work​

4.1 Sometimes the order is the point​

4.2 A generator pauses​

4.3 Pagination as a loop​

4.4 Breaking out cleans up​

False dependencies​

5.1 Two awaits that did not need to be in a row​

5.2 The same thing in a component tree​

5.3 A list is an unbounded Promise.all​

Work that fails or hangs​

6.1 Retry, with backoff and jitter​

6.2 A timeout that actually stops​

6.3 A timeout is not a failure​

Putting it together​

7.1 The layers​

7.2 Every wait has to listen​

7.3 The whole thing​

7.4 Running it​

7.5 What the pool can hide​

You already use this​

8.1 Three libraries, three defaults​

8.2 Idempotency is written into the defaults​

8.3 Who can tell a 404 from a 503​

8.4 The pool, in Node​

What concurrency costs​

9.1 Measured​

9.2 Who pays​

Where the limit lives​

See it in DevTools​

Take these 5 things from here​

You should now be able to​

Reading​

Demos​

The loop that waits

1.1 One at a time

1.2 All at once

1.3 The gap

How the operations depend on each other

2.1 Ask this first

2.2 Four answers

Independent work

3.1 A pool

3.2 When one of them fails

3.3 `all` does not stop the others

3.4 Identical work

Dependent work

4.1 Sometimes the order is the point

4.2 A generator pauses

4.3 Pagination as a loop

4.4 Breaking out cleans up

False dependencies

5.1 Two awaits that did not need to be in a row

5.2 The same thing in a component tree

5.3 A list is an unbounded `Promise.all`

Work that fails or hangs

6.1 Retry, with backoff and jitter

6.2 A timeout that actually stops

6.3 A timeout is not a failure

Putting it together

7.1 The layers

7.2 Every wait has to listen

7.3 The whole thing

7.4 Running it

7.5 What the pool can hide

You already use this

8.1 Three libraries, three defaults

8.2 Idempotency is written into the defaults

8.3 Who can tell a 404 from a 503

8.4 The pool, in Node

What concurrency costs

9.1 Measured

9.2 Who pays

Where the limit lives

See it in DevTools

Take these 5 things from here

You should now be able to

Reading

Demos