What Is Single-Flight Request Coalescing? A Complete Guide with Python Examples

Imagine you’ve built a URL shortener like Bitly. Everything works perfectly until one of your shortened links goes viral.

Thousands of users click the same link within seconds. Your Redis cache entry expires, and suddenly your database receives hundreds or thousands of identical queries.

The database isn’t doing anything complicated. It’s answering the same question repeatedly.

This is where single-flight request coalescing becomes useful.

Single-flight request coalescing is a concurrency optimization technique that combines simultaneous requests for the same resource into one shared operation, reducing duplicate work and protecting backend systems from unnecessary load.

In this guide, we’ll understand how it works, why it matters, how to implement it in Python, and where it fits into real-world systems.

What Is Single-Flight Request Coalescing?

Single-flight request coalescing, sometimes called in-flight request deduplication, ensures that when multiple requests need the same data at the same time, only one request performs the expensive operation.

The remaining requests wait for that operation to finish and then share its result.

Let’s say 1,000 users request the same product information simultaneously.

Without single-flight, your application might execute 1,000 identical database queries.

With single-flight, ideally, it executes one query and shares the response with all 1,000 users.

Here’s the difference:

Single-flight doesn’t eliminate all database queries. It eliminates redundant operations that overlap in time.

What Is a Cache Stampede, and Why Does It Happen?

A cache stampede happens when many requests simultaneously attempt to retrieve the same data after its cache entry becomes unavailable.

Consider a URL shortening service with this URL:

https://sho.rt/abc123

Its Redis entry stores:

Key:   url:abc123
Value: https://example.com/blog
TTL:   300 seconds

Under normal conditions, requests follow a simple flow:

1. A user clicks the shortened URL.
2. The application checks Redis.
3. Redis returns the original URL.
4. The application redirects the user.

No database query is needed.

But what happens when the cache entry expires?

Suppose 1,000 users click the link almost simultaneously.

Every request checks Redis, and every request encounters a cache miss. Without additional protection, they all attempt to query PostgreSQL.

This creates unnecessary database load, consumes connections, and may increase response times.

The problem gets worse when the underlying operation involves complex database joins, external APIs, or expensive calculations.

Even a fast database query can become a problem when thousands of identical requests execute together.

How Does Single-Flight Request Coalescing Work?

Single-flight solves the problem by introducing coordination between concurrent requests.

When the first request discovers that the cache is empty, it becomes the leader.

Other requests looking for the same resource become followers.

The leader performs the expensive database lookup, while followers wait for the result.

Once the operation finishes, every request receives the same result.

Let’s follow the process:

1. Request A asks for short code abc123.
2. Redis returns a cache miss.
3. Request A begins a database lookup and becomes the leader.
4. Requests B, C, and D arrive while the lookup is running.
5. Instead of starting new queries, they wait for Request A’s operation.
6. Request A retrieves the original URL and updates Redis.
7. All waiting requests receive the lookup result.

Notice that this coordination is based on the resource key.

Requests for abc123 can share one lookup, while requests for xyz789 can execute independently.

This prevents a single slow resource from blocking unrelated requests.

How Do You Implement Single-Flight in Python?

Python doesn’t provide a dedicated standard-library singleflight implementation like Go’s golang.org/x/sync/singleflight package.

However, we can implement the same behavior using asyncio, shared tasks, and an in-memory dictionary.

The idea is straightforward: store ongoing operations by key, and allow concurrent callers to await the same task.

Step 1: Create a reusable single-flight class

import asyncio
from typing import Awaitable, Callable, TypeVar

T = TypeVar("T")


class SingleFlight:
    def __init__(self):
        self._tasks: dict[str, asyncio.Task] = {}
        self._lock = asyncio.Lock()

    async def do(
        self,
        key: str,
        operation: Callable[[], Awaitable[T]],
    ) -> T:
        async with self._lock:
            task = self._tasks.get(key)

            if task is None:
                task = asyncio.create_task(operation())
                self._tasks[key] = task

                def cleanup(finished_task, *, task_key=key):
                    if self._tasks.get(task_key) is finished_task:
                        self._tasks.pop(task_key, None)

                task.add_done_callback(cleanup)

        return await asyncio.shield(task)

Let’s understand what’s happening.

The _tasks dictionary tracks operations currently running. The resource key identifies which requests can share a result.

The asyncio.Lock ensures that two requests cannot simultaneously create separate tasks for the same key.

The shared task represents the database lookup or another expensive asynchronous operation.

asyncio.shield() prevents cancellation of an individual waiting request from automatically cancelling the shared operation.

After the task finishes, the cleanup callback removes it from the dictionary.

This allows future calls to create a new operation when necessary.

One important detail: this is an in-process implementation. All callers must use the same SingleFlight instance within the same Python event loop.

It also doesn’t impose an execution timeout itself. Production code should enforce appropriate timeouts inside the underlying operation.

Step 2: Test it with concurrent requests

Let’s simulate a database query that takes 200 milliseconds.

import asyncio

singleflight = SingleFlight()
database_queries = 0


async def get_from_database(short_code: str):
    global database_queries

    database_queries += 1

    await asyncio.sleep(0.2)

    return "https://example.com/blog"


async def resolve_url(short_code: str):
    return await singleflight.do(
        short_code,
        lambda: get_from_database(short_code),
    )


async def main():
    results = await asyncio.gather(
        *[
            resolve_url("abc123")
            for _ in range(1000)
        ]
    )

    print("Total requests:", len(results))
    print("Database queries:", database_queries)


asyncio.run(main())

Expected output:

Total requests: 1000
Database queries: 1

We launched 1,000 concurrent requests, but only one database operation executed.

All requests received the same URL.

The result depends on concurrency: if some requests arrive after the first operation completes, another operation may run unless a persistent cache supplies the result.

How Do You Combine Single-Flight with Redis?

Single-flight becomes especially valuable when combined with caching.

Redis reduces repeated queries across time and across application instances.

Single-flight reduces duplicate operations among requests that arrive together while a lookup is already running.

These two mechanisms solve related but different problems.

Python example using Redis and PostgreSQL

Assume we’re using redis.asyncio and asyncpg.

import asyncio
import asyncpg
import redis.asyncio as redis

singleflight = SingleFlight()


async def get_original_url(
    short_code: str,
    redis_client,
    db_pool,
):
    cache_key = f"url:{short_code}"

    # Step 1: Read from Redis
    cached_url = await redis_client.get(cache_key)

    if cached_url is not None:
        return cached_url

    # Step 2: Share concurrent cache misses
    async def load_from_database():

        # Check Redis again
        cached = await redis_client.get(cache_key)

        if cached is not None:
            return cached

        # Only one in-process leader performs this lookup
        async with db_pool.acquire() as conn:
            original_url = await conn.fetchval(
                """
                SELECT long_url
                FROM urls
                WHERE short_code = $1
                  AND is_active = TRUE
                  AND (
                      expires_at IS NULL
                      OR expires_at > NOW()
                  )
                """,
                short_code,
            )

        if original_url is None:
            return None

        # Populate Redis for future requests
        await redis_client.set(
            cache_key,
            original_url,
            ex=300,
        )

        return original_url

    return await singleflight.do(
        cache_key,
        load_from_database,
    )

This implementation follows the cache-aside pattern.

On a cache hit, the application returns the URL immediately.

On a cache miss, single-flight ensures that overlapping requests for the same short code share one database lookup within that process.

Once the result is cached, future requests can use Redis directly.

For a real production service, add handling for Redis outages, database timeouts, URL updates, cache invalidation, and negative caching for missing URLs.

Also, if a URL has an expiration time, the cache TTL must not extend beyond that expiration. Cached redirects should respect disable and update operations.

Does Single-Flight Work Across Multiple Servers?

This is one of the most important limitations.

Our Python implementation coordinates requests within a single process.

Imagine deploying five application servers behind a load balancer.

                 Load Balancer
                       |
          +------------+------------+
          |            |            |
       Server A     Server B     Server C
          |            |            |
     SingleFlight  SingleFlight  SingleFlight
          |            |            |
          +------------+------------+
                       |
                   PostgreSQL

Each server maintains its own in-memory dictionary of ongoing operations.

If 1,000 requests are distributed across five independent server processes, each process might perform one database lookup for the same key.

That means the database could receive five queries instead of one.

This is still a meaningful reduction compared with 1,000 queries.

But it isn’t globally coordinated request deduplication.

How can you prevent duplicate work across servers?

For some high-cost operations, you can coordinate through distributed locking using Redis.

A simplified strategy is:

1. The first server acquires a Redis lock for the requested key.
2. That server executes the database lookup.
3. Other servers observe that another process is already handling the key.
4. They briefly wait and retry the cache lookup.
5. Once the result appears in Redis, they use it.

Distributed locking introduces additional complexity, including expired locks, crashed leaders, stale ownership, and the possibility that work runs more than once.

For ordinary URL lookups, I would usually begin with process-local single-flight and shared Redis caching rather than distributed locking.

How Much Database Load Can Single-Flight Reduce?

Suppose our application receives 10 million requests per day.

Assume 99% are redirects, giving us 9.9 million redirect requests daily.

A Redis cache with a hypothetical 95% hit rate leaves 495,000 redirect requests that need another source of data.

Single-flight can reduce some of those cache-miss queries further when requests for the same key overlap.

For example:

These numbers represent ideal simultaneous in-flight batches within one coalescing domain. Real results depend on request timing, number of application processes, cache behavior, and operation duration.

The important lesson is that single-flight works best when many concurrent requests need the same resource.

If every request asks for a different resource, single-flight offers little benefit.

What Happens If the Leader Request Fails?

A shared operation introduces another design question: what should followers receive if the leader’s database query fails?

In most single-flight implementations, followers receive the same outcome as the shared operation, including its exception.

This prevents duplicate work, but it also means one failed operation can affect multiple callers.

There are several precautions worth taking.

Use timeouts. A shared database query should not run indefinitely.

Avoid caching transient errors as successful results. A temporary PostgreSQL failure should not become a cached URL-not-found response.

Handle cancellation carefully. A disconnected user shouldn’t necessarily cancel an operation that hundreds of other requests are awaiting.

Use bounded retries. Retrying a failed database query from every follower could create another traffic spike.

Monitor shared failures. If a single operation fails, it might affect many requests even though only one database query occurred.

Single-flight improves efficiency, but it doesn’t replace proper failure handling.

Single-Flight vs. Caching vs. Rate Limiting

These concepts are often confused because all three can reduce backend pressure.

However, they serve different purposes.

These techniques can complement one another.

For example, a URL shortener might use Redis for caching, single-flight for cache stampede protection, and rate limiting to control abusive traffic.

No single technique solves every scalability problem.

When Should You Use Single-Flight?

Single-flight is useful for operations that are expensive, repeatable, and frequently requested concurrently.

Common examples include:

• URL shorteners: Prevent duplicate database lookups when a popular URL’s cache entry expires.
• Product catalogs: Avoid fetching the same product details thousands of times during a sale.
• Authentication systems: Coordinate concurrent refreshes for the same expired access token, using the right token and session as the key.
• External API integrations: Reduce duplicate requests to expensive or rate-limited third-party APIs.
• Configuration services: Prevent large groups of application instances or requests from repeatedly loading the same configuration.

Single-flight is generally less useful when every request has unique data, when operations are extremely cheap, or when the operation has side effects that require separate execution.

You should also avoid blindly sharing user-specific responses. The coalescing key must include any relevant authorization or personalization context.

What Should You Monitor in Production?

Adding single-flight without measuring it makes it difficult to know whether it actually helps.

Useful metrics include:

Pay particular attention to follower wait time.

Single-flight can reduce database load significantly, but requests still need to wait for the shared operation to finish.

The goal is not to make every request instant. The goal is to avoid unnecessary duplicate work while keeping latency within an acceptable range.

Frequently Asked Questions

Is single-flight the same as request deduplication?

Single-flight is a specific form of request deduplication focused on operations that are happening concurrently.

General request deduplication can include identifying repeated requests across much longer periods.

Single-flight typically shares only an operation that is currently in progress.

Does single-flight replace Redis?

No. Redis caches completed results so future requests can reuse them.

Single-flight coordinates concurrent callers while an operation is running.

Using both together is often more effective.

Can single-flight improve API response time?

It can improve overall latency under contention by preventing excessive backend work.

However, followers still wait for the leader. Their latency depends on the shared operation’s duration.

Does single-flight guarantee that only one database query runs?

Only within the coordination scope, while the operation is in flight.

A local Python implementation doesn’t coordinate across independent processes or servers.

Requests arriving after the shared operation completes may trigger new queries if no usable cache entry exists.

Is single-flight useful in Django or FastAPI?

Yes. The concept applies to both frameworks.

The implementation depends on whether the application uses asynchronous or synchronous execution.

For asynchronous FastAPI endpoints or async Django views, an asyncio implementation can be appropriate. For synchronous workers, you need synchronization primitives suitable for threads or processes.

What is the difference between single-flight and a distributed lock?

Single-flight allows multiple callers to share one result.

A distributed lock coordinates exclusive ownership of work across separate processes or machines.

A lock alone doesn’t automatically share the computed result with waiting callers.

Final Thoughts

When a backend starts receiving heavy traffic, the immediate reaction is often to add more servers, increase database capacity, or introduce additional caching layers.

But sometimes the problem isn’t a lack of hardware.

It’s that the system is doing the same work repeatedly.

Single-flight request coalescing addresses exactly that problem.

By sharing concurrent operations, it can protect databases from cache stampedes, reduce redundant backend requests, and improve system stability during traffic spikes.

For a production URL shortener, I would begin with PostgreSQL as the primary database, Redis for caching, and process-local single-flight to coordinate overlapping cache misses.

Then I would measure how often requests are coalesced and whether additional distributed coordination is genuinely necessary.

Good scalability engineering isn’t always about processing more work. Sometimes it’s about avoiding work that never needed to happen.

System Design, Python, Asyncio, Redis, Caching, Cache Stampede, Request Coalescing, Backend Engineering, Distributed Systems, API Performance