# How to Scale Long-Running API Requests

> When a single API call takes minutes to finish, it punishes both your users and your server. Here's the progression I walk through to turn long-running endpoints into something responsive, scalable, and operable - and when to reach for managed cloud services instead.

Published: 2026-05-23. Author: Milan Jovanović.

Canonical: https://milanjovanovic.tech/blog/how-to-scale-long-running-api-requests

A long-running endpoint should accept the work instead of doing it.
Validate the input, persist a job row with a `Pending` status, and return `202 Accepted` with an ID the client can poll.
When the in-process worker starts competing with your API for CPU and connections, move it behind a queue so the two sides scale independently.

Every system I've worked on eventually grows an endpoint that takes minutes to finish (or longer).
A report that aggregates years of data. A bulk import. A workflow that fans out to three external services and a database before it can answer.

You end up with two problems at once.
Your users sit on a spinner for several minutes, and your API holds that request open the entire time - burning a thread, a connection, and a slot in your concurrency budget.
A small traffic spike on that one endpoint could potentially take the rest of the API down with it.

I want to walk through the progression I use to fix this.
It's the same path the diagram below traces, from "the request just blocks" to a fully decoupled,
queue-backed worker pool - the shape I usually call an [**async API**](https://milanjovanovic.tech/blog/building-async-apis-in-aspnetcore-the-right-way).

Depending on your requirements, you might stop at any point along the way - but I want to make sure you understand the full path and the trade-offs at each step.

![System design progression for scaling long-running API requests, from synchronous request to queue-based competing consumers.](https://milanjovanovic.tech/blogs/mnw_195/scaling_long_running_requests.png)

## Step 0: The Naive Version

A user sends a request. The application server does the work. The work takes five minutes. The connection stays open the whole time.

There is nothing _wrong_ with this approach - it's just paying for correctness with availability.
The user experience is bad, and the blast radius is large: every long request you accept is a request you _can't_ accept somewhere else.

The first realization you need to internalize is that **the response time and the work duration don't have to be the same thing**.

<div className="centered">
  ![Diagram of a blocking API request, where the user sends a request and waits for the work to finish before getting a response.](https://milanjovanovic.tech/blogs/mnw_195/blocking_api_request.png)
</div>

## Step 1: Accept the Work, Don't Do It

The first move is to stop doing the work inside the request.

I add a `jobs` table that represents the work I _intend_ to do. The API endpoint now does three things:

1. Validate the request.
2. Insert a row into `jobs` with status `Pending`.
3. Return `202 Accepted` with a job ID.

A [**background processor**](https://milanjovanovic.tech/blog/running-background-tasks-in-asp-net-core) running inside the same API picks up `Pending` rows and works through them.
The client either polls a `GET /jobs/{id}` endpoint or - better - I push updates via [**SignalR**](https://milanjovanovic.tech/blog/adding-real-time-functionality-to-dotnet-applications-with-signalr),
[**Server-Sent Events**](https://milanjovanovic.tech/blog/server-sent-events-in-aspnetcore-and-dotnet-10), or email when the job is done.

![Diagram of an API request that accepts work and returns 202, with a background processor that picks up pending jobs and processes them asynchronously.](https://milanjovanovic.tech/blogs/mnw_195/202_accepted_background_processor.png)

This already buys you a lot.
The endpoint returns in milliseconds, the user gets a job ID they can track, and a spike of incoming requests just becomes a spike of rows in a table.
That table is cheap to write to.

But there's a ceiling here, and it's easy to hit.

## Step 2: Decouple the Worker From the API

The background processor in Step 1 still lives inside your API process.
It competes for the same CPU, memory, and connection pool as your real endpoints.
If processing gets heavy or slow, your API starts feeling it - the very thing you were trying to avoid.

The fix is to pull the background processor out into its own deployable, and put a queue between the two.

The API now publishes a message to the queue (and optionally writes the job row for tracking).
A pool of background workers consumes from the queue and does the actual work - the same shape I covered in [**event-driven architecture with RabbitMQ**](https://milanjovanovic.tech/blog/event-driven-architecture-in-dotnet-with-rabbitmq).
This is the **competing consumers** pattern, and it gives you something the previous step couldn't: **independent scaling**.

![System design progression for scaling long-running API requests, from synchronous request to queue-based competing consumers.](https://milanjovanovic.tech/blogs/mnw_195/scaling_long_running_requests_final.png)

Three things change once you cross this line:

- **The queue absorbs spikes.** Your API can keep accepting work at a constant rate while the workers drain at their own pace.
- **You scale workers separately from the API.** More throughput on background jobs doesn't mean more API instances.
- **Failures become normal.** A worker crash is just a message that goes back on the queue, not a 500 to your user.

You also get retryability, pause/resume, structured error handling, and a **dead-letter queue** for poison messages - effectively for free, because the queue infrastructure already provides them.

## What This Costs You

I'd be lying if I said this was a free upgrade.

You're now running a queue, a worker fleet, and a notification path. That's more moving parts to deploy, monitor, and alert on.
Your "is this done yet?" semantics are no longer obvious from the HTTP response - the client has to ask, or you have to tell them.
And every job needs to be [**idempotent**](https://milanjovanovic.tech/blog/the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it), because at-least-once delivery means your workers _will_ see duplicates.

If you only have one slow endpoint and modest traffic, this is overkill.
A simple "fire-and-forget with status polling" inside the same process is fine.
Don't reach for a queue until the pain justifies it.

## When I'd Use a Cloud Service Instead

You don't always need to assemble this from parts. A few alternatives I would consider:

- [**AWS SQS**](https://milanjovanovic.tech/blog/complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit) **+ Lambda** or [**Azure Service Bus**](https://milanjovanovic.tech/blog/messaging-made-easy-with-azure-service-bus) **+ Azure Functions** when I want the worker pool to scale to zero and I don't want to manage hosts.
- **Azure Durable Functions** or **AWS Step Functions** when the work is a multi-step workflow with timers, retries, and human approvals. Orchestration is what they're good at.
- [**Temporal**](https://temporal.io/) when the workflow is long-lived (hours, days) and I need first-class durable execution, versioning, and visibility across runs.

The trade-off is the usual one: less operational work, more vendor coupling, and a pricing model you need to model carefully when throughput grows.

## Summary

The progression is simple, and it generalizes:

1. **Don't do slow work inside the request.** Accept it, persist it, return `202`.
2. **Don't run workers inside the API.** Put a queue in between and scale the two sides independently.
3. **Tell the user when it's done.** Polling is fine, push is better.

This isn't a microservices argument. It's a separation between _accepting work_ and _doing work_ - two concerns that have very different scaling profiles and very different failure modes.

If you want the full implementation walkthrough, the [**video version is here**](https://youtu.be/U40HzU_KkDY).

Thanks for reading.

And stay awesome!

---

## Frequently asked questions

### How do you handle long-running API requests?

Stop doing the work inside the request. Validate the request, persist a job with a Pending status, and return 202 Accepted with a job ID. A background processor picks up pending jobs, and the client polls or receives a push notification when the work is done.

### Why should an API return 202 Accepted instead of doing the work?

Holding a request open for minutes burns a thread, a connection, and a slot in your concurrency budget, so a traffic spike on one slow endpoint can take the whole API down. Returning 202 makes the endpoint respond in milliseconds while the work happens elsewhere.

### What is the competing consumers pattern?

The API publishes a message to a queue and a pool of background workers consumes from it. The queue absorbs traffic spikes, workers scale independently of the API, and a worker crash just puts the message back on the queue instead of returning a 500.

### How does a client know when a background job is finished?

Either the client polls a GET /jobs/{id} endpoint for the job status, or the server pushes updates through SignalR, Server-Sent Events, or an email when the job completes. Polling is fine, push is better.

### What are the downsides of queue-based background processing?

You now run a queue, a worker fleet, and a notification path, which means more parts to deploy and monitor. Every job must be idempotent because at-least-once delivery produces duplicates. For one slow endpoint with modest traffic, in-process polling is enough.

### When should you use a managed cloud service instead of your own workers?

Use SQS with Lambda or Azure Service Bus with Azure Functions when you want workers that scale to zero, Durable Functions or Step Functions for multi-step workflows with timers and approvals, and Temporal for long-lived workflows needing durable execution. The cost is more vendor coupling.