# Outbox Pattern in Postgres: Concurrent Workers vs Reading the WAL

> An API can save an order and crash before publishing the message that tells other services about it. The outbox pattern saves the message with the order so a background worker can publish it later. I compare concurrent workers draining the outbox table with a reader that follows the Postgres write-ahead log, on throughput, latency, and crashes, and explain why LISTEN/NOTIFY doesn't replace them.

Published: 2026-11-07. Author: Milan Jovanović.

Canonical: https://milanjovanovic.tech/blog/outbox-pattern-postgres-workers-vs-wal

The **outbox pattern** saves a message in the same database transaction as the data change it describes, then uses a background worker to send it.
Concurrent workers can drain the outbox table with `FOR UPDATE SKIP LOCKED`, but new messages wait for the next poll.
Reading the Postgres write-ahead log with logical replication delivers them in about a millisecond at low load, at the cost of a replication slot and a cleanup job.

If you build distributed systems, you probably heard about the **Outbox pattern**.

Your API saves an order, then crashes before it can send the message announcing that order.

The order is in the database, but other services waiting for that message never hear about it.
The [**outbox pattern**](https://milanjovanovic.tech/blog/implementing-the-outbox-pattern) handles this by saving the message in a separate table,
in the same **database transaction** as the order.

Either both are saved or neither is.
This is the A in [**A**CID](https://en.wikipedia.org/wiki/ACID).
Atomicity.

A background worker reads the saved messages and publishes them to a message broker for other services to receive.
If the API crashes after saving the order, the message is still there for the worker to send.
Querying the table repeatedly for pending messages is called **polling**.

Polling is a good starting point for an outbox processor.
It's decently fast and easy to implement.
But the delay between checks is a tradeoff.
Checking more often means more database queries, while waiting longer delays new messages.

I want to reduce that wait by having Postgres **tell me** when there's something to process, and see how that compares with the polling workers.

## Concurrent Workers Draining the Table

For this walkthrough, we'll use an order API on Postgres that saves each `OrderCreated` message in an `outbox_messages` table, with [**RabbitMQ**](https://www.rabbitmq.com/) as the broker.

I want to run several workers at once, and Postgres row locks stop two of them from grabbing the same message.

Each worker starts a transaction and claims a batch with this query, using [**Dapper**](https://milanjovanovic.tech/blog/dapper-dotnet-guide):

```sql {6}
SELECT id, type, content
FROM outbox_messages
WHERE processed_on_utc IS NULL
ORDER BY occurred_on_utc
LIMIT 1000
FOR UPDATE SKIP LOCKED
```

`FOR UPDATE` locks the rows this worker picked until its transaction ends.
`SKIP LOCKED` makes the next worker take other rows instead of waiting.
The locks live in Postgres, so these workers can run in one process or in several.

The worker keeps that transaction open while it publishes with [**RabbitMQ.Client**](https://www.nuget.org/packages/RabbitMQ.Client), then sets `processed_on_utc` and commits once RabbitMQ confirms.

When a worker runs out of rows, it sleeps for one second with `Task.Delay` and tries again.

![The API writes an order and its message in one transaction. Workers poll and publish to RabbitMQ.](https://milanjovanovic.tech/blogs/mnw_219/outbox_polling.png)

If a worker crashes before it commits, those rows are still unprocessed, and a worker picks them up again.
Some of those messages may have already reached RabbitMQ, so the same message can arrive twice.

That makes this **at-least-once delivery**, so your consumers must be [**idempotent**](https://milanjovanovic.tech/blog/idempotent-consumer-handling-duplicate-messages).

## Why Not Just LISTEN/NOTIFY?

Postgres has a built-in way to wake those workers.
With [**`LISTEN`/`NOTIFY`**](https://www.postgresql.org/docs/current/sql-notify.html), one connection runs `LISTEN outbox_new`, and Postgres tells it when another connection runs `NOTIFY outbox_new` and commits.

My first implementation sent `NOTIFY` from a database trigger on the outbox table.
It needs no API changes and worked well at 100 orders per second.
But with 100 concurrent users, throughput fell from 8,070 orders per second with polling to 256.

The trigger runs inside the order transaction.
Before that transaction commits, Postgres takes a lock and holds it until after the commit.
Every transaction that sends a notification takes the same lock.

A commit normally waits for a disk flush, and concurrent transactions can share one, which is called **group commit**.
With the lock, each order waits for the one in front of it to finish its own flush.

![Without NOTIFY, commits share one disk write. With it, each waits for the lock.](https://milanjovanovic.tech/blogs/mnw_219/notify_commit_lock.png)

Sending `NOTIFY` after the commit, from a separate connection, avoids that lock.
At 100 users the API handled 7,674 orders per second, about 5% below polling, and at 100 orders per second a message took 1.91 ms from commit to consumer.

But now I'm running a notifier, a listener, and a fallback poll for notifications sent while the listener is disconnected, all to wake the same workers.
Postgres already has a record of every new row, so let's read that instead.

## Read the Outbox From the WAL

Postgres writes every change to the **write-ahead log (WAL)**.
[**Logical replication**](https://www.postgresql.org/docs/current/logical-replication.html) streams committed changes to my app, so there's no polling and no wake-up to miss.

![The WAL reader follows the log, publishes to RabbitMQ, and confirms its position.](https://milanjovanovic.tech/blogs/mnw_219/wal_reader.png)

You need `wal_level=logical`, which takes a restart on Postgres 18.
Then create a publication and a **replication slot**:

```sql
CREATE PUBLICATION outbox_pub FOR TABLE outbox_messages WITH (publish = 'insert');
SELECT pg_create_logical_replication_slot('outbox_slot', 'pgoutput');
```

The slot remembers how far my reader has confirmed, so it can resume after a restart.

Let's read the stream with an [**Npgsql replication connection**](https://www.npgsql.org/doc/replication.html):

```csharp {7,15}
await using var connection = new LogicalReplicationConnection(connectionString);
await connection.Open(cancellationToken);

var slot = new PgOutputReplicationSlot("outbox_slot");
var options = new PgOutputReplicationOptions("outbox_pub", PgOutputProtocolVersion.V1);

await foreach (var message in connection.StartReplication(slot, options, cancellationToken))
{
    if (message is InsertMessage insert)
    {
        await PublishAsync(insert, cancellationToken);
    }
    else if (message is CommitMessage commit)
    {
        connection.SetReplicationStatus(commit.TransactionEndLsn);
    }
}
```

`PublishAsync` publishes to RabbitMQ and waits for the confirm.
I only confirm a position after that, when the `CommitMessage` arrives, so a crash replays messages instead of losing them.

This loop only suits low rates, because waiting on every message drained about 1,600 per second.
So the reader I measured publishes first and waits for confirms, in order, in a separate task.

While the reader is down, the slot keeps all the server's WAL on disk, so a long outage can fill it.
You can cap that with [**`max_slot_wal_keep_size`**](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE), but Postgres invalidates a slot past the cap.

This reader is a prototype, and I haven't tested it through a failover or on managed Postgres.

## Workers vs the WAL Reader

The main benchmark comes from my [**scaling article**](https://milanjovanovic.tech/blog/scaling-the-outbox-pattern): I seeded a backlog, started the processor, and counted what RabbitMQ had after one minute, in **messages per second (MPS)**.
To keep it fair, the WAL reader also marks rows as processed in batches, so it does the same `UPDATE` as the workers.
This ran on one machine, with Postgres 18 and RabbitMQ 4 in Docker:

| Setup | MPS after 1 minute | Average commit to consumer, 100 orders/s | Average commit to consumer, 100 users |
| --- | --- | --- | --- |
| 1 worker | 14,516 to 15,050 | 500 ms | 8.5 s |
| 5 workers | 22,216 to 24,133 | 419 ms | 54 to 85 ms |
| WAL reader | 27,962 to 34,930 | 0.83 ms | 8.6 ms |

One worker already moves about 15,000 messages per second, and five get to about 23,000.
The WAL reader leads at about 31,000, and it reached about 57,600 without the `UPDATE`.
Both move thousands of messages per second, so throughput alone isn't a reason to switch.

Latency is the real difference.
New messages wait for a worker's one-second sleep, so five workers average 419 ms at 100 orders per second, while the reader stays under a millisecond.
At 100 users, one worker fell 8.5 seconds behind, five workers averaged 54 to 85 ms, and the reader held 8.6 ms.

A crash goes the other way.
A hard kill partway through 200,000 rows lost nothing, but the workers sent 4,536 duplicates and the WAL reader 31,095, because it replays from its last confirmed position.

Workers also scale out, because you can add more in any process.
A slot has one reader, which keeps commit order, but I can't scale it out.

## Summary

Keep polling if the delay is acceptable for your application.
Add workers when you need more throughput, because the row locks already handle the concurrency.

Read the WAL if you need new messages right away and can run a replication slot and a cleanup job.
If you use `NOTIFY` instead, send it after the commit and keep a fallback poll.

A worker can still crash after publishing but before committing its processed markers, causing a message to be sent again.
Consumers must be [**idempotent**](https://milanjovanovic.tech/blog/idempotent-consumer-handling-duplicate-messages): processing the same message twice must not repeat its effects.

Thanks for reading.

And stay awesome!

---

## Frequently asked questions

### How do concurrent outbox workers avoid publishing the same message twice?

Each worker starts a transaction and claims a batch with FOR UPDATE SKIP LOCKED. FOR UPDATE locks the selected rows until the transaction ends, and SKIP LOCKED makes other workers take different rows instead of waiting. The locks live in Postgres, so workers can run in one process or in several. A crash before the commit releases the locks, so another worker publishes those rows again, which makes delivery at least once.

### Why not send NOTIFY from a trigger inside the transaction?

A transaction that sends a notification takes a lock in Postgres and holds it until its commit is done. So these transactions commit one at a time, and each waits for the previous one to finish its disk write. In my k6 runs with 100 users, that cut the API from about 8,070 to 256 orders per second. Sending NOTIFY after the commit, from a separate connection, avoids the lock and held 7,674 orders per second, but it adds a notifier, a listener, and a fallback poll.

### Is a Postgres notification durable?

No. Postgres only delivers a notification to connections that are listening at that moment, so a listener that is disconnected when the commit happens never sees it. Keep a fallback poll, for example every 5 seconds, so idle workers check for messages even if they miss a notification.

### What does polling the outbox cost when there is nothing to process?

With five workers and no delay, the claim query ran about 289,000 times a minute and kept the Postgres container at 118 percent CPU. A one-second delay after an empty batch brought that to 305 queries a minute. A logical replication reader runs no outbox queries while idle.

### How many outbox messages per second can workers and a WAL reader publish?

Over a one-minute drain of a backlog, one worker published about 15,000 messages per second and five workers about 23,000. A logical replication reader doing the same batched UPDATE published about 31,000, with two runs at 27,962 and 34,930, and about 57,600 without the UPDATE. These are results from one machine, with Postgres and RabbitMQ in Docker.

### When should an outbox read the Postgres write-ahead log instead of using polling workers?

Read the write-ahead log when you need new messages right away and you can run a replication slot. At 100 orders per second, the commit-to-consumer average was 0.83 ms with the reader and 419 ms with five workers polling after a one-second sleep. It needs wal_level=logical, a slot that keeps WAL on disk while the reader is down, and a cleanup job, because the reader does not mark rows as processed by itself. After a hard kill it sent more duplicates, 31,095 against 4,536 of 200,000 rows, and neither design lost a message.