Why the Outbox Pattern Solves the Dual-Write Problem

The Outbox pattern solves the dual-write problem by removing the second write: you save the business data and the event in the same database transaction, using an outbox table, and a background processor publishes the events afterwards. The database transaction makes both writes atomic, so a committed order can never lose its event.

Every system that writes to a database and publishes to a message broker has the same crack running through it. Most teams discover it in production, as a ghost order or a shipment that never happened. This is the dual-write problem, and the Outbox pattern closes it with nothing more exotic than a database transaction.

The Dual-Write Problem

Picture this. A customer places an order. Your service saves the order to the database, then publishes an OrderPlaced event so the shipping module can start fulfillment.

Simple enough. Until it isn't.

The database write succeeds. The message broker call fails. Now you have an order sitting in the database, but the shipping module never hears about it. The customer waits. Nobody ships anything. You have a ghost order.

This is the dual-write problem: any time your application writes to two separate systems (a database and a message broker) without a shared transaction, one write can succeed while the other fails, leaving a consistency gap. There are three ways it can bite you:

1. DB succeeds, broker fails. Data is persisted, but downstream systems never get the event. Orders go unfulfilled. Notifications never send. State drifts silently.
2. Broker succeeds, DB fails. Other services react to an event for data that was never saved. The shipping module tries to fulfill an order that doesn't exist.
3. Distributed transaction bottleneck. You wrap both writes in a two-phase commit. It works - until the broker goes slow, and suddenly every write in your system is blocked.

This is not a theoretical concern. It shows up in production under load, during network blips, and especially during deployments. The insidious part is that it doesn't fail loudly. It fails silently, producing inconsistent state that you only notice hours later when a customer complains.

Why Other Solutions Fall Short

The first instinct is usually to add a retry.

If publishing to the broker fails, just try again. But retries don't solve the fundamental problem. Between the DB commit and the successful publish, your process might crash. The message is lost. You'd need to scan the database for "unpublished" records - and now you're reinventing the Outbox pattern anyway.

The second instinct is distributed transactions. Two-phase commit (2PC) can coordinate the database and the broker so both commit or both roll back. In theory, it sounds perfect. In practice, it has serious drawbacks:

• Most message brokers don't support 2PC at all (RabbitMQ, Kafka).
• It adds significant latency to every write.
• A coordinator failure can leave participants in a blocked state.
• It creates tight coupling between your database and your messaging infrastructure.

There's a third approach you'll sometimes see: publish first, save second. The reasoning goes, "if the DB save fails, at least we can compensate." But this is strictly worse. You've now published an event for data that doesn't exist. Every downstream consumer processes garbage. Compensation logic adds enormous complexity and rarely covers every edge case.

None of these approaches solve the core issue. You're trying to make two independent systems behave atomically, and that's fighting physics.

How the Outbox Pattern Works

The key insight behind the Outbox pattern is deceptively simple: stop writing to two systems.

Instead of saving your business data to the database and then publishing an event to the broker, you save both the business data and the event to the same database, in the same transaction.

That's it. That's the whole trick.

You add an outbox table to your database. When you save an order, you also insert a row into the outbox table describing the event you want to publish. Both writes happen in a single database transaction.

A separate background process - the outbox processor - polls that table, picks up unprocessed messages, publishes them to the message broker, and marks them as processed.

The flow looks like this:

The order insert and the outbox insert commit together. The database guarantees that. Either both rows are committed or neither is. The dual-write problem vanishes because there is no dual write anymore - there's only one write target.

The publishing step is decoupled and asynchronous. It introduces a small delay, but it introduces something far more valuable: reliability.

The Guarantee

Why does this actually work? Because a single database transaction is atomic by definition. Your relational database has decades of battle-tested ACID guarantees. By placing the outbox message inside the same transaction as the business data, you're leveraging guarantees that already exist rather than trying to invent new ones across system boundaries.

If the transaction commits, both the order and the outbox message are persisted. The processor will eventually pick it up and publish. If the transaction rolls back, neither exists. No ghost orders. No phantom events.

There is a trade-off, though.

The outbox processor might publish a message and then crash before marking it as processed. On the next run, it publishes the same message again. This means the Outbox pattern provides at-least-once delivery, not exactly-once. Your consumers must be idempotent - they need to handle receiving the same event twice without producing duplicate side effects.

This is a well-understood trade-off, and idempotency is far easier to implement than distributed transaction coordination. The idempotent consumer pattern shows how to handle duplicate messages cleanly.

A Minimal Example

In .NET with EF Core, the Outbox pattern requires two things: a place to store outbox messages and a mechanism to capture domain events before they leave the transaction.

The outbox entity is straightforward:

public sealed class OutboxMessage
{
    public Guid Id { get; set; }
    public string Type { get; set; } = string.Empty;
    public string Content { get; set; } = string.Empty;
    public DateTime OccurredOnUtc { get; set; }
    public DateTime? ProcessedOnUtc { get; set; }
}

An EF Core interceptor captures domain events and converts them into outbox rows before the transaction commits. AggregateRoot here is the base class whose DomainEvents collection entities raise events into; the shared kernel article shows the full base type.

public sealed class InsertOutboxMessagesInterceptor : SaveChangesInterceptor
{
    public override ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            InsertOutboxMessages(eventData.Context);
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private static void InsertOutboxMessages(DbContext context)
    {
        var outboxMessages = context.ChangeTracker
            .Entries<AggregateRoot>()
            .SelectMany(entry =>
            {
                var events = entry.Entity.DomainEvents.ToList();
                entry.Entity.ClearDomainEvents();
                return events;
            })
            .Select(domainEvent => new OutboxMessage
            {
                Id = Guid.NewGuid(),
                Type = domainEvent.GetType().FullName!,
                Content = JsonSerializer.Serialize(domainEvent, domainEvent.GetType()),
                OccurredOnUtc = DateTime.UtcNow
            })
            .ToList();

        context.Set<OutboxMessage>().AddRange(outboxMessages);
    }
}

When SaveChangesAsync is called, this interceptor collects every domain event from modified aggregates, serializes them into JSON, and adds them to the same DbContext. They all commit together. No second system involved.

One detail worth calling out: the Type column stores the full type name. The processor has to resolve the CLR type to deserialize the JSON later, and a bare class name won't round-trip across assemblies.

Wire the interceptor into the module's DbContext registration:

services.AddSingleton<InsertOutboxMessagesInterceptor>();

services.AddDbContext<OrderingDbContext>((sp, options) =>
    options
        .UseNpgsql(config.GetConnectionString("Database"))
        .AddInterceptors(
            sp.GetRequiredService<InsertOutboxMessagesInterceptor>()));

The background processor that publishes these messages is a separate concern. The critical part - the part that solves the dual-write problem - is entirely in the code above.

For the full end-to-end implementation, including the background worker, see implementing the Outbox pattern. And if you ever need serious throughput, I've written about scaling the Outbox to 2 billion messages per day.

When You Need It (and When You Don't)

The Outbox pattern is the right choice when losing an event has business consequences.

Reach for it when:

• You need reliable event publishing after database writes
• Modules in a modular monolith need to communicate through events
• You're publishing events to external brokers in a microservices architecture
• Consistency between your data and your events is non-negotiable

Skip it when:

• The events are purely informational - analytics pings, debug logs, metrics. If losing a few is acceptable, the added complexity isn't worth it.
• Fire-and-forget notifications where the occasional miss is tolerable.
• Your database supports Change Data Capture (CDC) and you'd rather capture changes at the log level. CDC solves a similar problem from a different angle and avoids the outbox table entirely.

The pattern adds a table, a background processor, and the idempotency requirement on consumers. That's real complexity. But for any system where "the event must go out if the data was saved," there's no simpler reliable solution.

Summary

The dual-write problem is one of those issues that's easy to overlook and painful to debug after the fact. It doesn't throw exceptions. It produces subtle data inconsistencies that erode trust in your system over time.

The Outbox pattern solves it by reducing two writes to one. Your database transaction becomes the single source of truth for both business data and events. Everything else - the publishing, the delivery, the processing - flows from that one atomic commit.

It's a small architectural decision with outsized impact on system reliability.

Thanks for reading, and stay awesome!