Leader Election Patterns in .NET

A leader lease gives one .NET instance temporary ownership of a role such as scheduling work. Other instances can acquire the role when that lease expires. The application still needs to reject writes from a former leader that resumes after a pause; election alone cannot guarantee exactly-once execution.

For applications already using PostgreSQL, a lease table avoids running another coordination service. The database serializes lease updates, and the protected writes below validate ownership in the same database transaction.

When Is a Database Lease Enough?

Consensus protocols such as Raft keep replicated state consistent despite node failures under their stated assumptions. Using an existing database moves that responsibility to the database and its failover configuration. It does not remove the consistency requirement.

This example assumes all contenders write to the same authoritative PostgreSQL primary. Promoting an asynchronous replica can lose acknowledged lease updates and roll back fencing tokens, so choose replication and failover settings that satisfy your safety requirements. Redis failover has a similar concern; a single atomic command is not a guarantee across data loss.

If your application already runs on Kubernetes, its Lease API is another coordination option. If the actual requirement is distributing independent work items, competing consumers or a clustered scheduler may fit better than electing a continuous leader role.

The Lease Table

Each role has one persistent row. Run this schema against a local PostgreSQL database before starting the application:

CREATE TABLE leader_lease (
    role          text PRIMARY KEY,
    holder_id     text NOT NULL,
    fencing_token bigint NOT NULL,
    expires_at    timestamptz NOT NULL
);

CREATE TABLE jobs (
    id                 bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    due_at             timestamptz NOT NULL,
    state              text NOT NULL DEFAULT 'pending',
    completed_by_token bigint
);

INSERT INTO jobs (due_at) VALUES (clock_timestamp());

The example's job changes a database row to completed. It does not send an email or call a payment gateway, because a database transaction cannot make those external effects atomic. For those workflows, write a durable command to an outbox and make the receiving operation idempotent.

Acquiring and Renewing Leadership

An upsert acquires an absent or expired lease and renews a live lease owned by the caller:

INSERT INTO leader_lease AS l (role, holder_id, fencing_token, expires_at)
VALUES ('scheduler', @holder, 1, clock_timestamp() + @lease)
ON CONFLICT (role) DO UPDATE
SET holder_id = @holder,
    fencing_token = CASE
        WHEN l.holder_id = @holder AND l.expires_at > clock_timestamp()
            THEN l.fencing_token
        ELSE l.fencing_token + 1
    END,
    expires_at = clock_timestamp() + @lease
WHERE l.expires_at <= clock_timestamp() OR l.holder_id = @holder
RETURNING fencing_token;

An unexpired renewal keeps the token stable. A takeover increments it, including when the same holder reacquires after expiry. An attempt by another holder while the lease remains live returns no row.

Keep the row permanently: deleting it would let the next insert restart the token at 1. Use a new holder ID for each process start so a restarted process cannot inherit the previous process's identity.

All expiry comparisons use the database's clock_timestamp(). Unlike PostgreSQL's now(), which is fixed at transaction start, it reads the current wall-clock time; see PostgreSQL time functions. This removes application-host clock skew from the comparison, but database clock changes can still affect lease timing.

A .NET Background Worker

Create the web project and install the driver:

dotnet new web -n LeaderDemo
cd LeaderDemo
dotnet add package Npgsql

Register the data source and worker in Program.cs:

using Npgsql;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton(sp => NpgsqlDataSource.Create(
    builder.Configuration.GetConnectionString("Postgres")
        ?? throw new InvalidOperationException("Configure ConnectionStrings:Postgres.")));
builder.Services.AddHostedService<LeaderWorker>();

var app = builder.Build();
app.MapGet("/", () => "Leader worker is running");
app.Run();

Add LeaderWorker.cs:

using Npgsql;

public sealed class LeaderWorker(
    NpgsqlDataSource dataSource,
    ILogger<LeaderWorker> logger) : BackgroundService
{
    private readonly string _holder = Guid.NewGuid().ToString("N");

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(10));
        try
        {
            do
            {
                try
                {
                    var token = await TryAcquireOrRenewAsync(stoppingToken);
                    if (token is { } fence)
                    {
                        var completed = await CompleteDueJobsAsync(fence, stoppingToken);
                        logger.LogInformation("Token {Token}: completed {Count} jobs", fence, completed);
                    }
                }
                catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
                {
                    break;
                }
                catch (Exception ex)
                {
                    // Do no more leader work until a later acquisition succeeds.
                    logger.LogWarning(ex, "Lease or job operation failed");
                }
            }
            while (await timer.WaitForNextTickAsync(stoppingToken));
        }
        catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
        {
            // Normal host shutdown.
        }
    }

    public async Task<long?> TryAcquireOrRenewAsync(CancellationToken ct)
    {
        const string sql = """
            INSERT INTO leader_lease AS l (role, holder_id, fencing_token, expires_at)
            VALUES ('scheduler', @holder, 1, clock_timestamp() + @lease)
            ON CONFLICT (role) DO UPDATE
            SET holder_id = @holder,
                fencing_token = CASE
                    WHEN l.holder_id = @holder AND l.expires_at > clock_timestamp()
                        THEN l.fencing_token
                    ELSE l.fencing_token + 1
                END,
                expires_at = clock_timestamp() + @lease
            WHERE l.expires_at <= clock_timestamp() OR l.holder_id = @holder
            RETURNING fencing_token;
            """;

        await using var cmd = dataSource.CreateCommand(sql);
        cmd.CommandTimeout = 5;
        cmd.Parameters.AddWithValue("holder", _holder);
        cmd.Parameters.AddWithValue("lease", TimeSpan.FromSeconds(30));
        return await cmd.ExecuteScalarAsync(ct) is long token ? token : null;
    }

    public async Task<int> CompleteDueJobsAsync(long token, CancellationToken ct)
    {
        await using var connection = await dataSource.OpenConnectionAsync(ct);
        await using var transaction = await connection.BeginTransactionAsync(ct);

        // Hold the lease row lock through the protected database update.
        await using var guard = new NpgsqlCommand("""
            SELECT fencing_token FROM leader_lease
            WHERE role = 'scheduler' AND holder_id = @holder
              AND fencing_token = @token AND expires_at > clock_timestamp()
            FOR UPDATE;
            """, connection, transaction);
        guard.CommandTimeout = 5;
        guard.Parameters.AddWithValue("holder", _holder);
        guard.Parameters.AddWithValue("token", token);
        if (await guard.ExecuteScalarAsync(ct) is not long)
            return 0;

        await using var update = new NpgsqlCommand("""
            UPDATE jobs SET state = 'completed', completed_by_token = @token
            WHERE id IN (
                SELECT id FROM jobs
                WHERE state = 'pending' AND due_at <= clock_timestamp()
                ORDER BY due_at, id LIMIT 100
            );
            """, connection, transaction);
        update.CommandTimeout = 5;
        update.Parameters.AddWithValue("token", token);
        var count = await update.ExecuteNonQueryAsync(ct);
        await transaction.CommitAsync(ct);
        return count;
    }
}

Configure ConnectionStrings:Postgres, apply the schema, and start two copies of the application against that database. They compete for the scheduler role; a follower simply waits for the next tick. The protected update validates the holder, token, and expiry after acquisition, so a delayed acquisition response cannot authorize stale work.

A local .NET 10 check against PostgreSQL 18 raced two LeaderWorker instances, expired the lease, and attempted a write with the old holder and token:

PostgreSQL 18: one of two contenders wins; renewal retains token; takeover increments; stale writer rejected; expired same holder receives new token.

The 30-second lease and 10-second interval are example values. Failover takes the remaining lease time plus acquisition scheduling and database latency; a blocked database can delay it longer. This worker renews between short batches, so it is not suitable for a long-running external job without a separate renewal loop and cooperative cancellation.

How Do You Stop a Stale Leader From Writing?

A process can pause after checking a lease and resume after another instance has taken over. That overlap is not bounded by the lease duration: the old process might resume much later. A local IsLeader flag cannot prevent its writes.

In this example, PostgreSQL row locking keeps lease validation and job completion in one transaction. Once the guard locks the lease row, a new acquisition waits until the job update commits or rolls back. This serializes protected database writes with leadership changes, even if the lease expires during the short transaction.

Every writer of those protected rows must follow that protocol. Keep transactions short, configure server-side idle-transaction timeouts, and monitor blocking; a paused transaction can delay takeover until the database releases its locks.

For a separate downstream resource, use a fencing token that the resource stores and checks atomically. Reject tokens below the highest accepted token for the protected resource. That only rejects an old leader after the resource has observed a newer token; strict validation against the current lease requires stronger coordination, as the shared-database transaction provides here.

A conditional update against a nullable completed_by_token on an arbitrary job is not enough to protect the whole leader role. Nor can attaching a token make a payment API enforce fencing if that API has no such contract. Use idempotency keys and durable job identity for external effects that must not be duplicated.

Shutdown and Alternative Coordination

The sample lets the lease expire on shutdown. For quicker graceful handoff, first stop and await all leader work, then shorten the expiry conditionally without deleting the row:

UPDATE leader_lease
SET expires_at = clock_timestamp()
WHERE role = 'scheduler' AND holder_id = @holder AND fencing_token = @token;

If that update fails, expiry still permits a later takeover. Do not release before in-flight work has finished, and never release a lease belonging to a newer holder.

PostgreSQL advisory locks are another option for work tied to a dedicated connection. The lock is released when the database recognizes that the session ended, which can take time after a network failure. They do not issue fencing tokens by themselves.

For independent queue rows, FOR UPDATE SKIP LOCKED can distribute work without a continuously elected leader. For scheduling, see preventing duplicate scheduled jobs, and give each scheduled execution a durable unique identity. Neither a lease nor queue ownership alone guarantees exactly-once external effects.

Summary

A database lease can coordinate a leader role when the database provides the consistency the application needs. Preserve the token history, increment it after an expired lease is reacquired, and validate leadership at the protected write boundary.

Test two contenders, a paused former leader, an expired lease, and database failover. The application needs a defined outcome for each case before leader-only work is safe to publish.