
<Sponsorships>

It's been a big year for API collaborations!
The [**Postman VS Code extension**](https://fnf.dev/4gjsqA6) helped us integrate our workflows, new templates got us started when we were stuck,
and the Postman Vault kept our data secure.
[**Postman's December drop**](https://fnf.dev/4gjsqA6) has all the details on these and more.

[**Get 1 Dometrain Course for free**](https://dometrain.com/courses?ref=milan-jovanovic&promo=newsletter-20241221&coupon_code=MILAN01)!
Dometrain is a learning platform by Nick Chapsas.
With more than 265.000 enrollments, there is no better place to invest in your learning as a .NET engineer.
Use code MILAN01 to get 20% off any course, and add more than one item to your basket to get one for free!
[**Get your free course**](https://dometrain.com/courses?ref=milan-jovanovic&promo=newsletter-20241221&coupon_code=MILAN01)!

</Sponsorships>

Most ASP.NET Core applications need to handle [**background processing**](running-background-tasks-in-asp-net-core) -
from sending reminder emails to running cleanup tasks.
While there are many ways to implement background jobs, [Quartz.NET](https://www.quartz-scheduler.net/)
stands out with its robust scheduling capabilities, persistence options, and production-ready features.

In this article, we'll look at:

- Setting up Quartz.NET with ASP.NET Core and proper observability
- Implementing both on-demand and recurring jobs
- Configuring persistent storage with PostgreSQL
- Handling job data and monitoring execution

Let's start with the basic setup and build our way up to a production-ready configuration.

## Setting Up Quartz With ASP.NET Core

First, let's set up Quartz with proper instrumentation.

We'll need to install some NuGet packages:

```powershell
Install-Package Quartz.Extensions.Hosting
Install-Package Quartz.Serialization.Json

# This might be in prerelease
Install-Package OpenTelemetry.Instrumentation.Quartz
```

Next, we'll configure the Quartz services and OpenTelemetry instrumentation and start the scheduler:

```csharp {15}
builder.Services.AddQuartz();

// Add Quartz.NET as a hosted service
builder.Services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing =>
    {
        tracing
            .AddHttpClientInstrumentation()
            .AddAspNetCoreInstrumentation()
            .AddQuartzInstrumentation();
    })
    .UseOtlpExporter();
```

This is all we need at the start.

## Defining and Scheduling Jobs

To define a [**background job**](scheduling-background-jobs-with-quartz-net), you have to implement the `IJob` interface.
All job implementations run as scoped services, so you can inject dependencies as needed.
Quartz allows you to pass data to a job using the `JobDataMap` dictionary.
It's recommended to only use primitive types for job data to avoid serialization issues.

When executing the job, there are a few ways to fetch job data:

- `JobDataMap` - a dictionary of key-value pairs
  - `JobExecutionContext.JobDetail.JobDataMap` - job-specific data
  - `JobExecutionContext.Trigger.TriggerDataMap` - trigger-specific data
- `MergedJobDataMap` - combines job data with trigger data

It's a best practice to use `MergedJobDataMap` to retrieve job data.

```csharp
public class EmailReminderJob(ILogger<EmailReminderJob> logger, IEmailService emailService) : IJob
{
    public const string Name = nameof(EmailReminderJob);

    public async Task Execute(IJobExecutionContext context)
    {
        // Best practice: Prefer using MergedJobDataMap
        var data = context.MergedJobDataMap;

        // Get job data - note that this isn't strongly typed
        string? userId = data.GetString("userId");
        string? message = data.GetString("message");

        try
        {
            await emailService.SendReminderAsync(userId, message);

            logger.LogInformation("Sent reminder to user {UserId}: {Message}", userId, message);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, "Failed to send reminder to user {UserId}", userId);

            // Rethrow to let Quartz handle retry logic
            throw;
        }
    }
}
```

One thing to note: `JobDataMap` isn't strongly typed. This is a limitation we have to live with, but we can mitigate it by:

1. Using constants for key names
2. Validating data early in the `Execute` method
3. Creating wrapper services for job scheduling

Now, let's discuss scheduling jobs.

Here's how to schedule one-time reminders:

```csharp {9}
public record ScheduleReminderRequest(
    string UserId,
    string Message,
    DateTime ScheduleTime
);

// Schedule a one-time reminder
app.MapPost("/api/reminders/schedule", async (
    ISchedulerFactory schedulerFactory,
    ScheduleReminderRequest request) =>
{
    var scheduler = await schedulerFactory.GetScheduler();

    var jobData = new JobDataMap
    {
        { "userId", request.UserId },
        { "message", request.Message }
    };

    var job = JobBuilder.Create<EmailReminderJob>()
        .WithIdentity($"reminder-{Guid.NewGuid()}", "email-reminders")
        .SetJobData(jobData)
        .Build();

    var trigger = TriggerBuilder.Create()
        .WithIdentity($"trigger-{Guid.NewGuid()}", "email-reminders")
        .StartAt(request.ScheduleTime)
        .Build();

    await scheduler.ScheduleJob(job, trigger);

    return Results.Ok(new { scheduled = true, scheduledTime = request.ScheduleTime });
})
.WithName("ScheduleReminder")
.WithOpenApi();

```

The endpoint schedules one-time email reminders using Quartz.
It creates a job with user data, sets up a trigger for the specified time, and schedules them together.
The `EmailReminderJob` receives a unique identity in the `email-reminders` group.

Here's a sample request you can use to test this out:

```
POST /api/reminders/schedule
{
    "userId": "user123",
    "message": "Important meeting!",
    "scheduleTime": "2024-12-17T15:00:00"
}
```

## Scheduling Recurring Jobs

For recurring background jobs, you can use [cron schedules](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html):

```csharp {4,9,27}
public record RecurringReminderRequest(
    string UserId,
    string Message,
    string CronExpression
);

// Schedule a recurring reminder
app.MapPost("/api/reminders/schedule/recurring", async (
    ISchedulerFactory schedulerFactory,
    RecurringReminderRequest request) =>
{
    var scheduler = await schedulerFactory.GetScheduler();

    var jobData = new JobDataMap
    {
        { "userId", request.UserId },
        { "message", request.Message }
    };

    var job = JobBuilder.Create<EmailReminderJob>()
        .WithIdentity($"recurring-{Guid.NewGuid()}", "recurring-reminders")
        .SetJobData(jobData)
        .Build();

    var trigger = TriggerBuilder.Create()
        .WithIdentity($"recurring-trigger-{Guid.NewGuid()}", "recurring-reminders")
        .WithCronSchedule(request.CronExpression)
        .Build();

    await scheduler.ScheduleJob(job, trigger);

    return Results.Ok(new { scheduled = true, cronExpression = request.CronExpression });
})
.WithName("ScheduleRecurringReminder")
.WithOpenApi();
```

Cron triggers are more powerful than simple triggers.
They allow you to define complex schedules like "every weekday at 10 AM" or "every 15 minutes".
Quartz supports cron expressions with seconds, minutes, hours, days, months, and years.

Here's a sample request if you want to test this:

```
POST /api/reminders/schedule/recurring
{
    "userId": "user123",
    "message": "Daily standup",
    "cronExpression": "0 0 10 ? * MON-FRI"
}
```

## Job Persistence Setup

By default, Quartz uses in-memory storage, which means your jobs are lost when the application restarts.
For production environments, you'll want to use a persistent store.
Quartz supports several [database providers](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html),
including SQL Server, PostgreSQL, MySQL, and Oracle.

Let's look at how to set up persistent storage with proper schema isolation:

```csharp {11}
builder.Services.AddQuartz(options =>
{
    options.AddJob<EmailReminderJob>(c => c
        .StoreDurably()
        .WithIdentity(EmailReminderJob.Name));

    options.UsePersistentStore(persistenceOptions =>
    {
        persistenceOptions.UsePostgres(cfg =>
        {
            cfg.ConnectionString = connectionString;
            cfg.TablePrefix = "scheduler.qrtz_";
        },
        dataSourceName: "reminders"); // Database name

        persistenceOptions.UseNewtonsoftJsonSerializer();
        persistenceOptions.UseProperties = true;
    });
});
```

A few important things to note here:

- The `TablePrefix` setting helps organize Quartz tables in your database - in this case, placing them in a dedicated `scheduler` schema
- You'll need to run the appropriate database scripts to create these tables
- Each database provider has its own [setup scripts](https://github.com/quartznet/quartznet/tree/main/database/tables) -
  check the Quartz documentation for your chosen provider

### Durable Jobs

Notice how we're configuring the `EmailReminderJob` with `StoreDurably`?
This is a powerful pattern that lets you define your jobs once and reuse them with different triggers.
Here's how to schedule a stored job:

```csharp
public async Task ScheduleReminder(string userId, string message, DateTime scheduledTime)
{
    var scheduler = await _schedulerFactory.GetScheduler();

    // Reference the stored job by its identity
    var jobKey = new JobKey(EmailReminderJob.Name);

    var trigger = TriggerBuilder.Create()
        .ForJob(jobKey)  // Reference the durable job
        .WithIdentity($"trigger-{Guid.NewGuid()}")
        .UsingJobData("userId", userId)
        .UsingJobData("message", message)
        .StartAt(scheduledTime)
        .Build();

    await scheduler.ScheduleJob(trigger);  // Note: just passing the trigger
}
```

This approach has several benefits:

- Job definitions are centralized in your startup configuration
- You can't accidentally schedule a job that hasn't been properly configured
- Job configurations are consistent across all schedules

## Summary

Getting **Quartz** set up properly in .NET involves more than just adding the NuGet package.

Pay attention to:

1. Proper job definition and data handling with `JobDataMap`
2. Setting up both one-time and recurring job schedules
3. Configuring persistent storage with proper schema isolation
4. Using durable jobs to maintain consistent job definitions

Each of these elements contributes to a reliable background processing system that can grow with your application's needs.
A good example of using background jobs is when you want to [**build asynchronous APIs**](building-async-apis-in-aspnetcore-the-right-way).

Good luck out there, and I'll see you next week.

---



<Sponsorships>

[**Get 1 Dometrain Course for free**](https://dometrain.com/courses?ref=milan-jovanovic&promo=newsletter-20241214&coupon_code=MILAN01)!
Dometrain is a learning platform by Nick Chapsas.
With more than 265.000 enrollments, there is no better place to invest in your learning as a .NET engineer.
Use code MILAN01 to get 20% off any course, and add more than one item to your basket to get one for free!
[**Get your free course**](https://dometrain.com/courses?ref=milan-jovanovic&promo=newsletter-20241214&coupon_code=MILAN01)!

Three things to know about [**Depot**](https://fnf.dev/4gtFbHV):\
• Up to 3x faster: Depot's Ultra Runners make GitHub Actions jobs 3x faster than GitHub's default runners.\
• Ideal for AI workloads, npm install, git clone, and other heavy file operations.\
• **Get one month free on any plan** — [**Sign up**](https://fnf.dev/4gtFbHV) and use code **DEPOTFRIENDS24** at checkout — valid through December 20.

</Sponsorships>

Every article about modular monoliths tells you to use public APIs between modules.
But they rarely tell you why these APIs exist or how to design them properly.

A modular monolith organizes an application into independent modules that have clear boundaries.
The module boundaries are logical and group related business capabilities together.

After building several large-scale modular monoliths, I've learned that public APIs are not just about clean code - they're about controlling chaos.
Let me show you what I mean.

## The Reality of Module Communication

Here's what nobody tells you about public APIs in modular monoliths: they represent intentional coupling points.
Yes, you read that right.
Public APIs don't eliminate coupling - they make it explicit and controllable.

When Module A needs something from Module B, you have three options:

1. Let Module A read directly from Module B's database
2. Let Module A access Module B's internal services
3. Create a public API that explicitly defines what Module A can do

<Image
  src="blogs/mnw_120/module_communication_options.png"
  width={2189}
  height={1462}
  alt="Module communication options: direct database access, calling internal services, calling public API."
/>

The first two options lead to chaos.
I've seen entire systems become unmaintainable because every module was freely accessing the data and services of other modules.

The previous options are examples of synchronous communication between modules.
But you can also implement [**asynchonrous module communication**](modular-monolith-communication-patterns) using messaging.
We have to adjust the technical implementation.
However, modules still have a public API in message contracts.

## Why We Need Public APIs

Public APIs serve three critical purposes:

1. **Contract Definition**: They explicitly state what other modules can and cannot do
2. **Dependency Control**: They force you to think about module dependencies
3. **Change Management**: They provide a stable interface while allowing internal changes

Here's a practical example. Imagine you have an Orders module and a Shipping module.

This is what you want to avoid:

```csharp
public class ShippingService
{
    private readonly OrdersDbContext _ordersDb; // Direct database access

    public async Task ShipOrder(string orderId)
    {
        // Directly reading from another module's database
        var order = await _ordersDb.Orders
            .Include(o => o.Lines)
            .FirstOrDefaultAsync(o => o.Id == orderId);

        // What happens if the Orders module changes its schema?
        // What if it moves to a different database?
    }
}
```

This is what you want to achieve instead:

```csharp
public class ShippingService
{
    private readonly IOrdersModule _orders; // Public API access

    public async Task ShipOrder(string orderId)
    {
        // Using the public API
        var order = await _orders.GetOrderForShippingAsync(orderId);

        // The Orders module can change its internals
        // as long as it maintains this contract
    }
}
```

## Controlling What Gets Exposed

The hardest part of designing public APIs is deciding what to expose.
Here's my rule of thumb:

1. Start with nothing public
2. Expose only what other modules actually need
3. Design the API around use cases, not data

Here's how this looks in practice:

```csharp
public interface IOrdersModule
{
    // Don't expose generic CRUD operations
    // Task<Order> GetOrderAsync(string orderId); // Bad

    // Instead, expose specific use cases
    Task<OrderShippingInfo> GetOrderForShippingAsync(string orderId);
    Task<OrderPaymentInfo> GetOrderForPaymentAsync(string orderId);
    Task<OrderSummary> GetOrderForCustomerAsync(string orderId);
}
```

## Protecting Your Module's Data

Public APIs aren't enough.
You also need to [**protect your module's data**](modular-monolith-data-isolation).
Here's what I've found works:

1. **Separate Schemas**: Each module gets its own database schema.

```sql
CREATE SCHEMA Orders;
CREATE SCHEMA Shipping;

-- Orders module can only access its schema
CREATE USER OrdersUser WITH DEFAULT_SCHEMA = Orders;
GRANT SELECT, INSERT, UPDATE, DELETE ON SCHEMA::Orders TO OrdersUser;

-- Shipping module can only access its schema
CREATE USER ShippingUser WITH DEFAULT_SCHEMA = Shipping;
GRANT SELECT, INSERT, UPDATE, DELETE ON SCHEMA::Shipping TO ShippingUser;
```

We can also lock down the user's access to a given schema to only allow reading and writing data.

2. **Different Connection Strings**: Each module gets its own database user with a respective connection string.

```csharp
builder.Services.AddDbContext<OrdersDbContext>(options =>
    options.UseSqlServer(builder.Configuration.GetConnectionString("OrdersConnection")));

builder.Services.AddDbContext<ShippingDbContext>(options =>
    options.UseSqlServer(builder.Configuration.GetConnectionString("ShippingConnection")));
```

If you want to learn more about this, check out this article about [**using multiple EF Core DbContexts**](using-multiple-ef-core-dbcontext-in-single-application).

3. **Read Models**: Create specific read models for other modules.

```csharp
internal class Order
{
    // Internal domain model with full complexity
}

public class OrderShippingInfo
{
    // Public DTO with only what shipping needs
    public string OrderId { get; init; }
    public Address ShippingAddress { get; init; }
    public List<ShippingItem> Items { get; init; }
}
```

## Dealing with Cross-Cutting Concerns

Some features naturally span multiple modules.
For example, when a customer views their order history, you might need data from the Orders, Shipping, and Payments modules.

Don't try to force this through module APIs.
Instead:

1. Create a separate query model
2. Use event-driven patterns to keep it updated
3. Own it in a dedicated module or one of the existing modules

```csharp
public class OrderHistoryModule
{
    public async Task<CustomerOrderHistory> GetOrderHistoryAsync(string customerId)
    {
        // Read from a dedicated read model that's kept
        // updated through events from other modules
        return await _orderHistoryRepository.GetCustomerHistoryAsync(customerId);
    }
}
```

## Summary

Public APIs in modular monoliths are not about preventing coupling - they're about controlling it.
Every public API is a contract that says: "Yes, these modules are coupled, and this is exactly how they depend on each other."

The goal isn't to eliminate dependencies between modules.
The goal is to make them explicit, controlled, and maintainable.

Get this right, and your modular monolith will be easier to maintain, test, and evolve.
Get it wrong, and you'll end up with a distributed big ball of mud.

Want to master building modular monoliths with clean APIs and event-driven patterns?
Check out my [**Modular Monolith Architecture**](/modular-monolith-architecture) course, where I'll show you how to build maintainable systems
using practical examples from real projects.

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

APIs can be frustrating — think inconsistent standards, incomplete docs, and unpredictable errors in production.
[**Postman's latest article**](https://shortclick.link/i6muq1) breaks down common struggles with APIs and how better tools and workflows can help.
[**Read the article**](https://shortclick.link/i6muq1) to learn how to simplify your workflow and tackle common API challenges.

[**Share your development expertise, get rewarded!**](https://ironsoftware.com/partners/affiliate/?utm_source=milan&utm_medium=email&utm_campaign=milan)
Empower developers by sharing your experiences with Iron Software libraries.
Your tutorials can help others find powerful tools and you'll earn exclusive rewards for sharing them!
[**Join the Developer Knowledge Network**](https://ironsoftware.com/partners/affiliate/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

I remember the days when managing NuGet packages across multiple projects was a real pain.
You know what I mean - you open a large solution and find out every project uses a different version of the same package.
Not fun!

Let me show you how **Central Package Management** (CPM) in .NET can fix this problem once and for all.

## The Problem We Need to Solve

I often work with solutions that have lots of projects.
It's not uncommon to have solutions with 30 or more projects.
Each one needs similar packages like Serilog or Polly.
Most test projects I create depend on xUnit.
Before CPM, keeping track of package versions was a mess:

- One project uses Serilog `4.1.0`
- Another uses Serilog `4.0.2`
- And somehow, a third one uses Serilog `3.1.1`

This causes real problems.
Different versions can behave differently, leading to weird bugs that are hard to track down.
I've wasted many hours fixing issues caused by version mismatches.

## How Central Package Management Helps

Think of CPM as a control center for all your package versions.
Instead of setting versions in each project, you set them once in one place.
Then, you just reference a package you want to use without specifying the version.
It's that simple.

Here's what you need to use [Central Package Management](https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management):

- NuGet 6.2 or newer
- .NET SDK 6.0.300 or newer
- If you use Visual Studio, you need version 2022 17.2 or newer

## Setting It Up

Let me show you how to set up CPM.
It's easier than you might think.

1. First, create a file called `Directory.Packages.props` in your solution's main folder:

```xml
<Project>
  <PropertyGroup>
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
    <PackageVersion Include="Serilog" Version="4.1.0" />
    <PackageVersion Include="Polly" Version="8.5.0" />
  </ItemGroup>
</Project>
```

Note the use of `PackageVersion` to define NuGet dependencies.

2. In your project files, you can list the packages using `PackageReference` without the version component:

```xml
<ItemGroup>
  <PackageReference Include="Newtonsoft.Json" />
  <PackageReference Include="AutoMapper" />
  <PackageReference Include="Polly" />
</ItemGroup>
```

That's it!
Now all your projects will use the same package versions.

## Cool Things You Can Do

### Need a Different Version for One Project?

Sometimes you might need a specific project to use a different version.
No problem!
Just add this to your project file:

```xml
<PackageReference Include="Serilog" VersionOverride="3.1.1" />
```

The `VersionOverride` property lets you define the specific version you want to use.

### Want a Package in Every Project?

If you have packages that every project needs, you can make them global.
Define a `GlobalPackageReference` in your props file:

```xml
<ItemGroup>
  <GlobalPackageReference Include="SonarAnalyzer.CSharp" Version="10.3.0.106239" />
</ItemGroup>
```

Now every project gets this package automatically!

## Migrating Existing Projects to Central Package Management

1. Create the `Directory.Packages.props` file at the solution root
2. Move all package versions from your `.csproj` files
3. Remove version attributes from `PackageReference` elements
4. Build your solution and fix any version conflicts
5. Test thoroughly before committing

Here's a Powershell script that will list all NuGet package versions in your solution:

```powershell
# Scan all .csproj files and aggregate unique package versions
$packages = Get-ChildItem -Filter *.csproj -Recurse |
    Get-Content |
    Select-String -Pattern '<PackageReference Include="([^"]+)" Version="([^"]+)"' -AllMatches |
    ForEach-Object { $_.Matches } |
    Group-Object { $_.Groups[1].Value } |
    ForEach-Object { @{
        Name = $_.Name
        Versions = $_.Group.ForEach({ $_.Groups[2].Value }) | Select-Object -Unique
    }} |
    Sort-Object { $_.Name }

# Display results
$packages | ForEach-Object {
    "$($_.Name) versions:"
    $_.Versions | ForEach-Object { "  $_" }
}
```

There's also a CLI tool called [CentralisedPackageConverter](https://github.com/Webreaper/CentralisedPackageConverter),
which you can use to automate the migration.
It will scan for all .NET project files within that folder tree, gather all the versioned references in the projects,
remove the versions from the project files, and write the entries to the `Directory.Packages.props` file.

```bash
# Install the tool globally
dotnet tool install CentralisedPackageConverter --global

# Convert your solution to use Central Package Management
central-pkg-converter /PATH_TO_YOUR_SOLUTION_FOLDER
```

## When Should You Use CPM?

I don't see a compelling reason for not using this by default.

I recommend using CPM when:

- You have many projects that share packages
- You're tired of fixing version-related bugs
- You want to make sure everyone uses the same versions

I recently added CPM to a solution with 30 projects.

Here's what happened:

- Fewer merge conflicts
- Caught version problems early
- Made it easier for new team members

This was especially helpful while migrating from .NET 8 to .NET 9.

You can combine CPM with [**build configuration and static code analysis**](improving-code-quality-in-csharp-with-static-code-analysis).

## Wrapping Up

My tips for success with **Central Package Management**:

1. When you add CPM to an existing solution, do it in its own change/PR
2. If you override a version, add a comment explaining why
3. Check your package versions regularly for updates
4. Only make packages global if you really need them everywhere

Since I started using Central Package Management, managing NuGet packages has become much easier.
It's like having a single source of truth for all your package versions.

Hope this was helpful.
See you next week.

---



<Sponsorships>

[**Make the Switch: IronPDF vs Aspose**](https://ironpdf.com/competitors/aspose-vs-ironpdf/?utm_source=milan&utm_medium=email&utm_campaign=milan)!
Why overcomplicate your projects?
IronPDF offers premium features like PDF compliance, pixel-perfect rendering, and seamless integration at a fraction of Aspose's cost.
Enjoy one year of free support, straightforward pricing, and a simpler API.
[**Switch & save extra 33%**](https://ironpdf.com/competitors/aspose-vs-ironpdf/?utm_source=milan&utm_medium=email&utm_campaign=milan).

Is API development a straightforward process?
We wish!
No matter, though. [**Postman's November Drop is here to help you stay on track with**](https://shortclick.link/xvvfnr):\
• The Postman VS Code extension.\
• Response-sharing on Slack and Teams.\
• Auto-generating documentation.

</Sponsorships>

Long-running business processes often involve multiple services working together.
Think about an e-commerce order: you need to process the payment, update inventory, and notify shipping.
Traditional distributed transactions using two-phase commit (2PC) seem like a solution, but they come with significant drawbacks.

The main issue?
Services can't make assumptions about how other services operate or how long they'll take.
What if the payment service needs manual approval?
What if the inventory check is delayed?
Holding database locks across multiple services for extended periods isn't practical and can lead to system-wide issues.

Let's look at how the Saga pattern solves these problems and implement it using MassTransit.

## Understanding the Saga Pattern

A Saga is a sequence of related local transactions where each step has a defined action and a compensating action if something goes wrong.
Instead of one big atomic transaction, we break the process into manageable steps that can be coordinated.

Here's a simple order processing flow:

<div className="centered">
  <Image
    src="blogs/mnw_118/order_flow.png"
    width={2313 / 4}
    height={2628 / 4}
    alt="Sequence diagram showing an order processing flow with multiple services."
  />
</div>

Each step is independent and can be compensated if needed.
If the inventory service reports items are out of stock, we can refund the payment.
This approach gives us flexibility and reliability without tight coupling.

## Implementing Sagas with MassTransit

[**MassTransit**](using-masstransit-with-rabbitmq-and-azure-service-bus) provides a state machine-based approach to implementing Sagas through its integration with Automatonymous.
Understanding how state machines work is crucial for implementing effective sagas.

### State Machine Fundamentals

A [state machine](https://masstransit.io/documentation/patterns/saga/state-machine) consists of several key components:

1. **States**: Represent the possible conditions of your saga instance
2. **Events**: Messages that can trigger state transitions
3. **Behaviors**: Actions that occur when events are received in specific states
4. **Instance**: Contains the data and current state for a specific saga

Here's the state machine diagram for our order processing saga:

<div className="centered">
  <Image
    src="blogs/mnw_118/order_saga_state_machine.png"
    width={1633 / 3}
    height={1728 / 3}
    alt="State diagram showing an order saga state machine."
  />
</div>

Every state machine automatically includes `Initial` and `Final` states.
The `Initial` state is where new saga instances begin, and the `Final` state marks the end of a saga's lifecycle.

### Defining the Saga Instance

The saga instance holds the data for a specific process:

```csharp
public class OrderState : SagaStateMachineInstance
{
    public Guid CorrelationId { get; set; }
    public string CurrentState { get; set; }

    // Business data
    public decimal OrderTotal { get; set; }
    public string? PaymentIntentId { get; set; }
    public DateTime? OrderDate { get; set; }
    public string? CustomerEmail { get; set; }
}
```

The `CorrelationId` uniquely identifies the saga instance, while `CurrentState` tracks its current state.
Any additional properties store business data needed for the process.

### Defining Events

Events are messages that can trigger state transitions. They must be correlated to a specific saga instance:

```csharp
public record OrderSubmitted
{
    public Guid OrderId { get; init; }
    public decimal Total { get; init; }
    public string Email { get; init; }
}

public record PaymentProcessed
{
    public Guid OrderId { get; init; }
    public string PaymentIntentId { get; init; }
}

public record InventoryReserved
{
    public Guid OrderId { get; init; }
}

public record OrderFailed
{
    public Guid OrderId { get; init; }
    public string Reason { get; init; }
}
```

### Building the State Machine

Let's implement the order processing flow as a state machine:

```csharp
public class OrderStateMachine : MassTransitStateMachine<OrderState>
{
    public OrderStateMachine()
    {
        Event(() => OrderSubmitted, x => x.CorrelateById(m => m.Message.OrderId));
        Event(() => PaymentProcessed, x => x.CorrelateById(m => m.Message.OrderId));
        Event(() => InventoryReserved, x => x.CorrelateById(m => m.Message.OrderId));
        Event(() => OrderFailed, x => x.CorrelateById(m => m.Message.OrderId));

        InstanceState(x => x.CurrentState);

        Initially(
            When(OrderSubmitted)
                .Then(context =>
                {
                    context.Saga.OrderTotal = context.Message.Total;
                    context.Saga.CustomerEmail = context.Message.Email;
                    context.Saga.OrderDate = DateTime.UtcNow;
                })
                .PublishAsync(context => context.Init<ProcessPayment>(new
                {
                    OrderId = context.Saga.CorrelationId,
                    Amount = context.Saga.OrderTotal
                }))
                .TransitionTo(ProcessingPayment)
        );

        During(ProcessingPayment,
            When(PaymentProcessed)
                .PublishAsync(context => context.Init<ReserveInventory>(new
                {
                    OrderId = context.Saga.CorrelationId
                }))
                .TransitionTo(ReservingInventory),
            When(OrderFailed)
                .TransitionTo(Failed)
                .Finalize()
        );

        During(ReservingInventory,
            When(InventoryReserved)
                .PublishAsync(context => context.Init<OrderConfirmed>(new
                {
                    OrderId = context.Saga.CorrelationId
                }))
                .TransitionTo(Completed)
                .Finalize(),
            When(OrderFailed)
                .PublishAsync(context => context.Init<RefundPayment>(new
                {
                    OrderId = context.Saga.CorrelationId,
                    Amount = context.Saga.OrderTotal
                }))
                .TransitionTo(Failed)
                .Finalize()
        );

        SetCompletedWhenFinalized();
    }

    public State ProcessingPayment { get; private set; }
    public State ReservingInventory { get; private set; }
    public State Completed { get; private set; }
    public State Failed { get; private set; }

    public Event<OrderSubmitted> OrderSubmitted { get; private set; }
    public Event<PaymentProcessed> PaymentProcessed { get; private set; }
    public Event<InventoryReserved> InventoryReserved { get; private set; }
    public Event<OrderFailed> OrderFailed { get; private set; }
}
```

The state machine defines the possible states and transitions.
Each step can trigger compensating actions if needed.
For example, if inventory reservation fails, we automatically trigger a payment refund.

### Implementing Message Consumers

Services interact with the saga by consuming and publishing messages.
Here's an example of a payment processing consumer:

```csharp
public class ProcessPaymentConsumer(
    IPaymentService paymentService,
    ILogger<ProcessPaymentConsumer> logger) : IConsumer<ProcessPayment>
{
    public async Task Consume(ConsumeContext<ProcessPayment> context)
    {
        try
        {
            var paymentResult = await paymentService.ProcessPaymentAsync(
                context.Message.OrderId,
                context.Message.Amount
            );

            if (paymentResult.Succeeded)
            {
                await context.Publish<PaymentProcessed>(new
                {
                    OrderId = context.Message.OrderId,
                    PaymentIntentId = paymentResult.PaymentIntentId
                });
            }
            else
            {
                await context.Publish<OrderFailed>(new
                {
                    OrderId = context.Message.OrderId,
                    Reason = paymentResult.FailureReason
                });
            }
        }
        catch (Exception ex)
        {
            logger.LogError(
                ex,
                "Failed to process payment for order {OrderId}",
                context.Message.OrderId);

            await context.Publish<OrderFailed>(new
            {
                OrderId = context.Message.OrderId,
                Reason = "Payment processing error"
            });
        }
    }
}
```

Each consumer handles a specific part of the business process and communicates back to the saga through events.
This separation of concerns allows each service to focus on its specific responsibility while the saga coordinates the overall process.

## Configuring MassTransit with PostgreSQL Persistence

To persist the saga state, we'll use PostgreSQL.
First, let's install the required packages:

```powershell
Install-Package MassTransit.EntityFrameworkCore
Install-Package Npgsql.EntityFrameworkCore.PostgreSQL
```

Create a `DbContext` for saga persistence:

```csharp
public class OrderSagaDbContext : SagaDbContext
{
    public OrderSagaDbContext(DbContextOptions options) : base(options)
    {
    }

    protected override IEnumerable<ISagaClassMap> Configurations
    {
        get
        {
            yield return new OrderStateMap();
        }
    }
}

public class OrderStateMap : SagaClassMap<OrderState>
{
    protected override void Configure(EntityTypeBuilder<OrderState> entity, ModelBuilder model)
    {
        entity.Property(x => x.CurrentState).HasMaxLength(64);
        entity.Property(x => x.CustomerEmail).HasMaxLength(256);
        entity.Property(x => x.PaymentIntentId).HasMaxLength(64);
    }
}
```

Configure MassTransit in your application:

```csharp
builder.Services.AddDbContext<OrderSagaDbContext>(options =>
    options.UseNpgsql(builder.Configuration.GetConnectionString("Postgres")));

builder.Services.AddMassTransit(x =>
{
    x.AddSagaStateMachine<OrderStateMachine, OrderState>()
        .EntityFrameworkRepository(r =>
        {
            r.ConcurrencyMode = ConcurrencyMode.Pessimistic;
            r.AddDbContext<DbContext, OrderSagaDbContext>();
            r.UsePostgres();
        });

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.Host(builder.Configuration.GetConnectionString("RabbitMQ"));
        cfg.ConfigureEndpoints(context);
    });
});
```

## Benefits of This Approach

Using sagas with MassTransit provides several advantages:

1. **Fault Tolerance**: Each step can be retried independently.
   And we can compensate failed operation, making our system more resilient.
2. **State Visibility**: The saga's state machine provides clear insight into where each process stands,
   making debugging and monitoring straightforward.
3. **Loose Coupling**: Services communicate through messages, allowing them to evolve independently while maintaining process integrity.
4. **Maintainability**: Changes can be made to individual steps without affecting others.

The state machine approach also makes the business process explicit.
Each state and transition is clearly defined, making it easier to understand and maintain the workflow.

## Takeaway

The Saga pattern with MassTransit provides a robust solution for managing distributed business processes.
Instead of dealing with distributed transactions, you get clear state management, automatic compensation for failures,
and the ability to handle long-running operations without blocking resources.

Want to dive deeper into event-driven architecture and distributed systems?
Check out my [**Modular Monolith Architecture**](/modular-monolith-architecture) course,
where we explore sagas, event-driven patterns, and other essential techniques for building maintainable systems.

Good luck out there, and see you next week.

**P.S.** You can find the code for the saga pattern implementation with MassTransit in [this repository](https://github.com/m-jovanovic/saga-pattern-masstransit).

---



<Sponsorships>

Even the smartest AI in the world won't save you from a productivity doom spiral if you don't get your APIs in order.
Don't spiral.
Do this instead — [**Check out Postman's November Drop to learn how the Postman VS Code extension can help**](https://shortclick.link/xvvfnr).
Avoid switching back and forth between tools.
Share responses on Slack and Teams.
Auto-generate your documentation.

[**Multiplayer**](https://dub.sh/e1TYNOD) auto-documents your system, from the high-level logical architecture down to the individual components, APIs, and dependencies.
Perfect for teams who want to speed up their workflows and consolidate their technical assets.
[**Try it for free here**](https://dub.sh/e1TYNOD).

</Sponsorships>

Most APIs follow a simple pattern.
The client sends a request.
The server does some work.
The server sends back a response.

This works well for fast operations like fetching data or simple updates.
But what about operations that take longer?

Think about processing large files, generating reports, or converting videos.
These operations can take minutes or even hours.

Making clients wait for these operations causes problems.

## Understanding Async APIs

The key to handling long-running operations is to change how we think about API responses.
An async API splits work into two parts:

- Accept the request
- Process it later

First, we accept the request and return a tracking ID immediately.
This gives users a quick response.
Then, we process the actual work in the background, which won't block other requests.
Users can check the status of their request using the tracking ID whenever they want.

This is different from `async`/`await` in C#.
That's about handling many requests at once (concurrently).
This is about handling long-running tasks better.
We're not just making the code asynchronous - we're making the entire operation asynchronous from the user's perspective.

## The Problem with Sync APIs

Let's see this in practice with image processing. A typical image upload API might look like this:

```csharp
[HttpPost]
public async Task<IActionResult> UploadImage(IFormFile file)
{
    if (file is null)
    {
        return BadRequest();
    }

    // Save original image
    var originalPath = await SaveOriginalAsync(file);

    // Generate thumbnails
    var thumbnails = await GenerateThumbnailsAsync(originalPath);

    // Optimize all images
    await OptimizeImagesAsync(originalPath, thumbnails);

    return Ok(new { originalPath, thumbnails });
}
```

The client must wait while we save the file, generate thumbnails, and optimize images.
On a slow connection or with a large file, this request could time out.
The server is also stuck processing one image at a time.

<Image
  src="blogs/mnw_117/sync_api_request.png"
  width={2460}
  height={1632}
  alt="Sequence diagram showing a synchronous API request."
/>

## A Better Way: Async Processing

Let's fix these problems. We'll split the work into two parts:

1. Accept the upload and return quickly
2. Do the heavy work in the background

<Image
  src="blogs/mnw_117/async_api_request.png"
  width={3275}
  height={2715}
  alt="Sequence diagram showing an asynchronous API request."
/>

### Uploading Images

Here's the new upload endpoint:

```csharp
[HttpPost]
public async Task<IActionResult> UploadImage(IFormFile? file)
{
    if (file is null)
    {
        return BadRequest("No file uploaded.");
    }

    if (!imageService.IsValidImage(file))
    {
        return BadRequest("Invalid image file.");
    }

    // Phase 1: Accept the work
    var id = Guid.NewGuid().ToString();
    var folderPath = Path.Combine(_uploadDirectory, "images", id);
    var fileName = $"{id}{Path.GetExtension(file.FileName)}";
    var originalPath = await imageService.SaveOriginalImageAsync(
        file,
        folderPath,
        fileName
    );

    // Queue Phase 2 for background processing
    var job = new ImageProcessingJob(id, originalPath, folderPath);
    await jobQueue.EnqueueAsync(job);

    // Return status URL immediately
    var statusUrl = GetStatusUrl(id);
    return Accepted(statusUrl, new { id, status = "queued" });
}
```

This new version only saves the original file during the HTTP request.
The heavy work moves to a background process.
The client immediately gets a status URL in the `Location` header instead of waiting.

### Checking Progress

Clients can check their image's status using a separate endpoint:

```csharp
[HttpGet("{id}/status")]
public IActionResult GetStatus(string id)
{
    if (!statusTracker.TryGetStatus(id, out var status))
    {
        return NotFound();
    }

    var response = new
    {
        id,
        status,
        links = new Dictionary<string, string>()
    };

    if (status == "completed")
    {
        response.links = new Dictionary<string, string>
        {
            ["original"] = GetImageUrl(id),
            ["thumbnail"] = GetThumbnailUrl(id, width: 200),
            ["preview"] = GetThumbnailUrl(id, width: 800)
        };
    }

    return Ok(response);
}
```

### Processing Images in Background

The real work happens in the background processor.
While the API handles new requests, a separate process works through the queued jobs.
This separation gives us flexibility in how we handle the processing.

For single-server deployments, we can use .NET's [**Channel**](lightweight-in-memory-message-bus-using-dotnet-channels) type to queue jobs in memory:

```csharp
public class JobQueue
{
    private readonly Channel<ImageProcessingJob> _channel;

    public JobQueue()
    {
        var options = new BoundedChannelOptions(1000)
        {
            FullMode = BoundedChannelFullMode.Wait
        };
        _channel = Channel.CreateBounded<ImageProcessingJob>(options);
    }

    public async ValueTask EnqueueAsync(ImageProcessingJob job,
        CancellationToken ct = default)
    {
        await _channel.Writer.WriteAsync(job, ct);
    }

    public IAsyncEnumerable<ImageProcessingJob> DequeueAsync(
        CancellationToken ct = default)
    {
        return _channel.Reader.ReadAllAsync(ct);
    }
}
```

For multi-server setups, we need a distributed queue like RabbitMQ or even Redis.

The background processor handles the time-consuming work:

```csharp
public class ImageProcessor : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        await foreach (var job in jobQueue.DequeueAsync(ct))
        {
            try
            {
                await statusTracker.SetStatusAsync(
                    job.Id,
                    "processing"
                );

                // Generate thumbnails
                await GenerateThumbnailsAsync(
                    job.OriginalPath,
                    job.OutputPath
                );

                // Optimize images
                await OptimizeImagesAsync(
                    job.OriginalPath,
                    job.OutputPath
                );

                await statusTracker.SetStatusAsync(
                    job.Id,
                    "completed"
                );
            }
            catch (Exception ex)
            {
                await statusTracker.SetStatusAsync(
                    job.Id,
                    "failed"
                );

                logger.LogError(ex, "Failed to process image {Id}", job.Id);
            }
        }
    }
}
```

The background processor needs to handle failures gracefully.
We can improve [**resilience**](building-resilient-cloud-applications-with-dotnet) by adding a retry policy with Polly.
Status updates keep users informed throughout the process.
Instead of just "processing", we tell them exactly what's happening.
This improves the user experience and helps with debugging.

## Beyond Polling: Real-Time Updates

Our status endpoint works, but it puts the burden on clients.
They must repeatedly check for updates, leading to unnecessary server load.
A client polling every second creates 60 requests per minute, yet most of these requests return the same status.

We can flip this model around.
Instead of clients asking for updates, the server can push updates when they happen.
This creates a more efficient and responsive system.

<Image
  src="blogs/mnw_117/async_api_request_with_push.png"
  width={3732}
  height={2964}
  alt="Sequence diagram showing an asynchronous API request with server push for status updates."
/>

[**SignalR and WebSockets**](adding-real-time-functionality-to-dotnet-applications-with-signalr) enable real-time communication between server and client.
When a job's status changes, the server immediately notifies interested clients.
This approach reduces network traffic and gives users instant feedback.

For longer-running jobs, email notifications make more sense.
Users don't need to keep their browsers open.
They can close the tab and come back when notified.
This works well for reports that take hours to generate or batch processes that run overnight.

Webhooks offer another option, especially for system-to-system communication.
When a job completes, your server can notify other systems.
This enables workflow automation and system integration without constant polling.

## Summary

Processing tasks asynchronously creates better experiences for everyone.
Users get immediate responses instead of watching spinning loading indicators.
They can start other tasks while waiting, and they'll know if something goes wrong.

The benefits extend beyond user experience.
Servers can handle more requests because they're not tied up with long-running tasks.
[**Background processors**](scheduling-background-jobs-with-quartz-net) can retry failed operations without affecting the main application.
You can even scale your processing separately from your web servers.

[**Error handling**](global-error-handling-in-aspnetcore-8) improves too.
When a long operation fails halfway through, you can save the progress and try again.
Users know exactly what's happening because they can check the status.
The system stays stable because one slow operation can't bring down your entire API.

That's all for today. Hope this was helpful.

---



<Sponsorships>

🎉 [**JetBrains Rider**](https://www.jetbrains.com/rider/?utm_campaign=rider_free&utm_content=site&utm_medium=cpc&utm_source=milan_jovanovic_newsletter)
is now free for non-commercial use.
Great news for all hobbyists, students, content creators, and open source contributors!
Now you can use Rider, a cross-platform .NET and game dev IDE, for non-commercial development for free.
[**Download and start today**](https://www.jetbrains.com/rider/?utm_campaign=rider_free&utm_content=site&utm_medium=cpc&utm_source=milan_jovanovic_newsletter)!

Remember when APIs were an afterthought?
We're [**not going back**](https://shortclick.link/djx5wv).
74% of devs surveyed in the 2024 State of the API Report are API-first, up from 66% in 2023.
What else? We're getting faster. 63% of devs can produce an API within a week, up from 47% in 2023.
Collaboration on APIs is still a challenge.
APIs are increasingly revenue generators.
Read the [**rest of the report**](https://shortclick.link/djx5wv) for fresh insights.

</Sponsorships>

[**Caching**](caching-in-aspnetcore-improving-application-performance) is essential for building fast, scalable applications.
ASP.NET Core has traditionally offered two caching options: in-memory caching and distributed caching.
Each has its trade-offs.
In-memory caching using `IMemoryCache` is fast but limited to a single server.
Distributed caching with `IDistributedCache` works across multiple servers using a backplane.

.NET 9 introduces `HybridCache`, a new library that combines the best of both approaches.
It prevents common caching problems like cache stampede.
It also adds useful features like tag-based invalidation and better performance monitoring.

In this week's issue, I'll show you how to use `HybridCache` in your applications.

## What is HybridCache?

The traditional caching options in ASP.NET Core have limitations.
In-memory caching is fast but limited to one server.
Distributed caching works across servers but is slower.

[HybridCache](https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid) combines both approaches and adds important features:

- Two-level caching (L1/L2)
  - L1: Fast in-memory cache
  - L2: Distributed cache (Redis, SQL Server, etc.)
- Protection against [cache stampede](https://en.wikipedia.org/wiki/Cache_stampede) (when many requests hit an empty cache at once)
- Tag-based cache invalidation
- Configurable serialization
- Metrics and monitoring

The L1 cache runs in your application's memory.
The L2 cache can be Redis, SQL Server, or any other distributed cache.
You can use HybridCache with just the L1 cache if you don't need distributed caching.

## Installing HybridCache

Install the `Microsoft.Extensions.Caching.Hybrid` NuGet package:

```powershell
Install-Package Microsoft.Extensions.Caching.Hybrid
```

Add `HybridCache` to your services:

```csharp
builder.Services.AddHybridCache(options =>
{
    // Maximum size of cached items
    options.MaximumPayloadBytes = 1024 * 1024 * 10; // 10MB
    options.MaximumKeyLength = 512;

    // Default timeouts
    options.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(30),
        LocalCacheExpiration = TimeSpan.FromMinutes(30)
    };
});
```

For custom types, you can add your own serializer:

```csharp
builder.Services.AddHybridCache()
    .AddSerializer<CustomType, CustomSerializer>();
```

## Using HybridCache

`HybridCache` provides several methods to work with cached data.
The most important ones are `GetOrCreateAsync`, `SetAsync`, and various remove methods.
Let's see how to use each one in real-world scenarios.

### Getting or Creating Cache Entries

The `GetOrCreateAsync` method is your main tool for working with cached data.
It handles both cache hits and misses automatically.
If the data isn't in the cache, it calls your factory method to get the data, caches it, and returns it.

Here's an endpoint that gets product details:

```csharp
app.MapGet("/products/{id}", async (
    int id,
    HybridCache cache,
    ProductDbContext db,
    CancellationToken ct) =>
{
    var product = await cache.GetOrCreateAsync(
        $"product-{id}",
        async token =>
        {
            return await db.Products
                .Include(p => p.Category)
                .FirstOrDefaultAsync(p => p.Id == id, token);
        },
        cancellationToken: ct
    );

    return product is null ? Results.NotFound() : Results.Ok(product);
});
```

In this example:

- The cache key is unique per product
- If the product is in the cache, it's returned immediately
- If not, the factory method runs to get the data
- Other concurrent requests for the same product wait for the first one to finish

### Setting Cache Entries Directly

Sometimes you need to update the cache directly, like after modifying data.
The `SetAsync` method handles this:

```csharp
app.MapPut("/products/{id}", async (int id, Product product, HybridCache cache) =>
{
    // First update the database
    await UpdateProductInDatabase(product);

    // Then update the cache with custom expiration
    var options = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromHours(1),
        LocalCacheExpiration = TimeSpan.FromMinutes(30)
    };

    await cache.SetAsync(
        $"product-{id}",
        product,
        options
    );

    return Results.NoContent();
});
```

Key points about `SetAsync`:

- It updates both L1 and L2 cache
- You can specify different timeouts for L1 and L2
- It overwrites any existing value for the same key

### Using Cache Tags

Tags are powerful for managing groups of related cache entries.
You can invalidate multiple entries at once using tags:

```csharp {7,18,31}
app.MapGet("/categories/{id}/products", async (
    int id,
    HybridCache cache,
    ProductDbContext db,
    CancellationToken ct) =>
{
    var tags = [$"category-{id}", "products"];

    var products = await cache.GetOrCreateAsync(
        $"products-by-category-{id}",
        async token =>
        {
            return await db.Products
                .Where(p => p.CategoryId == id)
                .Include(p => p.Category)
                .ToListAsync(token);
        },
        tags: tags,
        cancellationToken: ct
    );

    return Results.Ok(products);
});

// Endpoint to invalidate all products in a category
app.MapPost("/categories/{id}/invalidate", async (
    int id,
    HybridCache cache,
    CancellationToken ct) =>
{
    await cache.RemoveByTagAsync($"category-{id}", ct);

    return Results.NoContent();
});
```

Tags are useful for:

- Invalidating all products in a category
- Clearing all cached data for a specific user
- Refreshing all related data when something changes

### Removing Single Entries

For direct cache invalidation of specific items, use `RemoveAsync`:

```csharp
app.MapDelete("/products/{id}", async (int id, HybridCache cache) =>
{
    // First delete from database
    await DeleteProductFromDatabase(id);

    // Then remove from cache
    await cache.RemoveAsync($"product-{id}");

    return Results.NoContent();
});
```

`RemoveAsync`:

- Removes the item from both L1 and L2 cache
- Works immediately, no delay
- Does nothing if the key doesn't exist
- Is safe to call multiple times

Remember that `HybridCache` handles all the complexity of distributed caching, serialization, and stampede protection for you.
You just need to focus on your cache keys and when to invalidate the cache.

## Adding Redis as L2 Cache

To use [Redis](https://redis.io/) as your distributed cache:

1. Install the `Microsoft.Extensions.Caching.StackExchangeRedis` NuGet package:

```powershell
Install-Package Microsoft.Extensions.Caching.StackExchangeRedis
```

2. Configure Redis and `HybridCache`:

```csharp
// Add Redis
builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = "your-redis-connection-string";
});

// Add HybridCache - it will automatically use Redis as L2
builder.Services.AddHybridCache();
```

`HybridCache` will automatically detect and use Redis as the L2 cache.

## Summary

`HybridCache` simplifies caching in .NET applications.
It combines fast in-memory caching with distributed caching, prevents common problems like cache stampede,
and works well in both single-server and distributed systems.

Start with the default settings and basic usage patterns - the library is designed to be simple to use while solving complex caching problems.

Thanks for reading.

And stay awesome!

---



<Sponsorships>

You know that feeling when you have an idea in your head but can't think of the word for it?
Or you have documents in MongoDB Atlas but not the exact text to search them?
[**Vector search**](https://shortclick.link/1oo137) might be the answer.
Adding MongoDB Atlas Vector Search is pretty [**straightforward**](https://shortclick.link/1oo137), and can come in handy.

Auto-Generate [**enterprise-grade C# SDKs**](https://www.speakeasy.com/docs/sdk-design/csharp/oss-comparison-csharp?utm_source=net-weekly)
from OpenAPI specs.
Create customizable, strongly-typed C# SDKs with Speakeasy's SDK generator - designed for enterprise APIs where open-source solutions fall short.
Compare our SDK generator to popular OSS alternatives and [**create your first SDK for free**](https://www.speakeasy.com/docs/sdk-design/csharp/oss-comparison-csharp?utm_source=net-weekly).

</Sponsorships>

Functional programming patterns can feel academic and abstract.
Terms like "monads" and "functors" scare many developers away.
But beneath the intimidating terminology are practical patterns that can make your code safer and more maintainable.

C# has embraced many functional programming features over the years.

- Records for immutability
- LINQ for functional transformations
- Lambda expressions for first-class functions

These features aren't just syntax sugar - they help prevent bugs and make code easier to reason about.

Let's look at five practical patterns you can use in your C# projects today.

## Higher-Order Functions

Higher-order functions can take other functions as parameters or return them as results.
They let you write code that's more flexible and composable because you can pass behavior around like data.

Common examples of higher-order functions are LINQ's `Where` and `Select`, which take functions to transform data.

Let's refactor this validation example with higher-order functions:

```csharp
public class OrderValidator
{
    public bool ValidateOrder(Order order)
    {
        if (order.Items.Count == 0) return false;
        if (order.TotalAmount <= 0) return false;
        if (order.ShippingAddress == null) return false;
        return true;
    }
}

// What if we need:
// - different validation rules for different countries?
// - to reuse some validations but not others?
// - to combine validations differently?
```

Here's how higher-order functions make this more flexible:

```csharp
public static class OrderValidation
{
    public static Func<Order, bool> CreateValidator(string countryCode, decimal minimumOrderValue)
    {
        var baseValidations = CombineValidations(
            o => o.Items.Count > 0,
            o => o.TotalAmount >= minimumOrderValue,
            o => o.ShippingAddress != null
        );

        return countryCode switch
        {
            "US" => CombineValidations(
                baseValidations,
                order => IsValidUSAddress(order.ShippingAddress)),
            "EU" => CombineValidations(
                baseValidations,
                order => IsValidVATNumber(order.VatNumber)),
            _ => baseValidations
        };
    }

    private static Func<Order, bool> CombineValidations(params Func<Order, bool>[] validations) =>
        order => validations.All(v => v(order));
}

// Usage
var usValidator = OrderValidation.CreateValidator("US", minimumOrderValue: 25.0m);
var euValidator = OrderValidation.CreateValidator("EU", minimumOrderValue: 30.0m);
```

The higher-order function approach makes validators composable, testable, and easy to extend.
Each validation rule is a simple function that we can compose.

## Errors as Values

Error handling in C# often looks like this:

```csharp
public class UserService
{
    public User CreateUser(string email, string password)
    {
        if (string.IsNullOrEmpty(email))
        {
            throw new ArgumentException("Email is required");
        }

        if (password.Length < 8)
        {
            throw new ArgumentException("Password too short");
        }

        if (_userRepository.EmailExists(email))
        {
            throw new DuplicateEmailException(email);
        }

        // Create user...
    }
}
```

The problem?

- [Exceptions are expensive](https://youtu.be/E3dU9Y1CsnI)
- Callers often forget to handle exceptions
- The method signature lies - it claims to return a User but might throw

We can make errors explicit using the [OneOf](https://github.com/mcintyre321/OneOf) library.
It provides discriminated unions for C#, using a custom type `OneOf<T0, ... Tn>`.

```csharp
public class UserService
{
    public OneOf<User, ValidationError, DuplicateEmailError> CreateUser(string email, string password)
    {
        if (string.IsNullOrEmpty(email))
        {
            return new ValidationError("Email is required");
        }

        if (password.Length < 8)
        {
            return new ValidationError("Password too short");
        }

        if (_userRepository.EmailExists(email))
        {
            return new DuplicateEmailError(email);
        }

        return new User(email, password);
    }
}
```

By making the errors explicit:

- The method signature tells the whole truth
- Callers must handle all possible outcomes
- No performance overhead from exceptions
- The flow is easier to follow

Here's how you use it:

```csharp
var result = userService.CreateUser(email, password);

result.Switch(
    user => SendWelcomeEmail(user),
    validationError => HandleError(validationError),
    duplicateError => HandleError(duplicateError)
);
```

## Monadic Binding

A **monad** is a container for values - like `List<T>`, `IEnumerable<T>`, or `Task<T>`.
What makes it special is that you can chain operations on the contained values without dealing with the container directly.
This chaining is called monadic binding.

You use monadic binding daily with LINQ, but you might not know it.
It's what allows us to chain operations that transform data.

Map (`Select`) transforms values:

```csharp
// Simple transformations with Select (Map)
var numbers = new[] { 1, 2, 3, 4 };

var doubled = numbers.Select(x => x * 2);
```

Bind (`SelectMany`) transforms and flattens:

```csharp
// Operations that return multiple values use SelectMany (Bind)
var folders = new[] { "docs", "photos" };

var files = folders.SelectMany(folder => Directory.GetFiles(folder));
```

A popular example of applying monads in practice is the [**Result pattern**](functional-error-handling-in-dotnet-with-the-result-pattern),
which provides a clean way to chain operations that might fail.

## Pure Functions

Pure functions are predictable: they depend only on their inputs and don't change anything in the system.
No database calls, no API requests, no global state.
This constraint makes them easier to understand, test, and debug.

```csharp
// Impure - relies on hidden state
public class PriceCalculator
{
    private decimal _taxRate;
    private List<Discount> _activeDiscounts;

    public decimal CalculatePrice(Order order)
    {
        var price = order.Items.Sum(i => i.Price);

        foreach (var discount in _activeDiscounts)
        {
            price -= discount.Calculate(price);
        }

        return price * (1 + _taxRate);
    }
}
```

Here's the same example as a pure function:

```csharp
// Pure - everything is explicit
public static class PriceCalculator
{
    public static decimal CalculatePrice(
        Order order,
        decimal taxRate,
        IReadOnlyList<Discount> discounts)
    {
        var basePrice = order.Items.Sum(i => i.Price);

        var afterDiscounts = discounts.Aggregate(
            basePrice,
            (price, discount) => price - discount.Calculate(price));

        return afterDiscounts * (1 + taxRate);
    }
}
```

Pure functions are thread-safe, easy to test, and simple to reason about because all dependencies are explicit.

## Immutability

Immutable objects can't be changed after creation.
Instead, they create new instances for every change.
This simple constraint eliminates entire categories of bugs: race conditions, accidental modifications, and inconsistent state.

Here's an example of a mutable type:

```csharp
public class Order
{
    public List<OrderItem> Items { get; set; }
    public decimal Total { get; set; }
    public OrderStatus Status { get; set; }

    public void AddItem(OrderItem item)
    {
        Items.Add(item);
        Total += item.Price;
        // Bug: Thread safety issues
        // Bug: Can modify shipped orders
        // Bug: Total might not match Items
    }
}
```

Let's make this an immutable type:

```csharp
public record Order
{
    public ImmutableList<OrderItem> Items { get; init; }
    public OrderStatus Status { get; init; }
    public decimal Total => Items.Sum(x => x.Price);

    public Order AddItem(OrderItem item)
    {
        if (Status != OrderStatus.Created)
        {
            throw new InvalidOperationException("Can't modify shipped orders");
        }

        return this with
        {
            Items = Items.Add(item)
        };
    }
}
```

The immutable version:

- Is thread-safe by default
- Makes invalid states impossible
- Keeps data and calculations consistent
- Makes changes explicit and traceable

## Takeaway

[**Functional programming**](how-to-apply-functional-programming-in-csharp) isn't just about writing "cleaner" code.
These patterns fundamentally change how you handle complexity:

- **Push errors to compile time** - Catch problems before running the code
- **Make invalid states impossible** - Don't rely on documentation or conventions
- **Make the happy path obvious** - When everything is explicit, the flow is clear

You can adopt these patterns gradually.
Start with one class, one module, one feature.
The goal isn't to write purely functional code.
The goal is to write code that's safer, more predictable, and easier to maintain.

Hope this was helpful.
See you next week.

---



<Sponsorships>

Remember when APIs were an afterthought?
We're [**not going back**](https://shortclick.link/djx5wv).
74% of devs surveyed in the 2024 State of the API Report are API-first, up from 66% in 2023.
What else? We're getting faster. 63% of devs can produce an API within a week, up from 47% in 2023.
Collaboration on APIs is still a challenge.
APIs are increasingly revenue generators.
Read the [**rest of the report**](https://shortclick.link/djx5wv) for fresh insights.

New Report: [**State of Designer-Developer Collaboration 2024**](https://www.telerik.com/design-system/designer-developer-collaboration-survey-2024/?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=dt_survey_design_story_dev_collab_report).
Whether you're on the development or design end of creating web applications, it's easy to lose track of the world beyond your daily work.
This brand-new report will give you some interesting insights and in-depth analysis on the relationship between designers and developers,
including excellent ideas for making it more efficient and satisfying.
[**Check it out**](https://www.telerik.com/design-system/designer-developer-collaboration-survey-2024/?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=dt_survey_design_story_dev_collab_report).

</Sponsorships>

I see the same mistake happen over and over again.

Developers discover Clean Architecture, get excited about its principles, and then... they turn the famous Clean Architecture diagram into a project structure.

But here's the thing: **Clean Architecture is not about folders**.
It's about dependencies.

Simon Brown wrote a "missing chapter" for Uncle Bob's Clean Architecture book that addresses exactly this issue.
Yet somehow, this crucial message got lost along the way.

Today, I'll show you what Uncle Bob's Clean Architecture diagram really means and how you should actually organize your code.
We'll look at practical examples that you can use in your projects right now.

Let's clear up this common misconception once and for all.

## The Problem With Traditional Layering

Almost every .NET developer has built a solution that looks like this:

- `MyApp.Web` for controllers and views
- `MyApp.Business` for services and business logic
- `MyApp.Data` for repositories and data access

It's the default approach. It's what we see in tutorials. It's what we teach juniors.

And it's completely wrong.

### Why Layer-Based Organization Fails

When you organize code by technical layers, you scatter related components across multiple projects.
A single feature, like managing policies, ends up spread across your entire codebase:

- Policies controller in the Web layer
- Policy service in the Business layer
- Policy repository in the Data layer

Here's what you'll see when looking at the folder structure:

```text
📁 MyApp.Web
|__ 📁 Controllers
    |__ #️⃣ PoliciesController.cs
📁 MyApp.Business
|__ 📁 Services
    |__ #️⃣ PolicyService.cs
📁 MyApp.Data
|__ 📁 Repositories
    |__ #️⃣ PolicyRepository.cs
```

Here's a visual representation of the layer-based architecture:

<div className="centered">
  <Image
    src="blogs/mnw_114/layered_architecture.png"
    width={490 * 0.5}
    height={1247 * 0.5}
    alt="Feature Scattering in Layer-Based Architecture."
  />
</div>

This fragmentation creates several problems:

1. **Violates Common Closure Principle** - Classes that change together should stay together.
   When your "Policies" feature changes, you're touching three different projects.

2. **Hidden dependencies** - Public interfaces everywhere make it possible to bypass layers.
   Nothing stops a controller from directly accessing a repository.

3. **No business intent** - Opening your solution tells you nothing about what the application does.
   It only shows technical implementation details.

4. **Harder maintenance** - Making changes requires jumping between multiple projects.

The worst part? This approach doesn't even achieve what it promises.
Despite the separate projects, you often end up with a "big ball of mud" because public access modifiers allow any class to reference any other class.

### The Real Intent of Layers

Clean Architecture's circles were never meant to represent projects or folders.
They represent different levels of policy, with dependencies pointing inward toward business rules.

You can achieve this without splitting your code into artificial technical layers.

Let me show you a better way.

## Better Approaches to Code Organization

Instead of splitting your code by technical layers, you have two better options: **package by feature** or **package by component**.

Let's look at both.

### Package by Feature

Organizing by feature is a solid option.
Each feature gets its own namespace and contains everything needed to implement that feature.

```text
📁 MyApp.Policies
|__ 📁 RenewPolicy
    |__ #️⃣ RenewPolicyCommand.cs
    |__ #️⃣ RenewPolicyHandler.cs
    |__ #️⃣ PolicyValidator.cs
    |__ #️⃣ PolicyRepository.cs
|__ 📁 ViewPolicyHistory
    |__ #️⃣ PolicyHistoryQuery.cs
    |__ #️⃣ PolicyHistoryHandler.cs
    |__ #️⃣ PolicyHistoryViewModel.cs
```

Here's a diagram representing this structure:

<div className="centered">
  <Image
    src="blogs/mnw_114/feature_folder_architecture.png"
    width={490 * 0.5}
    height={1247 * 0.5}
    alt="Vertical slice architecture for package by feature organization."
  />
</div>

This approach:

- Makes features explicit
- Keeps related code together
- Simplifies navigation
- Makes it easier to maintain and modify features

If you want to learn more, check out my article about [**vertical slice architecture**](vertical-slice-architecture).

### Package by Component

A component is a cohesive group of related functionality with a well-defined interface.
Component-based organization is more coarse-grained than feature folders.
Think of it as a mini application that handles one specific business capability.

This is very similar to how I define modules in a [**modular monolith**](what-is-a-modular-monolith).

Here's what a component-based organization looks like:

```text
📁 MyApp.Web
|__ 📁 Controllers
    |__ #️⃣ PoliciesController.cs
📁 MyApp.Policies
|__ #️⃣ PoliciesComponent.cs     // Public interface
|__ #️⃣ PolicyService.cs         // Implementation detail
|__ #️⃣ PolicyRepository.cs      // Implementation detail
```

The key difference? Only `PoliciesComponent` is public.
Everything else is internal to the component.

<div className="centered">
  <Image
    src="blogs/mnw_114/component_architecture.png"
    width={491 * 0.5}
    height={1247 * 0.5}
    alt="Feature Scattering in Layer-Based Architecture."
  />
</div>

This means:

- No bypassing layers
- Clear dependencies
- Real encapsulation
- Business intent visible in the structure

### Which One Should You Choose?

Choose **Package by Feature** when:

- You have many small, independent features
- Your features don't share much code
- You want maximum flexibility

Choose **Package by Component** when:

- You have clear business capabilities
- You want strong encapsulation
- You might split into microservices later

Both approaches achieve what Clean Architecture really wants: proper dependency management and business focus.

Here's a side-by-side comparison of these architectural approaches:

<figure>
  <div className="centered">
    <Image
      src="blogs/mnw_114/architecture_comparison.png"
      width={1934}
      height={1247}
      alt="Comparison between layered, vertical slice and component architectural approaches."
    />
  </div>
  <figcaption>
    Greyed-out types are internal to the defining assembly.
  </figcaption>
</figure>

In the Missing Chapter of Clean Architecture, Simon Brown argues strongly for package by component.
The key insight is that components are the natural way to slice a system.
They represent complete business capabilities, not just technical features.

My recommendation?
Start with package by component.
Within the component, organize around features.

## Practical Examples

Let's transform a typical layered application into a clean, component-based structure.
We'll use an insurance policy system as an example.

### The Traditional Way

Here's how most developers structure their solution:

```csharp
// MyApp.Data
public interface IPolicyRepository
{
    Task<Policy> GetByIdAsync(string policyNumber);
    Task SaveAsync(Policy policy);
}

// MyApp.Business
public class PolicyService : IPolicyService
{
    private readonly IPolicyRepository _repository;

    public PolicyService(IPolicyRepository repository)
    {
        _repository = repository;
    }

    public async Task RenewPolicyAsync(string policyNumber)
    {
        var policy = await _repository.GetByIdAsync(policyNumber);
        // Business logic here
        await _repository.SaveAsync(policy);
    }
}

// MyApp.Web
public class PoliciesController : ControllerBase
{
    private readonly IPolicyService _policyService;

    public PoliciesController(IPolicyService policyService)
    {
        _policyService = policyService;
    }

    [HttpPost("renew/{policyNumber}")]
    public async Task<IActionResult> RenewPolicy(string policyNumber)
    {
        await _policyService.RenewPolicyAsync(policyNumber);
        return Ok();
    }
}
```

The problem?
Everything is public.
Any class can bypass the service and go straight to the repository.

### The Clean Way

Here's the same functionality organized as a proper component:

```csharp
// The only public contract
public interface IPoliciesComponent
{
    Task RenewPolicyAsync(string policyNumber);
}

// Everything below is internal to the component
internal class PoliciesComponent : IPoliciesComponent
{
    private readonly IRenewPolicyHandler _renewPolicyHandler;

    // Public constructor for DI
    public PoliciesComponent(IRenewPolicyHandler renewPolicyHandler)
    {
        _renewPolicyHandler = renewPolicyHandler;
    }

    public async Task RenewPolicyAsync(string policyNumber)
    {
        await _renewPolicyHandler.HandleAsync(policyNumber);
    }
}

internal interface IRenewPolicyHandler
{
    Task HandleAsync(string policyNumber);
}

internal class RenewPolicyHandler : IRenewPolicyHandler
{
    private readonly IPolicyRepository _repository;

    internal RenewPolicyHandler(IPolicyRepository repository)
    {
        _repository = repository;
    }

    public async Task HandleAsync(string policyNumber)
    {
        var policy = await _repository.GetByIdAsync(policyNumber);
        // Business logic for policy renewal here
        await _repository.SaveAsync(policy);
    }
}

internal interface IPolicyRepository
{
    Task<Policy> GetByIdAsync(string policyNumber);
    Task SaveAsync(Policy policy);
}
```

The key improvements are:

1. **Single public interface** - Only `IPoliciesComponent` is public. Everything else is internal.

2. **Protected dependencies** - No way to bypass the component and access the repository directly.

3. **Clear dependencies** - All dependencies flow inward through the component.

4. **Proper encapsulation** - Implementation details are truly hidden.

This is how you would register the services with dependency injection:

```csharp
services.AddScoped<IPoliciesComponent, PoliciesComponent>();
services.AddScoped<IRenewPolicyHandler, RenewPolicyHandler>();
services.AddScoped<IPolicyRepository, SqlPolicyRepository>();
```

This structure enforces Clean Architecture principles through compiler-checked boundaries, not just conventions.

The compiler won't let you bypass the component's public interface.
That's much stronger than hoping developers follow the rules.

## Best Practices and Limitations

Let's discuss something that is often overlooked: the practical limitations of enforcing Clean Architecture in .NET.

### The Limits of Encapsulation

The `internal` keyword in .NET provides protection within a single assembly.
Here's what that means in practice:

```csharp
// In a single project:
public interface IPoliciesComponent { } // Public contract
internal class PoliciesComponent : IPoliciesComponent { }
internal class PolicyRepository { }

// Someone could still do this:
public class BadPoliciesComponent : IPoliciesComponent
{
    public BadPoliciesComponent()
    {
        // Nothing stops them from creating a bad implementation
    }
}
```

While `internal` helps, it doesn't prevent all architectural violations.

### The Trade-offs

Some teams split their code into separate assemblies for stronger encapsulation:

```plaintext
MyCompany.Policies.Core.dll
MyCompany.Policies.Infrastructure.dll
MyCompany.Policies.Api.dll
```

This comes with trade-offs:

1. **More complex build process** - Multiple projects need to be compiled and referenced.
2. **Harder navigation** - Jumping between assemblies in the IDE is slower.
3. **Deployment complexity** - More DLLs to manage and deploy.

### A Pragmatic Approach

Here's what I recommend:

1. **Use a single assembly**

   - Keep related code together
   - Use `internal` for implementation details
   - Make only the component interfaces public
   - Add `sealed` to prevent inheritance when possible

2. **Enforce through architecture testing**

   - Add architecture tests to verify dependencies
   - Automatically check for architectural violations
   - Fail the build if someone bypasses the rules

```csharp
[Fact]
public void Controllers_Should_Only_Depend_On_Component_Interfaces()
{
    var allTypes = Types.InAssembly(Assembly.GetExecutingAssembly());

    TestResult? result = allTypes
        .That()
        .ResideInNamespace("MyApp.Controllers")
        .Should()
        .OnlyHaveDependenciesOn(
            allTypes
                .That()
                .HaveNameEndingWith("Component")
                .Or()
                .HaveNameStartingWith("IPolicy")
                .GetTypes()
                .Select(t => t.FullName!)
                .ToArray())
        .GetResult();

    result.IsSuccessful.Should().BeTrue();
}
```

Want to learn more about enforcing architecture through testing?
Check out my article on [**architecture testing**](enforcing-software-architecture-with-architecture-tests).

Remember: Clean Architecture is about managing dependencies, not about achieving perfect encapsulation.
Use the tools the language gives you, but don't over-complicate things chasing an impossible ideal.

## Conclusion

Clean Architecture isn't about projects, folders, or perfect encapsulation.

It's about:

- Organizing code around business capabilities
- Managing dependencies effectively
- Keeping related code together
- Making boundaries explicit

Start with a single project.
Use components.
Make interfaces public and implementations internal.
Add architecture tests if you need more control.

And remember: **pragmatism beats purism**.
Your architecture should help you ship features faster, not slow you down with artificial constraints.

Want to learn more?
Check out my [**Pragmatic Clean Architecture**](/pragmatic-clean-architecture) course,
where I'll show you how to build maintainable applications with proper boundaries, clear dependencies, and business-focused components.

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

Looking for a developer-friendly, affordable eSignature solution?
[**BoldSign**](https://boldsign.com/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign) offers highly responsive and secure API integration for your .NET applications with transparent pricing—making it the perfect
[**alternative to DocuSign**](https://boldsign.com/docusign-alternative/?utm_source=milanjovanovic&utm_medium=newsletter_ad&utm_campaign=boldsign).
Start your free trial today.

Transform your team's API development.
See how [**Postman Flows**](https://app.zuddl.com/p/a/event/b718e16b-9456-4da3-9af1-f990b252ceb5?utm_source=influencer&utm_medium=Newsletter&utm_campaign=Flows_collab&utm_term=Milan_Jovanovic&utm_content=Webinar)
eliminates bottlenecks & enables seamless collaboration.
Join the finale of our "Visualize, Accelerate, & Collaborate" series to supercharge your workflow.
[**Register now**](https://app.zuddl.com/p/a/event/b718e16b-9456-4da3-9af1-f990b252ceb5?utm_source=influencer&utm_medium=Newsletter&utm_campaign=Flows_collab&utm_term=Milan_Jovanovic&utm_content=Webinar).

</Sponsorships>

Idempotency is a crucial concept for REST APIs that ensures the reliability and consistency of your system.
An idempotent operation can be repeated multiple times without changing the result beyond the initial API request.
This property is especially important in distributed systems, where network failures or timeouts can lead to repeated requests.

Implementing idempotency in your API brings several benefits:

- It prevents unintended duplicate operations
- It improves reliability in distributed systems
- It helps handle network issues and retries gracefully

In this week's issue, we'll explore how to implement idempotency in ASP.NET Core APIs, ensuring your system remains robust and reliable.

## What is Idempotence?

Idempotence, in the context of web APIs, means that making multiple identical requests should have the same effect as making a single request.
In other words, no matter how many times a client sends the same request, the server-side effect should only occur once.

The [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110) standard about HTTP Semantics offers a definition we could use.
Here's what it says about **idempotent methods**:

<figure>

> A request method is considered "idempotent" if the intended effect on the server of multiple identical requests with
> that method is the same as the effect for a single such request.
>
> Of the request methods defined by this specification,
> PUT, DELETE, and safe request methods [(GET, HEAD, OPTIONS, and TRACE) - author's note] are idempotent.

  <figcaption>
    _&mdash; [RFC 9110 (HTTP Semantics), Section 9.2.2, Paragraph 1](https://www.rfc-editor.org/rfc/rfc9110#section-9.2.2-1)_
  </figcaption>
</figure>

However, the following paragraph is quite interesting.
It clarifies that the server can implement "other non-idempotent side effects" that don't apply to the resource.

<figure>

> ... the idempotent property only applies to what has been requested by the user;
> a server is free to log each request separately, retain a revision control history, or implement other non-idempotent side effects for each idempotent request.

  <figcaption>
    _&mdash; [RFC 9110 (HTTP Semantics), Section 9.2.2, Paragraph 2](https://www.rfc-editor.org/rfc/rfc9110#section-9.2.2-2)_
  </figcaption>
</figure>

The benefits of implementing idempotency extend beyond just adhering to HTTP method semantics.
It significantly improves the reliability of your API, especially in distributed systems where network issues can lead to retried requests.
By implementing idempotency, you prevent duplicate operations that could occur due to client retries.

## Which HTTP Methods are Idempotent?

Several HTTP methods are inherently idempotent:

- `GET`, `HEAD`: Retrieve data without modifying the server state.
- `PUT`: Update a resource, resulting in the same state regardless of repetition.
- `DELETE`: Remove a resource with the same outcome for multiple requests.
- `OPTIONS`: Retrieve communication options information.

`POST` is not inherently idempotent, as it typically creates resources or processes data.
Repeated `POST` requests could create multiple resources or trigger multiple actions.

However, we can implement idempotency for `POST` methods using custom logic.

**Note**: While `POST` requests aren't naturally idempotent, we can design them to be.
For example, checking for existing resources before creation ensures that repeated `POST` requests don't result in duplicate actions or resources.

## Implementing Idempotency in ASP.NET Core

To implement idempotency, we'll use a strategy involving idempotency keys:

1. The client generates a unique key for each operation and sends it in a custom header.
2. The server checks if it has seen this key before:
   - For a new key, process the request and store the result.
   - For a known key, return the stored result without reprocessing.

This ensures that retried requests (e.g., due to network issues) are processed only once on the server.

We can implement idempotency for controllers by combining an `Attribute` and `IAsyncActionFilter`.
Now, we can specify the `IdempotentAttribute` to apply idempotency to a controller endpoint.

**Note**: When a request fails (returns 4xx/5xx), we don't cache the response.
This allows clients to retry with the same idempotency key.
However, this means a failed request followed by a successful one with the same key will succeed - make sure this aligns with your business requirements.

```csharp
[AttributeUsage(AttributeTargets.Method)]
internal sealed class IdempotentAttribute : Attribute, IAsyncActionFilter
{
    private const int DefaultCacheTimeInMinutes = 60;
    private readonly TimeSpan _cacheDuration;

    public IdempotentAttribute(int cacheTimeInMinutes = DefaultCacheTimeInMinutes)
    {
        _cacheDuration = TimeSpan.FromMinutes(cacheTimeInMinutes);
    }

    public async Task OnActionExecutionAsync(
        ActionExecutingContext context,
        ActionExecutionDelegate next)
    {
        // Parse the Idempotence-Key header from the request
        if (!context.HttpContext.Request.Headers.TryGetValue(
                "Idempotence-Key",
                out StringValues idempotenceKeyValue) ||
            !Guid.TryParse(idempotenceKeyValue, out Guid idempotenceKey))
        {
            context.Result = new BadRequestObjectResult("Invalid or missing Idempotence-Key header");
            return;
        }

        IDistributedCache cache = context.HttpContext
            .RequestServices.GetRequiredService<IDistributedCache>();

        // Check if we already processed this request and return a cached response (if it exists)
        string cacheKey = $"Idempotent_{idempotenceKey}";
        string? cachedResult = await cache.GetStringAsync(cacheKey);
        if (cachedResult is not null)
        {
            IdempotentResponse response = JsonSerializer.Deserialize<IdempotentResponse>(cachedResult)!;

            var result = new ObjectResult(response.Value) { StatusCode = response.StatusCode };
            context.Result = result;

            return;
        }

        // Execute the request and cache the response for the specified duration
        ActionExecutedContext executedContext = await next();

        if (executedContext.Result is ObjectResult { StatusCode: >= 200 and < 300 } objectResult)
        {
            int statusCode = objectResult.StatusCode ?? StatusCodes.Status200OK;
            IdempotentResponse response = new(statusCode, objectResult.Value);

            await cache.SetStringAsync(
                cacheKey,
                JsonSerializer.Serialize(response),
                new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = _cacheDuration }
            );
        }
    }
}

internal sealed class IdempotentResponse
{
    [JsonConstructor]
    public IdempotentResponse(int statusCode, object? value)
    {
        StatusCode = statusCode;
        Value = value;
    }

    public int StatusCode { get; }
    public object? Value { get; }
}
```

**Note**: There's a small [**race condition**](solving-race-conditions-with-ef-core-optimistic-locking) window between checking and setting the cache.
For absolute consistency, we should consider using a distributed lock pattern, though this adds complexity and latency.

Now, we can apply this attribute to our controller actions:

```csharp {6}
[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    [HttpPost]
    [Idempotent(cacheTimeInMinutes: 60)]
    public IActionResult CreateOrder([FromBody] CreateOrderRequest request)
    {
        // Process the order...

        return CreatedAtAction(nameof(GetOrder), new { id = orderDto.Id }, orderDto);
    }
}
```

**Idempotency with Minimal APIs**

To implement idempotency with Minimal APIs, we can use an `IEndpointFilter`.

```csharp
internal sealed class IdempotencyFilter(int cacheTimeInMinutes = 60)
    : IEndpointFilter
{
    public async ValueTask<object?> InvokeAsync(
        EndpointFilterInvocationContext context,
        EndpointFilterDelegate next)
    {
        // Parse the Idempotence-Key header from the request
        if (TryGetIdempotenceKey(out Guid idempotenceKey))
        {
            return Results.BadRequest("Invalid or missing Idempotence-Key header");
        }

        IDistributedCache cache = context.HttpContext
            .RequestServices.GetRequiredService<IDistributedCache>();

        // Check if we already processed this request and return a cached response (if it exists)
        string cacheKey = $"Idempotent_{idempotenceKey}";
        string? cachedResult = await cache.GetStringAsync(cacheKey);
        if (cachedResult is not null)
        {
            IdempotentResponse response = JsonSerializer.Deserialize<IdempotentResponse>(cachedResult)!;
            return new IdempotentResult(response.StatusCode, response.Value);
        }

        object? result = await next(context);

        // Execute the request and cache the response for the specified duration
        if (result is IStatusCodeHttpResult { StatusCode: >= 200 and < 300 } statusCodeResult
            and IValueHttpResult valueResult)
        {
            int statusCode = statusCodeResult.StatusCode ?? StatusCodes.Status200OK;
            IdempotentResponse response = new(statusCode, valueResult.Value);

            await cache.SetStringAsync(
                cacheKey,
                JsonSerializer.Serialize(response),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(cacheTimeInMinutes)
                }
            );
        }

        return result;
    }
}

// We have to implement a custom result to write the status code
internal sealed class IdempotentResult : IResult
{
    private readonly int _statusCode;
    private readonly object? _value;

    public IdempotentResult(int statusCode, object? value)
    {
        _statusCode = statusCode;
        _value = value;
    }

    public Task ExecuteAsync(HttpContext httpContext)
    {
        httpContext.Response.StatusCode = _statusCode;

        return httpContext.Response.WriteAsJsonAsync(_value);
    }
}
```

Now, we can apply this endpoint filter to our Minimal API endpoint:

```csharp {4}
app.MapPost("/api/orders", CreateOrder)
    .RequireAuthorization()
    .WithOpenApi()
    .AddEndpointFilter<IdempotencyFilter>();
```

An alternative to the previous two implementations is implementing idempotency logic in a custom middleware.

## Best Practices and Considerations

Here are the key things I always keep in mind when implementing idempotency.

Cache duration is tricky.
I aim to cover reasonable retry windows without holding onto stale data.
A reasonable cache time typically ranges from a few minutes to 24-48 hours, depending on your specific use case.

Concurrency can be a pain, especially in high-traffic APIs.
A thread-safe implementation using a distributed lock works great.
It keeps things in check when multiple requests hit at once.
But this should be a rare occurrence.

For distributed setups, Redis is my go-to.
It's perfect as a shared cache, keeping idempotency consistent across all your API instances.
Plus, it handles distributed locking.

What if a client reuses an idempotency key with a different request body?
I return an error in this case.
My approach is to hash the request body and store it with the idempotency key.
When a request comes in, I compare the request body hashes.
If they differ, I return an error.
This prevents misuse of idempotency keys and maintains the integrity of your API.

## Summary

Implementing idempotency in REST APIs enhances service reliability and consistency.
It ensures identical requests yield the same result, preventing unintended duplicates and gracefully handling network issues.

While our implementation provides a foundation, I recommend adapting it to your needs.
Focus on critical operations in your APIs, especially those that modify the system state or trigger important business processes.

By embracing idempotency, you're building more robust and user-friendly APIs.

That's all for today.

See you next week.

---



<Sponsorships>

[**Telerik UI for ASP.NET Core Grid**](https://www.telerik.com/blogs/editing-data-telerik-ui-aspnet-core-grid-no-code-required?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=core-general-awareness):
Enable data editing without code.
Multiple options for customizing user experience.
Configure the grid easily to meet your UX needs.
[**Learn more here**](https://www.telerik.com/blogs/editing-data-telerik-ui-aspnet-core-grid-no-code-required?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=core-general-awareness).

Webinar: [**Accelerating with AI using Postman Flows**](https://app.zuddl.com/p/a/event/aa49e0b7-9b7b-43d2-afce-282fcf08a756?utm_source=influencer&utm_medium=Newsletter&utm_campaign=Flows_AI&utm_term=Milan_Jovanovic&utm_content=Webinar).
Second in our series on visualizing, accelerating, and collaborating with Postman Flows.
Learn how AI enhances API development and speeds up your workflow.
[**Register here for free**](https://app.zuddl.com/p/a/event/aa49e0b7-9b7b-43d2-afce-282fcf08a756?utm_source=influencer&utm_medium=Newsletter&utm_campaign=Flows_AI&utm_term=Milan_Jovanovic&utm_content=Webinar).

</Sponsorships>

When developing HTTP APIs, providing consistent and informative error responses is crucial for a smooth developer experience.
**Problem Details** in ASP.NET Core offers a standardized solution to this challenge, ensuring your APIs communicate errors effectively and uniformly.

In this article, we'll explore the latest developments in **Problem Details**, including:

- The new [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) that refines the Problem Details standard
- Using the .NET 8 `IExceptionHandler` for global exception handling
- Using the `IProblemDetailsService` for customizing Problem Details

Let's dive into these features and see how they can improve your API's error handling.

## Understanding Problem Details

Problem Details is a machine-readable format for specifying errors in HTTP API responses.
HTTP status codes don't always contain enough details about errors to be helpful.
The Problem Details specification defines a JSON (and XML) document format to describe problems.

Problem Details includes:

- `type`: A URI reference that identifies the problem type
- `title`: A short, human-readable summary of the problem type
- `status`: The HTTP status code
- `detail`: A human-readable explanation specific to this occurrence of the problem
- `instance`: A URI reference that identifies the specific occurrence of the problem

[RFC 9457](https://www.rfc-editor.org/rfc/rfc9457), which replaces [RFC 7807](https://www.rfc-editor.org/rfc/rfc7807),
introduces improvements such as clarifying the use of the type field and providing guidelines for extending **Problem Details**.

Here's an example Problem Details response:

```json
Content-Type: application/problem+json

{
  "type": "https://tools.ietf.org/html/rfc9110#section-15.5.5",
  "title": "Not Found",
  "status": 404,
  "detail": "The habit with the specified identifier was not found",
  "instance": "PUT /api/habits/aadcad3f-8dc8-443d-be44-3d99893ba18a"
}
```

## Implementing Problem Details

Let's see how to implement Problem Details in ASP.NET Core.
We want to return a Problem Details response for unhandled exceptions.
By calling `AddProblemDetails`, we're configuring the application to use the Problem Details format for failed requests.
With `UseExceptionHandler`, we introduce an exception handling middleware to the request pipeline.
By adding `UseStatusCodePages`, we're introducing a middleware that will convert error responses with an empty body to a Problem Details response.

```csharp {4,9,12}
var builder = WebApplication.CreateBuilder(args);

// Adds services for using Problem Details format
builder.Services.AddProblemDetails();

var app = builder.Build();

// Converts unhandled exceptions into Problem Details responses
app.UseExceptionHandler();

// Returns the Problem Details response for (empty) non-successful responses
app.UseStatusCodePages();

app.Run();
```

When we encounter an unhandled exception, it will be translated to a Problem Details response:

```json
Content-Type: application/problem+json

{
  "type": "https://tools.ietf.org/html/rfc9110#section-15.6.1",
  "title": "An error occurred while processing your request.",
  "status": 500
}
```

Now, let's explore how we can customize this response.

## Global Error Handling

We have a few options for [implementing global error handling](global-error-handling-in-aspnetcore-8).
The most popular approach is creating a custom exception handling middleware.
You wrap the API request in a `try-catch` statement and return a response based on any caught exception.

With .NET 8, we can use the `IExceptionHandler` that runs in the built-in exception handling middleware.
This handler allows you to tailor the Problem Details response for specific exceptions.
Returning `true` from the `TryHandleAsync` method short-circuits the pipeline and returns the API response.
If we return `false`, the next handler in the chain attempts to handle the exception.

We can map different exception types to appropriate HTTP status codes, providing more precise error information to API consumers.

Here's an example `CustomExceptionHandler` implementation:

```csharp
internal sealed class CustomExceptionHandler : IExceptionHandler
{
    public async ValueTask<bool> TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        int status = exception switch
        {
            ArgumentException => StatusCodes.Status400BadRequest,
            _ => StatusCodes.Status500InternalServerError
        };
        httpContext.Response.StatusCode = status;

        var problemDetails = new ProblemDetails
        {
            Status = status,
            Title = "An error occurred",
            Type = exception.GetType().Name,
            Detail = exception.Message
        };

        await httpContext.Response.WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}

// In Program.cs
builder.Services.AddExceptionHandler<CustomExceptionHandler>();
```

## Using The ProblemDetailsService

Calling `AddProblemDetails` registers a default implementation of the `IProblemDetailsService`.
The `IProblemDetailsService` will set the response status code based on the `ProblemDetails.Status`.

Here's how we can use it in the `CustomExceptionHandler`:

```csharp
public class CustomExceptionHandler(IProblemDetailsService problemDetailsService) : IExceptionHandler
{
    public async ValueTask<bool> TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        var problemDetails = new ProblemDetails
        {
            Status = exception switch
            {
                ArgumentException => StatusCodes.Status400BadRequest,
                _ => StatusCodes.Status500InternalServerError
            },
            Title = "An error occurred",
            Type = exception.GetType().Name,
            Detail = exception.Message
        };

        return await problemDetailsService.TryWriteAsync(new ProblemDetailsContext
        {
            Exception = exception,
            HttpContext = httpContext,
            ProblemDetails = problemDetails
        });
    }
}
```

This approach seems very similar to the previous one, where we wrote to the response body.
However, using the `IProblemDetailsService` gives an easy way to customize all Problem Details responses.

We can return Problem Details in controllers using the `Problem` method, or `Results.Problem` in Minimal APIs.
These methods respect the configured Problem Details customizations (more on this in the next section).

```csharp
IdentityUser identityUser = new() { UserName = registerUserDto.UserName, Email = registerUserDto.Email };
IdentityResult result = await userManager.CreateAsync(identityUser, registerUserDto.Password);

if (!result.Succeeded)
{
    // return Results.Problem - Minimal APIs
    return Problem(
        type: "Bad Request",
        title: "Identity failure",
        detail: result.Errors.First().Description,
        statusCode: StatusCodes.Status400BadRequest);
}
```

## Customizing Problem Details

We can pass a delegate to the `AddProblemDetails` method to set the `CustomizeProblemDetails`.
You can use this to add extra information to all Problem Details responses.

This is an excellent place for solving cross-cutting concerns, like setting the `instance` value and adding diagnostics information.

```csharp
builder.Services.AddProblemDetails(options =>
{
    options.CustomizeProblemDetails = context =>
    {
        context.ProblemDetails.Instance =
            $"{context.HttpContext.Request.Method} {context.HttpContext.Request.Path}";

        context.ProblemDetails.Extensions.TryAdd("requestId", context.HttpContext.TraceIdentifier);

        Activity? activity = context.HttpContext.Features.Get<IHttpActivityFeature>()?.Activity;
        context.ProblemDetails.Extensions.TryAdd("traceId", activity?.Id);
    };
});
```

This customization adds the request path, a request ID, and a trace ID to every Problem Details response, enhancing debuggability and traceability of errors.

```json {7-9}
Content-Type: application/problem+json

{
  "type": "https://tools.ietf.org/html/rfc9110#section-15.5.5",
  "title": "Not Found",
  "status": 404,
  "instance": "PUT /api/habits/aadcad3f-8dc8-443d-be44-3d99893ba18a",
  "traceId": "00-63d4af1807586b0d98901ae47944192d-9a8635facb90bf76-01",
  "requestId": "0HN7C8PRNMGIA:00000001"
}
```

You can use the `traceId` to find the distributed traces and logs in a monitoring system like Seq.

<div className="bordered">
  <Image
    src="blogs/mnw_112/seq_tracing.png"
    width={1919}
    height={905}
    alt="Seq user interface showing a distributed trace with the same trace identifier as the problem details response."
  />
</div>

## Handling Specific Exceptions (Status Codes)

.NET 9 introduces a simpler way to map exceptions to status codes.
Great news for fans of throwing exceptions.
You can use the `StatusCodeSelector` to define the mappings.
This makes it easier to maintain consistent error responses across your API.

```csharp
app.UseExceptionHandler(new ExceptionHandlerOptions
{
    StatusCodeSelector = ex => ex switch
    {
        ArgumentException => StatusCodes.Status400BadRequest,
        NotFoundException => StatusCodes.Status404NotFound,
        _ => StatusCodes.Status500InternalServerError
    }
});
```

If you use this together with an `IExceptionHandler` that sets the `StatusCode`, then the `StatusCodeSelector` is ignored.

## Takeaway

Implementing Problem Details in your ASP.NET Core APIs is more than just a best practice -
it's a standard for improving the developer experience of your API consumers.
By providing consistent, detailed, and well-structured error responses, you make it easier for clients to understand and handle error scenarios gracefully.

As you implement these practices in your own projects, you'll discover even more ways to tailor Problem Details to your specific needs.
I shared what worked well for my use cases.

{/* Update this when the course is out */}
Problem Details is just one of the best practices covered in my upcoming [**REST APIs course**](/rest-apis-in-aspnetcore).
Check it out if you're looking for a comprehensive guide.

Good luck out there, and I'll see you next week.

---


Previous issues of The .NET Weekly