
<Sponsorships>

Today's issue is sponsored by [**Fiverr.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)
Fiverr is your one-stop solution for finding skilled freelance developers.
Build custom APIs and integrations tailored to your needs.
Seamlessly connect with payment providers (Stripe, PayPal) and automation platforms (Zapier) for streamlined operations.
Check it out [**here!**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)

And by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

</Sponsorships>

If you're building a **.NET** application, chances are high that you'll need to call an **external API** over **HTTP**.

The easy way to make HTTP requests in .NET is to use the `HttpClient` to send those requests.
And it's a great abstraction to work with, especially with the methods supporting **JSON** payloads and responses.

Unfortunately, it's easy to misuse the `HttpClient`.

**Port exhaustion** and **DNS behavior** are some of the most common problems.

So here's what you need to know about working with `HttpClient`:

- How not to use `HttpClient`
- How to simplify configuration with `IHttpClientFactory`
- How to configure **typed clients**
- Why you should avoid **typed clients** in singleton services
- When to use which option

Let's dive in!

## The Naive Way To Use HttpClient

The simplest way to work with the `HttpClient` is to just create a new instance, set the required properties and use it to send requests.

What could possibly go wrong?

`HttpClient` instances are meant to be **long-lived**, and reused throughout the lifetime of the application.

Each instance uses its own **connection pool** for isolation purposes, but also to prevent **port exhaustion**.
If a server is under high load, and your application is constantly creating new connections, it could lead to exhausting the available ports.
This will cause an exception at runtime, when trying to send a request.

So how can you avoid this?

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;

    public GitHubService(IOptions<GitHubSettings> settings)
    {
        _settings = settings.Value;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        using var client = new HttpClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## The Smart Way To Create HttpClient Using IHttpClientFactory

Instead of managing the `HttpClient` lifetime yourself, you can use an `IHttpClientFactory` to create the `HttpClient` instance.

Simply call the `CreateClient` method and use the returned `HttpClient` instance to send your HTTP requests.

Why is this a better approach?

The expensive part of the `HttpClient` is the actual message handler - `HttpMessageHandler`.
Each `HttpMessageHandler` has an internal HTTP **connection pool** that can be reused.

The `IHttpClientFactory` will **cache** the `HttpMessageHandler` and reuse it when creating a new `HttpClient` instance.

An important note here is that `HttpClient` instances created by `IHttpClientFactory` are meant to be **short-lived**.

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;
    private readonly IHttpClientFactory _factory;

    public GitHubService(
        IOptions<GitHubSettings> settings,
        IHttpClientFactory factory)
    {
        _settings = settings.Value;
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        using var client = _factory.CreateClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Reducing Code Duplication With Named Clients

Using `IHttpClientFactory` will solve most of the issues of manually creating an `HttpClient`.
However, we still need to configure the default request parameters every time we obtain a new `HttpClient` from the `CreateClient` method.

You can configure a **named client** by calling the `AddHttpClient` method and passing in the desired name.
The `AddHttpClient` accepts a delegate that you can use to configure the default parameters on the `HttpClient` instance.

```csharp
services.AddHttpClient("github", (serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

The main difference is you now have to obtain the client by passing the name of the client to `CreateClient`.

But the use of the `HttpClient` looks a lot simpler:

```csharp
public class GitHubService
{
    private readonly IHttpClientFactory _factory;

    public GitHubService(IHttpClientFactory factory)
    {
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        using var client = _factory.CreateClient("github");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Replacing Named Clients With Typed Clients

The downside of using **named clients** is having to resolve an `HttpClient` by passing in a name every time.

There's a better way to achieve the same behavior by configuring a **typed client**.
You can do this by calling the `AddClient<TClient>` method and configuring the service that will consume the `HttpClient`.

Under the hood, this is still using a **named client**, where the name is the same as the type name.

And this will also register `GitHubService` with a **transient lifetime**.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

Inside of `GitHubService`, you inject and use the typed `HttpClient` instance which will have all of the configuration applied.

No more dealing with `IHttpClientFactory` and creating `HttpClient` instances manually.

```csharp
public class GitHubService
{
    private readonly HttpClient client;

    public GitHubService(HttpClient client)
    {
        _client = client;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Why You Should Avoid Typed Clients In Singleton Services

You could run into a **problem** if you inject a **typed client** into a **singleton service**.
Since the **typed client** is **transient**, injecting it in a **singleton service** will cause it to be cached for the lifetime of the **singleton service**.

This will prevent the **typed client** from reacting to DNS changes.

If you want to use a **typed client** in a **singleton service**, the recommened approach is using `SocketsHttpHandler` as the primary handler,
and configuring the `PooledConnectionLifetime`.

Since the `SocketsHttpHandler` will handle connection pooling, you can disable recycling at the `IHttpClientFactory` level by setting `HandlerLifetime` to `Timeout.InfiniteTimeSpan`.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    return new SocketsHttpHandler()
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(15)
    };
})
.SetHandlerLifetime(Timeout.InfiniteTimeSpan);
```

## When Should You Use Which Option?

I showed you a few possible options for working with `HttpClient`.

But which one should you use and when?

Microsoft was kind enough to provide us with a set of best practices and [recommended use](https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient-guidelines#recommended-use)
for `HttpClient`.

- Use a `static` or **singleton** `HttpClient` instance with a `PooledConnectionLifetime` configured, since this solves both port exhaustion and tracking DNS changes
- Use `IHttpClientFactory` if you want to move the configuration to one place, but remember that clients are meant to be **short-lived**
- Use a **typed client** if you want the `IHttpClientFactory` configurability

I prefer working with a **typed client**, and I'm mindful of the fact that it's configured as a **transient service**.

Thanks for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

If you're building a scalable application, it's a common requirement to offload some work in your application to a **background job**.

Here are a few examples of that:

- Publishing email notifications
- Generating reports
- Updating a cache
- Image processing

How can you create a recurring **background job** in .NET?

[**Quartz.NET**](https://www.quartz-scheduler.net/) is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

There are three concepts you need to understand in **Quartz.NET**:

- **Job** - the actual background task you want to run
- **Trigger** - the trigger controlling when a job runs
- **Scheduler** - responsible for coordinating jobs and triggers

Let's see how we can use **Quartz.NET** to create and schedule **background jobs**.

## Adding The Quartz.NET Hosted Service

The first thing we need to do is install the **Quartz.NET** NuGet package.
There are a few to pick from, but we're going to install the `Quartz.Extensions.Hosting` library:

```powershell
Install-Package Quartz.Extensions.Hosting
```

The reason we're using this library is because it integrates nicely with .NET using an `IHostedService` instance.

To get the **Quartz.NET** hosted service up and running, we need two things:

- Add the required services with the DI container
- Add the hosted service

```csharp
services.AddQuartz(configure =>
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});
```

**Quartz.NET** will create jobs by fetching them from the DI container.
This also means you can use scoped services in your jobs, not just singleton or transient services.

Setting the `WaitForJobsToComplete` option to `true` will ensure that **Quartz.NET** waits for the jobs to complete gracefully before exiting.

## Creating Background Jobs With `IJob`

To crate a background job with **Quartz.NET** you need to implement the `IJob` interface.

It only exposes a single method - `Execute` - where you can place the code for your background job.

A few things worth noting here:

- We're using DI to inject the `ApplicationDbContext` and `IPublisher` services
- The job is decorated with `DisallowConcurrentExecution` to prevent running the same job concurrently

```csharp
[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List<OutboxMessage> messages = await _dbContext
            .Set<OutboxMessage>()
            .Where(m => m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject<IDomainEvent>(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}
```

Now that the **background job** is ready, we need to register it with the **DI** container and add a trigger that will run the job.

## Configuring the Job

I mentioned at the start that there are three key concepts in **Quartz.NET**:

- Job
- Trigger
- Scheduler

We already implemented the `ProcessOutboxMessagesJob` background job in the previous section.

The **Quartz.NET** library will take care of the scheduler.

And this leaves us with configuring the **trigger** for our `ProcessOutboxMessagesJob`.

```csharp
services.AddQuartz(configure =>
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob<ProcessOutboxMessagesJob>(jobKey)
        .AddTrigger(
            trigger => trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule => schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});
```

We need to uniquely identify our **background job** with a `JobKey`.
I like to keep it simple and use the job name.

Calling `AddJob` will register the `ProcessOutboxMessagesJob` with DI and also with Quartz.

After that we configure a trigger for this job by calling `AddTrigger`.
You need to associate the job with the trigger by calling `ForJob`, and then you configure the schedule for the background job.
In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.

**Quartz** also has support for configuring triggers using [cron expressions.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html)

## Job Persistence

By default, Quartz configures all jobs using the `RAMJobStore` which is the most performant because it keeps all of its data in RAM.
However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.

It could be useful to have a persistent job store in some scenarios and there's a built in `AdoJobStore` which works with SQL databases.
You need to create a set of database tables for Quartz.NET to use.

You can learn more about this in the [job stores documentation.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html)

## Takeaway

**Quartz.NET** makes running **background jobs** in .NET easy, and you can use all the power of DI in your **background jobs**.
It's also flexible for various scheduling requirements with configuration via code or using cron expressions.

There's some room for improvement to make scheduling jobs easier and reduce boilerplate:

- Add an extension method to simplify configuring jobs with a simple schedule
- Add an extension method to simplify configuring jobs with a cron schedule from application settings

If you want to see a tutorial on using **Quartz.NET**, I made an in-depth video about [**using Quartz for processing Outbox messages**](https://youtu.be/XALvnX7MPeo).

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Tower.**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)
Try Tower, a [**powerful Git client for Windows**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly) focused on productivity.
Make the complex simple with helpful features like Undo, Drag and Drop and Remote Services Integration!

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

Do you want to streamline your software development process and accelerate your release cycles?

Imagine being able to automatically build, test, and deploy your .NET applications with every code change.

With **CI/CD**, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.

And it's never been easier to get started with **CI/CD**.

**GitHub Actions** are completely free and simple to use.

So here's what we'll cover:

- Introduction to **CI/CD** & **GitHub Actions**
- Creating a **build & test pipeline** for **.NET**
- Creating a **deployment pipeline** for **Azure App Service**

Let's dive in.

## What Is Continuous Integration And Delivery?

I'll try to briefly explain what **CI/CD** is, before we take a look at **GitHub Actions**.

**CI/CD** is a method to increase the frequency of delivering new features by adding automation to your software development workflow.

**Continuous Integration ("CI")** refers to the automation process of syncing new code to a repository.
Any new changes to the application code are immediately built, tested and merged.

**Continuous Delivery, or Deployment, ("CD")** refers to the process of automating the deployment part of the workflow.
When you make a change which gets merged to the repository, this step takes care of deploying those changes to the production environment (or any other environment).

## Continuous Integration With GitHub Actions

If you're using **GitHub**, getting started with **Continuous Integration** has never been easier.

You can use [**GitHub Actions**](https://github.com/features/actions) to automate your build, test, and deployment **pipeline**.
You can create workflows that build and test every commit to your repository, or deploy to production when a new tag is created.

To create a **GitHub Action**, you write a _workflow_ to be triggered when some _event_ occurs in your repository.
An example event is a commit to the main branch, creation of a tag, or you can manually run the workflow.

Here's a **GitHub Actions** workflow to build and test a .NET project:

```yaml
name: Build & Test 🧪

on:
  push:
    branches:
      - main

env:
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Install dependencies 📂
        run: dotnet restore WebApi

      - name: Build 🧱
        run: dotnet build WebApi --configuration Release --no-restore

      - name: Test 🧪
        run: dotnet test WebApi --configuration Release --no-build
```

Let's unwrap what is happening here:

- Defining an event to trigger the workflow
- Setting up the **.NET SDK** with the version from `env.DOTNET_VERSION`
- Restoring, building and testing the project using the `dotnet` CLI tool

You can add this to your **GitHub** repository today, and start getting instant feedback when you commit code to the repository.

When a workflow run fails due to a build error or a failed test, you'll get an email notification.

## Continuous Delivery To Azure With GitHub Actions

**Continuous Integration** is a great way to start out with **CI/CD**, but the real value lies in **automating** your **deployment** process.

Imagine this:

- You make a change to your codebase
- The commit triggers the **deployment pipeline**
- A few minutes later your changes are live in production

Usually it's a little more nuanced, because we need to think about configuration, database migrations, etc.
But try to see the big picture here.

If you're running your application in the cloud, for example on **Azure**, chances are there's an existing **GitHub Action** you can use.

Here's a **deployment pipeline** I use to publish my application to an **Azure App Service** instance:

```yaml
name: Publish 🚀

on:
  push:
    branches:
      - main

env:
  AZURE_WEBAPP_NAME: web-api
  AZURE_WEBAPP_PACKAGE_PATH: './publish'
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Build and Publish 📂
        run: |
          dotnet restore WebApi
          dotnet build WebApi -c Release --no-restore
          dotnet publish WebApi -c Release --no-build
            --output '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'

      - name: Deploy to Azure 🌌
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          publish-profile: ${{ secrets.AZURE_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
```

This workflow is pretty similar to the previous one, with the differences being:

- Adding a publish step and configuring the output path
- Using the `azure/webapps-deploy@v2` action to deploy to **Azure**

If you need to safely and securely expose secret values in your workflows, you can use **GitHub secrets**.
You can define the secrets on GitHub, and use them in actions without having to add them to source control.

In the deployment workflow I'm using `secrets.AZURE_PUBLISH_PROFILE` to access my publish profile for the App Service instance.

## In Summary

**Continuous Integration and Delivery** can transform your development process by increasing the speed at which you release changes.

Try adding up how much time you spend on deployments.
I'm pretty sure you'll be surprised by the time savings potential of automating them.

And the good part is you will typically set up your **build and deployment pipelines** once, and then continue benefiting from them for the lifetime of your project.

If you want a step by step guide, I made a video showing [**how to implement a CI/CD pipeline from scratch.**](https://youtu.be/QP0pi7xe24s)
And the **source code** is also public, so you can add the **GitHub Actions** workflow file to your project.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Refactoring.**](https://refactoring.club/?utm_source=milan)
Refactoring is a weekly newsletter [**about making great software and working well with humans.**](https://refactoring.club/?utm_source=milan)
It is read every week by 35,000+ developers, managers and leaders at companies like Google, Meta, and Amazon.
Check it out [**here.**](https://refactoring.club/?utm_source=milan)

And by [**Jam.**](https://jam.dev/)
Reduce time spent on bug reporting with [**Jam**](https://jam.dev/) by 60-80%.
Auto-capture dev logs & network requests in one link, access Instant Replays for hard-to-repro bugs and integrate with Jira, Slack & more.
Free Team Tier for the rest of 2023.
[**Get Jam Now.**](https://jam.dev/)

</Sponsorships>

Most software applications today are built around the concept of **multi-tenancy**.

One application serves multiple customers, while keeping their data **isolated**.

You can approach **multi-tenancy** in two ways:

- **Single database** and **logical isolation** of tenants
- **Multiple databases** and **physical isolation** of tenants

Which option you decide to use will depend mostly on your requirements.
Some industries like healthcare require a high degree of **data isolation**, and using a **database per tenant** is a must.

So how are we going to **implement multi-tenancy** support with **EF Core**?

We can use **Query Filters** to apply a **tenant filter** to all database queries.

Implement it once and you can almost forget about it.

Let's see what are some of the problems we need to solve.

## How To Use EF Core Query Filters

If you want an in-depth dive into **Query filters**, take a look at the newsletter issue where I talked about
[**using query filters with EF Core.**](how-to-use-global-query-filters-in-ef-core)

Here's a quick refresher on **Query filters**:

- Configure the **query filter** by calling `HasQueryFilter` for your entity
- **EF** will apply it to all queries for that entity
- You can turn it off with `IgnoreQueryFilters`
- Only **one** query filter **per entity** is allowed

And here's a simple example:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

All queries to the `Order` table will include an `IsDeleted = FALSE` condition.

## Single Database Multi-Tenancy With EF Core

You will need two things to **implement multi-tenancy** on a **single database**:

- A way to know **who** the **current tenant** is
- A way to **filter** the **data** for that **tenant** only

The typical approach for **multi-tenancy** on a **single database** is having a `TenantId` column in your tables.
And then filtering on that column when querying the database.

You can use the **Query filters** feature in **EF Core** to apply a **global filter** for some entity.

Inside of the `OnModelCreating` method we configure the query filter on the `Order` entity:

```csharp
public class OrdersDbContext : DbContext
{
    private readonly string _tenantId;

    public OrdersDbContext(
        DbContextOptions<OrdersDbContext> options,
        TenantProvider tenantProvider)
        : base(options)
    {
        _tenantId = tenantProvider.TenantId;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder
            .Entity<Order>
            .HasQueryFilter(o => o.TenantId == _tenantId);
    }
}
```

We're using the `TenantProvider` class to get the **current tenant** value.

Here's what the `TenantProvider` implementation looks like:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;

    public TenantProvider(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];
}
```

The `TenantId` is coming from the HTTP request header in this example.

A few other options to get the `TenantId` are:

- Query string - `api/orders?tenantId=example-tenant-id`
- JWT Claim
- API Key

If you want a more **secure implementation** you should go with **JWT Claims** or **API Keys** to provide the `TenantId` value.

## Separate Databases Multi-Tenancy With EF Core

What if we want to **isolate** each tenant to a **separate database**?

Here are the changes we need to make:

- Applying different **connection string per tenant**
- Resolving the connection string for each tenant _somehow_

You can't use **Query filters** here, since we are working with different databases.

So you will need to store the **tenant information** and **connection strings** somewhere.

A simple example would be store them in the **application settings**:

```json
"Tenants": {
    { "Id": "tenant-1", "ConnectionString": "Host=tenant1.db;Database=tenant1" },
    { "Id": "tenant-2", "ConnectionString": "Host=tenant2.db;Database=tenant2" }
}
```

You can then register an `IOptions` instance with a list of `Tenant` objects.

And we need to slightly modify the `TenantProvider` class to return a **connection string** for the current tenant:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;
    private readonly TenantSettings _tenantSettings;

    public TenantProvider(
        IHttpContextAccessor httpContextAccessor,
        IOptions<TenantSettings> tenantsOptions)
    {
        _httpContextAccessor = httpContextAccessor;
        _tenants = tenantsOptions.Value;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];

    public string GetConnectionString()
    {
        return _tenantSettings.Tenants.Single(t => t.Id == TenantId);
    }
}
```

And the last part is registering your `DbContext` to **dynamically resolve** the **connection string** for the current **tenant**.

```csharp
builder.Services.AddDbContext<OrdersDbContext>((sp, o) =>
{
    var tenantProvider = sp.GetRequiredService<TenantProvider>();

    var connectionString = tenantProvider.GetConnectionString();

    o.UseSqlServer(connectionString);
});
```

On every request, we create a new `OrdersDbContext` and connect to the appropriate database for that tenant.

You should definitely consider storing the tenant **connection strings** in a **secure** place like **Azure Key Vault**.

## Closing Thoughts

I hope you now have a better understanding of how to build a **multi-tenant system** with **EF Core**.

I showed you the bare bones implementation, which you can improve to make it more robust and secure.

Building **multi-tenant systems** isn't easy, but when you understand the basic principles it shouldn't be too difficult either.

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel**](https://u.icepanel.io/1eec0e6f).
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** diagrams are a great way to communicate how you are planning to build a software system or how an existing software system works.

However, the majority of software architecture **diagrams** I've seen are a total mess.

If only there was a standard way to visualize your software architecture...

Enter the **C4 model**, which stands for **context**, **containers**, **components**, and **code**.

The **C4 model** is a lightweight approach to describing your **software architecture**.

And it's essentially just two things, a predefined set of common **abstractions** and four diagram types:

- **Context diagram** - high-level view of the system
- **Container diagram** - shows the objects running inside a system
- **Component diagram** - shows the building blocks that make each container
- **Code diagram** - rarely used, typically UML or ER diagram

Let's see how we can use the **C4 model** to visualize our **software architecture**.

## First You Need Abstractions

Before we dive into the **C4 diagrams**, we first need to understand the ubiquitous language of the **C4 model**.

> A software system is made up of one or more containers (applications and data stores),
> each of which contains one or more components, which in turn are implemented by one or more code elements (classes, interfaces, objects, functions, etc).
> And people may use the software systems that we build.

The **C4 model** defines a set of **abstractions** that you can use to describe your software systems:

- Person - the end user using your system
- Software system - highest level of abstraction that delivers value to end users
- Container - applications and data stores that make up a system
- Component - building blocks/modules that make up the container

With these high-level concepts in place, let's take a look at how to use them in our **C4 diagrams**.

## System Context Diagram

The **System Context diagram** is a high-level view of your software system.

It shows your software system as the central part, and any external systems and users that your system interacts with.

It should be **technology agnostic**, and the focus on the people and software systems instead of low-level details.

<Image
  src="blogs/mnw_037/system_context_diagram.jpg"
  width={2116}
  height={1172}
/>

The intended audience for the **System Context Diagram** is _everybody_.

If you can show it to non-technical people and they are able to understand it, then you know you're on the right track.

## Container Diagram

When you zoom into one software system, you get to the **Container diagram**.

Your software system is comprised of multiple running parts - **containers**.

A **container** can be a:

- Web application
- Single-page application
- Database
- File system
- Object store
- Message broker

You can look at a **container** as a **deployment unit** that executes code or stores data.

The **Container diagram** shows the high-level view of the software architecture and the **major technology choices**.

<Image src="blogs/mnw_037/container_diagram.jpg" width={1972} height={1732} />

The **Container diagram** is intended for technical people inside and outside of the software development team:

- Software architects
- Developers
- Operations/support staff

## Component Diagram

Next you can zoom into an individual **container** to decompose it into its building blocks.

The **Component diagram** show the individual **components** that make up a **container**:

- What each of the components are
- The technology and implementation details

<Image src="blogs/mnw_037/component_diagram.jpg" width={2100} height={1524} />

The **Component diagram** is intended for software architects and developers.

## Code Diagram

Finally, you can zoom into each **component** to show how it is implemented with **code**, typically using a UML class diagram or an ER diagram.

This level is rarely used as it goes into too much technical detail for most use cases.

However, there are supplementary diagrams that can be useful to fill in missing information by showcasing:

- Sequence of events
- Deployment information
- How systems interact at a higher level

It's only recommended for the most **important or complex components**.

Of course, the target audience are software architects and developers.

## Takeaway

The **C4 model** is a tool for visual and verbal communication, enabling your team to talk in the same language and have a common understanding of the design.

It's valuable for many reasons, to name a few:

- Product and business people benefit from understanding user behavior at the Context level
- Context and Container diagrams provide a simple view of how software systems work
- New developers can quickly understand system flows

Now that you understand what the **C4 model** is, there's one question left to answer.

Should you use the **C4 model** to express your **software architecture**?

I can't give you a definitive answer.

I used it with a lot of success.

But I think you should at least give it a try.

You can learn more about the **C4 model** [here.](https://c4model.com/)

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** is a blueprint for how you should structure your system.
You can follow this blueprint strictly, or you can give yourself varying levels of freedom.

But when deadlines are tight, and you start cutting corners, that beautiful software architecture you built crumbles like a house of cards.

How can you **enforce** your **software architecture**?

By writing **architecture tests**.

**Architecture tests** are automated tests that verify the structure and design of your code.

You can use them to enforce your software architecture and the direction of dependencies of your projects.

In this week's issue I'll explain how to:

- Write architecture tests
- Enforce architecture
- Enforce design rules

Let's dive in!

## Writing Architecture Tests

You write **architecture tests** the same as any unit test in your application.
There's an excellent library for writing architecture tests that already implements the boilerplate code we need to start writing tests.

We're going to use the `NetArchTest.Rules` library for writing architecture tests.

First, you have to install the NuGet package:

```powershell
Install-Package NetArchTest.Rules
```

And now you can use it to write rules in your test project.

The starting point for writing architecture tests is the static `Types` class, which you can use to load a set of types.

Once you have loaded your types you can further filter them to find a more specific set of types.

Some of the available filtering methods:

- `ResideInNamespace`
- `AreClasses`
- `AreInterfaces`
- `HaveNameStartingWith`
- `HaveNameEndingWith`

Finally, when you are satisfied with your selection, you can write the rule you want to enforce by calling `Should` or `ShouldNot`
and applying the condition you want to check.

Here's an example checking that all classes in the domain assembly are sealed:

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .That()
  .AreClasses()
  .Should()
  .BeSealed()
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Architecture Rules

**Architecture tests** are particularly useful to **enforce software architecture rules** in a layered architecture or **Modular Monolith**.

Let's take the example of the Clean architecture:

- Domain should not have any dependencies
- Application should not depend on Infrastructure
- Infrastructure should depend on Application and Domain

Here's how you can write tests for enforcing architecture rules.

**Domain should not have any dependencies**

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .ShouldNot()
  .HaveDependencyOnAny("Application", "Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Application should not depend on Infrastructure**

```csharp
var result = Types
  .InAssembly(AplicationAssembly)
  .Should()
  .NotHaveDependencyOn("Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Infrastructure should depend on Application and Domain**

How the `NetArchTest.Rules` library works is by scanning the imported namespaces of your types.

Because of this, writing negative conditions like in the previous two examples is straightforward.

But writing positive conditions has to be scoped to a more specific set of types.

For example, we can validate this dependency by checking that all repositories must have a dependency on the `Domain` namespace.

```csharp
var result = Types
  .InAssembly(InfrastructureAssembly)
  .HaveNameEndingWith("Repository")
  .Should()
  .HaveDependencyOn("Domain")
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Design Rules

Another valuable **use case** for **architecture tests** is **enforcing design rules** in your application.

Design rules are more specific than project references, and focus on the implementation details of your classes.

Here are some **design rules** that you can enforce:

- Services must be internal
- Entities and Value objects must be sealed
- Controllers can't depend on repositories directly
- Command (or query) handlers must follow a naming convention

The possibilities are endless, and it's up to you how many design rules you want to enforce.

Here's how you can write tests for enforcing design rules.

**Command handlers must end with `CommandHandler`**

```csharp
var result = Types
    .InAssembly(ApplicationAssembly)
    .That()
    .ImplementInterface(typeof(ICommandHandler))
    .Should()
    .HaveNameEndingWith("CommandHandler")
    .GetResult();

Assert.True(result.IsSuccessful);
```

**Controllers can't directly reference repositories**

```csharp
var result = Types
    .InAssembly(ApiAssembly)
    .That()
    .HaveNameEndingWith("Controller")
    .ShouldNot()
    .HaveDependencyOn("Infrastructure.Repositories")
    .GetResult();

Assert.True(result.IsSuccessful);
```

## Takeaway

**Architecture tests** are an easy way to **enforce software architecture** and design rules with automated tests.

One of the best investments you can make as a software engineer is writing automated tests.
You write the tests once, and use them to verify your system forever.
Granted, you also have to maintain the tests over time as your system grows.

Manually enforcing software architecture with pair programming and constant PR reviews is:

- Error prone
- Time consuming
- Not cost effective

**Architecture tests** really shine here, since you can write them quickly and reduce the cost of enforcing your software architecture rules to zero.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

We all want to build **robust** and **reliable** applications that can scale indefinitely and handle any number of requests.

But with **distributed systems** and **microservices architectures** growing in complexity, it's becoming increasingly harder to **monitor** the **health** of our applications.

It's vital that you have a system in place to receive quick feedback of your application **health.**

That's where **health checks** come in.

**Health checks** provide a way to monitor and verify the health of various components of an application including:

- Databases
- APIs
- Caches
- External services

Here's what I'll show you in this week's newsletter:

- [What are health checks](#what-are-health-checks)
- [Adding a custom health check](#adding-custom-health-checks)
- [Using existing health check libraries](#using-existing-health-check-libraries)
- [Customizing the health checks response format](#formatting-health-checks-response)

Let's see how to implement **health checks** in **ASP.NET Core**.

## What Are Health Checks?

**Health checks** are a proactive mechanism for monitoring and verifying the **health** and **availability** of an application in **ASP.NET Core.**

ASP.NET Core has **built-in support** for implementing **health checks.**

Here's the basic configuration, which registers the health check services and adds the `HealthCheckMiddleware` to respond at the specified URL.

```csharp {3,7}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks("/health");

app.Run();
```

The health check returns a `HealthStatus` value indicating the health of the service.

There are three distinct `HealthStatus` values:

- `HealthStatus.Healthy`
- `HealthStatus.Degraded`
- `HealthStatus.Unhealthy`

You can use the `HealthStatus` to indicate the different states of your application.

For example, if the application is functioning slower than expected you can return `HealthStatus.Degraded`.

## Adding Custom Health Checks

You can create **custom health checks** by implementing the `IHealthCheck` interface.

For example, you can implement a check to see if your **SQL** database is available.

It's important to use a query that can complete quickly in the database, like `SELECT 1`.

Here's a **custom health check** implementation example in the `SqlHealthCheck` class:

```csharp
public class SqlHealthCheck : IHealthCheck
{
    private readonly string _connectionString;

    public SqlHealthCheck(IConfiguration configuration)
    {
        _connectionString = configuration.GetConnectionString("Database");
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            using var sqlConnection = new SqlConnection(_connectionString);

            await sqlConnection.OpenAsync(cancellationToken);

            using var command = sqlConnection.CreateCommand();
            command.CommandText = "SELECT 1";

            await command.ExecuteScalarAsync(cancellationToken);

            return HealthCheckResult.Healthy();
        }
        catch(Exception ex)
        {
            return HealthCheckResult.Unhealthy(
                context.Registration.FailureStatus,
                exception: ex);
        }
    }
}
```

After you implement the **custom health check**, you need to register it.

The previous call to `AddHealthChecks` now becomes:

```csharp {2}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
```

We're giving it a custom name and setting which status to use as the failure result in `HealthCheckContext.Registration.FailureStatus`.

But stop and think for a moment.

Do you want to implement a **custom health check** on your own for **every external service** that you have?

Of course not! There's a better solution.

## Using Existing Health Check Libraries

Before you start implementing a custom **health check** for everything, you should first see if there's already an **existing library.**

In the [`AspNetCore.Diagnostics.HealthChecks`](https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks) repository you can find
a wide collection **health check** packages for frequently used services and libraries.

Here are just a few examples:

- SQL Server - `AspNetCore.HealthChecks.SqlServer`
- Postgres - `AspNetCore.HealthChecks.Npgsql`
- Redis - `AspNetCore.HealthChecks.Redis`
- RabbitMQ - `AspNetCore.HealthChecks.RabbitMQ`
- AWS S3 - `AspNetCore.HealthChecks.Aws.S3`
- SignalR - `AspNetCore.HealthChecks.SignalR`

Here's how to add health checks for **PostgreSQL** and **RabbitMQ**:

```csharp {3,4}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
    .AddNpgSql(pgConnectionString)
    .AddRabbitMQ(rabbitConnectionString)
```

## Formatting Health Checks Response

By default, the endpoint returning you **health check** status will return a string value representing a `HealthStatus`.

This isn't practical if you have **multiple health checks** configured, as you'd want to view the health status individually per service.

To make matters worse, if one of the services is failing the entire response will return `Unhealthy` and you don't know what's causing the issue.

You can solve this by providing a `ResponsWriter`, and there's an existing one in the `AspNetCore.HealthChecks.UI.Client` library.

Let's install the **NuGet** package:

```powershell
Install-Package AspNetCore.HealthChecks.UI.Client
```

And you need to slightly update the call to `MapHealthChecks` to use the `ResponseWriter` coming from this library:

```csharp {3-6}
app.MapHealthChecks(
    "/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });
```

After making these changes, here's what the response from the health check endpoint looks like:

```json
{
  "status": "Unhealthy",
  "totalDuration": "00:00:00.3285211",
  "entries": {
    "npgsql": {
      "data": {},
      "duration": "00:00:00.1183517",
      "status": "Healthy",
      "tags": []
    },
    "rabbitmq": {
      "data": {},
      "duration": "00:00:00.1189561",
      "status": "Healthy",
      "tags": []
    },
    "custom-sql": {
      "data": {},
      "description": "Unable to connect to the database.",
      "duration": "00:00:00.2431813",
      "exception": "Unable to connect to the database.",
      "status": "Unhealthy",
      "tags": []
    }
  }
}
```

## Takeaway

Application monitoring is important to track availability, resource usage, and changes to performance in your application.

I've used **health checks** before to implement **failover scenarios** in a **cloud deployment**.
When one application instance stops responding with a healthy result, a new one is created to continue serving requests.

It's easy to monitor the health of your ASP.NET Core applications by **exposing health checks** for your services.

You can decide to implement **custom health checks**, but first consider if there are **existing solutions**.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

What happens when a **message is retried** in an event-driven system?

It happens more often than you think.

The **worst case scenario** is that the **message is processed twice**, and the **side effects** can also be applied more than once.

Do you want your bank account to be double charged?

I'll assume the answer is no, of course.

You can use the **Idempotent Consumer** pattern to solve this problem.

In this week's issue I will show you:

- How the Idempotent Consumer pattern works
- How to implement an Idempotent Consumer
- The tradeoffs you need to consider

Let's see why the **Idempotent Consumer** pattern is valuable.

## How The Idempotent Consumer Pattern Works

What's the idea behind the **Idempotent Consumer pattern**?

> An idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

We want to avoid handling the same message more than once.

This would require **Exactly-once** message delivery guarantees from our messaging system.
And this is a really hard problem to solve in distributed systems.

A looser delivery guarantee is **At-least-once**, where we are aware that retries can happen and we can receive the same message more than once.

The **Idempotent Consumer** pattern works well with **At-least-once** message delivery, and solves the **problem of duplicate messages.**

Here's what the algorithm looks like from the moment we receive a message:

1. Was the message already processed?
2. If yes, it's a duplicate and there's nothing to do
3. If not, we need to handle the message
4. We also need to store the message identifier

<Image
  src="blogs/mnw_034/idempotent_consumer_algorithm.png"
  width={400}
  height={634}
/>

We need a **unique identifier** for every **message** we receive, and a table in the database to store processed messages.

However, it's interesting how we choose the implement the message handling and storing of the processed message identifier.

You can implement the idempotent consumer as a decorator around a regular message handler.

I'll show you two implementations:

- Lazy idempotent consumer
- Eager idempotent consumer

## Lazy Idempotent Consumer

The **lazy idempotent consumer** matches the flow shown in the algorithm above.

Lazy refers to how we store the message identifer to mark it as processed.

In the happy path, we handle the message and store the message identifier.

If the message handler throws an exception, we never store the message identifier and the consumer can be executed again.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        if (_messageRepository.IsProcessed(message.Id))
        {
            return;
        }

        await _decorated.Handle(message);

        _messageRepository.Store(message.Id);
    }
}
```

## Eager Idempotent Consumer

The **eager idempotent consumer** is slightly different from the lazy implementation, but the end result is the same.

In this version, we eagerly store the message identifier in the database and then continue to handle the message.

If the handler throws an exception, we need to perform cleanup in the database and remove the eagerly stored message identifier.

Otherwise, we risk leaving the system in an inconsistent state since the message was never handled correctly.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        try
        {
            if (_messageRepository.IsProcessed(message.Id))
            {
                return;
            }

            _messageRepository.Store(message.Id);

            await _decorated.Handle(message);
        }
        catch (Exception e)
        {
            _messageRepository.Remove(message.Id);

            throw;
        }
    }
}
```

## In Summary

Idempotency is an interesting problem to solve in a software system.

Some operations are **naturally idempotent**, and we don't need the overhead of the **Idempotent Consumer** pattern.

However, for those operations that aren't naturally idempotent, the **Idempotent Consumer** is a great solution.

The high-level algorithm is simple, and you can take two approaches in the implementation:

- Lazy storing of message identifiers
- Eager storing of message identifiers

I prefer to use the **lazy approach**, and only **store the message identifier** in the database when the **handler completes successfully.**

It's easier to reason about and there is one less call to the database.

Thanks for reading.

Hope that was helpful.

---



<Sponsorships>

This week's issue is sponsored by the [**Why You Should Join Newsletter.**](https://whyyoushouldjoin.substack.com/)
[**Why You Should Join**](<(https://whyyoushouldjoin.substack.com/)>) is a free, monthly newsletter doing deep dives on the most promising startups to join.
They receive inside info from Sequoia, Index, and [**other top VCs**](https://whyyoushouldjoin.substack.com/about) about which companies are doing best and recommend just one each month.
Prior suggestions [**Warp**](https://whyyoushouldjoin.substack.com/p/warp) and [**Pinecone**](https://whyyoushouldjoin.substack.com/p/pinecone)
have since raised significant follow-on funding - [**subscribe for free**](https://whyyoushouldjoin.substack.com/) so you don't miss the next Uber or Airbnb.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

**EF Core** is getting many new and exciting features in the upcoming version.

**EF7** introduced support for returning **scalar types** using **SQL** queries.

And now we're getting support for **querying unmapped types** with **raw SQL queries** in **EF8.**

This is exactly what **Dapper** offers out of the box, and it's good to see **EF Core** catching up.

In this week's newsletter, I'm going to cover how to use **EF Core** for:

- [Raw SQL queries](#ef-core-and-sql-queries)
- [Composing SQL queries with LINQ](#composing-sql-queries-with-linq)
- [Executing data modifications with SQL](#sql-queries-for-data-modifications)

Let's dive in!

## EF Core And SQL Queries

**EF7** added support for **raw SQL queries** returning scalar types.
**EF8** is taking this a step further with raw SQL queries that can return any mappable type, without having to include it in the **EF model**.

You can query unmapped types with the `SqlQuery` and `SqlQueryRaw` methods.

The `SqlQuery` method uses string interpolation to parameterize the query, protecting against SQL injection attacks.

Here's an example query returning an `OrderSummary` list:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .ToListAsync();
```

This will be the **SQL** sent to the database:

```sql
SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
```

The type used for the query result can have a parameterized constructor.
The property names don't need to match the column names in the database, but they do have to match the names of the values in the result set.

You can also execute raw SQL queries and return results with:

- Views
- Functions
- Stored procedures

## Composing SQL Queries With LINQ

An interesting thing about `SqlQuery` is that it returns `IQueryable`, which can be further composed with **LINQ.**

You can add a `Where` statement after calling `SqlQuery`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>("SELECT * FROM OrderSummaries AS o")
    .Where(o => o.CreatedOn >= startDate)
    .ToListAsync();
```

However, the generated **SQL** isn't optimal:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o
) AS s
WHERE s.CreatedOn >= @p0
```

Another possibility is to combine an `OrderBy` statement with `Skip` and `Take`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .OrderBy(o => o.Id)
    .Skip(10)
    .Take(5)
    .ToListAsync();
```

This would be the generated **SQL** for the previous query:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
) AS s
ORDER BY s.Id
OFFSET @p1 ROWS FETCH NEXT @p2 ROWS ONLY
```

In case you're wondering, the performance is similar to **LINQ** queries using `Select` projections.

I ran some benchmarks, and didn't notice any significant performance improvement.

This feature will be very useful if you're more comfortable with writing **SQL** or you want to fetch data from views, functions, and stored procedures.

## SQL Queries For Data Modifications

If you want to modify data in the database with **SQL**, you will typically write a query that doesn't return a result.

The SQL query can be an `UPDATE` or `DELETE` statement, or even a stored procedure call.

You can use the `ExecuteSql` method to execute this type of query with **EF Core**:

```csharp
var startDate = new DateOnly(2023, 1, 1);

dbContext.Database.ExecuteSql(
    $"UPDATE Orders SET Status = 5 WHERE CreatedOn >= {startDate}");
```

`ExecuteSql` also protects from SQL injection by parameterizing arguments, just like `SqlQuery`.

With **EF7** you can write the above query with **LINQ** and the `ExecuteUpdate` method.
There's also the `ExecuteDelete` method for deleting records.

## In Summary

**EF7** introduced support for raw SQL queries returning **scalar** values.

**EF8** will add support for **raw SQL queries** returning **unmapped types** with `SqlQuery` and `SqlQueryRaw`.

I like the direction that **EF** is going, introducing more flexibility for querying the database.

The performance isn't as good as **Dapper**, unfortunately.
But it's close enough that network costs will play the bigger factor.

I will probably be using only **EF** moving forward since it covers more use cases.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

Rate limiting is a technique to limit the number of requests to a server or an API.

A limit is introduced within a given time period to prevent server overload and protect against abuse.

In ASP.NET Core 7, we have a built-in rate limiter middleware that's easy to integrate into your API.

We're going to cover four rate limiter algorithms:

- [Fixed window](#fixed-window-limiter)
- [Sliding window](#sliding-window-limiter)
- [Token bucket](#token-bucket-limiter)
- [Concurrency](#concurrency-limiter)

Let's see how we can work with rate limiting.

## What Is Rate Limiting?

Rate limiting is about restricting the number of requests to an API, usually within a specific time window or based on other criteria.

This is practical for a few reasons:

- Prevents overloading of servers or applications
- Improves security and guards against DDoS attacks
- Reduces costs by preventing unnecessary resource usage

In a multi-tenant application, each unique user can have a limitation on the number of API requests.

## Configuring Rate Limiting

ASP.NET Core 7 introduced built-in rate limiting middleware in the `Microsoft.AspNetCore.RateLimiting` namespace.

To add rate limiting to your application, you first need to register the rate limiting services:

```csharp
builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // We'll talk about adding specific rate limiting policies later.
});
```

I suggest updating the `RejectionStatusCode` to `429 (Too Many Requests)` because it's more correct.
The default value is `503 (Service Unavailable)`.

And you also have to apply the `RateLimitingMiddleware`:

```csharp
app.UseRateLimiter();
```

That's everything you'll need.

Let's see the rate limiting algorithms we can use.

## Fixed Window Limiter

The `AddFixedWindowLimiter` method configures a fixed window limiter.

The `Window` value determines the time window.

When a time window expires, a new one starts, and the request limit is reset.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Sliding Window Limiter

The sliding window algorithm is similar to the fixed window, but it introduces segments in a window.

Here's how it works:

- Each time window is divided into multiple segments
- The window slides one segment each segment interval
- The segment interval is (window_time)/(segments_per_window)
- When a segment expires, the requests taken in that segment are added to the current segment

The `AddSlidingWindowLimiter` method configures a sliding window limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddSlidingWindowLimiter("sliding", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.SegmentsPerWindow = 2;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Token Bucket Limiter

The token bucket algorithm is similar to the sliding window, but instead of adding back the requests from the expired segment,
a fixed number of tokens are added after each replenishment period.

The total number of tokens can never exceed the token limit.

The `AddTokenBucketLimiter` method configures a token bucket limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 100;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
        options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        options.TokensPerPeriod = 20;
        options.AutoReplenishment = true;
    });
});
```

When `AutoReplenishment` is `true`, an internal timer will execute every `ReplenishmentPeriod` and replenish the tokens.

## Concurrency Limiter

The concurrency limiter is the most straightforward algorithm, and it just limits the number of concurrent requests.

The `AddConcurrencyLimiter` method configures a concurrency limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddConcurrencyLimiter("concurrency", options =>
    {
        options.PermitLimit = 10;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

There's no time component involved in this case. The only parameter is the number of concurrent requests.

## Using Rate Limiting In Your API

Now that we have configured our rate limiting policies, let's see how we can use them in our API.

There are slight differences between controllers and minimal API endpoints, so I'll cover them in separate examples.

**Controllers**

To add rate limiting on a controller we use the `EnableRateLimiting` and `DisableRateLimiting` attributes.

`EnableRateLimiting` can be applied on the controller or on the individual endpoints.

```csharp
[EnableRateLimiting("fixed")]
public class TransactionsController
{
    private readonly ISender _sender;

    public TransactionsController(ISender sender)
    {
        _sender = sender;
    }

    [EnableRateLimiting("sliding")]
    public async Task<IActionResult> GetTransactions()
    {
        return Ok(await _sender.Send(new GetTransactionsQuery()));
    }

    [DisableRateLimiting]
    public async Task<IActionResult> GetTransactionById(int id)
    {
        return Ok(await _sender.Send(new GetTransactionByIdQuery(id)));
    }
}
```

In the previous example:

- All endpoints in the `TransactionsController` will use a **fixed window** policy
- The `GetTransactions` endpoint will use a **sliding window** policy
- The `GetTransactionById` endpoint won't have any rate limiting applied

**Minimal APIs**

In a Minimal API endpoint you can configure the rate limit policy by calling `RequireRateLimiting` and specifying the policy name.

We're using the **token bucket** policy in this example.

```csharp
app.MapGet("/transactions", async (ISender sender) =>
{
    return Results.Ok(await sender.Send(new GetTransactionsQuery()));
})
.RequireRateLimiting("token");
```

## Closing Thoughts

It's great that we can quickly introduce **rate limiting** in ASP.NET Core.

You can choose from one of the existing rate limiter algorithms:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here are some resources if you want to learn more about rate limiting:

- [Rate Limiting pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern)
- [Announcing Rate Limiting for .NET](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/)

I'm excited to try out rate limiting in my projects.

That's all for today.

Have an awesome Saturday!

---


Previous issues of The .NET Weekly