
<Sponsorships>

[**Dotnetjobs**](https://dotnet-jobs.com/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnet-jobs.com/).

</Sponsorships>

In this week's newsletter I want to talk about the `yield` keyword in **C#**.
I think it's a powerful **C#** feature and I wanted to highlight the benefits.

The `yield` keyword tells the compiler that the method in which it appears
is an **iterator block**. An iterator block, or method, returns an `IEnumerable`
as the result. And the `yield` keyword is used to return the values for the
`IEnumerable`.

An interesting thing aboug `IEnumerable` is that it is lazily evaluted.
Calling a method with an iterator block doesn't run any code. It's only
when the `IEnumerable` is iterated over, or enumerated, that we get
the actual values. I'll talk about this more later.

Let's see how we can start using the `yield` keyword!

## How To Use The Yield Keyword

The `yield` keyword on it's own doesn't do anything, you have to combine
it with the `return` or `break` statement:

- `yield return` - provides the next value of the iterator
- `yield-break` - signals the end of iteration

In every project I worked on, there's a piece of code similar to the
following. You create a list to hold the results, add elements to the
list, and return the list in the end.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    var result = new List<SoftwareEngineer>();

    for(var i = 0; i < 10; i++)
    {
        result.Add(new SoftwareEngineer
        {
            Id = i
        });
    }

    return result;
}
```

You can simplify the method using the `yield return` statement, and
completely remove the intermediate list required to hold the results.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    for(var i = 0; i < 10; i++)
    {
        yield return new SoftwareEngineer
        {
            Id = i
        };
    }
}
```

However, it's important to note these two implementation are fundamentally
different from each other. In the first example, the entire list is
populated and materialized. In the second example, the `IEnumerable`
returned will not be materialized and you have to either iterate over
it inside a `foreach` loop or call `ToList()`.

## Stopping Iteration With Yield Break

You can use the `yield break` statement to stop iteration and exit
the iterator block. Typically you would do this when a certain
condition is met, or you only want to return a specific set of values
from the iterator block.

Here's an example where this would be useful:

```csharp
Console.WriteLine(string.Join(", ", TakeWhilePositive(new[] { 1, 2, -3, 4 })));
// Output: 1, 2

public IEnumerable<int> TakeWhilePositive(IEnumerable<int> numbers)
{
    foreach(int num in numbers)
    {
        if (num > 0)
        {
            yield return num;
        }
        else
        {
            yield break;
        }
    }
}
```

## Working With IAsyncEnumerable

In **C# 8** we got the `IAsyncEnumerable` type which allows us to
iterate over a collection asynchronously with the `yield` statement.

For example, this can be useful when you want to call a thid-party
API multiple times to fetch some data. A common situation is when
you get a list of users from the database, and then have to call
an external storage service to get profile picture information.

Without `IAsyncEnumerable` you would have to do something like this:

```csharp
public async Task<IEnumerable<User>> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);
    }

    return users;
}

// And you would call the method like this.
var users = await GetUsersAsync();

foreach(var user in users)
{
    Console.WriteLine(user);
}
```

Now, consider this same example with the use of `IAsyncEnumerable`:

```csharp
public async IAsyncEnumerable<User> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);

        yield return user;
    }
}

// And you would call the method like this.
await foreach(var user in GetUsersAsync())
{
    Console.WriteLine(user);
}
```

The second implementation will iterate over the users returned from
the database when they are yielded by the `IAsyncEnumerable`.

## When Should I Use Yield?

I've found a few interesting practical applications for the `yield` keyword.
One example is when implementing Domain-Driven Design value objects.

Value objects need to support structural equality. They need to implement
a method that returns all of the equality components. Here's an example of
that using the `yield return` statement:

```csharp
public class Address
{
    public string City { get; init; }

    public string Street { get; init; }

    public string Zip { get; init; }

    public string Country { get; init; }

    public IEnumerable<object> GetEqualityComponents()
    {
        yield return City;
        yield return Street;
        yield return Zip;
        yield return Country;
    }
}
```

---



<Sponsorships>

Have some [**C# and .NET experience**](https://packt.link/PJqza), and want to get started building apps?
Get introduced to the most common technologies like gRPC, Blazor, GraphQL, Cosmos DB, SignalR, .NET MAUI, and more in
[**Apps and Services with .NET 7**](https://packt.link/PJqza).

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to introduce you to an interesting feature
in **EF Core** called **Compiled Queries**.

If you have queries that you execute frequently in your application with a
different set of parameters, it can be helpful to explicitly compile the
query and reuse it throughout the lifetime of your application.

**Compiled Queries** are more performant than standard EF queries, because they
can take advantage of some additional optimizations.

Let me show you how to use **Compiled Queries**, and how much performance
improvement to expect!

## How To Create Compiled Queries

Lets define a simple class that will represent out data model that we will
use for writing **EF** queries:

```csharp
public class Newsletter
{
    public long Id { get; init; }

    public string Title { get; init; }

    public int ReadTimeInMinutes { get; init; }
}
```

How would we write a simple query to fetch a `Newsletter` by the `Id`?
I think you can do this in your sleep.

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id);
```

Now, how do we convert this query into a **Complied Query**?

There are a few steps involved:

- Create a **Compiled Query** by calling `EF.CompileQuery`
- Store the **Compiled Query** in a static field, so that it can be reused
- Execute the database query using the **Compiled Query**

You can define the **Compiled Query** in a static field inside of the `AppDbContext`.
And then expose a method that will accept an argument, and pass it to the
**Compiled Query** to invoke it.

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, long, Newsletter?> GetNewsletter =
        EF.CompileQuery(
            (dbContext, id) =>
                dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id));

    public Newsletter? GetNewsletter(long id)
    {
        return GetNewsletter(this, id);
    }
}
```

This is how we would call the method which invokes the **Compiled Query**:

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.GetNewsletter(id);
```

I ran some benchmarks, with the following setup:

- **EF Core 7**
- **SQL Server 2022**
- Table with 10,000 records

The **Compiled Query** was consistently around **10% faster**.

I also tried running a no-tracking query by calling `AsNoTracking()` and
observed similar results.

## Why Are Compiled Queries Faster?

So we can conclude that **Compiled Queries** are faster. But why is that?

Let's examine what happens when we execute an **EF** LINQ query. Before **EF**
can convert the query into valid SQL that can be executed in the database,
it needs to compile the query. The compiled query is cached and **EF** will be
able to reuse that cached query. In some situations the query needs to be
recompiled, introducing additional performance costs.

When we explicitly compile the query by calling `EF.CompileQuery`, we
can utilize some optimization techniques that aren't available at runtime.

Note that **Compiled Queries** only improve the performance of the in-memory
portion of executing an EF query. The round trip time and materializing
results from the database remain unaffected.

## Can We Make Compiled Queries Asynchronous?

I showed you how to write a synchronous **Compiled Query**. But due to
performance considerations we almost always want to execute database
queries asynchronously.

Here's how we can create an asynchronous **Compiled Query**:

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, string, Task<Newsletter?>> GetByTitle =
        EF.CompileAsyncQuery(
            (AppDbContext context, string title) =>
                context.Set<Newsletter>().FirstOrDefault(c => c.Title == title));

    public async Task<Newsletter?> GetNewsletterByTitleAsync(string title)
    {
        return await GetByTitle(this, title);
    }
}
```

It's interesting that we aren't writing an asynchronous query in the expression
passed to `EF.CompileAsyncQuery`. It will be converted to an asynchronous query
during compilation.

## Compiled Queries Aren't a Silver Bullet

You might be tempted to go and convert all of your **EF** queries into
**Compiled Queries**, to squeeze out that last little bit of performance.
I urge you not do it. **Compiled Queries** are a useful tool, but they
aren't the solution to all your problems.

Instead, I think we should use **Compiled Queries** sparingly, only in
situations where we really need to do these kinds of micro-optimizations.

Although **Compiled Queries** seem great, we can't deny they increase
the complexity of our code considerably. If you think the slight
performance improvement gained from using **Compiled Queries** justifies
the increase in complexity, then by all means, you should use them.
Otherwise, I would look for other ways to improve performance.

---



<Sponsorships>

[**Dotnetjobs**](https://dotnet-jobs.com/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnet-jobs.com/).

</Sponsorships>

In this week's newsletter I will show you how to easily add **validation**
to the strongly typed configuration objects injected with `IOptions`.

The **Options pattern** allows us to use classes to provide strongly typed
configuration values in our application at runtime.

But you have no guarantee that the configuration values injected with
`IOptions` will be correctly read from the application settings.

Let's see how we can introduce validation for `IOptions`, and make sure the application settings are correct.

## Strongly Typed Configuration

I first want to define a simple class that will represent our strongly
typed configuration. Let's say we want to integrate with the **GitHub API**,
so we create a `GitHubSettings` class to hold our configuration:

```csharp
public class GitHubSettings
{
    public string AccessToken { get; init; }

    public string RepositoryName { get; init; }
}
```

Inside of our `appsettings.json` file we need to create a section to
hold our configuration values:

```json
"GitHubSettings": {
    "AccessToken": "access-token-value",
    "RepositoryName": "youtube-projects"
}
```

And with this in place, we can configure our `GitHubSettings`:

```csharp
builder.Services.Configure<GitHubSettings>(
    builder.Configuration.GetSection("GitHubSettings"));
```

Finally, our `GitHubSettings` is properly configured and we can inject
it with `IOptions<GitHubSettings>`.

## What Could Go Wrong?

If we leave the implementation like this, we're moving the responsibility
for providing the correct configuration values to the developer. I'm not
saying we are the problem, but I've forgotten to add application settings
a few times. I'm sure this happened to you also.

Here are just a few things that can go wrong:

- Passing an incorrect section name to `IConfiguration.GetSection`
- Forgetting to add the settings values in `appsettings.json`
- Typo in a property name in the class or in the configuration
- Unbindale properties without a setter
- Data type mismatch resulting in incompatible values

Depending on which one of these mistakes is made, the application will
behave differently at runtime.

The best case scenario is that the incorrect application settings cause
a runtime exception, and you realize you have a problem and fix it.

The worst case scenario, and this happens more often than you may think,
is that the application silently fails. The application settings aren't
correctly set on the value provided by `IOptions`, but you don't get a
runtime exception. The problem may go undetected for some time.

How do we solve this?

## Validation For The Options Pattern

There is a simple way to introduce **validation** to the settings class
using **data annotations**. We just add the validation attributes that
we need to the properties of the settings class.

For example, we can add the `Required` attribute to the `GitHubSettings`
properties:

```csharp
public class GitHubSettings
{
    [Required]
    public string AccessToken { get; init; }

    [Required]
    public string RepositoryName { get; init; }
}
```

We have to slightly change how we configure the `GitHubSettings`:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations();
```

A few things to note here:

- `AddOptions` - returns an `OptionsBuilder<TOptions>` that binds to
  the `GitHubSettings` class
- `BindConfiguration` - binds the values from the configuration section
- `ValidateDataAnnotations` - enables **validation** using **data annotations**

With this in place, if we try to inject `GitHubSettings` with any of
the properties missing a value, we will get a runtime exception.

You can also define a **custom delegate** for the **validation** logic, instead
of using data annotations:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .Validate(gitHubSettings =>
    {
        if (string.IsNullOrEmpty(gitHubSettings.AccessToken))
        {
            return false;
        }

        return true;
    });
```

## Running Validation At Application Start

It would be great if we could run **validation** on the configuration values
when our application is starting, instead of at runtime.

We can do that by calling `ValidateOnStart` method when configuring
our settings class:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations()
    .ValidateOnStart(); // 👈 the magic happens here
```

When we start the application, the validation will run on `GitHubSettings`
and an exception is thrown if validation fails. The validation exception
will look something like this:

```yaml
Unhandled exception. Microsoft.Extensions.Options.OptionsValidationException:
  DataAnnotation validation failed for 'GitHubSettings' members:
```

This shortens the feedback loop, and you will know right away that you have
a problem. This is much better than finding out that you have a problem at
runtime, like in the previous examples.

## Closing Thoughts

The **Options pattern** is very flexible and allows us to use strongly typed
settings in ASP.NET Core.

If you want to see how to implement the [**Options pattern**](https://youtu.be/wxYt0motww0),
I made a [**video about it where I go into the details.**](https://youtu.be/wxYt0motww0)
I covered the differences between `IOptions`, `IOptionsSnapshot` and `IOptionsMonitor`.

And now you know how to use the `ValidateOnStart` method, which was introduced
in **.NET 6**, to validate your application settings on app start up. This allows
you to learn about configuration issues as soon as possible, instead of at runtime.

I also made a video showing how to add [validation to the Option pattern](https://youtu.be/qRruEdjNVNE).

---



In this week's newsletter I will share 5 simple tips on how
you can be a better software engineer in 2023.

I find it a little amusing that the last newsletter of the year
and is also coming out on the last day of the year.

Here are 5 tips for being a better software engineer in 2023:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

I'm confident that you will become a better software engineer if you apply one of these, but I challenge you to work on all of them in the coming year.

Let's dive in.

## 1. Keep Learning And Acquiring New Skills

The field of software engineering is constantly evolving, so it's
important to stay up-to-date with [**new technologies**](https://youtu.be/dDasAmowFts)
and best practices.

We have a new .NET release every year, and it's easy to get lost
with the latest news. How I stay up to date is through online courses,
attending conferences and meetups, or working on [**personal projects**](https://youtu.be/Ru6_b50wdfo).

Personal projects transition best to my day job, and I don't shy away
from exploring new topics. I shared many of my personal projects on [**GitHub**](https://github.com/m-jovanovic),
maybe you can find some inspiration over there.

## 2. Invest In Code Quality

In addition to writing clean and readable code, it's important to also
consider the overall quality of your code.

This includes things like performance, security, and maintainability.
By writing high-quality code, you'll be able to build systems that are
more reliable, scalable, and easier to maintain over time.

One way to write clean and quality code is [**with the help of static code analysis**](https://youtu.be/0nVT1gM4vPg).

Investing in code quality will pay dividends in the later stages of any project.

## 3. Work On Complex Systems

If you aspire to be senior engineer or software architect, you have to
work on complex systems. You need to be in a position to solve the toughest problems.

What do I consider to be a [**complex system**](https://youtu.be/Ru6_b50wdfo)?

That's difficult to say, but here are some rough guidelines:

- Microservices
- Event-driven systems
- High performance systems

Of course, you don't need a [**fancy architecture**](https://youtu.be/Ru6_b50wdfo) to work in a complex system.

If you are working in a business domain with many domain rules,
I consider that a complex system also.

You should strive to always be in a position to work on a challenging project,
this will help you grow.

## 4. Be Comfortable In The Cloud

The cloud is here to stay, and you should be familiar with at least one
of the major cloud providers:

- Microsoft Azure
- Amazon Web Services
- Google Cloud Platform

Most of them give you free credits to get started and explore the services they offer.

I stayed away from [**cloud development**](https://youtu.be/QP0pi7xe24s) for too long in my career, and now I wish I started sooner.

You want to be [**comfortable in the cloud**](https://youtu.be/QP0pi7xe24s) to be a better software engineer.

## 5. Take Care Of Yourself

Being a software engineer is mentally and physically demanding, and it's
important to take care of yourself in order to maintain a healthy work-life balance.

This includes getting enough sleep, eating well, and taking breaks when needed.

Taking care of yourself can also help you stay focused and productive in your work.

I like to take short breaks every hour or so, and not think about work for a few minutes.
This helps me replenish my energy, and allows me to continue to operate on a high level.

## What I'm Doing To Make 2023 Amazing

What I'm Doing To Make 2023 Amazing
Making New Year's resolutions is popular at the start of the year. The problem is most people get too excited about it, but then proceed to not accomplish anything in a few months.

I've found the simplest rule to always make progress is taking action. I try to make a small step forward every day. And when I look back on the year, I realize I made a lot of progress.

Again, my 5 tips so you can be a better software engineer in 2023 are:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

Where I failed most in 2022 was taking care of myself, and this will be one of my main improvement points.

I also want to learn many new things, so I can share them with you in this newsletter and on my social media channels.

I wish you a very happy and prosperous New Year.
Write lots of code, squash many bugs,
and may you have green unit tests year round.

Stay awesome! 🎁\
Milan

---



<Sponsorships>

Coordinating deployments across 37,000 stores and 500,000 database
instances for 6+ years with zero incidents? Impossible! But it's
the reality for one Fortune 100 fast-food empire who chose a NoSQL
point-of-sale database designed to smartly scale.
[**Learn how RavenDB excels as a point-of-sale DB**](https://ravendb.net/whitepapers/ravendb-point-of-sale-database/?utm_campaign=mnw_newsletter_17&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_whitepaper&utm_content=ad_link).

</Sponsorships>

In the world of software development, there are countless approaches
and methodologies to choose from. It's easy to get swayed with the
latest trends, and loose sight of architectural principles that really matter.

One of the more popular ones is **Clean Architecture**, a design approach
that prioritizes maintainability, scalability, flexibility, and productivity.

In this week's newsletter, we will explore the key benefits of using
**Clean Architecture** and how it can help your team build better software.

Let's dive in.

## What Is Clean Architecture?

**Clean Architecture**, also known as "The Onion Architecture," was first
introduced by Robert C. Martin (aka "Uncle Bob") in his book
"Clean Architecture: A Craftsman's Guide to Software Structure and Design".

At its core, **Clean Architecture** is a way of organizing a software system
in a way that separates the concerns of the various components,
making it easier to understand and maintain.

In **Clean Architecture**, the core of the system is the **"inner circle"**,
which contains the business rules and logic.

Surrounding this **inner circle** are layers of abstraction,
each one representing a different concern.

<Image src="blogs/mnw_017/clean_architecture.png" width={1920} height={1080} />

The typical outer layers are **Infrastructure** and **Presentation** layers.
The **Infrastructure** layer handles external concerns such as APIs and databases.
While the **Presentation** layer exposes an interface for clients to interact with
the application.

The key principle of **Clean Architecture** is that the **inner circle**
should not depend on the outer layers. Instead, the outer layers should
depend on the **inner circle**. This helps to ensure that the **core** of the
system is flexible and easy to modify, without worrying about the impact
on other parts of the system.

## Benefits Of Using Clean Architecture

I want to highlight some of the key benefits of using **Clean Architecture**.

### Improved Maintainability

One of the primary benefits of using **Clean Architecture** is improved **maintainability**.
By separating the concerns of the various components and enforcing the dependency rule,
it becomes much easier to understand and modify the code.
Depending on abstractions allows you to design your business logic in a flexible way,
without having to know the implementation details.

### Modularity and Separation of Concerns

**Clean Architecture** helps to create a clear **separation of concerns**
within the codebase. Each layer has a specific purpose and is decoupled
from the others, making it easier to understand and modify individual
components without affecting the rest of the system. This modularity
also makes it easier to reuse components in other projects.

### Testability

**Clean Architecture** also makes it **easier to test** and debug the code.
Because the inner circle is independent of the outer layers,
it's easier to write unit tests that focus specifically on the business
rules. This can help to catch errors early on in the development
process and reduce the overall testing effort.

### Loose Coupling of Components

**Clean Architecture** also promotes **loose coupling** between the various
components of the system. This means that it's easier to swap out
external dependencies or make other modifications without affecting
the core business logic. This can be especially useful when it comes
to upgrading or replacing technology.

### Increased Flexibility

Another key benefit of **Clean Architecture** is increased **flexibility**.
By separating the concerns of the various components, it's easier
to modify and adapt the code to changing requirements. This can be
especially useful in fast-paced environments where requirements
are constantly evolving.

### Improved Team Productivity

**Clean Architecture** can help to improve team **productivity**.
By establishing clear separation of responsibilities and well-defined
boundaries, it's easier for team members to understand their roles
and responsibilities. This can improve communication and collaboration,
leading to more efficient and effective work.

## Clean Architecture In The Real World

This all sounds nice in theory, but how does **Clean Architecture**
perform in the real world?

I have used **Clean Architecture** on roughly 10 projects in the
last 5 years, and I've had a lot of success with it. It was
easy to add new features, and scale the applications when
necessary. **Clean Architecture** can easily be broken down
into multiple modules or services, if performance is suffering
and there is a need to scale out.

One problem with the **Clean Architecture** is that it is **easy to overengineer**.

Dogmatism is a real issue, as I see many people with strong
opinions of what **Clean Architecture** should be.
I've been guilty of this myself in the past.

Recently, I try to be more **pragmatic** when using **Clean Architecture**.
I apply what I like, and give myself the flexibility of **"breaking"**
**Clean Architecture** if I think it will simplify things in the long run.

## Closing Thoughts

By following the principles of **Clean Architecture**, you can create a flexible
and maintainable codebase that is well-suited to evolving requirements and technology.

However, it's important to be **pragmatic** with **Clean Architecture** and
allow yourself to be flexible in the design, in order to simplify things in the long run.

If you want to see how to apply **Clean Architecture** in practice,
I have a
[**playlist with more than 20 videos on Clean Architecture**](https://youtu.be/tLk4pZZtiDY?list=PLYpjLpq5ZDGstQ5afRz-34o_0dexr1RGa).

---



<Sponsorships>

New to C# and .NET, or want a refresher on modern .NET?\
[**C# 11 and .NET 7 - Modern Cross-Platform Development Fundamentals**](https://packt.link/402oY)
is a concise, beginner-friendly introduction to the C# language,
.NET libraries, and ASP.NET Core.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

Did you know you can turn **PostgreSQL** into a fully-fledged **Document database**?

**Marten** is a **.NET** library that allows developers to use the **PostgreSQL**
database as both a **document database** and a fully-featured **event store**.

You don't need to install anything else to be able to use **PostgreSQL**
as a **document database**, outside of the Nuget package. **Marten** relies
on the **JSONB** support available since **PostgreSQL** 9.4.

In this week's newsletter, I want to introduce you to the basics of working
with **Marten** and show you how easy it is to get started.

Let's dive in.

## Installing And Configuring Marten

What are you going to need to start using **PostgreSQL** as a **Document datbase**?

Other than a running instance of **PostgreSQL**, of course, you will need
to install the **Marten** Nuget package:

```csharp
dotnet add package Marten
```

**Marten** can build the required database schema and necessary tables on the fly,
and I suggest using this approach in development.
For a production environment, you definitely want to apply schema
changes on your own with migration scripts.

To register **Marten** with dependency injection, you need to call the `AddMarten`
method.

Here's an example **Marten** configuration inside of a **.NET 7** application:

```csharp
builder.Services.AddMarten(options =>
{
    options.Connection(builder.Configuration.GetConnectionString("Marten"));
});
```

This will register a few services with dependency injection:

- `IDocumentStore` - used to create sessions, generate schema migrations, and do bulk inserts
- `IDocumentSession` - used for read and write operations
- `IQuerySession` - used for read operations

Let's see how we can work with documents using **Marten**.

## Storing Documents With Marten

Storing documents in the database is very straightforward. You need to create
a new `DocumentStore` instance, and open an `IDocumentSession` which exposes
methods for storing and persisting documents.

Let's see how we can store a `Product` document:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.OpenSession();

var product = new Product
{
    Name = "C# 11 and .NET 7 - Modern Cross-Platform Fundamentals",
    Price = 46.87
};

session.Store(product);

await session.SaveChangesAsync();
```

We're creating a new `DocumentStore` instance which we use to open
a session to **PostgreSQL**. And then we just call `Store` and pass in
the `Product` instance. Note that **Marten** will populate the
`Product.Id` at this point. **Marten** can populate keys of `Guid`,
`int`, `long`, and other data types. It uses the HiLo algorithm
for numeric keys. Finally, when we call `SaveChangesAsync` the
`Product` is serialized into **JSON** and persisted as a document.

An important thing to be aware of is that the `IDocumentSession`
created by `OpenSession` doesn't track changes on the entities automatically.
You need to create a session with dirty checking enabled by
calling `DirtyTrackedSession` on the `DocumentStore` to enable
automatic change detection.

## Querying Documents With Marten

**Marten** has rich support for querying documents in the database.
You can write and execute queries using **LINQ**, which you are
familiar with if you worked with **EF Core**. And you can also
write and execute **SQL** queries, because it's still a **PostgreSQL**
database underneath.

Here's an example query to return products that have a higher price
than the one which is specified:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.QuerySession();

var products = session.Query<Product>().Where(p => p.Price > 9.99).ToList();
```

**Marten** also has support for:

- Including related documents
- Batched queries
- Paging
- Full text search

## Advanced Options With Marten

**Marten** can utilize the full capabilities **PostgreSQL** has to offer,
notably transactions and indexing. **Marten** sessions are transactional
by default, either all of the documents are persisted together or
none of them are. And you can configure indexes on your documents
for faster queries.

**Marten** isn't just a **document database** on top of **PostgreSQL**!

You have fully-fledged support for **event sourcing** with **Marten**,
as well as projections. This makes it a perfect choice for
implementing **CQRS**. But this is a topic for a separate newsletter.

## Closing Thoughts

I'm absolutely amazed with [Marten](https://martendb.io/) and what it has to offer.
And **PostgreSQL** is also my favorite database, so it's like a match
made in heaven. I don't get too excited about learning new
technologies, but **Marten** has been an endless source
of joy this past week.

I still need to explore a few more topics before I can consider it
for production use:

- Schema migrations
- Relationships and foreign keys
- Advanced configuration options

Considering that **PostgreSQL** is cheaper than most **document databases**,
I think using **Marten** is an interesting alternative. And if you are
familiar with **SQL** databases, you can still use all of that knowledge.

---



<Sponsorships>

Keeping your business logic and data in-sync doesn't have to be a drag.
Database configuration-as-code and auto-indexes mean your logic is always
querying against what you expect after every release.
[**Take off with RavenDB today**](https://ravendb.net/learn/bootcamp/?utm_campaign=mnw_newsletter_15&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_bootcamp&utm_content=ad_link)
and fly faster tomorrow.

</Sponsorships>

In this week's newsletter we are going to explore **Minimal APIs**,
which were introduced in **.NET 6**.

**Minimal APIs** were introduced to remove some of the ceremony
of creating traditional APIs with controllers.
To define an endpoint, you can use the new extension
methods, such as `MapGet` to define a **GET** endpoint.

I see one big issue with **Minimal APIs**, and that is the lack
of clear guidance around how to structure applications
built with **Minimal APIs**.

In this newsletter, I want to offer a few solutions for that problem.

Let's dive in.

## How To Create Minimal APIs?

Let's define a simple **Minimal API** application with two endpoints.
We're going to create one `GET` endpoint for getting a list of products.
And one `POST` endpoint for saving a product to the database.

We're using the powerful **DI** feature that allows us to inject services
as expression arguments, which you can see in the two expressions below
where we are injecting the `AppDbContext`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

var app = builder.Build();

app.MapGet("/products", async (AppDbContext dbContext) =>
{
    return Results.Ok(await dbContext.Products.ToListAsync());
});

app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
{
    dbContext.Products.Add(product);

    await dbContext.SaveChangesAsync();

    return Results.Ok(product);
});

app.Run();
```

And with this we have a functioning **Minimal API** that we can develop further
as we continue to add more endpoints.

## The Problem With Maintaining Minimal APIs

There is one potential problem with structuring our **Minimal APIs** like
in the previous example. If we keep adding the **Minimal API** endpoints
in the same file, our API will become hard to maintain as it grows
in complexity. How can we solve the maintance problem with **Minimal APIs**?

One solution can be to use extension methods to encapsulate the
definiton of the **Minimal API** endpoints.

Here's an example of that:

```csharp
public static class ProductsModule
{
    public static void RegisterProductsEndpoints(this IEndpointRouteBuilder  endpoints)
    {
        endpoints.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        endpoints.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

And then inside of `Program` we need to register the endpoints:

```csharp
app.RegisterProductsEndpoints();
```

You can see that this simplifies our **Minimal API** definition,
and we also have our endpoints grouped by feature in one place.
I think this improve the maintainability of **Minimal APIs**,
but it comes at a cost. And that cost is having to define
extension methods for each group of endpoints you want to encapsulate,
and then you have to remember to call that extensions method in `Program`.

Can we do better?

## Structuring Minimal API Projects With Modules

I want to introduce you to an interesting open source library
called [Carter](https://github.com/CarterCommunity/Carter),
which has a concept of modules that we can use to group endpoints.

Here's how we can define our `ProductsModule` with **Carter**:

```csharp
public class ProductsModule : ICarterModule
{
    public void AddRoutes(IEndpointRouteBuilder app)
    {
        app.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

This takes care of configuring our **Minimal API** endpoints, but we still
need to tell the framework to use these endpoints. We have to slightly
modify the `Program` to register the required **Carter** services by
calling `AddCarter`, and also map our endpoints by calling `MapCarter`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

builder.Services.AddCarter();

var app = builder.Build();

app.MapCarter();

app.Run();
```

When we want to define additional **Minimal API** endpoints we just need to
implement a new `ICarterModule`, and register our endpoints. **Carter** will
automatically take care of registering the new endpoints after that.

## Would I Use Minimal APIs In a Real Project?

I think **Minimal APIs** have evolved nicely since they were first introduced
in **.NET 6**. I would be careful with using them in very large applications,
but I'm definitely going to explore options for using them on smaller projects.

A good use case can be for building a microservice that has a limited
number of endpoints.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_14&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter we will talk about running **background tasks** in **ASP.NET Core**.
After reading this newsletter, you will be able to set up a **background task**
and have it up and running within minutes.

**Background tasks** are used to offload some work in your application to the background,
outside of the normal application flow. A typical example can be asynchronously
processing messages from a queue.

I will show you how to create a simple **background task** that runs once and completes.

And you will also see how to configure a continuous **background task**, that repeats after a specific period.

Let's dive in.

## Background Tasks With IHostedService

You can define a **background task** by implementing the `IHostedService` interface.
It has only two methods.

Here's what the `IHostedService` interface looks like:

```csharp
public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);

    Task StopAsync(CancellationToken cancellationToken);
}
```

All you have to do is implement the `StartAsync` and `StopAsync` methods.

Inside of `StartAsync` you would usually perform the background processing.
And inside of `StopAsync` you would perform any cleanup that is necessary,
such as disposing of resources.

To configure the **background task** you have to call the `AddHostedService` method:

```csharp
builder.Services.AddHostedService<MyBackgroundTask>();
```

Calling `AddHostedService` will configure the **background task**
as a **singleton** service.

So does dependency injection still work in `IHostedService` implementations?\
Yes, but you can only inject **transient** or **singleton** services.

However, I don't like to implement the `IHostedService` interface myself.
I prefer using the `BackgroundService` class instead.

## Background Tasks With BackgroundService

The `BackgroundService` class already implements the `IHostedService` interface,
and it has an `abstract` method that you need to override - `ExecuteAsync`.
When you are using the `BackgroundService` class, you only have to think about
the operation you want to implement.

Here's an example **background task** that runs **EF** migrations:

```csharp
public class RunEfMigrationsBackgroundTask : BackgroundService
{
    private readonly IServiceProvider _serviceProvider;

    public RunEfMigrationsBackgroundTask(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = _serviceProvider.CreateScope();

        await using AppDbContext dbContext =
            scope.ServiceProvider.GetRequiredService<AppDbContext>();

        await dbContext.Database.MigrateAsync(stoppingToken);
    }
}
```

The **EF** `DbContext` is a **scoped** service, which we can't inject directly
inside of `RunEfMigrationsBackgroundTask`. We have to inject an instance of
`IServiceProvider` which we can use to create a custom service scope,
so that we can resolve the scoped `AppDbContext`.

I would _not recommend_ running the `RunEfMigrationsBackgroundTask` in production.
**EF** migrations can easily fail and you'll run into problems.
However, I think it's perfectly fine for local development.

## Periodic Background Tasks

Sometimes we want run a **background task** continuously, and have it
perform some operation on repeat. For example, we want consume messages
from a queue every ten seconds. How do we build this?

Here's an example `PeriodicBackgroundTask` to get you started:

```csharp
public class PeriodicBackgroundTask : BackgroundService
{
    private readonly TimeSpan _period = TimeSpan.FromSeconds(5);
    private readonly ILogger<PeriodicBackgroundTask> _logger;

    public PeriodicBackgroundTask(ILogger<PeriodicBackgroundTask> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using PeriodicTimer timer = new PeriodicTimer(_period);

        while (!stoppingToken.IsCancellationRequested &&
               await timer.WaitForNextTickAsync(stoppingToken))
        {
            _logger.LogInformation("Executing PeriodicBackgroundTask");
        }
    }
}
```

We're using a [PeriodicTimer](https://learn.microsoft.com/en-us/dotnet/api/system.threading.periodictimer?view=net-6.0)
to asynchronously wait for a given period, before executing our **background task**.

## What If You Need A More Robust Solution?

It should be obvious by now that `IHostedService` is useful when you need
simple **background tasks** that are running while your application is running.

What if you want to have a scheduled **background task** that runs at 2AM every day?

You can probably build something like this yourself, but there are existing solutions
that you should consider first.

Here are two popular solutions for running **background tasks** that I worked with before:

- [Quartz](https://www.quartz-scheduler.net/)
- [Hangfire](https://www.hangfire.io/)

I also have an example of [using Quartz for processing Outbox messages](https://youtu.be/XALvnX7MPeo)
on my YouTube channel that you can take a look at.

---



<Sponsorships>

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_13&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter, we're going to explore the new
`ExecuteUpdate` and `ExecuteDelete` methods that were released with **EF7**.

`ExecuteUpdate` allows us to write a query and run a bulk update
operation on the entities matching that query.

Similarly, `ExecuteDelete` allows us to write a query and delete
the entities matching that query.

We can significantly improve performance using the new methods in some scenarios, and I'm going to show you what those scenarios are.

Let's dive in.

## Updating And Deleting Entities Before EF Core 7

If you want to update a collection of entities before **EF7**,
you need to load the entities into memory using the `DatabaseContext`.

The **EF ChangeTracker** will then track any changes made to these entities.
When you are ready to commit the changes to the database,
you simply call the `SaveChanges` method.

Here's an example where we load a few notifications,
and we want to snooze them so they aren't sent:

```csharp
var notifications = dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ToList();

foreach(var notification in notifications)
{
    notification.Snoozed = true;
}

dbContext.SaveChanges();
```

**EF7** will generate the following **SQL** statement to update the records in the database:

```sql
UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_1;

...

UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_N;
```

Notice that for every notification we end up with one **SQL UPDATE** statement.
This won't scale well as the number of notifications increases.

## Updating Entities With ExecuteUpdate

With **EF7**, we now have access to the new `ExecuteUdpate` method.
It also has an asynchronous version - `ExecuteUpdateAsync`.

How do you use it?

You need to write a query that will select the records you want to update,
and then call the `ExecuteUpdate` method on the resulting `IQueryable`.

Let's rewrite the previous example using the new approach:

```csharp
dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ExecuteUpdate(s => s.SetProperty(
        n => n.Snoozed,
        n => true));
```

In the call to `ExecuteUpdate` we call the `SetProperty` method to specify
which properties we want to update, and what values we want to set.
The `SetProperty` method can be called multiple times, if you need to update more than one property.

In this case, **EF7** will generate the following **SQL** query:

```sql
UPDATE n
SET n.[Snoozed] = TRUE
FROM [Notifications] AS n
WHERE n.[Snoozed] = FALSE;
```

Notice that this time we only have one **SQL** query being sent to the database.
This is a major performance improvement. It can be as much as 10x faster
than the old version, from my testing.

## Deleting Entities With ExecuteDelete

Let's also see how we can do bulk deletes using the `ExecuteDelete` and `ExecuteDeleteAsync` methods.

Again, you have to write a query that will select the records you want to delete,
and then call the `ExecuteDelete` method on the resulting `IQueryable`.

If you want to delete all snoozed notifications:

```csharp
dbContext
    .Notifications
    .Where(n => n.Snoozed)
    .ExecuteDelete();
```

And **EF7** will generate the following **SQL** query:

```sql
DELETE FROM n
FROM [Notifications] AS n
WHERE n.[Snoozed] = TRUE;
```

I think this will be incredibly useful when you want to
delete records in the database based on a specific condition.

## Transactions, Change Tracking And Query Filters With Bulk Methods

You need to be aware how transactions and change tracking
work with the new bulk methods. `ExecuteUpdate` and `ExecuteDelete` will
immediately go to database, and run the **SQL** query.

**What does this mean for transactions?**

If you want to run a bulk method together with other updates
applied with `SaveChanges`, by default they won't run in the same transaction.
You need to open an explicit **transaction** using the `DatabaseContext` to keep everything consistent.

**What does this mean for change tracking?**

`ExecuteUpdate` and `ExecuteDelete` run directly on the database, without loading any entities into memory.
**EF7** will not track these entities in the `ChangeTracker`.

If you have any database interceptors defined, they won't execute
after calling one of the bulk update methods.
This also means that if you override `SaveChanges`
to add custom behavior, it won't be called.

**Do Query Filters still work?**

Yes, **query filters** will be **correctly applied** when calling `ExecuteUpdate` or `ExecuteDelete`.

## When Should You Use The New Bulk Methods?

I think this is an excellent new addition to **EF7**,
and it solves a real problem when you need to run
a typical **UPDATE** or **DELETE** query with a
**WHERE** statement applied.

Previously, you had to write raw **SQL** and execute it using something like **Dapper**.

I will likely use this approach when it applies to my projects.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to show you how you can use
the powerful **options pattern** in **ASP.NET Core 7**.

The **options pattern** uses classes to provide **strongly typed settings**
in your application at runtime.

The values for the **options** instance can come from multiple sources.
The typical use case is to provide the settings from application configuration.

You can configure the **options pattern** in a few different ways in **ASP.NET Core**.
I want to discuss some of the approaches and their potential benefits.

Let's dive in.

## Creating The Options Class

I want to set the stage first, by creating the **options** class and
explaining what settings we want to bind to it.

We want to configure JWT Authentication for our application,
so we decided to create the `JwtOptions` class to hold that configuration:

```csharp
public class JwtOptions
{
    public string Issuer { get; init; }
    public string Audience { get; init; }
    public string SecretKey { get; init; }
}
```

And let's imagine that inside of our `appsettings.json` file
we have the following configuration values:

```json
"Jwt": {
    "Issuer": "Gatherly",
    "Audience": "Gatherly",
    "SecretKey": "dont-tell-anyone!"
}
```

Alright, that's looking good. Now I want to show you a few ways to bind
the values from JSON to our `JwtOptions` class.

## Setting Up Options Pattern Using IConfiguration

The most straightforward approach is to use the `IConfiguration` instance
that we can access while registering services.

We need to call the `IServiceCollection.Configure<TOptions>` method,
and specify the `JwtOptions` as the generic argument:

```csharp
builder.Services.Configure<JwtOptions>(
    builder.Configuration.GetSection("Jwt"));
```

It doesn't get simpler than this, does it?

The only downside is that we are limited to the configuration
values provided through application configuration.

This can be extended to include environment variables and user secrets also.

## Setting Up Options Pattern Using IConfigureOptions

If you want a more robust approach, I have you covered.
We're going to use the `IConfigureOptions` interface to define a class
to configure our **strongly typed options**.

There are two steps that we need to follow in this case:

- Create the `IConfigureOptions` implementation
- Call `IServiceCollection.ConfigureOptions<TOptions>` with our `IConfigureOptions`
  implementation as the generic argument

To start off, we will create the `JwtOptionsSetup` class:

```csharp
public class JwtOptionsSetup : IConfigureOptions<JwtOptions>
{
    private const string SectionName = "Jwt";
    private readonly IConfiguration _configuration;

    public JwtOptionsSetup(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    public void Configure(JwtOptions options)
    {
        _configuration
            .GetSection(SectionName)
            .Bind(options);
    }
}
```

We wrote more code, to achieve the same thing. Was it worth it?

Perhaps, if you consider that we now have access to **dependency injection** in the `JwtOptionsSetup` class.
This means that we can resolve other services that we can use to get the configuration values.

We also need to tell the application to use the `JwtOptionsSetup` class:

```csharp
builder.Services.ConfigureOptions<JwtOptionsSetup>();
```

When we try to inject our `JwtOptions` somewhere, the `JwtOptionsSetup.Configure`
method will be called first the calculate the correct values.

## Injecting Options With IOptions

We've seen a few examples for how to configure the **options pattern** with the `JwtOptions` class.

But how do we actually use the **options pattern**?

Easy, you just need to inject `IOptions<JwtOptions>` from the constructor.

I'll just show the `JwtProvider` constructor here, for brevity.

```csharp
public JwtProvider(IOptions<JwtOptions> options)
{
    _options = options.Value;
}
```

The actual `JwtOptions` instance is available on the `IOptions<JwtOptions>.Value` property.

The `IOptions` instance that we injected here is configured
as a **Singleton** in dependency injection. This is very important to be aware of.

## What About IOptionsSnapshot and IOptionsMonitor?

If you want to use the latest configuration values every time
you inject an **options** class, then injecting `IOptions` won't work.

However, you can use the `IOptionsSnapshot` interface instead:

- It provides the latest configuration snapshot (cached per request)
- It is registered as a **Scoped** service
- It detects configuration changes after application start

You can also use the `IOptionsMonitor` which retrieves the current
option values at any time, and it's a **Singleton** service.

## Wrapping up

The **options pattern** gives us a way to use strongly typed configuration classes in our application.

We can configure the options class in a simple way with [`IConfiguration`](#setting-up-options-pattern-using-iconfiguration),
or we can create an [`IConfigureOptions`](#setting-up-options-pattern-using-iconfigureoptions)
implementation if we need something more powerful.

When it comes to using the **options pattern**, we have three approaches:

- [`IOptions`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptions-1?view=dotnet-plat-ext-7.0)
- [`IOptionsSnapshot`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionssnapshot-1?view=dotnet-plat-ext-7.0)
- [`IOptionsMonitor`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionsmonitor-1?view=dotnet-plat-ext-7.0)

Deciding which of them to use in your application depends on what kind of behavior you want.
If you don't need to support refreshing configuration values
after application start, [`IOptions`](#injecting-options-with-ioptions) is a perfect solution.

---


Previous issues of The .NET Weekly