# Source Generated JSON Serialization using fast path in ASP.NET Core

In this post I explore how ASP.NET Core works together with the source generated JSON serializer. In .NET 8 the streaming JSON serialization can be enabled to use serialization-optimization mode with ASP.NET Core. However, using the default settings does not enable this fast-path. In this post I will explore how to enable the fast-path serialization by exploring the inner workings of `JsonTypeInfo.cs`. For this post I use .NET 8 with the corresponding ASP.NET Core release.

The .NET 8 runtime comes with three built-in JSON serializer [modes](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/reflection-vs-source-generation?pivots=dotnet-8-0):

- Reflection
- Source generation (Metadata-based mode)
- Source generation (Serialization-optimization mode)

*Reflection* mode is the default one. This mode is used when someone invokes `JsonSerializer.(De)SerializeAsync` without additional arguments passed. *Source Generation (Metadata-based mode)* as the name suggests generates source code at compile time that contains the metadata that the *Reflection* would need to discover during its first invocation. This mode helps to enable JSON serialization in environments where reflection otherwise would not be possible, for example in case of AOT published applications. *Source generation (Serialization-optimization mode)* also generates code at compile time, but besides the metadata it also uses a generated *fast-path* method when serializing objects.

From .NET 8 an ASP.NET Core application [can leverage](https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/#json) the source generated serializers.

## Performance Overview

Here I compare the source generating modes to *Reflection* mode. The *Metadata-based mode* is typically faster for the first invocation, as it does not have to generate the corresponding metadata for the types. Further invocations with the same types (and options) will not realize performance gains. The *Serialization-optimization mode* uses a fast-path for serialization, which can be faster for certain types. Based on my measurements the *Serialization-optimization mode* is faster than the *Metadata-based mode* by 0-20% depending on the shape of the data.

In an ASP.NET Core application the performance gain on a single request with the fast-path is likely to be outweighed by the network jitter. However, the application throughput will increase as less CPU resources are required per request. This allows more requests to be served compared to the other modes.

## Setup

### Metadata-based mode

Enabling source generated serialization consists of two steps:

1. Creating a partial context class derived from `JsonSerializerContext`, that is attributed with types to be source generated.
1. This context type should be set as the `TypeInfoResolver` or added to the [TypeInfoResolverChain](https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonserializeroptions.typeinforesolverchain?view=net-8.0).

In an ASP.NET Core WebApi application:

```csharp
using System.Text.Json.Serialization;

var builder = WebApplication.CreateBuilder(args);
builder.Services.ConfigureHttpJsonOptions(options =>
{
    // Set the TypeInfoResolver 👇
    options.SerializerOptions.TypeInfoResolver = MyContext.Default;
        options.SerializerOptions.Encoder = null;
});

var app = builder.Build();
app.MapGet("/plain", WithPlain);
app.Run();

static MyDto WithPlain() => new MyDto();

public class MyDto { public string Description { get; set; } = "hello"; }

// Derive a partial class from JsonSerializerContext 👇
[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase)]
[JsonSerializable(typeof(MyDto))]
public partial class MyContext : JsonSerializerContext { }
```

### Serialization-optimization mode

Enabling the fast-path with *Serialization-optimization mode* requires a few further options:

1. Most settings of `JsonSourceGenerationOptions` must be compatible with the serialization options used by ASP.NET Core's serializer options.
1. ASP.NET Core's `options.SerializerOptions.Encoder` should be set to `null` (by default `SystemTextJsonOutputFormatter` adds a `JavaScriptEncoder.UnsafeRelaxedJsonEscaping` encoder.
1. `DefaultBufferSize` must be at least twice the size of the serialized data. However, it cannot take an arbitrarily large value as it is pooled by `ArrayPool` which has an upper 'limit' on array sizes that are reasonable to pool. Another consideration against large buffer sizes is to have the pooled arrays allocated on the Small Object Heap (SOH) instead of the Large Object Heap (LOH).

## JsonTypeInfo Internals

When a web application is started, the `ApiExplorer` of ASP.NET Core creates routes for the endpoints. As part of this process the return types' type info is fetched and passed to the request delegate factory.
During this process the `JsonSerializerOptions` caches `JsonTypeInfo`. The first time a `JsonTypeInfo` is created it gets configured. The *Configure* method of `JsonTypeInfo` determines if the object is compatible with ASP.NET Core's `JsonSerializerOptions`. When it is compatible, and the `JsonTypeInfo` has a serializer handler (a delegate for the generated fast-path code), it sets the `CanUseSerializeHandler` property to *true*.

Every time `SerializeAsync` is invoked the method checks if `CanUseSerializeHandlerInStreaming` is enabled. When enabled *fast-path* serialization mode is used. However, by default it is disabled, hence the metadata-based serialization is used. 
Before `SerializeAsync` returns the result to its invoker, the above `CanUseSerializeHandler` property is tested. When it has the value of *true*, the `OnRootLevelAsyncSerializationCompleted` called is called. This method compares the size of current serialized response with the default buffer size:

```csharp
private void OnRootLevelAsyncSerializationCompleted(long serializationSize)
{
    if (_canUseSerializeHandlerInStreamingState != 2)
    {
        if ((ulong)serializationSize > (ulong)(base.Options.DefaultBufferSize / 2))
        {
            _canUseSerializeHandlerInStreamingState = 2;
        }
        else if ((uint)_serializationCount < 10u && Interlocked.Increment(ref _serializationCount) == 10)
        {
            Interlocked.CompareExchange(ref _canUseSerializeHandlerInStreamingState, 1, 0);
        }
    }
}
```

Any time the response is larger than half of the buffer, value *2* is set to `_canUseSerializeHandlerInStreamingState` field. This represents an *end state*, which disables future *fast-path* serialization as `CanUseSerializeHandlerInStreaming` will return *false*.
Otherwise, the method increments a counter. When a counter reaches *10*, `_canUseSerializeHandlerInStreamingState` field is set to the value of *1*. In this state `CanUseSerializeHandlerInStreaming` returns true, that makes `SerializeAsync` to leverage the *fast-path* mode. In this state debugger breakpoints set in the generated code will be hit.

During serialization both the *fast-path* mode and the metadata mode rent an array in the size defined by the default buffer size. This array is rented from `ArrayPool<byte>.Shared`. A key difference is that the metadata mode regularly flushes the buffer while the fast-path mode writes all data to the buffer before it is flushed to the response stream.

The final detail I glossed over is how `JsonTypeInfo`'s compatibility with ASP.NET Core's `JsonSerializerOptions` is determined. This is decided during the *configuration* of the `JsonTypeInfo`. When compatibility is determined, `JsonTypeInfo` reads its `Options.CanUseFastPathSerializationLogic` property. This property is not debugger browsable, because it caches its value for the rest of the lifetime of the object. Having it evaluated too early (for example by being displayed in the debugger) might result in an invalid state.

The `CanUseFastPathSerializationLogic` property itself determines if the `Options` is compatible with all the chained `JsonContext`s, by invoking their `IsCompatibleWithOptions` method and passing itself as an argument. Each context then compares its corresponding settings (configured with `[JsonSourceGenerationOptions]` attribute) with ASP.NET Core's `SerializerOptions` as shown here:

```csharp
bool IBuiltInJsonTypeInfoResolver.IsCompatibleWithOptions(JsonSerializerOptions options)
{
    JsonSerializerOptions generatedSerializerOptions = GeneratedSerializerOptions;
    if (generatedSerializerOptions != null 
        && options.Converters.Count == 0 
        && options.Encoder == null 
        && !JsonHelpers.RequiresSpecialNumberHandlingOnWrite(options.NumberHandling) 
        && options.ReferenceHandlingStrategy == ReferenceHandlingStrategy.None 
        && !options.IgnoreNullValues 
        && options.DefaultIgnoreCondition == generatedSerializerOptions.DefaultIgnoreCondition 
        && options.IgnoreReadOnlyFields == generatedSerializerOptions.IgnoreReadOnlyFields
        && options.IgnoreReadOnlyProperties == generatedSerializerOptions.IgnoreReadOnlyProperties 
        && options.IncludeFields == generatedSerializerOptions.IncludeFields 
        && options.PropertyNamingPolicy == generatedSerializerOptions.PropertyNamingPolicy)
	{
		return options.DictionaryKeyPolicy == null;
	}
	return false;
}
```

The reason why the `Encoder` property must be `null` is because otherwise the compatibility would evaluate to false, hence the source generated serializers do not (yet) support encoders. On the other hand, ASP.NET Core sets the `JavaScriptEncoder.UnsafeRelaxedJsonEscaping` encoder when `SystemTextJsonOutputFormatter` is added to the output formatters. This encoder would [not encode](https://learn.microsoft.com/en-us/dotnet/api/system.text.encodings.web.javascriptencoder.unsaferelaxedjsonescaping?view=net-8.0) some of the HTML-sensitive characters.

## Appendix

### Performance Measurements Benchmarks

Legend:

* MyDto is a type with 2 properties.
* MyDtoLarge is a type with 24 properties, but its serialized size matches MyDto.
* `DefaultBuffeSize` is set to 2MB with *Buffer* suffix, allowing fast-path serialization. Otherwise `DefaultBuffeSize` is left as default at 16384 bytes.
* *Reflection* mode is used with *NoSG* suffix. Otherwise, *source generation mode* is used.

#### Serializing a single 864 KB DTO:

| Method           | Mean     | Error   | StdDev  | Allocated |
|----------------- |---------:|--------:|--------:|----------:|
| MyDto            | 581.0 us | 4.85 us | 4.05 us |     465 B |
| MyDtoLarge       | 610.2 us | 8.30 us | 7.76 us |     465 B |
| MyDtoBuffer      | 535.1 us | 3.77 us | 3.34 us |      33 B |
| MyDtoLargeBuffer | 532.9 us | 5.25 us | 4.91 us |      33 B |
| MyDtoNoSG        | 590.6 us | 5.27 us | 4.68 us |     465 B |
| MyDtoLargeNoSG   | 586.1 us | 5.74 us | 5.37 us |     465 B |

In this case the additonal performance is only gained in the *Buffer* cases, where the `DefaultBufferSize` is large enough to enable the fast-path mode. Otherwise MyDto and MyDtoLarge performs similar to the non-source generated results.

#### Serializing a list of these DTOs (1 KB each), but in total larger to 16384 bytes:

| Method           | Mean       | Error    | StdDev   | Allocated |
|----------------- |-----------:|---------:|---------:|----------:|
| MyDto            |   758.4 us |  6.50 us |  6.08 us |     482 B |
| MyDtoLarge       | 1,217.2 us |  9.90 us |  9.26 us |     500 B |
| MyDtoBuffer      |   618.5 us |  4.28 us |  3.80 us |    2082 B |
| MyDtoLargeBuffer |   920.3 us | 16.23 us | 16.67 us |    2082 B |
| MyDtoNoSG        |   752.7 us |  6.81 us |  6.37 us |     482 B |
| MyDtoLargeNoSG   | 1,223.0 us | 11.73 us | 10.97 us |     500 B |

This is a similar case as above considering the buffer size. Also note, that having significantly more properties to serialize (while in total containing the same amount of data) slows down serialization. This is show by the differences between the *Large* and regular DTO's mean execution time.

#### Single entity serialization (1 KB):

| Method           | Mean       | Error    | StdDev   | Gen0   | Allocated |
|----------------- |-----------:|---------:|---------:|-------:|----------:|
| MyDto            |   687.2 ns |  5.44 ns |  5.09 ns | 0.0048 |      32 B |
| MyDtoLarge       |   984.3 ns | 16.35 ns | 15.29 ns | 0.0038 |      32 B |
| MyDtoBuffer      |   682.0 ns |  9.27 ns |  7.74 ns | 0.0048 |      32 B |
| MyDtoLargeBuffer |   953.7 ns |  3.07 ns |  2.56 ns | 0.0048 |      32 B |
| MyDtoNoSG        |   796.7 ns |  5.05 ns |  4.48 ns | 0.0734 |     464 B |
| MyDtoLargeNoSG   | 1,286.2 ns | 11.79 ns | 10.45 ns | 0.0725 |     464 B |

Here the *Buffer* cases do not gain further performance as all source-generated results (MyDto, MyDtoLarge, MyDtoBuffer, MyDtoLargeBuffer) fit in the default buffer using the fast-path mode. However, we can see significant gains over the non-source generated results (MyDtoNoSG, MyDtoLargeNoSG).