Hello, I am Laszlo

Output Caching gRPC is Bad Idea

06/10/2023

6 minutes

In this post I investigate how to enable ASP.NET Core's Output Caching for an ASP.NET Core gRPC service. Primarily, this is done for the sake of curiosity. The concept, design and code below should be avoided in production grade applications.

Output Caching

Output caching is a new type of caching introduced with ASP.NET Core 7.0. It positioned between response caching and in-memory (or distributed) caching. With output caching the server responses are cached on the server. However, it gives less flexibility on choosing custom keys or caching strategy compared to in-memory caching provided by .NET runtime.

Output cache varying allows caching based on keys composed from query parameters, header parameters or by chosen value given the HTTP request. While output cache provides features like cache revalidation and cache invalidation, in this post I will not explore them in detail. Output caching is typically used for caching Razor pages, static HTML content, etc. in GET and HEAD HTTP requests. The default output caching policy behavior bypasses caching responses for authenticated requests or responses that sets cookies.

gRPC

gRPC for .NET is a remote procedure call (RPC) framework which allows interoperable client-server application communication. A typical gRPC service exposes functions that may be called by a client application. gRPC for .NET provides an abstraction layer that allows to handle remote procedure calls into local function invocations (or method invocation in case of C# language). gRPC is built on HTTP/2 carrying messages serialized into Protocol Buffers binary format.

gRPC provides features such as request-response processing, streaming, bidirectional streaming.

Output Caching gRPC is Bad Idea

Combining these two technologies is possible due to the common layer of transport, HTTP. Output cache is a solution for caching responses for requests that are rather static and cause no changes in the state of the server-side resource (stateless). gRPC is designed for client-server communication, which is rarely static, and often causes server-side state changes.

Output caching by default bypasses authenticated requests. It also requires special handling of responses that are tailored for certain request parameters or clients. Contrary gRPC is built for individual client-server messaging, where individual requests may result vastly different response from the server. These two technologies may compose together, but they are semantically incompatible.

Forcing Output Caching with gRPC

The above section should give the idea that these technologies are built on HTTP but solving completely different problems. Combining them is rather a technical exercise than a reasonable solution for any production grade application.

With all the precautions, one may add the built-in output caching to a gRPC service and notice it does not cache any response. The first and foremost reason for that is gRPC using POST requests, that are not cached by the default policy. One may create its own policy to override the defaults. To do that create a type implementing IOutputCachePolicy interface. This interface defines three methods:

• CacheRequestAsync: should set properties on OutputCacheContext to drive the behavior of the caching middleware: whether a request-response should be served from cache, or if the response may be stored in cache

• ServeFromCacheAsync is invoked when a cached response is being returned

• ServeResponseAsync is invoked when a non-cached response is being returned

One such policy can be implemented as the following:

internal sealed class GrpcPolicy : IOutputCachePolicy
{
    public ValueTask CacheRequestAsync(OutputCacheContext context, CancellationToken cancellation)
    {
        var isAuthenticatd = IsAuthenticated(context.HttpContext);
        context.EnableOutputCaching = true;
        context.AllowCacheLookup = !isAuthenticatd;
        context.AllowCacheStorage = !isAuthenticatd;
        return ValueTask.CompletedTask;

        bool IsAuthenticated(HttpContext context)
        {
            return !StringValues.IsNullOrEmpty(context.Request.Headers.Authorization)
                || context.User?.Identity?.IsAuthenticated == true;
        }
    }

    public ValueTask ServeFromCacheAsync(OutputCacheContext context, CancellationToken cancellation)
    {
        var trailersFeature = context.HttpContext.Features.Get<Microsoft.AspNetCore.Http.Features.IHttpResponseTrailersFeature>();
        trailersFeature.Trailers.Add("Grpc-Status", new Microsoft.Extensions.Primitives.StringValues("0"));
        return ValueTask.CompletedTask;
    }

    public async ValueTask ServeResponseAsync(OutputCacheContext context, CancellationToken cancellation)
    {
        await context.HttpContext.Response.BodyWriter.FlushAsync();

        var trailersFeature = context.HttpContext.Features.Get<Microsoft.AspNetCore.Http.Features.IHttpResponseTrailersFeature>();
        if (!trailersFeature.Trailers.TryGetValue("Grpc-Status", out var result) || result.First() != "0")
        {
            context.AllowCacheStorage = false;
            return;
        }
    }
}

The above policy has the following behavior:

• CacheRequestAsync enables the output cache behavior for non-authenticated requests

• ServeResponseAsync flushes the body writer. The output caching middleware replaces the response stream on the HttpContext.Response object. Then uses this stream later to return a cached response. The flush method call makes sure the gRPC response is captured in the output cache's stream (OutputCacheStream). gRPC uses trailers to return the status of the response. ServeResponseAsync sets false to AllowCacheStorage property when the response shows an error in the status.

• ServeFromCacheAsync is extended by adding the "OK" status code to the trailers when the response is served from the cache. Note, that ServeResponseAsync makes sure that only successful requests are cached. The reason for manually adding the Grpc-Status trailer is that - as trailer - it is not captured by the output caching stream. Note, that in a production grade code, one would need to handle cases where the Grpc-Status is returned as a header.

To add enable policy to a gRPC service use the CacheOutput extension method. The following code snippet caches successful gRPC responses for 10 seconds:

app.MapGrpcService<SuperServiceImpl>().CacheOutput(builder => builder
  .AddPolicy<GrpcPolicy>()
  .Expire(TimeSpan.FromSeconds(10))
  .Cache(), true);

Conclusion

While it is technically achievable to combine the output caching middleware with gRPC, it is semantically incorrect. Even with the above policy, several aspects of caching are disregarded; for example, it is not possible to set a Vary based on content or vary by query. Instead of trying to address these concerns, using a memory cache is a better alternative when it comes to caching responses for gRPC requests.