Deadlocking Pipes

I/O pipelines are a special constructs that has been added to .NET at its renaissance. Pipes help to solve the problem of buffering and parsing an incoming/outgoing stream of byte data. This is an inherently difficult problem to implement considering performance aspects. The data chunks received as the input stream are unlikely to be delimited on message boundaries. That means a single data chunk might contain only partial message or multiple messages and a partial message. Handling all use-cases, taking care about buffering, increasing buffer size (if needed) or reducing buffer size, reducing excessive memory allocations are challenging to be implemented by hand. Fortunately, System.IO.Pipelines help to solve this problem.

Problem

The official documentation for System.IO.Pipelines shows a basic usage of pipes. It creates a pipe, and then uses a reader and a writer to demonstrate the usage of the pipe. Finally, it uses await Task.WhenAll(reading, writing); to await both tasks completing. Reading the full documentation, it should be clear that using pipes require great care from the developer's point of view.

In the writer implementation of the above sample a while loop is used to write data into the pipe. When the write completes, or an exception occurs the code breaks out of the loop.

Find out more


DotNet gRPC Getting Started

In the this series of posts I will look into the details of gRPC in ASP.NET Core. In this first post I will be creating a simple service and a corresponding client. In future posts I will focus on the internal implementation of ASP.NET Core's gRPC extension. gRPC (gRPC Remote Procedure Calls is an open source remote procedure call implementation that is based on modern (web) standards. gRPC leverages HTTP2 as transport protocol, and uses Protocol Buffer as a data format and interface definition language. It is typically used for back channel (service-to-service) communication due to its efficiency. However, the efficiency comes at a cost: debugging/decoding messages are not as straightforward as with other protocols.

Creating a gRPC Service

This post provides a getting started and a look into the internals of Grpc.AspNetCore nuget package. The official documentation is spot on to get started:

  • create a proto file: the proto file defines the messages exchanged by server and the client as well as the operations that a client may invoke on a server

  • create a new asp.net core project (using dotnet CLI: dotnet new webapp)

  • add Grpc.AspNetCore nuget package to the project (dotnet add package Grpc.AspNetCore)

  • add the proto file as a Server gRPC service

Find out more


HttpClient Diagnostics

HttpClient has the capability to propagate correlation Id-s in the HTTP headers of traceparent and tracestate. Every recent .NET release had changes in this scope, for the last few releases the followings has changed:

  • Automatic Id propagation

  • AspNet Core creates a new parent activity for each request (by default)

  • Actvitity's DefaultIdFormat changed in .NET 5

  • ActivitySource introduced

In .NET 6 HttpClient allows greater control on how traceIds and spanIds are propagated on downstream HTTP calls. It accomplishes this with the help of DistributedContextPropagator, which comes with a few built in propagator strategies. In this post I will look into testing these strategies work with OpenTelemetry and Jaeger.

Under the hood

Find out more


Regex Unleashed

In the series of these posts, I take a regular task, and implement it multiple times in an iteratively manner, while also improving performance.

The task is the following, given an input string with placeholders in it, and a map of placeholders to actual values. Replace the placeholders in the input string with the actual text values from the map and return the final string. For example, given input string Hello [%placeholder%]! and map [%placeholder%] => World should return Hello World!. The placeholder's [%placeholder%] part is referred as the key, and World part as the value in this post.

To further clarify the task, these constraints are also true:

  • placeholder keys are delimited with [% at the beginning and %] at the end

  • placeholders are recursive (a placeholder's value may contain another (or the same) placeholder's key, but it should be processed recursively)

  • input text contains no placeholder that are embedded by other placeholders (such as hello [%outer[%inner%]example%])

  • the key of a placeholder contain letters and underscores

  • the same fixed set of placeholders are used on multiple input strings

  • placeholders are used at most once per input text

Find out more