Hello, I am Laszlo

Deadlocking Pipes

07/30/2022

4 minutes

I/O pipelines are a special constructs that has been added to .NET at its renaissance. Pipes help to solve the problem of buffering and parsing an incoming/outgoing stream of byte data. This is an inherently difficult problem to implement considering performance aspects. The data chunks received as the input stream are unlikely to be delimited on message boundaries. That means a single data chunk might contain only partial message or multiple messages and a partial message. Handling all use-cases, taking care about buffering, increasing buffer size (if needed) or reducing buffer size, reducing excessive memory allocations are challenging to be implemented by hand. Fortunately, System.IO.Pipelines help to solve this problem.

Problem

The official documentation for System.IO.Pipelines shows a basic usage of pipes. It creates a pipe, and then uses a reader and a writer to demonstrate the usage of the pipe. Finally, it uses await Task.WhenAll(reading, writing); to await both tasks completing. Reading the full documentation, it should be clear that using pipes require great care from the developer's point of view.

In the writer implementation of the above sample a while loop is used to write data into the pipe. When the write completes, or an exception occurs the code breaks out of the loop.

async Task FillPipeAsync(Socket socket, PipeWriter writer)
{
    const int minimumBufferSize = 512;

    while (true)
    {
        // Allocate at least 512 bytes from the PipeWriter.
        Memory<byte> memory = writer.GetMemory(minimumBufferSize);
        try
        {
            // ...
        }
        catch (Exception ex)
        {
            LogError(ex);
            break;
        }

        // ...
    }

     // By completing PipeWriter, tell the PipeReader that there's no more data coming.
    await writer.CompleteAsync();
}

The sample code for basic usage omitting irrelevant details for this discussion.

Someone implementing the writer may take this sample and customize it. A critical aspect of it is the exception handling, which is quite opinionated in the sample. However, simply removing the catch block might cause a deadlock when using await Task.WhenAll(reading, writing); to await both the reader and the writer. The reason for that is if the writer throws an exception, the writer never calls CompleteAsync, which means the reader will be expecting more data on the pipe. Although the Task of the writer has already completed with an error, the reader is still awaiting data on the pipe. Because of this, Task.WhenAll will never complete with success or with an exception either.

Notice, that CompleteAsync has multiple overloads, one of which takes an Exception argument.

The problem is that the sample above does not communicate this critical flaw. Reading the code, a developer might feel comfortable with removing or rewriting the try-catch block.

Solutions

The less obvious solution to this problem is to create a custom logic to await both tasks. One may get inspiration for that from ORDERING BY COMPLETION, AHEAD OF TIME by Jon Skeet. Instead of awaiting tasks in completion order, one could simply await the first failing task or all successfully completed tasks. While this seems code fun to write as a kata, I would rather suggest fixing the writing Task.

In this case the solution is simple: surround the writer logic with a try-catch block along with a finally block. Make sure the CompleteAsync is called in the finally block. This will communicate to the reader of the code that calling CompleteAsync is required before returning from this method. Alternatively, the catch block itself may call CompleteAsync passing the exception caught.