Hello, I am Laszlo

IAsyncEnumerable WithCancellation

08/16/2025

5 minutes

I recently used IAsyncEnumerable in C# the first time. In this post, I will summarize some of my learnings based on .NET 9.

There is an excellent introduction to Iterating with Async Enumerables that introduces they key concepts from the iterators point of view and then details the internals of an async enumerator.

Key Concepts

IAsync­Enumerable<T> allows creating iterators that produce elements in an asynchronous fashion. Although it has been around for a few years, it is not commonly seen in application source code. Typically, iteration happens over a collection where elements are already computed, making synchronous iteration adequate, like iterating a list of integers. There are also cases where the elements of an iterator can computed synchronously, for example the runtime provides Enumerble.Range or Enumerable.Repeat iterators. Additionally, Select extension can project each element of a sequence into a new shape, like a Task<T>, then a consumer awaits these tasks to complete concurrently.

IAsync­Enumerable<T> offers two interesting use-cases:

1. Creating an iterator with yield return that invokes and awaits asynchronous operations in a streamlined way.

2. Streaming data asynchronously, such as returning an async enumerable source by ASP.NET Core as a stream of data.

For example, an iterator might need to invoke a HTTP service at each iteration, like iterate through all pages of a website and returning the number of words on each page. Without a complete map of all the pages, it can only be iterated page-by-page by following the hyperlinks. While downloading a webpage can be done SendAsync or GetAsync async methods of the HTTP client.

I find many sources online that compare IAsync­Enumerable<T> with Task<IEnumerable<T>>, however, I think its behavior matches more closely to an IEnumerable<Task<T>>, where each task is only started and awaited when the corresponding element is consumed.

One trivial example could be the following RangeAsync() method:

var token = new CancellationToken(true);
await foreach(var item in RangeAsync())
   Console.WriteLine(item);

async IAsyncEnumerable<int> RangeAsync()
{
    for(int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

A consumer of an IAsync­Enumerable<T> can iterate the asynchronous sequence with the await foreach C# language syntax.

Cancellation Support

To support cancellation, one could add a CancellationToken token = default parameter to the RangeAsync method. In this case, the compiler will issue a warning to attribute the parameter with [EnumeratorCancellation]:

async IAsyncEnumerable<int> RangeAsync([EnumeratorCancellation]CancellationToken token = default)
{
    for(int i = 0; i < 10; i++)
    {
        await Task.Delay(100, token);
        yield return i;
    }
}

This allows the compiler generated state machine to accept and use a cancellation token parameter. There are 3 ways to pass a token:

1. Directly to the method:

await foreach(var item in RangeAsync(token))
   Console.WriteLine(item);

2. Using the WithCancellation extension:

await foreach(var item in RangeAsync().WithCancellation(token))
   Console.WriteLine(item);

3. Directly to the method and using the WithCancellation

await foreach(var item in RangeAsync(token1).WithCancellation(token2))
   Console.WriteLine(item);

There is a re-usability (re-iterability) difference between the 3 approaches, but they all behave (more-or-less) to the same. The compiler will pass a token (or a combined token) to the RangeAsync method. While the WithCancellation gives the impression that the compiler generated code could cancel the iteration, it does not do so. Let's see this in an example:

Take a non-cooperative version of the RangeAsync method (the token parameter is not used):

async IAsyncEnumerable<int> RangeAsync([EnumeratorCancellation]CancellationToken token = default)
{
    for(int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

All three ways to iterate RangeAsync will result in the same behavior (iterating all elements), regardless of whether the token is cancelled or not; or the WithCancellation extension is used or not. The reason for this is explained in a reply this a discussion.

If the compiler generated state machine would cooperate in the cancellation, it would not allow the implementer of the iterator to fully control the behavior in case of a cancellation. For example, an iterator might want to catch an OperationCancelledException or TaskCancelledException, to do some cleanup and recover from the exception before completing or returning the next element:

async IAsyncEnumerable<int> RangeAsync([EnumeratorCancellation]CancellationToken token = default)
{
    for(int i = 0; i < 10; i++)
    {
        try
        {
            await Task.Delay(10, token);
        }
        catch(TaskCanceledException e)
        {
            Console.WriteLine("Cancelled");
            break;
        }
        yield return i;
    }
}

In the case above when TaskCanceledException happens a message is written to the console before gracefully returning from the iterator using the break keyword.Handling exceptions in iterators has its own challenges though. Notice, that the try-catch block is only wrapping the async operation, but not the yield return expression. This is a limitation from the C# compiler for both sync and async iterators. When the try-catch block includes the yield-return statement, the following error is issued by the compiler: error CS1626: Cannot yield a value in the body of a try block with a catch clause.

Conclusion

In conclusion, IAsyncEnumerable<T> provides a powerful way to handle asynchronous iteration in C#. It allows for more efficient and responsive applications, especially when dealing with I/O-bound operations or streaming data. By understanding how to implement and use IAsyncEnumerable<T> with cancellation support, developers can create more robust and user-friendly applications. Remember to handle exceptions properly and be aware of the limitations when using yield return within try-catch blocks. With these insights, you can effectively leverage IAsyncEnumerable<T> in your projects.