Hello, I am Laszlo

Thread Counters for Performance Metrics

12/28/2020

3 minutes

In the previous posts I have looked into unit testing custom performance counters and interpreting the built-in CPU counter. In this post I will investigate on using some of the thread related counters. When using dotnet-counters we get a couple of useful counters:

Number of Active Timers                            0
ThreadPool Completed Work Items / sec             17
ThreadPool Queue Length                          703
ThreadPool Threads Count                           9

On the above sample we can see there are 703 work items queued on the ThreadPool, there are 9 threads in the pool, and 17 items has been completed since the last update, ~1 sec. This is a quite long queue compared to the work items completed in the last sec - should raise some concerns. Indeed my test application starts up a 1000 tasks, where each task uses the same lock to request mutual exlcusion for a code path which calls Thread.Sleep(). Another counter shows the contention on the locked object:

Monitor Lock Contention Count / sec               13

GetAvailableThreads

During the lifetime the application there are a couple of methods to access threadpool counters in-process. The most common one is ThreadPool.GetAvailableThreads() method which is available since .NET Framework 1.1, and it is part of the netstandard too.

It has two out arguments, the workers count and IO count. According to the documentation the worker count: workerThreads contains the number of additional worker threads that can be started. Its initial value is 32766 on my machine, and decreases as more and more threads are started up. It seems as a thread is free to pick up new work items (while the queue is empty), the value increases. So an idle thread in the threadpool is not considered a worker.

ThreadCount

Since .NET Core 3 release there is a new property: ThreadPool.ThreadCount which returns the number of threadpool threads. The value of the property matches the one returned by dotnet-counters, ThreadPool Threads Count. The value of the property increases as more and more threadpool threads are started. When a thread becomes idle (because the thread pool queue is empty), the ThreadCount property shows that the ThreadPool does not kill the thread right away, it keeps it around (for pooling purposes) and kills it only after a timeout period. This is slightly different to GetAvailableThreads()'s workerThreads, which resumes right away.

Conclusion

The GetAvailableThreads() method has been around in .NET from the beginning hence it can be used in all application types. I personally find the value returned for the workerThreads argument a bit difficult to work with, when interpreting application metrics. On the other side ThreadCount property is only added with .net core 3.0, but I find it easier to work with as it reflects the true number of threads in the threadpool. The consideration on ThreadCount is that some of those threads may be idle for a short period before they get disposed.