• Dictionary<TKey, TValue> is a Hash table mapping a set of keys to a set of values.
• Each key in the dictionary must be unique according to the IEqualityComparer<TKey> (used to determine the equality of the keys in the dictionary).
• Retrieving a value by using its key has a complexity of O(1) (very fast).

Typical use-case

Most of the time, I use a dictionary to map a model to one of its properties; Let’s say an Id or Name. For example, given the model:

public sealed record class Person(Guid Id, string Name, uint Age);

And given a collection of people from somewhere:

IEnumerable<Person> people = Enumerable.Range(1, 10)
    .Select(n => new Person(Guid.NewGuid(), "P-" + n.ToString(), (uint)n));

We may now want to make people available for fast lookups based on the Id of a given person.

Well that’s easy, let’s new-up a Dictionary and add our people to it:

Dictionary<Guid, Person> dic = new();
foreach(Person p in people)
{
    dic[p.Id] = p;
    // or if you don't like the index syntax then you can instead do:
    // dic.Add(p.Id, p);
}

We can also create a dictionary directly from the IEnumerable<Person> by doing:

Dictionary<Guid, Person> dic = Enumerable.Range(1, 10)
    .Select(n => new Person(Guid.NewGuid(), "P-" + n.ToString(), (uint)n))
    .ToDictionary(p => p.Id, p => p);

Okay not the most verbose code to complain about but if you take into account that every time you want to Add, TryGet or check for existence (Contains) of a given value you need to specify the TKey explicitly then you will be able to also hear my OCD screaming to make things simpler.

What if…

Would it not be more elegant if we could just add a person and let the dictionary get the Id from it? Something similar to:

EasyDictionary<Guid, Person> dic = new (...);
foreach(Person p in people)
{
    dic.Add(p)
}

In order to do this, we would need some sort of a key selector which for a given TValue would return a TKey. In other words, we need a Func<TValue, TKey> which in our example would be a Func<Person, Guid>.

Now that we know what a key selector would look like, we can start thinking about the structure of our new abstraction:

public sealed class EasyDictionary<TKey, TValue>
{
    public EasyDictionary(Func<TValue, TKey> keySelector)
    {
        KeySelector = keySelector;
    }

    public Func<TValue, TKey> KeySelector { get; }
    ...
}

All we have to do now is put a Dictionary<TKey, TValue> inside our wrapper, implement IReadOnlyDictionary<TKey, TValue> as well as ICollection<TValue> and then redirect all the required methods and properties to our internal dictionary.

Job done!

I am going to spare you the unexciting code here and instead, point you to the completed version in EasyDictionary available as part of the Easy.Common NuGet package.

Armed with our much smarter dictionary, let us see what we can do with it (look at the unit tests for more examples):

Person[] people = Enumerable.Range(1, 10)
    .Select(n => new Person(Guid.NewGuid(), "P-" + n.ToString(), (uint)n))
    .ToArray();

EasyDictionary<Guid, Person> easyDic = new(p => p.Id);
foreach(Person p in people)
{
    easyDic.Add(p);
}

// Let's lookup an existing value.
Person firstPerson = people[0];

easyDic.Contains(firstPerson);                          // true
easyDic.ContainsKey(firstPerson.Id);                    // true

easyDic[firstPerson.Id];                                // firstPerson

easyDic.TryGetValue(firstPerson.Id, out Person yep);    // true (yep is firstPerson)

easyDic.Remove(firstPerson.Id);                         // true (removed)
easyDic.Remove(firstPerson);                            // false (no longer there)

easyDic.TryGetValue(firstPerson.Id, out Person _);      // false (no longer there)

We can also enumerate over the items:

foreach (Person p in easyDic) { /* do what you want with p */ }

But what about over-writing an existing item? Well, we can use AddOrReplace for that:

Person olderPerson = new(
    firstPerson.Id,
    firstPerson.Name,
    firstPerson.Age + 41);

easyDic.AddOrReplace(olderPerson); // true;

Bonus point

We can even throw a bunch of extension methods to be able to do:

EasyDictionary<Guid, Person> dic = Enumerable.Range(1, 10)
    .Select(n => new Person(Guid.NewGuid(), "P-" + n.ToString(), (uint)n))
    .ToEasyDictionary(p => p.Id);

Now this is tranquillity!

The IDs are base-32 encoded increments of a long (which is seeded based on the DateTime.UtcNow.Ticks) using the characters 1 to 9 and A to V.

Let’s look at the code:

internal static class CorrelationIdGenerator
{
    private static readonly string _encode32Chars = "0123456789ABCDEFGHIJKLMNOPQRSTUV";

    private static long _lastId = DateTime.UtcNow.Ticks;

    public static string GetNextId() => GenerateId(Interlocked.Increment(ref _lastId));

    private static unsafe string GenerateId(long id)
    {
        char* charBuffer = stackalloc char[13];

        charBuffer[0] = _encode32Chars[(int)(id >> 60) & 31];
        charBuffer[1] = _encode32Chars[(int)(id >> 55) & 31];
        charBuffer[2] = _encode32Chars[(int)(id >> 50) & 31];
        charBuffer[3] = _encode32Chars[(int)(id >> 45) & 31];
        charBuffer[4] = _encode32Chars[(int)(id >> 40) & 31];
        charBuffer[5] = _encode32Chars[(int)(id >> 35) & 31];
        charBuffer[6] = _encode32Chars[(int)(id >> 30) & 31];
        charBuffer[7] = _encode32Chars[(int)(id >> 25) & 31];
        charBuffer[8] = _encode32Chars[(int)(id >> 20) & 31];
        charBuffer[9] = _encode32Chars[(int)(id >> 15) & 31];
        charBuffer[10] = _encode32Chars[(int)(id >> 10) & 31];
        charBuffer[11] = _encode32Chars[(int)(id >> 5) & 31];
        charBuffer[12] = _encode32Chars[(int)id & 31];

        return new string(charBuffer, 0, 13);
    }
}

If we invoke it like so:

CorrelationIdGenerator.GetNextId();
Thread.Sleep(50);
CorrelationIdGenerator.GetNextId();
Thread.Sleep(150);
CorrelationIdGenerator.GetNextId();

We get:

0HLH7QN5JTC87
0HLH7QN5JTC88
0HLH7QN5JTC89

Overall, this code is very efficient in how it constructs a string representation of an ID based on a long by avoiding the .ToString() call which (according to the comments on the code) would be ~310% and ~600% slower on x64 and x86 respectively.

It also avoids allocating memory on the heap avoiding frequent GC collections (when invoked on a hot path) by allocating a char[] on the stack, populating it with the encoded values then passing the char pointer to the string constructor.

Impressive stuff, but…

Allocating memory on the stack (using stackalloc) has long been used as a performance trick; Not only it can be faster than heap allocation but it also avoids GC. However, there are some caveats:

• stackalloc is explicitly unsafe (hence why I used 'safe' in the title of this article) which means you would have to compile your assembly as such (by using the unsafe flag). Doing so means that your assembly may be black-listed by some environments where loading of unsafe assemblies are not permitted. e.g. SQLServer (although there are workarounds). (Also note that as of C#7.2 unsafe is no longer required as long as you use Span when using stackalloc)

• The amount of data you are allowed to allocate is very limited. Managed stacks are limited to 1MB in size and can be even less when running under IIS. Each stack frame takes a chunk of that space so the deeper the stack the less memory will be available.

• If an exception is thrown during allocation, you will get an StackOverflowException which is best known as an exception that cannot be caught. To be fair, Kestrel allocates a small block of memory when generating an ID, therefore, the chances of overflowing is rare; However, I still prefer to be on the safe side if and when I can.

With the above points in mind, I got curious to see if I can avoid using unsafe and:

1. Measure the performance impact of going without it.
2. Try close the gap in the performance as much as possible and bring it to a reasonable level.

A bit about how we are going to measure the performance

If you have not yet read my previous articles I suggest having a look so that you can get familiar with some of the terms, tools and my overall approach to profiling.

• Practical Parallelization in C#
• High Performance Logging using log4net
• Counting Lines of a Text File in C#, the Smart Way
• Stuff Every .NET App Should be Logging at Startup

We are going to start by doing some coarse-grained/high-level measurements just so that we get a general idea of the behaviour of GC and the running time for our baseline algorithm before we start playing with things.

We will be running a .NET Core 2.1.5 application hosted on Windows Server 2016. Our machine is a dual CPU Xeon processor with 12 cores (24 logical) clocking at 2.67GHz.

Also note the GC mode of our application will be set to Workstation to emphasise the impact of the GC collections on the process (refer to the links above for finding out the difference between Workstation and Server). The overall result of our benchmark should still be valid with Server mode enabled.

Let’s start by generating 100 million IDs while measuring the execution time and the GC behaviour. We will repeat this 10 times and look at the average values across all executions.

Here is the code for generating the IDs:

Stopwatch sw = Stopwatch.StartNew();
for(int i = 0; i < 100_000_000; i++)
{
    string id = CorrelationIdGenerator.GetNextId();
}
sw.Stop();

using Process process = Process.GetCurrentProcess();

Console.WriteLine("Execution time: {0}\r\n  - Gen-0: {1}, Gen-1: {2}, Gen-2: {3}",
        sw.Elapsed.ToString(),
        GC.CollectionCount(0),
        GC.CollectionCount(1),
        GC.CollectionCount(2));

This not-so-fancy code starts a timer, generates a lot of IDs then measures how many times GC collections occurred across the 3 generations.

1. Stackalloc

With all the above out of the way, here’s the average values for our baseline:

- Execution time: 00:00:06.0511525
- Gen-0: 890, Gen-1: 0, Gen-2: 0

Okay, so it took around 6 seconds and 890 Gen0 GC to generate the IDs. If you are wondering why we have those Gen0 collections despite allocating on the stack, well, the answer is; We are only allocating the char[] on the stack but we still need to allocate 100 million string objects on the heap.

Now that we know how our baseline behaves, let us measure how we would have done if we had allocated the char[] on the heap instead of the stack.

2. Heap allocation

All we have to do for this version is remove the unsafe keyword and new up a char[], with everything else unchanged.

private static string GenerateId(long id)
{
    var buffer = new char[13];

    buffer[0] = _encode32Chars[(int)(id >> 60) & 31];
    buffer[1] = _encode32Chars[(int)(id >> 55) & 31];
    buffer[2] = _encode32Chars[(int)(id >> 50) & 31];
    buffer[3] = _encode32Chars[(int)(id >> 45) & 31];
    buffer[4] = _encode32Chars[(int)(id >> 40) & 31];
    buffer[5] = _encode32Chars[(int)(id >> 35) & 31];
    buffer[6] = _encode32Chars[(int)(id >> 30) & 31];
    buffer[7] = _encode32Chars[(int)(id >> 25) & 31];
    buffer[8] = _encode32Chars[(int)(id >> 20) & 31];
    buffer[9] = _encode32Chars[(int)(id >> 15) & 31];
    buffer[10] = _encode32Chars[(int)(id >> 10) & 31];
    buffer[11] = _encode32Chars[(int)(id >> 5) & 31];
    buffer[12] = _encode32Chars[(int)id & 31];

    return new string(buffer, 0, 13);
}

Right, let us measure this version:

- Execution time: 00:00:06.1883507
- Gen-0: 1780, Gen-1: 0, Gen-2: 0

Allocating on the heap seems to be about 140 milliseconds slower than allocating on the stack; However, we have doubled the number of _GC_s. You may now be saying:

Dude, it doesn’t matter. Gen0 GCs are super fast…

And you would be right; Both Gen0 and Gen1 collections are fast but unlike the Gen2 collections, they are blocking; So, the more IDs you generate, the more frequent blocking collections wasting CPU cycles reducing your overall throughput. This may not be a problem for your application and if it is not then fair enough but there are scenarios where you need to reduce allocations as much as possible.

Okay, what is the most simple method of reducing those allocations? Reusing the char[] you say? Sure, let’s try that…

3. Re-using the buffer

What we need to do here is declare a field for our re-usable char[] buffer and overwrite it for every ID we generate.

private static readonly char[] _buffer = new char[13];

private static string GenerateId(long id)
{
    char[] buffer = _buffer;
    ...
}

And this is how it does:

- Execution time: 00:00:05.5621535
- Gen-0: 890, Gen-1: 0, Gen-2: 0

Excellent! Awesome gains! Super fast! No allocation! Let’s wrap up!

But wait a second

The unsafe version was thread-safe and our version is not. What if we want to generate the IDs from different threads and in parallel?

No worries, all we have to do is ensure access to the char[] is synchronised across all threads and for that we are going to use a simple lock, let’s go!

4. Locking the buffer

With the rest of the code unchanged, Let us lock the body of our method like so:

private static string GenerateId(long id)
{
    lock(_buffer)
    {
        char[] buffer = _buffer;
        ...
    }
}

And run the benchmark:

- Execution time: 00:00:07.7254997
- Gen-0: 890, Gen-1: 0, Gen-2: 0

Okay, no more allocation other than the actual string representation of our IDs. As you can see the cost of using the lock is an extra second or so of overhead.

But this is not a valid test

Even though we now have a thread-safe method, we have so far only generated the IDs from a single thread and that is not representative of how our code can and will be used out in the wild.

A more realistic scenario

We are going to sprinkle a bit of parallelism to our benchmarking code by generating 1 billion IDs across 12 threads in parallel.

Here is the updated code for our benchmarker:

const int ITERATION_COUNT = 1_000_000_000;
const int THREAD_COUNT = 12;

Stopwatch sw = Stopwatch.StartNew();

ParallelEnumerable
    .Range(1, ITERATION_COUNT)
    .WithDegreeOfParallelism(THREAD_COUNT)
    .WithExecutionMode(ParallelExecutionMode.ForceParallelism)
    .ForAll(_ => {
        string id = CorrelationIdGenerator.GetNextId();
    });

sw.Stop();

using Process process = Process.GetCurrentProcess();
Console.WriteLine("Execution time: {0}\r\n  - Gen-0: {1}, Gen-1: {2}, Gen-2: {3}",
        sw.Elapsed.ToString(),
        GC.CollectionCount(0),
        GC.CollectionCount(1),
        GC.CollectionCount(2));

5. Stackalloc (Parallel)

Now that we have updated our benchmarker, we need to run it again for our baseline to see how it does in parallel.

- Execution time: 00:00:58.9092805
- Gen-0: 8980, Gen-1: 5, Gen-2: 0

Okay, so 8,980 Gen0, 5 Gen1 and 00:00:58.9092805 is what we are up against.

6. New buffer (Parallel)

As a reminder, this is the version where we are creating a new char[] each time we are generating an ID. So let us see the performance of this allocaty method in parallel:

- Execution time: 00:01:04.6934614
- Gen-0: 17959, Gen-1: 19, Gen-2: 0

Again, we can see the high number of allocations except this time the Gen1 allocations are also increasing.

7. Locking the buffer (Parallel)

Not to worry, we have it all under control. We will be avoiding all those nasty allocations thanks to our friend Mr lock! Let’s go:

- Execution time: 00:04:14.3642255
- Gen-0: 9008, Gen-1: 25, Gen-2: 2

Yep, looks like Mr lock was not much of a friend after all! not only it took us 4 times longer but also 25 Gen1 and 2 Gen2 collections. But why? Isn’t acquiring a lock fast? Well, that depends; Acquiring a non-contended lock is extremely fast (~20 ns) but if contended, the context switch depending on the number of competing threads can quickly increase this to microseconds which in our case resulted in a noticeable performance hit.

Indeed, if we profile this app using dotTrace (refer to my previous blog posts for more details) we can see that we spent a whopping 75% of our total execution time waiting in Lock contention (click to enlarge):

Those 12 threads are barely utilising the CPU. Let’s compare that with our stackalloc version:

See how nice and healthy those threads are? Those context switches are wasting the CPU cycles. If only we could somehow reduce it…

8. Spin Locking

Guess what? We can avoid those context switches by using a SpinLock.

SpinLock lets you lock without incurring the cost of a context switch, at the expense of keeping a thread spinning (uselessly busy) - Albahari

And this is how we are going to use it in our code:

private static readonly char[] _buffer = new char[13];
private static SpinLock _spinner = new SpinLock(false);

private static string GenerateId(long id)
{
    bool lockTaken = false;
    try
    {
        _spinner.Enter(ref lockTaken);

        char[] buffer = _buffer;
        ...
        return new string(buffer, 0, 13);
    } finally
    {
        if (lockTaken)
        {
            _spinner.Exit();
        }
    }
}

<warning>IMPORTANT</warning>

You should almost never use a SpinLock as it comes with many gotchas one of which is that unlike a lock (aka Monitor) a SpinLock is not re-entrant; So, if you cannot figure out why I have set its constructor parameter to false or why the _spinner field has not been declared as readonly, then it means you should not be using SpinLock until you fully understand how it works (I still do not!) and once you do, you should totally erase any memory of SpinLock from your head. Seriously, DO NOT USE IT! Nevertheless, you can have a look at THIS and THIS to satisfy your curiosity.

Let’s see what difference SpinLock makes:

- Execution time: 00:01:34.5641002
- Gen-0: 8962, Gen-1: 9, Gen-2: 0

It looks like this time we did much better by not having to do as many context switches as we did when we were locking; But let us have a closer look at what our process was actually doing:

Even though we had no contentions, we spent only 13.5% of our total execution time running and the rest spinning and wasting precious CPU time. NOT GOOD!

The Evil Shared State

Sharing the char[] across our worker threads does not seem to be a good choice after all. How can we eliminate this shared resource and at the same time avoid allocating? If only there was a way for each thread to have its own buffer to work with…

Well, .NET has a facility which allows us to do just that.

9. ThreadStatic

Any static field marked with a ThreadStatic attribute is attached to a thread and each thread will then have its own copy of it. You can think of it as a container which holds a separate instance of our field for every thread.

A static field marked with ThreadStaticAttribute is not shared between threads. Each executing thread has a separate instance of the field, and independently sets and gets values for that field. If the field is accessed on a different thread, it will contain a different value. MSDN

When using ThreadStatic, we are not required to specify an initial value for our field; if we do, then the initialisation occurs once the static constructor of the class is executed, therefore, the value is only set for the thread executing that constructor; When accessed in all subsequent threads, it will be initialised to its default value. So, in our case, before each thread tries to use the buffer for the first time, we need to make sure the field for that thread is initialised to a new char[] otherwise its value will be null.

This is what our code looks like with ThreadStatic:

[ThreadStatic]
private static char[] _buffer;

private static string GenerateId(long id)
{
    if (_buffer is null)
    {
        _buffer = new char[13];
    }

    char[] buffer = _buffer;
    ...
}

Let’s see how this one does:

- Execution time: 00:01:11.0138624
- Gen-0: 8981, Gen-1: 7, Gen-2: 0

Okay, we are getting there! We reduced the execution time by around 20 seconds. Let us see how our threads performed:

Now we are talking! As you can see from the image, all our threads are showing a healthy utilisation and overall spent 94.6% of their time running. This is what we wanted as eliminating that shared buffer also removes any contention between our worker threads enabling them to each do their work independently with no bottleneck.

Okay good, but can we do better? Just a little bit faster?

10. ThreadLocal

Starting from .NET4.0, ThreadLocal provides Thread Local Storage (TLS) of data. It also improves on ThreadStatic by providing a strongly typed, locally scoped container to store values specific to each thread. Unlike ThreadStatic, you can mark both instance and static fields with it. It also allows you to use a factory method to create or initialise its value lazily and on the first access from each thread.

Side note

I have used this ThreadLocal trick many times in the past including in Easy.MessageHub (a high-performance Event Aggregator which you can read about HERE) to avoid locking and remove contention between threads.

Armed with our new friend, I present to you the final version of our code:

private static readonly ThreadLocal<char[]> _buffer =
    new ThreadLocal<char[]>(() => new char[13]);

private static string GenerateId(long id)
{
    char[] buffer = _buffer.Value;

    buffer[0] = Encode_32_Chars[(int)(id >> 60) & 31];
    buffer[1] = Encode_32_Chars[(int)(id >> 55) & 31];
    buffer[2] = Encode_32_Chars[(int)(id >> 50) & 31];
    buffer[3] = Encode_32_Chars[(int)(id >> 45) & 31];
    buffer[4] = Encode_32_Chars[(int)(id >> 40) & 31];
    buffer[5] = Encode_32_Chars[(int)(id >> 35) & 31];
    buffer[6] = Encode_32_Chars[(int)(id >> 30) & 31];
    buffer[7] = Encode_32_Chars[(int)(id >> 25) & 31];
    buffer[8] = Encode_32_Chars[(int)(id >> 20) & 31];
    buffer[9] = Encode_32_Chars[(int)(id >> 15) & 31];
    buffer[10] = Encode_32_Chars[(int)(id >> 10) & 31];
    buffer[11] = Encode_32_Chars[(int)(id >> 5) & 31];
    buffer[12] = Encode_32_Chars[(int)id & 31];

    return new string(buffer, 0, buffer.Length);
}

Let’s take it for a spin:

- Execution time: 00:00:58.7741476
- Gen-0: 8980, Gen-1: 5, Gen-2: 0

And just like that, we managed to beat the stackalloc version by 135ms!

Here’s the profiling snapshot where once again we can see a clean and healthy utilisation across all our threads.

Bonus point

Shortly after publishing this article, Ben Adams the author of the original pull request for Kestrel’s ID generation reminded me of the new String.Create method which allows us to avoid both stack and heap allocation for our buffer and write directly to the string memory.

You will need to be on .NET Core 2.1 or higher and know the length of your string in advance. The method then allocates a string and gives you a Span<char> allowing you to essentially write directly into the string’s memory on the heap.

This is how that version would look like:

public static string GenerateId(long id) =>
    string.Create(13, id, _writeToStringMemory);

private static readonly SpanAction<char, long> _writeToStringMemory =
    // DO NOT convert to method group otherwise will allocate
    (span, id) => WriteToStringMemory(span, id);

private static void WriteToStringMemory(Span<char> span, long id)
{
    span[0] = Encode_32_Chars[(int) (id >> 60) & 31];
    span[1] = Encode_32_Chars[(int) (id >> 55) & 31];
    span[2] = Encode_32_Chars[(int) (id >> 50) & 31];
    span[3] = Encode_32_Chars[(int) (id >> 45) & 31];
    span[4] = Encode_32_Chars[(int) (id >> 40) & 31];
    span[5] = Encode_32_Chars[(int) (id >> 35) & 31];
    span[6] = Encode_32_Chars[(int) (id >> 30) & 31];
    span[7] = Encode_32_Chars[(int) (id >> 25) & 31];
    span[8] = Encode_32_Chars[(int) (id >> 20) & 31];
    span[9] = Encode_32_Chars[(int) (id >> 15) & 31];
    span[10] = Encode_32_Chars[(int) (id >> 10) & 31];
    span[11] = Encode_32_Chars[(int) (id >> 5) & 31];
    span[12] = Encode_32_Chars[(int) id & 31];
}

Ready to see the result?

- Execution time: 00:01:00.1041126
- Gen-0: 8980, Gen-1: 6, Gen-2: 0

And here’s how the process was doing:

Hmm… this is not what I expected. Could it be that we are spending a little too much time Bounds Checking? Let’s give the JIT a little help by reversing the buffer population:

private static void WriteToStringMemory(Span<char> span, long id)
{
    span[12] = Encode_32_Chars[(int) id & 31];
    ...
    span[0] = Encode_32_Chars[(int) (id >> 60) & 31];
}

And how about now?

- Execution time: 00:00:56.6800000
- Gen-0: 8980, Gen-1: 5, Gen-2: 0

Excellent!

Conclusion

As we discussed before, our focus was to avoid using unsafe whilst trying to minimise any potential drop in the performance. The goal was not to be faster than the baseline but as it turned out we managed to shave a little bit off which is also nice.

I have included the final version of this method (IDGenerator) in Easy.Common. You can also find the benchmark code HERE.

I hope you enjoyed this article. Happy ThreadLocaling all the things!

So in this article we are going to have a look at how you can (micro)-optimise this seemingly easy task.

Sample Data

Before we start we need to have some sort of sample data we can use to benchmark and measure the performance of our algorithms.

We will be running our algorithms against 5 text files I have produced with each increasing in size starting from 6MB going all the way to 37GB. Every file consists of a word in each line also every smaller file is a subset of its larger brother (or sister or gender-neutral sibling).

Here’s the summary of our files:

Name     Size(KB)     Lines
1.txt    6,516        714,584
2.txt    12,995       1,410,082
3.txt    370,180      32,245,593
4.txt    3,721,999    332,799,453
5.txt    37,089,272   3,261,066,928

No Cheating

When benchmarking disk I/O operations it is important to remember to clear the Windows File Cache before each run otherwise instead of reading the data directly from the file we will read it from the RAM which can skew the results dramatically; Therefore, we will use RAMMap from the Sysinternals suite to clear the cache before each run.

Okay now that we have all of the formality out of the way let’s start counting some lines!

The Simple Way

The simplest way to get the number of lines in a text file is to combine the File.ReadLines method with System.Linq.Enumerable.Count. The ReadLines method returns an IEnumerable<string> on which we can then call the Count method to get the result. Here’s how:

public long CountLinesLINQ(FileInfo file) =>
    File.ReadLines(file.FullName).Count();

Let us benchmark this nice and short method and see how it does against our files.

private void Benchmark(FileInfo file)
{
    Console.WriteLine(nameof(CountLinesLINQ));

    Stopwatch sw = Stopwatch.StartNew();
    int count = CountLinesLINQ(file);
    sw.Stop();

    Console.WriteLine("{0} | {1} | Gen0 - {2} | Gen1 - {3} | Gen2 - {4}",
        file.Name,
        sw.Elapsed.ToString(),
        GC.CollectionCount(0).ToString("N0"),
        GC.CollectionCount(1).ToString("N0"),
        GC.CollectionCount(2).ToString("N0"));
}

All we are doing here is run the method once for each of our input files (remembering to clear the cache before each run) and output the time taken as well as the number of times GC kicked in across all generations.

On my machine with a 1TB 7200 RPM 64MB cache HDD, Here’s what I get:

CountLinesLINQ
    1.txt | 00:00:00.5933717 | Gen0 - 4     | Gen1 - 0 | Gen2 - 0
    2.txt | 00:00:00.6265670 | Gen0 - 8     | Gen1 - 0 | Gen2 - 0
    3.txt | 00:00:03.4404236 | Gen0 - 210   | Gen1 - 0 | Gen2 - 0
    4.txt | 00:00:30.1371443 | Gen0 - 2,274 | Gen1 - 3 | Gen2 - 0
    5.txt System.OverflowException: Arithmetic operation resulted in an overflow.

Oops! Everything was going well until we got an OverflowException, why?

Well, the System.Linq.Enumerable.Count extension method on File.ReadLines returns an Int32 which has a max value of 2,147,483,647 and as we saw earlier, there are over 3 billion lines in the 5.txt file so no surprise there.

Also notice the number of GC_s that took place, we didn’t have any _Gen2 collection and that’s mainly due to the fact that we don’t need to hold on to each line read from the file so nothing survives the Gen1 collection but even without the expensive Gen2 collection we are still causing Gen0 and Gen1 which are both blocking operations requiring all user threads to be suspended.

Let’s first fix that OverflowException. You may have realised that we should have used the System.Linq.Enumerable.LongCount as opposed to System.Linq.Enumerable.Count which as the name suggests returns an Int64 but why not avoid LINQ altogether.

Slightly Less Simple Way

In the LINQ version, all the File.ReadLine does is new-up a StreamReader and yield return every line read from the file as an IEnumerable<string> which we then call System.Linq.Enumerable.Count on, so why don’t we do that ourselves this time avoiding the OverflowException:

public long CountLinesReader(FileInfo file)
{
    var lineCounter = 0L;
    using(StreamReader reader = new(file.FullName))
    {
        while(reader.ReadLine() != null)
        {
            lineCounter++;
        }
        return lineCounter;
    }
}

Again nothing really complicated here, we have the lineCounter declared as Int64 which we then increment every time a line is read from the reader. Okay, let’s benchmark it:

CountLinesReader
    1.txt | 00:00:00.5690903 | Gen0 - 4      | Gen1 - 0  | Gen2 - 0
    2.txt | 00:00:00.6243011 | Gen0 - 8      | Gen1 - 0  | Gen2 - 0
    3.txt | 00:00:03.2662413 | Gen0 - 210    | Gen1 - 0  | Gen2 - 0
    4.txt | 00:00:29.3749685 | Gen0 - 2,274  | Gen1 - 2  | Gen2 - 0
    5.txt | 00:04:28.1865611 | Gen0 - 22,663 | Gen1 - 25 | Gen2 - 2

The results are almost identical albeit slightly faster compared to the LINQ version but this time we managed to handle the 5.txt file. So can we do better? Is there a way to count the lines without allocating anything on the heap so that GC doesn’t have to run? After all we don’t really care about the content of the lines. Hmm…

What If…

what if we go through the file one character at the time and every time we encounter an end-of-line character we increment our counter. Sounds pretty easy but before jumping to the implementation let’s remember that there are 3 different flavors of end-of-line characters:

• Carriage Return (\r) used in Macintosh
• Line Feed (\n) used in UNIX
• Carriage Return Line Feed (\r\n) used in MS-DOS, Windows, OS/2

Even though \r\n is the most used end-of-line character these days but we still need to make sure our algorithm can handle other flavors.

So what if we do this:

private const char CR = '\r';
private const char LF = '\n';
private const char NULL = (char)0;

public static long CountLinesSmarter(Stream stream)
{
    Ensure.NotNull(stream, nameof(stream));

    long lineCount = 0L;

    byte[] byteBuffer = new byte[1024 * 1024];
    char prevChar = NULL;
    bool pendingTermination = false;

    int bytesRead;
    while ((bytesRead = stream.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
    {
        for (int i = 0; i < bytesRead; i++)
        {
            char currentChar = (char)byteBuffer[i];

            if (currentChar == NULL) { continue; }
            if (currentChar == CR || currentChar == LF)
            {
                if (prevChar == CR && currentChar == LF) { continue; }

                lineCount++;
                pendingTermination = false;
            }
            else
            {
                if (!pendingTermination)
                {
                    pendingTermination = true;
                }
            }
            prevChar = currentChar;
        }
    }

    if (pendingTermination) { lineCount++; }
    return lineCount;
}

Take your time to study what it actually does but all it is doing is incrementing a counter whenever it sees an end-of-line character.

Now that we have a working algorithm why don’t we also sprinkle some C#7 goodness over it! We can rewrite the body of the for loop as a switch statement with some pattern matching case blocks. You can read more about this feature HERE.

That then gives us:

public static long CountLinesSmarter(Stream stream)
{
    Ensure.NotNull(stream, nameof(stream));

    long lineCount = 0L;

    byte[] byteBuffer = new byte[1024 * 1024];
    char prevChar = NULL;
    bool pendingTermination = false;

    int bytesRead;
    while ((bytesRead = stream.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
    {
        for (int i = 0; i < bytesRead; i++)
        {
            char currentChar = (char)byteBuffer[i];
            switch (currentChar)
            {
                case NULL:
                case LF when prevChar == CR:
                    continue;
                case CR:
                case LF when prevChar != CR:
                    lineCount++;
                    pendingTermination = false;
                    break;
                default:
                    if (!pendingTermination)
                    {
                        pendingTermination = true;
                    }
                    break;
            }
            prevChar = currentChar;
        }
    }

    if (pendingTermination) { lineCount++; }
    return lineCount;
}

Decisions Decisions

In terms of readability, both versions look pretty similar but which one should we pick? Is there any performance difference between the two versions? hmm…

Profiling Time

Let’s take our dotTrace profiler for a spin. We are going to use the line-by-line profiling which will give us the most accurate timing information on each statement we execute.

First the version with the if statement:

And now the one with the switch statement:

I let you go through the result at your leisure but a simple summing up of the numbers shows us that the switch version executes 15% fewer statements than the if version. So there we go, we know which version to pick.

If                  Switch
---------------------------
1                   1
1                   1
1                   1
1                   1
1                   1
3,636               3,636
3,635               3,635
3,811,330,537       3,811,330,537
3,811,326,902       3,811,326,902
3,811,326,902       3,811,326,902
3,811,326,902       3,811,326,902
3,811,326,902       332,799,453
665,598,906         332,799,453
332,799,453         332,799,453
332,799,453         332,799,453
332,799,453         3,145,727,996
3,145,727,996       332,799,453
332,799,453         3,478,527,449
3,478,527,449       1
1                   1
1
--------------------------------------
27,677,697,586      23,533,571,231

Refactor

Our Switch algorithm doesn’t look too bad but the fact that we have created a state-machine looks less satisfying to me so why don’t we refactor it slightly and see if we can reduce the number of comparisons we make.

public static long CountLinesSmarter(Stream stream)
{
    Ensure.NotNull(stream, nameof(stream));

    long lineCount = 0L;

    byte[] byteBuffer = new byte[1024 * 1024];
    char detectedEOL = NULL;
    char currentChar = NULL;

    int bytesRead;
    while ((bytesRead = stream.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
    {
        for (int i = 0; i < bytesRead; i++)
        {
            currentChar = (char)byteBuffer[i];

            if (detectedEOL != NULL)
            {
                if (currentChar == detectedEOL)
                {
                    lineCount++;
                }
            }
            else if (currentChar == LF || currentChar == CR)
            {
                detectedEOL = currentChar;
                lineCount++;
            }
        }
    }

    // We had a NON-EOL character at the end without a new line
    if (currentChar != LF && currentChar != CR && currentChar != NULL)
    {
        lineCount++;
    }
    return lineCount;
}

This time we first try detecting the end-of-line character and then once we have it we can just check for that character and increment our counter without having to keep track of the previous character.

Let us see what the profiler says this time:

I like the smell of less statements to execute in a for loop!

Micro-Micro-Optimisation

By looking at the profiling sessions above you may jump to the conclusion that since we are spending too much time in the for loop why not unroll it.

For those unfamiliar with the concept of Loop Unrolling, it is an optimisation technique with the goal of reducing the overhead of a loop by executing fewer iterations but doing more work per iteration. So if we were to unroll the above implementation we can come up with something like:

public static long CountLinesMaybe(Stream stream)
{
    Ensure.NotNull(stream, nameof(stream));

    long lineCount = 0L;

    byte[] byteBuffer = new byte[1024 * 1024];
    const int BytesAtTheTime = 4;
    char detectedEOL = NULL;
    char currentChar = NULL;

    int bytesRead;
    while ((bytesRead = stream.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
    {
        int i = 0;
        for (; i <= bytesRead - BytesAtTheTime; i += BytesAtTheTime)
        {
            currentChar = (char)byteBuffer[i];

            if (detectedEOL != NULL)
            {
                if (currentChar == detectedEOL) { lineCount++; }

                currentChar = (char)byteBuffer[i + 1];
                if (currentChar == detectedEOL) { lineCount++; }

                currentChar = (char)byteBuffer[i + 2];
                if (currentChar == detectedEOL) { lineCount++; }

                currentChar = (char)byteBuffer[i + 3];
                if (currentChar == detectedEOL) { lineCount++; }
            }
            else
            {
                if (currentChar == LF || currentChar == CR)
                {
                    detectedEOL = currentChar;
                    lineCount++;
                }
                i -= BytesAtTheTime - 1;
            }
        }

        for (; i < bytesRead; i++)
        {
            currentChar = (char)byteBuffer[i];

            if (detectedEOL != NULL)
            {
                if (currentChar == detectedEOL) { lineCount++; }
            }
            else
            {
                if (currentChar == LF || currentChar == CR)
                {
                    detectedEOL = currentChar;
                    lineCount++;
                }
            }
        }
    }

    if (currentChar != LF && currentChar != CR && currentChar != NULL)
    {
        lineCount++;
    }
    return lineCount;
}

This abomination would certainly result in a better performance if we were not I/O bound but in our case any performance gained by unrolling the loop is dwarfed by the time taken to read the bytes from the disk. Therefore, we are going to stick to the non-unrolled version.

What we have come up with so far is by no means anywhere close to the most optimal solution specially if you are prepared to use unsafe combined with byte-masking but it is pretty much as far as I am prepared to go with optimising the algorithm for now as my main goal was to eliminate allocation.

Moment of Truth

Now that we have our candidate algorithm let’s run the benchmarks:

CountLinesSmart
    1.txt | 00:00:00.5498381 | Gen0 - 0 | Gen1 - 0 | Gen2 - 0
    2.txt | 00:00:00.5834317 | Gen0 - 0 | Gen1 - 0 | Gen2 - 0
    3.txt | 00:00:03.0332798 | Gen0 - 0 | Gen1 - 0 | Gen2 - 0
    4.txt | 00:00:25.3517502 | Gen0 - 0 | Gen1 - 0 | Gen2 - 0
    5.txt | 00:04:13.1127603 | Gen0 - 0 | Gen1 - 0 | Gen2 - 0

Voila! Not only we improved the overall execution time but also we managed to be nice to the GC by not causing any collections.

How does it compare to wc -l?

In case you are wondering how fast you can count these lines in Linux (using time wc -l <file-name>), here are some numbers:

time wc -l <file-name>
    1.txt | real 0m00.680s | user 0m0.000s | sys 0m0.031s
    2.txt | real 0m00.719s | user 0m0.015s | sys 0m0.061s
    3.txt | real 0m03.218s | user 0m0.015s | sys 0m0.046s
    4.txt | real 0m25.966s | user 0m0.015s | sys 0m0.046s
    5.txt | real 4m13.669s | user 0m0.015s | sys 0m0.000s

And finally, I have added this implementation to Easy.Common. You can use it by calling the CountLines extension method on any Stream. The source is HERE and unit tests are HERE.

Unless all you build are HelloWorld applications which never get deployed anywhere you almost certainly have come across cases where either you or your support team have had to figure out:

• What is causing that OutOfMemory exception?
• Why is there such a performance degradation after the last release?
• Why is the DLL I have referenced not doing what it is supposed to?
• What system is my app running in?
• What arguments the app has started with?
• What version of JSON.NET is the app using?
• …

Over the years I have found myself and my colleagues spending precious time trying to deal with finding the answers to such questions over and over again particularly at times where there are production issues and downtime could cost the business a lot of money. So today I have decided to share with you a helper class I built a while back which can help answer such questions more proactively. The idea is to log the report generated by this class at the start of your application and if required on demand or at certain intervals.

The class is very simple to use and currently supports Windows, Linux and OSX:

DiagnosticReport report = DiagnosticReport.Generate();
Console.WriteLine(report.ToString());

Which then generates a report that looks like:

/
|Diagnostic Report generated at: 13-11-2017 18:14:41.174 (+00:00) in: 849.7042 milliseconds.
|
|System|..................................................................
|
|    . OS - Name         : Microsoft Windows 10.0.14393
|    . OS - Type         : Windows (Server)
|    . OS - 64Bit        : True
|    . CLR Runtime       : .NET Framework 4.7.2110.0
|    . Machine Name      : MyServer.
|    . FQDN              : MyServer
|    . User              : MyServer\admin
|    . CPU               : Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|    . CPU Core Count    : 2
|    . Installed RAM     : 16GB
|    . System Directory  : C:\Windows\system32
|    . Current Directory : C:\Users\admin\SampleApp
|    . Runtime Directory : C:\Windows\Microsoft.NET\Framework64\v4.0.30319\
|    . Uptime            : 20Day 12Hour 00Min 08Sec
|
|Process|......................................................................................................
|
|    . Id                  : 348
|    . Name                : LINQPad.UserQuery
|    . Started             : 13-11-2017 18:13:18.161 (+00:00)
|    . Loaded In           : 00:01:23.1225614
|    . Interactive         : True
|    . Optimized           : True
|    . 64Bit Process       : True
|    . Server GC           : True
|    . Large Address Aware : True
|    . WorkingSet          : 42MB
|    . Threads             : 88
|    . IO Threads          : 24 <-> 1,000
|    . Worker Threads      : 24 <-> 32,767
|    . File Version        : 1.0.0.0
|    . Product Version     : 1.0.0.0
|    . Language            : Language Neutral
|    . Copyright           : Copyright © Nima Ara 2017
|    . Original File Name  : LINQPad.UserQuery.exe
|    . File Name           : C:\Users\admin\AppData\Local\LINQPad\ProcessServer5AnyCPUB\LINQPad.UserQuery.exe
|    . Module Name         : LINQPad.UserQuery.exe
|    . Module File Name    : C:\Users\admin\AppData\Local\LINQPad\ProcessServer5AnyCPUB\LINQPad.UserQuery.exe
|    . Product Name        : A Sample Query
|    . CommandLine         : C:\Users\admin\AppData\Local\LINQPad\ProcessServer5AnyCPUB\LINQPad.UserQuery.exe
|                            C:/Program Files/LINQPad5/LINQPad.EXE
|                            C:/Program Files/LINQPad5/
|                            C:/Program Files/LINQPad5/LINQPad.config
|                            LINQPad.ivddgmnj.Server.2
|                            ivddgmnj
|                            4328
|                            True
|
|Drives|.......................................................................................
|
|     -----------------------------------------------------------------------------------------
|    | Name | Type      | Format | Label             | Capacity(GB) | Free(GB) | Available(GB) |
|    |------|-----------|--------|-------------------|--------------|----------|---------------|
|    | A:\  | Removable |        |                   | 0            | 0        | 0             |
|    | C:\  | Fixed     | NTFS   |                   | 127          | 108      | 108           |
|    | D:\  | Fixed     | NTFS   | Temporary Storage | 14           | 13       | 13            |
|    | E:\  | CDRom     |        |                   | 0            | 0        | 0             |
|     -----------------------------------------------------------------------------------------
|
|Assemblies|........................................................................................................................................
|
|    001| Name      : mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscorlib.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.NET/Framework64/v4.0.30319/mscorlib.dll
|
|    002| Name      : System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
|
|    003| Name      : System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Drawing\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Drawing.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
|
|    004| Name      : System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
|
|    005| Name      : System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Windows.Forms\v4.0_4.0.0.0__b77a5c561934e089\System.Windows.Forms.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
|
|    006| Name      : System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
|
|    007| Name      : System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
|
|    008| Name      : System.Xml.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.Linq.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml.Linq/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.Linq.dll
|
|    009| Name      : System.Data.Linq, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Data.Linq\v4.0_4.0.0.0__b77a5c561934e089\System.Data.Linq.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Data.Linq/v4.0_4.0.0.0__b77a5c561934e089/System.Data.Linq.dll
|
|    010| Name      : System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_64/System.Data/v4.0_4.0.0.0__b77a5c561934e089/System.Data.dll
|
|    011| Name      : Microsoft.CSharp, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
|       . GAC       : True
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.CSharp\v4.0_4.0.0.0__b03f5f7f11d50a3a\Microsoft.CSharp.dll
|       . CodeBase  : file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/Microsoft.CSharp/v4.0_4.0.0.0__b03f5f7f11d50a3a/Microsoft.CSharp.dll
|
|    012| Name      : LINQPad, Version=1.0.0.0, Culture=neutral, PublicKeyToken=21353812cd2a2db5
|       . GAC       : False
|       . 64Bit     : True
|       . Optimized : False
|       . Framework : .NETFramework,Version=v4.5.2
|       . Location  : C:\Program Files\LINQPad5\LINQPad.exe
|       . CodeBase  : file:///C:/Program Files/LINQPad5/LINQPad.EXE
|
|    013| Name      : LINQPad.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=21353812cd2a2db5
|       . GAC       : False
|       . 64Bit     : True
|       . Optimized : False
|       . Framework : .NETFramework,Version=v4.5.2
|       . Location  :
|       . CodeBase  : file:///C:/Program Files/LINQPad5/LINQPad.exe
|
|    014| Name      : query_viulpd, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
|       . GAC       : False
|       . 64Bit     : True
|       . Optimized : False
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Users\admin\AppData\Local\Temp\2\LINQPad5\_tqnmweww\query_viulpd.dll
|       . CodeBase  : file:///C:/Users/admin/AppData/Local/Temp/2/LINQPad5/_tqnmweww/query_viulpd.dll
|
|    015| Name      : Easy.Common, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
|       . GAC       : False
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NETFramework,Version=v4.5
|       . Location  : C:\Users\admin\AppData\Local\Temp\2\LINQPad5\_tqnmweww\shadow_ebuhxn\easy.common.dll
|       . CodeBase  : file:///C:/Users/admin/AppData/Local/Temp/2/LINQPad5/_tqnmweww/shadow_ebuhxn/easy.common.dll
|
|    016| Name      : System.Runtime.InteropServices.RuntimeInformation, Version=4.0.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
|       . GAC       : False
|       . 64Bit     : True
|       . Optimized : True
|       . Framework : .NET 2, 3 or 3.5
|       . Location  : C:\Users\admin\AppData\Local\Temp\2\LINQPad5\netstandard2\System.Runtime.InteropServices.RuntimeInformation.dll
|       . CodeBase  : file:///C:/Users/admin/AppData/Local/Temp/2/LINQPad5/netstandard2/System.Runtime.InteropServices.RuntimeInformation.dll
|
|Environment-Variables|..................................................................................................................................................................................................
|
|    001. ALLUSERSPROFILE           : C:\ProgramData
|    002. APPDATA                   : C:\Users\admin\AppData\Roaming
|    003. CLIENTNAME                : SomeClient
|    004. CommonProgramFiles        : C:\Program Files\Common Files
|    005. CommonProgramFiles(x86)   : C:\Program Files (x86)\Common Files
|    006. CommonProgramW6432        : C:\Program Files\Common Files
|    007. COMPUTERNAME              : MyServer
|    008. ComSpec                   : C:\Windows\system32\cmd.exe
|    009. HOMEDRIVE                 : C:
|    010. HOMEPATH                  : \Users\admin
|    011. LOCALAPPDATA              : C:\Users\admin\AppData\Local
|    012. LOGONSERVER               : \\MyServer
|    013. NUMBER_OF_PROCESSORS      : 2
|    014. OS                        : Windows_NT
|    015. Path                      : C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\LINQPad5;C:\Users\admin\AppData\Local\Microsoft\WindowsApps;
|    016. PATHEXT                   : .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
|    017. PROCESSOR_ARCHITECTURE    : AMD64
|    018. PROCESSOR_IDENTIFIER      : Intel64 Family 6 Model 63 Stepping 2, GenuineIntel
|    019. PROCESSOR_LEVEL           : 6
|    020. PROCESSOR_REVISION        : 3f02
|    021. ProgramData               : C:\ProgramData
|    022. ProgramFiles              : C:\Program Files
|    023. ProgramFiles(x86)         : C:\Program Files (x86)
|    024. ProgramW6432              : C:\Program Files
|    025. PSModulePath              : C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules;C:\Program Files\Microsoft Monitoring Agent\Agent\PowerShell\
|    026. PUBLIC                    : C:\Users\Public
|    027. SESSIONNAME               : RDP-Tcp#58
|    028. SystemDrive               : C:
|    029. SystemRoot                : C:\Windows
|    030. TEMP                      : C:\Users\admin\AppData\Local\Temp\2
|    031. TMP                       : C:\Users\admin\AppData\Local\Temp\2
|    032. USERDOMAIN                : MyServer
|    033. USERDOMAIN_ROAMINGPROFILE : MyServer
|    034. USERNAME                  : admin
|    035. USERPROFILE               : C:\Users\admin
|    036. windir                    : C:\Windows
|
|Networks|.....................................................................
|
|    . Windows IP Configuration
|      -------------------------
|      | Host               : MyServer
|      | Domain             :
|      | DHCP Scope         :
|      | Node Type          : Hybrid
|      | Is WINS Proxy      : False
|
|    . Microsoft Hyper-V Network Adapter #2
|      -------------------------------------
|      | Name               : Ethernet 2
|      | MAC                : 00:0D:3B:A3:D6:AC
|      | Type               : Ethernet
|      | Status             : Up
|      | Supports Multicast : True
|      | Is Receive Only    : False
|      | Speed              : 10,000 Mbit/s
|      | IP Addresses       : [1.2.3.4]
|
|    . Software Loopback Interface 1
|      ------------------------------
|      | Name               : Loopback Pseudo-Interface 1
|      | MAC                :
|      | Type               : Loopback
|      | Status             : Up
|      | Supports Multicast : True
|      | Is Receive Only    : False
|      | Speed              : 1,073 Mbit/s
|
|    . Microsoft ISATAP Adapter #2
|      ----------------------------
|      | Name               : isatap.MyServer.a8.internal.cloudapp.net
|      | MAC                : 00:00:00:00:00:00:00:E0
|      | Type               : Tunnel
|      | Status             : Down
|      | Supports Multicast : False
|      | Is Receive Only    : False
|      | Speed              : 0 Mbit/s
|
|    . Teredo Tunneling Pseudo-Interface
|      ----------------------------------
|      | Name               : Teredo Tunneling Pseudo-Interface
|      | MAC                : 00:00:00:00:00:00:00:E0
|      | Type               : Tunnel
|      | Status             : Up
|      | Supports Multicast : False
|      | Is Receive Only    : False
|      | Speed              : 0 Mbit/s
|
\

What does it produce?

The above report was generated in about 849 milliseconds on its first run and contains various information that are relevant to your application. This information has been categorised into six main sections which you can find in more detail at the bottom of this article but here’s an overview:

1. System
2. Process
3. Drives
4. Assemblies
5. Environment Variables
6. Networking

If you are not interested in generating the full report, you have the option to only generate a specific section:

DiagnosticReport drivesReport = DiagnosticReport.Generate(DiagnosticReportType.Drives);

or combine multiple sections:

DiagnosticReport drivesAndNetworkingReport = DiagnosticReport.Generate(
    DiagnosticReportType.Drives | DiagnosticReportType.Networking);

or only exclude a specific section:

DiagnosticReport partialReport = DiagnosticReport.Generate(
       DiagnosticReportType.Full &~ DiagnosticReportType.EnvironmentVariables);

It is as simple as that!

Where do I find it?

You can see the source of the class in DiagnosticReport and can start generating your own reports by referencing the Easy.Common NuGet package from your .NET or .NET Core app.

What information is covered?

System

Contains information relating to the system under which the application is running:

• Name of the operating system
• Type of the operating system e.g Windows, Linux or OSX
• Flag indicating whether the operating system is 64-bit capable
• Version of the CLR
• machine name
• Fully Qualified Domain Name (FQDN)
• User under which the process is running
• Processor name
• Number of processor cores including HyperThreading
• Number of installed RAM
• Location of the System directory
• Location of the current directory the application is running
• Location where the CLR is installed
• Duration the system has been up

Process

Contains information relating to the current process:

• Process ID (PID)
• Process name
• Time at which the process was started
• Time taken for the process to be loaded
• Flag indicating whether the process is running in User Interactive mode (This will be false for a Windows Service or a service such as IIS that runs without a UI)
• Flag indicating whether the application has been compiled in Release mode
• Flag indicating whether the process is 64-bit
• Flag indicating whether the GC mode is Server
• Flag indicating whether the process is Large Address Aware
• Current value of Working Set memory (RAM) in use by the process. This value includes both Shared and Private memory
• Number of threads owned by the process
• Minimum and maximum number of IO Completion Port threads
• Minimum and maximum number of Thread Pool Worker threads
• File version of the process
• Version of the product the process is distributed with
• Default language for the process
• Copyright notices that apply to the process
• Name of the file the process was created as
• File name representing the process
• Name of the process module
• File representing the process module
• Name of the product the process is distributed with
• CommandLine including any arguments passed into the process

Drives

Contains information relating to the system drives including fixed as well as removable ones:

• Name of the drive
• Type of the drive e.g. Fixed, CDRom etc.
• Format of the drive e.g. NTFS
• Label of the drive
• Capacity of the drive
• Total amount of free space available on the drive (not just what is available to the current user)
• Amount of free space available on the drive (this value takes into account disk quotas)

Assemblies

Contains information relating to the assemblies which have been loaded by the process:

• Full name of the assembly
• Flag indicating whether the assembly has been loaded from the GAC
• Flag indicating whether the assembly is 64-bit
• Flag indicating whether the assembly has been compiled in Release mode
• framework for which the assembly has been compiled
• Location from which the assembly has been loaded from
• Path to the location where the assembly was found. (If the assembly was downloaded from the web, this value may start with HTTP)

Environment Variables

Contains the name and value of the environment variables available to the process.

Networking

Contains information relating to the networking infrastructure available to the process:

• Windows IP Configuration including:

  • Host name
  • DNS details

• Adapter information:

  • Name & description
  • Status
  • Speed
  • Physical address (MAC)
  • IP addresses
  • DHCP details

Enjoy!

Despite this low barrier of entry, many developers fail to achieve the best performance out of their multi-threaded solutions. This is sometimes due to frequent or unnecessary locking/synchronization or at other times due to incorrectly partitioning their data or using an inappropriate pattern.

In this article we are going to look at how we can squeeze the best performance out of an easily parallelizable problem by rewriting the same basic implementation using some of the most widely used multi-threading tools available in .NET. We will be covering different .NET technologies such as PLINQ, BlockingCollection, Parallel class as well as TPL Dataflow in conjunction with patterns such as MapReduce, ActorModel as well as ProducerConsumer while trying to achieve optimal parallelization and minimize the cost of synchronization between the threads.

The Scenario

The example scenario we will be solving is to count the number of words in a text file which is the IMDB database for movie plots downloaded from HERE. The file (as of today) is 432MB with 8,151,603 lines consisting of 69,275,414 words and 432,045,742 characters. So our job is to figure out what the top n words and their respective counts are in this file. The overall work required to calculate the result can be summarized as:

1. Read every line in the file
2. Convert each line into words
3. Exclude invalid words e.g. “a”, “and”, “with” etc.
4. Count and track the occurrence of each word
5. Sort the words from the highest count to the lowest
6. Return the top n words e.g. 100

Our final result should display something like this:

life   128,714
new    119,587
man    96,257
time   89,828
world  88,863
young  87,279
film   82,800
love   80,002
family 75,530
home   74,265
story  73,211
old    62,433
day    62,319
finds  61,394
father 60,055
...

Approach

We will be measuring the following for each of the patterns we implement:

1. The total execution time of the program
2. The count of times Garbage Collection occurs across different generations i.e. 0, 1 and 2
3. The process’s Peak Working Set memory which is the maximum amount of physical memory (RAM) used by the process.

In addition to the above, for each implementation I will also include a Time-Line profiling snapshot of the process using the awesome dotTrace profiler developed by JetBrains (available as a 10 day trial). This allows us to analyze the context under which our thread(s) were executing and the level of concurrency and CPU utilization we achieved for that implementation.

You can use other tools and profilers such as the Visual Studio Performance Profiling, ANTS Performance Profiler by RedGate or the Concurrency Visualizer extension for Visual Studio. There is also a free new comer built using C# and WPF called CodeTrack; It looks quite promising and I highly suggest taking a look and having a play.

In order to increase our application’s throughput, we will set the GC mode to Server. A Server GC mode results in less frequent GC across all generations but it also means that each CPU core will now have its own heap as well as a dedicated thread responsible for collecting every dead references in that heap so on my 24-core server we will have 24 threads dedicated to GC but that is not all, thanks to the Background Server GC which was introduced in .NET4.5 the number of threads can increase even more! This feature is used to offload a significant subset of the Gen-2 GC by using dedicated background threads resulting in a significant reduction of the total pause times on the user threads but remember, this only applies to Gen-2 as both Gen-0 and Gen-1 collections still require all user threads to be suspended.

Also remember that under Server GC, heap sizes are much larger compared to the Workstation GC so don’t be alarmed if you see large memory usage in our implementations.

Note:

In order to keep the focus of this article on the concurrency costs of each implementation rather than the behavior of GC, I have excluded all of the GC threads from our profiling results. This includes the Finalizer as well as the background GC threads. But to investigate how the GC performed for each of the algorithms, you can include all those threads in the snapshot file provided for each implementation (by installing & running dotTrace).

I will be running the application on a 24-core dual CPU workstation with 24GB of RAM and the input file is being read from a 7200 RPM HDD with 64MB of cache. Each pattern will be profiled individually and the Windows File Cache will be cleared between each run.

Finally, You can find the code HERE. Okay, now that we have all those covered, let’s get started!

1. Sequential

private static IDictionary<string, uint> GetTopWords()
{
    Dictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);

    // process lines as IEnumerable<string> read one after another from the disk.
    foreach (var line in File.ReadLines(InputFile.FullName))
    {
        foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        {
            if (!IsValidWord(word)) { continue; }
            TrackWordsOccurrence(result, word);
        }
    }

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

private static void TrackWordsOccurrence(IDictionary<string, uint> wordCounts, string word)
{
    if (wordCounts.TryGetValue(word, out uint count))
    {
        wordCounts[word] = count + 1;
    }
    else
    {
        wordCounts[word] = 1;
    }
}

private static bool IsValidWord(string word) =>
    !InvalidCharacters.Contains(word[0]) && !StopWords.Contains(word);

There is nothing complicated about this code, all we are doing here is reading a bunch of lines splitting them into words, doing some filtering and keeping track of each word in a Dictionary<string, uint>. We then order the result by the count of each word in descending order and return n entries from the result.

Running the above produces the following:

Execution time: 87,025 ms
Gen-0: 27, Gen-1: 10, Gen-2: 4
Peak WrkSet: 617,037,824

As we can see, the program achieved an average CPU utilization of 4.0%, took 87,025 milliseconds out of which 0.6% was spent on GC. 91.3% of the main thread time was in the Running state and 8.7% in Waiting. The process reached a peak memory of ~600MB. Since this implementation is a straight-forward single-threaded execution, there was no locking involved therefore no cost associated to locking-contention.

2. Sequential (LINQ)

Next, the same sequential algorithm implemented using LINQ:

private static IDictionary<string, uint> GetTopWords()
{
    return File.ReadLines(InputFile.FullName)
        .SelectMany(l => l.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        .Where(IsValidWord)
        .ToLookup(x => x, StringComparer.InvariantCultureIgnoreCase)
        .Select(x => new { Word = x.Key, Count = (uint)x.Count() })
        .OrderByDescending(kv => kv.Count)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Word, kv => kv.Count);
}

Here we are projecting each line into a sequence of words, filter out the invalid words and then group the words using ToLookup (or GroupBy) to get their count; finally we order them and take the top n items.

Here’s the result:

Execution time: 92,174 ms
Gen-0: 6, Gen-1: 4, Gen-2: 3
Peak WrkSet: 4,579,205,120

This time, the program achieved an average CPU utilization of 7.6%, took 92,174 milliseconds out of which 8.7% was spent on GC. 82.5% of all our threads time were in the Running state and 17.5% in Waiting. Exploring the snapshot file reveals that around 40% was used doing LINQ operations so there is no surprise to see the inferior performance (due to LINQ’s overhead). Just as above, there is no locking involved. Also note our process resulted in a whopping 4.6GB of peak memory.

3. PLINQ (Naive)

As the title suggests here we will build a hybrid of PLINQ and a single-threaded solution. The code is similar to the sequential LINQ with the addition of .AsParallel() which does all the magic!

The work relating to reading each line, projecting the words and filtering will be distributed across all the available system cores and the work relating to keeping track of each of the word’s occurrence is done on the main thread. In other words the main thread in our foreach loop pulls each line from the disk all the way through the PLINQ parallelized pipeline.

private static IDictionary<string, uint> GetTopWords()
{
    var words = File.ReadLines(InputFile.FullName)
        .AsParallel()
        .SelectMany(l => l.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        .Where(IsValidWord);

    Dictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);
    foreach (var word in words)
    {
        TrackWordsOccurrence(result, word);
    }

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

Running the above results in the following:

Execution time: 80,620 ms
Gen-0: 10, Gen-1: 7, Gen-2: 4
Peak WrkSet: 1,677,287,424

This time, the program achieved an average CPU utilization of 17.5%, took 80,620 milliseconds out of which 1.0% was spent on GC. 17% of all our threads time were in the Running state and 83% in Waiting. Exploring the snapshot file reveals that around 18% was spent doing LINQ operations there is also very little locking contention (0.07%) since the main part of the app which requires synchronization is actually running on the main thread. Also note our process resulted in a 1.7GB of peak memory.

Okay, so not much of an improvement but be ready as things are about to change!

A bit about Partitioning

Before continuing any further I feel I should mention an important aspect of PLINQ which is responsible for splitting and assigning each input element to a worker thread. This is known as Partitioning and there are currently four different partitioning strategies in TPL all of which are supported by PLINQ; These are:

1. Chunk partitioning
2. Range partitioning
3. Hash partitioning
4. Striped partitioning

Chunk partitioning uses dynamic element allocation as opposed to static element allocation (which is used by both Range and Hash partitioning). Dynamic allocation allows partitions to be created on the fly as opposed to being allocated in advance which means that a dynamic partitioner needs to request elements more frequently from the input sequence resulting in more synchronization; In contrast, static partitioning allows elements to be assigned all at once which results in less overhead and no additional synchronization after the initial work required for creating the partitions. Note that Parallel.ForEach only supports dynamic partitioning (unless you implement one yourself).

PLINQ chooses Range partitioning for any sequence which can be indexed into i.e IList<T> (or ParallelEnumerable.Range) which usually results in a better performance unless the amount of processing required for each element is not constant. For example in a scenario where there is a non-uniform processing time (think of ray-tracing), threads can finish work on their assigned partitions early and sit there idle unable to process more items. For such scenarios with non-uniform processing times, Chunk partitioning may be a better strategy. It is possible to instruct PLINQ to choose Chunk partitioning for an indexable sequence by using Partitioner.Create(myArray, true).

Hash partitioning is used by query operators that require comparing elements (e.g. GroupBy, Join, GroupJoin, Intersect, Except, Union and Distinct). Hash partitioning can be inefficient in that it must precalculate the hashcode for each element (so that elements with identical codes are assigned and processed on the same thread).

Striped partitioning is the strategy used by SkipWhile and TakeWhile and is optimized for processing items at the head of a data source. In striped partitioning, each of the n worker threads is allocated a small number of items (sometimes 1) from each block of n items. The set of items belonging to a single thread is called a stripe, hence the name. A useful feature of this scheme is that there is no inter-thread synchronization required as each worker thread can determine its data via simple arithmetic. This is really a special case of range partitioning and only works on arrays and types that implement IList<T>.

Here is a diagram showing an overview of the most widely used Chunk and Range partitioning taken from Albahari’s must read Threading in C#.

For more details on Partitioning, I highly suggest having a look at Partitioning in PLINQ, Chunk partitioning vs range partitioning in PLINQ, Partitioning of Work and Custom Partitioners for PLINQ and TPL.

Okay, now that we know a little more about partitioning we can look at more PLINQ implementations.

4. PLINQ

This time we are going to let PLINQ do all the work by letting it partition the input sequence of lines (using Chunk partitioning) and see if we can do any better than what we have done so far.

private static IDictionary<string, uint> GetTopWords()
{
    return File.ReadLines(InputFile.FullName)
        .AsParallel()
        .SelectMany(l => l.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        .Where(IsValidWord)
        .ToLookup(x => x, StringComparer.InvariantCultureIgnoreCase)
        .AsParallel()
        .Select(x => new { Word = x.Key, Count = (uint)x.Count() })
        .OrderByDescending(kv => kv.Count)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Word, kv => kv.Count);
}

Running the above results in the following:

Execution time: 22,480 ms
Gen-0: 4, Gen-1: 2, Gen-2: 2
Peak WrkSet: 4,579,401,728

And just like that the program managed to achieve an average CPU utilization of 60.7%, took 22,480 milliseconds out of which 6.1% was spent on GC. 62.4% of all our threads time were in the Running state and 37.6% in Waiting. Exploring the snapshot file reveals that around 47% was spent executing LINQ operations and 14.5% on locking contention. Note the size of the WorkingSet is still at around 4.5GB.

Despite the great reduction in execution time we can still observe a substantial cost in locking contention. A large part of this cost is incurred during the grouping of the words (the step calling ToLookup(x => x, StringComparer.InvariantCultureIgnoreCase)).

5. PLINQ (MapReduce)

As we saw above, despite the simplicity and succinctness of the PLINQ implementation, we still incurred a high cost of locking so now we will use a different flavour of PLINQ which allows us to perform MapReduce. We are going to have a map phase where our problem can be split into smaller independent steps which can then be executed on any thread without the need for synchronization.

Each worker thread will then do the work of splitting the lines and excluding the invalid words and when the time comes to group them all back together, each thread will then merge reduce the result by locking over the shared collection which will happen per partition.

Let us see how this can be implemented using the Aggregate method in PLINQ:

private static IDictionary<string, uint> GetTopWords()
{
    return File.ReadLines(InputFile.FullName)
        .AsParallel()
        .Aggregate(
/*#1*/      () => new Dictionary<string, uint>(StringComparer.InvariantCultureIgnoreCase),
/*#2*/      (localDic, line) =>
            {
                foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
                {
                    if (!IsValidWord(word)) { continue; }
                    TrackWordsOccurrence(localDic, word);
                }
                return localDic;
            },
/*#3*/      (finalResult, localDic) =>
            {
                foreach (var pair in localDic)
                {
                    var key = pair.Key;
                    if (finalResult.ContainsKey(key))
                    {
                        finalResult[key] += pair.Value;
                    }
                    else
                    {
                        finalResult[key] = pair.Value;
                    }
                }
                return finalResult;
            },
/*#4*/      finalResult => finalResult
                .OrderByDescending(kv => kv.Value)
                .Take((int)TopCount)
                .ToDictionary(kv => kv.Key, kv => kv.Value)
        );
}

Still confusing? Here are the main steps again:

• Step #1: First we supply a Dictionary<string, uint> for each of the threads running our code so this means every thread will have a local collection to work with, eliminating the need for locking.

• Step #2: Here we have the main logic which will be executed by each thread for every line in our file, the scary looking Func<Dictionary<string, uint>, string, Dictionary<string, uint>> allows the dictionary declared at step #1 to be passed as localDic along with the line read from the disk to the body of the delegate. After the thread is done adding the words to the dictionary it will return the localDic so that it can be available to the same thread operating on the next line in the same partition.

• Step #3: Once every thread has finished processing all the items in its partition then all the items on its local dictionary is then merged (reduced) into a final result-set. Hence, the delegate is of type Func<Dictionary<string, uint>, Dictionary<string, uint>>. By this step our MapReduce implementation is actually complete however the Aggregate method offers us one final step.

• Step #4: This is where you have the chance to do any modification to the final result before returning it. In our case we are doing a simple ordering followed by a Take and projection back into a dictionary.

So let us see how we did this time:

Execution time: 21,915 ms
Gen-0: 12, Gen-1: 5, Gen-2: 3
Peak WrkSet: 1,082,503,168

This time the program achieved an average CPU utilization of 45.9%, it took 21,915 milliseconds out of which only 1.4% was spent on GC. 55.1% of all our threads time were in the Running state and 44.9% in Waiting. So despite the lower CPU utilization in comparison to the previous step the lower time in GC did actually improve the overall execution time. Exploring the snapshot file reveals that around 37% was spent executing LINQ operations and 6.9% on locking contention. You can also see that our peak memory usage was 4x smaller than the previous method. So while we may not have improved the execution time significantly we have managed to achieve more by using less CPU as well as a much lower memory footprint.

Not too shabby ha! :-)

6. PLINQ (ConcurrentDictionary)

PLINQ’s biggest advantage/convenience is thanks to its ability to collate the result of the parallelized work into a single output sequence and that is indeed what we have used until now. For this pattern we are going to do things a bit differently; First, we are not going to use PLINQ’s result collation. Instead, we are going to give it a function and tell it to run multiple threads each applying that function to our data. The method allowing us to achieve this is ForAll. That was the first part, now for the second part we are going to choose a different data type for tracking the count of words.

Sometimes choosing the right datatype (in our case collection), can make all the difference. There is no better example to demonstrate this than what we will see in this section. We are going to pick the ConcurrentDictionary as our collection and let it handle all the synchronization for us.

private static IDictionary<string, uint> GetTopWords()
{
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);

    File.ReadLines(InputFile.FullName)
        .AsParallel()
        .ForAll(line =>
        {
            foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
            {
                if (!IsValidWord(word)) { continue; }
                result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
            }
        });

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

There is not much to the code except the use of AddOrUpdate. Each thread is using this method to Add a given word with a count of 1, if the word was already added, it then takes the value (count) of that word and adds 1 to it. All of this is done in a thread-safe manner and any synchronization is handled by the ConcurrentDictionary. Okay, how did we do this time?

Execution time: 19,229 ms
Gen-0: 16, Gen-1: 8, Gen-2: 4
Peak WrkSet: 863,531,008

This time the program achieved an average CPU utilization of 53%, it took 19,229 milliseconds out of which only 7% was spent on GC. 65.7% of all our threads time were in the Running state and 34.3% in Waiting. Exploring the snapshot file reveals that around 33% was spent executing LINQ operations and 10.3% in locking.

7. PLINQ (ProducerConsumer)

In this final section of using PLINQ to achieve parallelism, we are going to cover my favorite pattern which I frequently use to deal with problems where I have one or more producers producing data and one or more workers consuming them.

For this algorithm, we have a single producer reading and adding (producing) the lines into some sort of a collection. We then have a bunch of consumers taking (consuming) and processing each line. The collection which allows such dance between the producer and its consumers is called the BlockingCollection<T> which by default is very similar to a [ConcurrentQueue](<https://msdn.microsoft.com/en-us/library/dd267265(v=vs.110).aspx>). There is more to `BlockingCollection` than I will cover today but the bits we are interested in are the _Add_ method which the producer will use to add each line to the collection; The `GetConsumingEnumerable()` which blocks each thread invoking it until there is an item in the collection and finally the `CompleteAdding` method which notifies the consumers that no more item will be added by the publisher(s) therefore unblocking every thread that is waiting on `GetConsumingEnumerable()`.

As far as PLINQ is concerned we are telling it to use at most 12 threads and also to not buffer the input so that as soon as the publisher adds a line, PLINQ can pass that to one of its 12 consumers and through the rest of the pipeline.

private static IDictionary<string, uint> GetTopWords()
{
    const int WorkerCount = 12;
    const int BoundedCapacity = 10_000;
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);

    // Setup the queue
    BlockingCollection<string> blockingCollection = new(BoundedCapacity);

    // Declare the worker
    Action<string> work = line =>
    {
        foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        {
            if (!IsValidWord(word)) { continue; }
            result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
        }
    };

    Task.Run(() =>
    {
        // Begin producing
        foreach (var line in File.ReadLines(InputFile.FullName))
        {
            blockingCollection.Add(line);
        }
        blockingCollection.CompleteAdding();
    });

    // Start consuming
    blockingCollection
        .GetConsumingEnumerable()
        .AsParallel()
        .WithDegreeOfParallelism(WorkerCount)
        .WithMergeOptions(ParallelMergeOptions.NotBuffered)
        .ForAll(work);

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

With the output of:

Execution time: 18,858 ms
Gen-0: 16, Gen-1: 9, Gen-2: 4
Peak WrkSet: 857,141,248

This time the program achieved an average CPU utilization of 30.7%, it took 18,858 milliseconds out of which only 7.2% was spent on GC. 67.9% of all our threads time were in the Running state and 32.1% in Waiting. Exploring the snapshot file reveals that around 15% was spent executing LINQ operations and 2.2% in locking contention.

As you can see, we managed to achieve very good concurrency by only using half of our cores and reduced the overall Waiting time and locking cost.

I cannot think of any other ways to squeeze any more performance juice out of PLINQ; (I am happy to be challenged by anyone on this, let me know in the comments) therefore, we are now going to start playing with the Parallel class.

8. Parallel.ForEach (MapReduce)

The Parallel class has a bunch of methods for us to use but the one we are interested in is the Parallel.ForEach. In it’s simplest form it accepts an IEnumerable<T> and an Action<T> which it then invokes in parallel for every item in the collection. Using this method however is not suitable for our use-case as we would need to synchronize access to our dictionary responsible for tracking the occurrence of each word therefore diminishing any benefits of parallelization. Instead, we are going to use the overload of the method which enables us to implement MapReduce. Here’s the code:

private static IDictionary<string, uint> GetTopWords()
{
    Dictionary<string, uint> result = new (StringComparer.InvariantCultureIgnoreCase);
    Parallel.ForEach(
        File.ReadLines(InputFile.FullName),
        new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount },
        () => new Dictionary<string, uint>(StringComparer.InvariantCultureIgnoreCase),
        (line, state, index, localDic) =>
        {
            foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
            {
                if (!IsValidWord(word)) { continue; }
                TrackWordsOccurrence(localDic, word);
            }
            return localDic;
        },
        localDic =>
        {
            lock (result)
            {
                foreach (var pair in localDic)
                {
                    var key = pair.Key;
                    if (result.ContainsKey(key))
                    {
                        result[key] += pair.Value;
                    }
                    else
                    {
                        result[key] = pair.Value;
                    }
                }
            }
        }
    );

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

The implementation is very similar to the MapReduce we implemented using PLINQ and as you saw before, the main idea behind this pattern is to ensure each thread has it’s local data to work with and then when all the threads have processed all their items they will then merge (reduce) their results into a single sequence therefore greatly reducing synchronization.

Here is what this implementation gets us:

Execution time: 19,606 ms
Gen-0: 23, Gen-1: 13, Gen-2: 7
Peak WrkSet: 942,002,176

Okay, not bad. We have an average CPU utilization of 35.9%, an execution time of 19,606 milliseconds out of which 3.4% was spent on GC. 40.5% of all our threads time were in the Running state and 59.5% in Waiting. Looking at the snapshot we can see that the cost of locking contention is a whopping 35%, why? This is mainly due to the Chunk Partitioning if we were using Range Partitioning, we would only need to lock over the result once per thread at the end of its partition but for that we would need to load all the lines into an indexable collection such as an Array or an IList which would be fine if you had all the available RAM. In our case however, I was happy to pay the cost of having to lock more frequently (due to much smaller chunks).

9. Parallel.ForEach (ConcurrentDictionary)

In this section we are going to again use the ConcurrentDictionary and see what it gets us when used with Parallel.ForEach.

private static IDictionary<string, uint> GetTopWords()
{
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);
    Parallel.ForEach(
        File.ReadLines(InputFile.FullName),
        new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount },
        (line, state, index) =>
        {
            foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
            {
                if (!IsValidWord(word)) { continue; }
                result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
            }
        }
    );

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

Which outputs:

Execution time: 19,622 ms
Gen-0: 13, Gen-1: 6, Gen-2: 3
Peak WrkSet: 853,209,088

I don’t know about you but I quite like the ConcurrentDictionary! We have an average CPU utilization of 52.8%, an execution time of 19,622 milliseconds out of which 6.7% was spent on GC. 61.6% of all our threads time were in the Running state and 38.4% in Waiting. Looking at the snapshot reveals the cost of locking contention to be 9.4%.

Despite the great result we can see from the profiling graph that we have over loaded the CPU. How can I tell? Well each of those pink sections on the graph is showing us the time intervals where the thread is ready to run on the next available CPU core; Therefore, long ready intervals (the time the thread is waiting so that it can switch its state from Waiting to Running) can mean thread starvation and CPU overload. To address it, we can try reducing the number of threads by playing with the MaxDegreeOfParallelism. So remember that CPU overloading/thread starvation can and will hurt the performance.

10. ProducerConsumer with Tasks

We talked about the ProducerConsumer pattern in the PLINQ section, what we are doing here is pretty much identical to what we did before except we are going to use Tasks to execute our consumers.

Note the most important thing to remember about this pattern is that once the worker threads finish processing each item they get from the queue, they will stay blocked waiting for more items until the queue is marked as complete. So depending on the size of your input, you need to make sure that when you are assigning worker threads using Tasks, you mark them with the [TaskCreationOptions.LongRunning])(https://msdn.microsoft.com/en-us/library/system.threading.tasks.taskcreationoptions(v=vs.110).aspx) flag otherwise the Hill Climbing aka Thread Injection algorithm employed by TPL thinks that threads in the ThreadPool are blocked so in order to prevent against dead-locks, it kicks in and inject extra threads to the ThreadPool at a rate of one thread every 500 milliseconds which is not what we want. By using this flag you are telling TPL to spin up a dedicated thread for each of your workers/consumers.

private static IDictionary<string, uint> GetTopWords()
{
    const int WorkerCount = 12;
    const int BoundedCapacity = 10_000;
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);

    // Setup the queue
    BlockingCollection<string> blockingCollection = new(BoundedCapacity);

    // Declare the worker
    Action work = () =>
    {
        foreach (var line in blockingCollection.GetConsumingEnumerable())
        {
            foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
            {
                if (!IsValidWord(word)) { continue; }
                result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
            }
        }
    };

    // Start the workers
    var tasks = Enumerable.Range(1, WorkerCount)
        .Select(n =>
            Task.Factory.StartNew(
                work,
                CancellationToken.None,
                TaskCreationOptions.LongRunning,
                TaskScheduler.Default))
        .ToArray();

    // Begin producing
    foreach (var line in File.ReadLines(InputFile.FullName))
    {
        blockingCollection.Add(line);
    }
    blockingCollection.CompleteAdding();
    // End of producing

    // Wait for workers to finish their work
    Task.WaitAll(tasks);

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

Which outputs:

Execution time: 31,979 ms
Gen-0: 15, Gen-1: 8, Gen-2: 4
Peak WrkSet: 935,489,536

We have an average CPU utilization of 26.5%, an execution time of 31,979 milliseconds out of which 4.3% was spent on GC. 59.1% of all our threads time were in the Running state and 40.9% in Waiting. Looking at the snapshot we can see that cost of locking contention was at 2%.

11. ProducerConsumer with Tasks (Easier)

As I mentioned before, I use this pattern regularly therefore, over the years I have built an opinionated wrapper around it to reduce boiler plate as well as providing easy exception handling and control over the producers and consumers. It can also be used as a single-thread worker (sequencer) which is another pattern of its own that I may write about in another article. It is part of my Easy.Common library available on NuGet. So let us implement what we did in the previous section by using the ProducerConsumerQueue.

private static IDictionary<string, uint> GetTopWords()
{
    const int WorkerCount = 12;
    const int BoundedCapacity = 10_000;
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);

    // Declare the worker
    Action<string> work = line =>
    {
        foreach (var word in line.Split(Separators, StringSplitOptions.RemoveEmptyEntries))
        {
            if (!IsValidWord(word)) { continue; }
            result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
        }
    };

    // Setup the queue
    ProducerConsumerQueue<string> pcq = new(work, WorkerCount, BoundedCapacity);
    pcq.OnException += (sender, ex) => Console.WriteLine("Oooops: " + ex.Message);

    // Begin producing
    foreach (var line in File.ReadLines(InputFile.FullName))
    {
        pcq.Add(line);
    }
    pcq.CompleteAdding();
    // End of producing

    // Wait for workers to finish their work
    pcq.Completion.Wait();

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

With the following output:

Execution time: 39,124 ms
Gen-0: 14, Gen-1: 7, Gen-2: 4
Peak WrkSet: 934,985,728

We have an average CPU utilization of 25.1%, an execution time of 39,124 milliseconds out of which 3.4% was spent on GC. 53.8% of all our threads time were in the Running state and 46.2% in Waiting. The snapshot shows the cost of locking contention as 2%.

12. TPL Dataflow

Finally and for the sake of completion we are going to implement the same solution using TPL Dataflow which provides a pipeline based execution and is great for implementing ActorModel scenarios as it provides (among other things) message passing. I am not going to focus much on Dataflow or ActorModel as there is a lot to cover.

We are going to start by creating the bufferBlock to which we push our lines, the block is then linked to splitLineToWordsBlock which as its name suggests is responsible for turning our lines into words. The splitLineToWordsBlock itself is linked to the batchWordsBlock which batches up our words into arrays of length 5000. Finally, each batch is then fed through the trackWordsOccurrencBlock which tracks the occurrence of each word.

Once we have all our blocks linked together, all we have to do is read our lines and feed them through the bufferBlock; When we are done pushing all the lines, we mark the bufferBlock as complete and wait for the trackWordsOccurrencBlock to finish consuming all its batches. Finally, just like the previous steps we order the result and return the top n entries.

private static IDictionary<string, uint> GetTopWords()
{
    const int WorkerCount = 12;
    ConcurrentDictionary<string, uint> result = new(StringComparer.InvariantCultureIgnoreCase);
    const int BoundedCapacity = 10_000;

    BufferBlock<string> bufferBlock = new(new DataflowBlockOptions { BoundedCapacity = BoundedCapacity });

    TransformManyBlock<string, string> splitLineToWordsBlock = new(
        line => line.Split(Separators, StringSplitOptions.RemoveEmptyEntries),
        new ExecutionDataflowBlockOptions
        {
            MaxDegreeOfParallelism = 1,
            BoundedCapacity = BoundedCapacity
        });

    BatchBlock<string> batchWordsBlock = new(5_000);

    ActionBlock<string[]> trackWordsOccurrencBlock = new(words =>
        {
            foreach (var word in words)
            {
                if (!IsValidWord(word)) { continue; }
                result.AddOrUpdate(word, 1, (key, oldVal) => oldVal + 1);
            }
        },
        new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = WorkerCount });

    DataflowLinkOptions defaultLinkOptions = new () { PropagateCompletion = true };
    bufferBlock.LinkTo(splitLineToWordsBlock, defaultLinkOptions);
    splitLineToWordsBlock.LinkTo(batchWordsBlock, defaultLinkOptions);
    batchWordsBlock.LinkTo(trackWordsOccurrencBlock, defaultLinkOptions);

    // Begin producing
    foreach (var line in File.ReadLines(InputFile.FullName))
    {
        bufferBlock.SendAsync(line).Wait();
    }

    bufferBlock.Complete();
    // End of producing

    // Wait for workers to finish their work
    trackWordsOccurrencBlock.Completion.Wait();

    return result
        .OrderByDescending(kv => kv.Value)
        .Take((int)TopCount)
        .ToDictionary(kv => kv.Key, kv => kv.Value);
}

Which outputs:

Execution time: 31,447 ms
Gen-0: 18, Gen-1: 10, Gen-2: 4
Peak WrkSet: 868,356,096

We have an average CPU utilization of 21.6%, an execution time of 31,447 milliseconds out of which 4.3% was spent on GC. 25.1% of all our threads time were in the Running state and 74.9% in Waiting. Looking at the snapshot we can see the cost of locking contention was at 0.01% which is not surprising as our entire processing was pipelined.

Conclusion

Here is a quick a summary of what each implementation achieved:

We saw that both the Parallel class and PLINQ can efficiently partition tasks and execute them across all the available cores in our system. We also learned how choosing a different data type, library or even different overloads of the same library can help us achieve better performance and CPU utilization.

We then looked at some of the other approaches to achieving parallelism mainly ProducerConsumer and ActorModel which may not have necessarily performed well for our DataParallelism scenario but can be suitable for TaskParallelism problems you may face in the future.

Finally, in order to help you choose between PLINQ and Parallel, I would like to mention an important difference between the two which goes back to how they were designed.

Parallel uses a concept referred to as Replicating Tasks which means a loop will start with one task for processing the loop body and if more threads are freed up to assist with the processing, additional tasks will be created to run on those threads which minimizes resource consumption in scenarios where most of the threads in the ThreadPool are busy doing other work. PLINQ on the other hand requires that a specific number of threads are actively involved for the query to make progress. This difference in design is the reason why there are ParallelOptions.MaxDegreeOfParallelism & PLINQ’s WithDegreeOfParallelism.

I hope you enjoyed reading this article as much as I enjoyed writing it and I invite you to share the patterns and techniques that you use for multi-threading and parallelism in the comments section below.

Have fun and let’s:

What are they?

Local Functions enable the definition of functions inside other functions. You can think of them as helper functions that would only make sense within a given context. Such context could be a function, constructor or a property getter or setter. They are transformed by the compiler to private methods meaning there is no overhead in calling them.

The main goal of Local Functions is encapsulation so the compiler is enforcing that such functions cannot be called from anywhere else in the class. Consider the following overly simplified example:

public string Address
{
    get
    {
        return $@"{GetHouseNumber()}, {GetStreet()},
                  {GetCity()},
                  {GetCountry()}";

        int GetHouseNumber() => 10;
        string GetStreet() => "Liberty St";
        string GetCity() => "Amsterdam";
        string GetCountry() => "Jamaica";
    }
}

Here we have a getter property which is composing an address comprised of 4 different segments. Each segment is being retrieved via a separate method, we are also taking advantage of the Expression Bodied and String Interpolation syntax introduced as part of C# 6 to keep our property clear and condensed.

Had we declared those methods as private, they could be invoked by any member in the class so by using Local Functions we have encapsulated their usage.

But we could do this with Lambdas, no?

Yes, if you ignore the constraint forcing us to declare our lambdas before using them, you could technically achieve the same result:

public string Address
{
    get
    {
        Func<int> getHouseNumber = () => 10;
        Func<string> getStreet = () => "Liberty St";
        Func<string> getCity = () => "Amsterdam";
        Func<string> getCountry = () => "Jamaica";

        return $@"{getHouseNumber()}, {getStreet()},
                  {getCity()},
                  {getCountry()}";
    }
}

BUT, doing so is not such a good idea for the following reasons:

• Allocation, By declaring those lambdas you are essentially allocating objects for the delegates on the heap so if your property is being invoked many times then you will end up putting non trivial unnecessary pressure on the GC.
• No support for ref, out, params or Optional Parameters.
• No recursion, in order to be able to call a Lambda recursively you would need to declare it in 2 steps which does not seem elegant to me:

private uint GetFactorial(uint number)
{
    var factorial = default(Func<uint, uint>);
    factorial = n =>
    {
        if (n < 2) { return 1; }
        return n * factorial(n - 1);
    };

    return factorial(number);
}

Limitations

Local Functions can be generic, asynchronous, dynamic and they can even have access to variables accessible in their enclosing scope however despite their flexibility they come with the following limitations:

• Cannot be static, you cannot declare a Local Function as static. (no longer true as of C# 8)
• No attributes allowed, Unlike normal methods you cannot use attributes inside a Local Function. This means you cannot use Caller Information so the following example does not compile:

private void Greet()
{
    string GetCaller([CallerMemberName] string name = null) => name;
    Console.WriteLine("Hello " + GetCaller());
}

This abstraction provides easier methods of communicating with a HTTP server in full asynchrony as well as allowing to set default headers for each and every request. This is all awesome an all but there are dark secrets about this class that if not addressed can cause serious performance and at times mind boggling bugs! In this article we will explore the problems subtleties that one would need to be aware of when working with the HttpClient class.

So what’s wrong?

The HttpClient class implements IDisposable suggesting any object of this type must be disposed of after use; With that in mind, let us have a look at how I would use this class assuming that I am not aware of the problem:

Uri endpoint = new("http://localhost:1234/");
for (int i = 0; i < 10; i++)
{
    using HttpClient client = new();
    string response = await client.GetStringAsync(endpoint);
    Console.WriteLine(response);
}

So here we are sending 10 requests to an endpoint sequentially and assuming there is a listener serving the requests on port 1234 or any other endpoint you choose to hit, you will see 10 responses written to the Console; This is all going well, right? WRONG!

Let us run the command: netstat -abn on CMD which should return (depending on the endpoint you hit):

...
TCP    [::1]:40968            [::1]:1234             TIME_WAIT
TCP    [::1]:40969            [::1]:1234             TIME_WAIT
TCP    [::1]:40970            [::1]:1234             TIME_WAIT
TCP    [::1]:40971            [::1]:1234             TIME_WAIT
TCP    [::1]:40972            [::1]:1234             TIME_WAIT
TCP    [::1]:40973            [::1]:1234             TIME_WAIT
TCP    [::1]:40975            [::1]:1234             TIME_WAIT
TCP    [::1]:40976            [::1]:1234             TIME_WAIT
TCP    [::1]:40977            [::1]:1234             TIME_WAIT
TCP    [::1]:40978            [::1]:1234             TIME_WAIT
...

What is this you ask? Well, this is showing us that our application has opened up 10 sockets to the server so one for every request but more importantly even though our application has now ended, the OS has 10 sockets still occupied and in the TIME__WAIT state.

This is due to the way TCP/IP has been designed to work as connections are not closed immediately to allow the packets to arrive out of order or re-transmitted after the connection has been closed. TIMEWAIT indicates that the local endpoint (the one on our side) has closed the connections but the connections are being kept around so that any delayed packets can be handled correctly. Once this happens the connections will then be removed after their timeout period of 4 minutes. Remember, we sent 10 requests to the same endpoint and yet we have 10 individual sockets still busy for at least 4 minutes!

The above example is an overly simplified one but have a look at this:

public class ProductController : ApiController
{
    public async Task<Product> GetProductAsync(string id)
    {
        using HttpClient httpClient = new();
        string result = await httpClient.GetStringAsync("http://somewhere/api/...");
        return new Product { Name = result };
    }
}

Doing this per incoming request will eventually result in SocketException, don’t believe me? Just run a load test, sit back and watch how many requests you can serve before you run out of sockets!

What can we do?

Well, the first obvious thing that comes to mind is reusing our client instead of creating a new one for every request but as you will see later in the post, that can cause yet another problem. Before we get to that point, let us first find out if we can even re-use a single instance. Is the HTTPClient thread-safe? The answer is YES it is, at least the following methods have been documented to be thread-safe:

• CancelPendingRequests
• DeleteAsync
• GetAsync
• GetByteArrayAsync
• GetStreamAsync
• GetStringAsync
• PostAsync
• PutAsync
• SendAsync

However the following are not thread-safe and cannot be changed once the first request has been made:

• BaseAddress
• Timeout
• MaxResponseContentBufferSize

In fact on the same documentation page, under the Remarks section, it explains:

HttpClient is intended to be instantiated once and re-used throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors.

Okay so is that it? Create and reuse a single instance of our client and happy days? Well, NO! There is yet another very subtle but serious issue you may face.

A Singleton HttpClient does not respect DNS changes

Re-using an instance of HttpClient means that it holds on to the socket until it is closed so if you have a DNS record update occurring on the server the client will never know until that socket is closed and let me tell you DNS records change for different reasons all the time, for example a fail-over scenario is just one of them (albeit in such case the connection/socket would have faulted and closed) or an Azure deployment when you swap different instances e.g. Production/Staging in this case your client would still be hitting the old instance! In fact there is an issue on the dotnet/corefx repo about this behaviour.

HTTPClient (for valid reasons) does not check the DNS records when the connection is open so how can we fix this? One naive easy workaround is to set the keep-alive header to false so the socket will be closed after each request, this obviously results in sub-optimal performance but if you do not care, then there is your answer; However, I think we can do better.

There is the lesser known ServicePoint class which holds the solution to our problem. This class is responsible for managing different properties of a TCP connection and one of such properties is the ConnectionLeaseTimeout. This guy as its name suggests specifies how long (in ms) the TCP socket can stay open. By default the value of this property is set to -1 resulting in the socket staying open indefinitely (relatively speaking) so all we have to do is set it to a more realistic value:

ServicePointManager.FindServicePoint(endpoint)
    .ConnectionLeaseTimeout = (int)TimeSpan.FromMinutes(1).TotalMilliseconds;

The above override needs to be applied once for each endpoint. Note the method only cares about the host, schema and port everything else is ignored.

Almost There…

So far we have taken care of force-closing the connections after a given period but that was just the first part. If our singleton client opens another connection it may still be pointed to the old server, why you ask? well all the DNS entries are cached which by default does not refresh for 2 minutes. So we also need to reduce the cache timeout which we can do by setting the DnsRefreshTimeout on the ServicePointManager class like so:

ServicePointManager.DnsRefreshTimeout = (int)1.Minutes().TotalMilliseconds;

I wanted to have a better abstraction and not have to remember to do all of this on every request, I also wanted the abstraction to implement an interface for dependency injection between my services.

RestClient

RestClient is a thread-safe wrapper around HttpClient and internally keeps a cache of endpoints that it has already sent a request to and if it is asked to send a request to an endpoint it does not have in its cache, it updates the ConnectionLeaseTimeout for that endpoint. Here is a simple usage example:

// This is to show that IRestClient implements IDisposable
// just like HttpClient, you should not dispose it per request.
using IRestClient client = new RestClient();
await client.SendAsync(new HttpRequestMessage(HttpMethod.Get, new Uri("http://localhost/api")));

Now you can safely hold on to the client and/or register it using your favorite IoC container and inject it where ever you require.

The class supports the same constructors as HttpClient and also provides a safe way of setting its default properties:

var defaultHeaders = new Dictionary<string, string>
{
    {"Accept", "application/json"},
    {"User-Agent", "foo-bar"}
};

using IRestClient client = new RestClient(defaultHeaders, timeout: 15.Seconds(), axResponseContentBufferSize: 10);
client.DefaultRequestHeaders.Count.ShouldBe(defaultHeaders.Count);
client.DefaultRequestHeaders["Accept"].ShouldBe("application/json");
client.DefaultRequestHeaders["UserAgent"].ShouldBe("foo-bar");

client.Endpoints.ShouldBeEmpty();
client.MaxResponseContentBufferSize.ShouldBe((uint)10);
client.Timeout.ShouldBe(15.Seconds());

The code is on GitHub and is available on NuGet as part of the Easy.Common library used in my other projects.

Update 2019

Starting from .NET Core 2.1, Microsoft addressed some of the issues covered in this article by making available HttpClientFactory. Despite the various features offered in this class, in my opinion, there is a little too much of ceremony involved with using this type also we would still need to deal with setting the DNS refresh timeout ourselves; Therefore, I still prefer to use RestClient in my projects.

HttpClient was also overhauled in .NET Core 2.1 with a rewritten HttpMessageHandler called SocketsHttpHandler which results in significant performance improvements it also introduces the PooledConnectionLifetime property which allows us to set the connection timeout without having to set the ConnectionLeaseTimeout for each endpoint.

As of version 3.0.0 of Easy.Common, the RestClient no longer needs to set the ConnectionLeaseTimeout when running on .NET Core 2.1 or higher.

Have fun and happy RESTing.

BTW

This post was inspired by YOU’RE USING HTTPCLIENT WRONG AND IT IS DESTABILIZING YOUR SOFTWARE by Simon Timms and Singleton HttpClient by Ali Ostad as well as various great posts by Darrel Miller.

A stopped clock is exactly accurate twice a day, a clock a minute slow is never accurate at any time. But the clock a minute slow is always precise to the nearest minute, whereas a stopped clock has no useful precision at all.

So we know that DateTime in .NET is very precise. why? well it is represented as a struct which internally keeps track of the number of ticks since a particular start date and those ticks are represented as a 64 bit number; And just for the context, 1 tick equals 100 nanoseconds and every second equals to 10,000,000 ticks so with such structure one could theoretically represent sub-microsecond times.

Of course having a precise DateTime is not all the story let us think about it for a second, I can represent the distance between point A and B as a high-precision floating point by using a double for example: 156.34214233234517974 meters but do I really have a method of measuring that distance to such accuracy? The same goes for DateTime; Hans Passant explains:

DateTime.UtcNow is accurate to 15.625 milliseconds and stable over very long periods thanks to the time service updates. Going lower than that just doesn’t make much sense, you can’t get the execution guarantee you need in user mode to take advantage of it.

You need lots of bigger tricks to get it accurate down to a millisecond. You can only get a guarantee like that for code that runs in kernel mode, running at interrupt priority so it cannot get pre-empted by other code and with its code and data pages page-locked so it can’t get hit with page faults Commercial solutions use a GPS radio to read the clock signal of the GPS satellites, backed up by an oscillator that runs in an oven to provide temperature stability.

How accurate is DateTime?

With all of the above in mind and remembering that obtaining a time matching an atomic clock is almost impossible, we can start by measuring the (relative) accuracy of DateTime.UtcNow by:

var duration = TimeSpan.FromSeconds(5);
var distinctValues = new HashSet<DateTime>();
var stopWatch = Stopwatch.StartNew();

while (stopWatch.Elapsed < duration)
{
    distinctValues.Add(DateTime.UtcNow);
}

Console.WriteLine("Samples: " + distinctValues.Count);
Console.WriteLine($"Accuracy: {stopWatch.Elapsed.TotalMilliseconds / distinctValues.Count:0.000000} ms");

he code above is telling us how many times DateTime.UtcNow is giving us distinct values in a period of 5 seconds which is the same as showing us how many times its value was updated in that period.

Running the code on my laptop returns:

Samples: 320
Accuracy: 15.625348 ms

And on my server:

Samples: 4980
Accuracy: 1.005322 ms

Both machines are running Windows Server 2012 but as you saw one is more accurate than the other so it seems the accuracy depends on the hardware as well as software as we will cover shortly.

Can we get more accurate?

Absolutely! We are going to get some help from StopWatch which has a very high resolution and it is going to help us manually tune our DateTime in order to achieve a much higher accuracy.

The idea is to start with a base time using DateTime and then every time we need to obtain the value of current time we use the Stopwatch to supplement our base time. All good? not really!

One slight problem with Stopwatch is that after a while (30 minutes) it can get out of sync meaning it can get too far from the system time so in order compensate for that we need to reset our start time every let’s say 10 seconds. Sounds complicated? not really, here is the code:

public sealed class Clock
{
    private readonly long _maxIdleTime = TimeSpan.FromSeconds(10).Ticks;
    private const long TicksMultiplier = 1000 * TimeSpan.TicksPerMillisecond;

    private DateTime _startTime = DateTime.UtcNow;
    private double _startTimestamp = Stopwatch.GetTimestamp();

    public DateTime UtcNow
    {
        get
        {
            double endTimestamp = Stopwatch.GetTimestamp();

            var durationInTicks = (endTimestamp - _startTimestamp) / Stopwatch.Frequency * TicksMultiplier;
            if (durationInTicks >= _maxIdleTime)
            {
                _startTimestamp = Stopwatch.GetTimestamp();
                _startTime = DateTime.UtcNow;
                return _startTime;
            }

            return _startTime.AddTicks((long)durationInTicks);
        }
    }
}

So let us run our test again to measure how accurate our hybrid clock is:

var clock = new Clock();
var duration = TimeSpan.FromSeconds(5);
var distinctValues = new HashSet<DateTime>();
var stopWatch = Stopwatch.StartNew();

while (stopWatch.Elapsed < duration)
{
    distinctValues.Add(clock.UtcNow);
}

Console.WriteLine("Samples: " + distinctValues.Count);
Console.WriteLine($"Accuracy: {stopWatch.Elapsed.TotalMilliseconds / distinctValues.Count:0.000000} ms");

Which gives us:

Samples: 11469019
Accuracy: 0.000436 ms

Great! so can you go and start using this code? Yes and No! you see this code is not thread-safe so if you are planning on using it in a multi-threaded scenario then continue reading….

Making it thread-safe

We can easily make our clock thread-safe by using ThreadLocal. This guy allows each thread to have its own instance of the _startTime and _startTimestamp instead of having to resort to a lock. So the modified clock now looks like:

public sealed class Clock : IDisposable
{
    private readonly long _maxIdleTime = TimeSpan.FromSeconds(10).Ticks;
        private const long TicksMultiplier = 1000 * TimeSpan.TicksPerMillisecond;

    private readonly ThreadLocal<DateTime> _startTime =
        new ThreadLocal<DateTime>(() => DateTime.UtcNow, false);

    private readonly ThreadLocal<double> _startTimestamp =
        new ThreadLocal<double>(() => Stopwatch.GetTimestamp(), false);

    public DateTime UtcNow
    {
        get
        {
            double endTimestamp = Stopwatch.GetTimestamp();

            var durationInTicks = (endTimestamp - _startTimestamp.Value) / Stopwatch.Frequency * TicksMultiplier;
            if (durationInTicks >= _maxIdleTime)
            {
                _startTimestamp.Value = Stopwatch.GetTimestamp();
                _startTime.Value = DateTime.UtcNow;
                return _startTime.Value;
            }

            return _startTime.Value.AddTicks((long)durationInTicks);
        }
    }

    public void Dispose()
    {
        _startTime.Dispose();
        _startTimestamp.Dispose();
    }
}

Is that it? well, yes but….!

Can we do better?

Starting from Windows 8 and Windows Server 2012, Microsoft has made available for us a very nice API. That API is GetSystemTimePreciseAsFileTime. This method obtains the current system time with the highest possible level of precision (we are talking about <1us); Why not take advantage of this method in our hybrid clock.

The final version of our clock now looks like:

/// <summary>
/// This class provides a high resolution clock by using the new API available in <c>Windows 8</c>/
/// <c>Windows Server 2012</c> and higher. In all other operating systems it returns time by using
/// a manually tuned and compensated <c>DateTime</c> which takes advantage of the high resolution
/// available in <see cref="Stopwatch"/>.
/// </summary>
public sealed class Clock : IDisposable
{
    private readonly long _maxIdleTime = TimeSpan.FromSeconds(10).Ticks;
    private const long TicksMultiplier = 1000 * TimeSpan.TicksPerMillisecond;

    private readonly ThreadLocal<DateTime> _startTime =
        new ThreadLocal<DateTime>(() => DateTime.UtcNow, false);

    private readonly ThreadLocal<double> _startTimestamp =
        new ThreadLocal<double>(() => Stopwatch.GetTimestamp(), false);


    [DllImport("Kernel32.dll", CallingConvention = CallingConvention.Winapi)]
    private static extern void GetSystemTimePreciseAsFileTime(out long filetime);

    /// <summary>
    /// Creates an instance of the <see cref="Clock"/>.
    /// </summary>
    public Clock()
    {
        try
        {
            long preciseTime;
            GetSystemTimePreciseAsFileTime(out preciseTime);
            IsPrecise = true;
        } catch (EntryPointNotFoundException)
        {
            IsPrecise = false;
        }
    }

    /// <summary>
    /// Gets the flag indicating whether the instance of <see cref="Clock"/> provides high resolution time.
    /// <remarks>
    /// <para>
    /// This only returns <c>True</c> on <c>Windows 8</c>/<c>Windows Server 2012</c> and higher.
    /// </para>
    /// </remarks>
    /// </summary>
    public bool IsPrecise { get; }

    /// <summary>
    /// Gets the date and time in <c>UTC</c>.
    /// </summary>
    public DateTime UtcNow
    {
        get
        {
            if (IsPrecise)
            {
                long preciseTime;
                GetSystemTimePreciseAsFileTime(out preciseTime);
                return DateTime.FromFileTimeUtc(preciseTime);
            }

            double endTimestamp = Stopwatch.GetTimestamp();

            var durationInTicks = (endTimestamp - _startTimestamp.Value) / Stopwatch.Frequency * TicksMultiplier;
            if (durationInTicks >= _maxIdleTime)
            {
                _startTimestamp.Value = Stopwatch.GetTimestamp();
                _startTime.Value = DateTime.UtcNow;
                return _startTime.Value;
            }

            return _startTime.Value.AddTicks((long)durationInTicks);
        }
    }

    /// <summary>
    /// Releases all resources used by the instance of <see cref="Clock"/>.
    /// </summary>
    public void Dispose()
    {
        _startTime.Dispose();
        _startTimestamp.Dispose();
    }
}

Now we are done!

When working on any application, I have the habit of logging as much information as possible not just about the process but also about the assemblies which the process is referencing and one important part of such information is the version of .NET framework the assembly has been built for e.g. .NET 2, .NET 4.5.1 etc. Note this is different from the run-time version.

The not so accurate

If you look at the assembly object you can find the ImageRuntimeVersion property which gets you the string representation of the CLR saved in the file containing the assembly manifest (which by the way you can view by using a decompiler such as ILSpy or dotPeek) however, this is not accurate as I explain below.

The problem

The ImageRuntimeVersion property returns either .NET 2.0 or .NET 4.0 regardless of what framework version you have built your assembly against; Meaning that if you have an assembly built in VisualStudio to target .NET 4.5.1, the value returned by the ImageRuntimeVersion is .NET 4.0. Not so helpful is it.

What do we do?

There is some hope otherwise why write this article!? Assemblies targeting .NET 4 and above are marked with the TargetFrameworkAttribute which conveniently includes the target framework we are after. This of course means that if your assembly is targeting a framework version less than .NET 4 i.e. .NET 2, 3 and 3.5, then as far as I am aware you are out of luck!

So armed with that knowledge one can obtain the version by calling an extension method such as:

public static string GetFrameworkVersion(this Assembly assembly)
{
    var targetFrameAttribute = assembly.GetCustomAttributes(true)
        .OfType<TargetFrameworkAttribute>().FirstOrDefault();
    if (targetFrameAttribute == null)
    {
        return ".NET 2, 3 or 3.5";
    }

    return targetFrameAttribute.FrameworkDisplayName.Replace(".NET Framework", ".NET");
}

As it is the case in most scenarios the source of latency is usually due to the .NET Garbage Collection, so we always try very hard to minimize object allocations specially in our performance-critical components.

We have a messaging API which is used by many of our micro services. Some of those services use the API to publish thousands of messages per second to the market so it is important to keep the latency as low as possible.

In my quest to reduce latency and CPU consumption of our components, I came across a very interesting and probably not so well known performance gotcha!

Problem

The API has a method which accepts an IDictionary<string, string> and publishes them to a message bus like so:

public void Publish(IDictionary<string, string> data)
{
    foreach (var item in data)
    {
        _publisher.Publish(item);
    }
}

Nothing really complicated right? WRONG!

What is going on?

Let’s do a quick and dirty profiling of an even simpler example:

void Main()
{
    Dictionary<int, int> dic = new();

    var sw = Stopwatch.StartNew();
    for (var i = 0; i < 100000000; i++)
    {
        foreach (var item in dic) { ; }
    }
    sw.Stop();

    // compose the result
    StringBuilder result = new();
    result.AppendFormat("[Time]\t{0}", sw.Elapsed);
    result.AppendLine();
    result.AppendFormat("[Gen-0]\t{0}", GC.CollectionCount(0));
    result.AppendLine();
    result.AppendFormat("[Gen-1]\t{0}", GC.CollectionCount(1));
    result.AppendLine();
    result.AppendFormat("[Gen-2]\t{0}", GC.CollectionCount(2));

    Console.WriteLine(result.ToString());
}

What we have here is an empty dictionary where both the keys and values are of type int. We are then simply enumerating the dictionary a hundred million times. Note we are not doing anything in the body of the foreach so that we can focus only on the enumeration.

Finally, we record the total time it took for the enumeration as well as the number of times GC performed collections of our GEN 0 to GEN 2. Running the above on a machine with a Quad-Core 3.4GHz and 8GB of RAM prints:

[Time]  00:00:01.8128806
[Gen-0] 0
[Gen-1] 0
[Gen-2] 0

So it took almost 2 seconds and resulted in no garbage collection, so far so good. Now let us try again but this time change our dic pointer to an IDictionary<int, int> instead:

... same as above
    IDictionary<int, int> dic = new Dictionary<int, int>();
... same as above

Which prints:

[Time]  00:00:04.3902405
[Gen-0] 765
[Gen-1] 2
[Gen-2] 1

This time it took more than twice than enumerating a Dictionary<int, int> and resulted in 765 collections of GEN 0, 2 collections of GEN 1 and 1 collection of GEN 2. This is MADNESS!

And here is why

Let us look at the IL generated in the case when we are using a Dictionary<int, int>:

... not important
IL_0013:  ldloc.0     // dic
IL_0014:  callvirt    System.Collections.Generic.Dictionary<System.Int32,System.Int32>.GetEnumerator
IL_0019:  stloc.s     04
IL_001B:  br.s        IL_0029
IL_001D:  ldloca.s    04
IL_001F:  call        System.Collections.Generic.Dictionary<System.Int32,System.Int32>+Enumerator.get_Current
IL_0024:  stloc.s     05 // item
...
IL_0029:  ldloca.s    04
IL_002B:  call        System.Collections.Generic.Dictionary<System.Int32,System.Int32>+Enumerator.MoveNext
IL_0030:  brtrue.s    IL_001D
IL_0032:  leave.s     IL_0043
IL_0034:  ldloca.s    04
IL_0036:  constrained. System.Collections.Generic.Dictionary<,>.Enumerator
... not important

Here we can see a call to System.Collections.Generic.Dictionary<System.Int32,System.Int32>+Enumerator.get_Current to get the current Enumerator and on every iteration we have a call to System.Collections.Generic.Dictionary<System.Int32,System.Int32>+Enumerator.MoveNext to move the Enumerator forward.

Let us compare that with the IL for the case of an IDictionary<int, int>:

... not important
IL_0013:  ldloc.0     // dic
IL_0014:  callvirt    System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<System.Int32,System.Int32>>.GetEnumerator
IL_0019:  stloc.s     04
IL_001B:  br.s        IL_0029
IL_001D:  ldloc.s     04
IL_001F:  callvirt    System.Collections.Generic.IEnumerator<System.Collections.Generic.KeyValuePair<System.Int32,System.Int32>>.get_Current
IL_0024:  stloc.s     05 // item
...
IL_0029:  ldloc.s     04
IL_002B:  callvirt    System.Collections.IEnumerator.MoveNext
IL_0030:  brtrue.s    IL_001D
IL_0032:  leave.s     IL_0041
... not important

Very interesting! This time we have a callvirt to System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<System.Int32,System.Int32>>.GetEnumerator followed by more callvirt to System.Collections.Generic.IEnumerator.

Aha! Allow me to explain in case it is not that obvious.

Virtual Call / Interface Dispatch

The first issue is the Interface Dispatch (the parts that do a non-inlineable callvirt to System.Collections.Generic.IEnumerator) which is in contrast to the case of Dictionary<int, int> where we are doing fast, non-virtual and inlineable calls.

Boxing

The second issue and by far the most significant performance hit is due to _boxing. Instead of returning a Dictionary<int, int>.Enumerator, we are boxing the struct and return an IEnumerator<KeyValuePair<int, int>> which explains the high number of GC collections across our heap.

How about IList, IReadOnlyList, ICollection & IEnumerable vs List?

IList<T> and IReadOnlyList<T> implement ICollection<T> which in turn implements IEnumerable<T> and therefore using any of those interfaces for the enumeration incurs the similar performance cost we covered above.

This is even more clear if you look at the source of List<T>:

public Enumerator GetEnumerator() => new Enumerator(this);

IEnumerator<T> IEnumerable<T>.GetEnumerator() => new Enumerator(this);

System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => new Enumerator(this);

You can see in the last two GetEnumerator()s the Enumerator struct is being boxed to a generic an a non generic IEnumerator.

Summary

It is a good idea to abstract yourself from an implementation, by that I mean programming against an interface rather than a concrete implementation however there are times where this can have a performance impact and what we covered in this article was one of such examples. In our case the problem was fixed by simply switching to a Dictionary<string, string> which is the cost we are prepared to pay in return of a more efficient API.