We have a ASP.NET application and suffered from high CPU issue occasionally - for years. It’s in production code and hard to reproduce. Fortunately, we got two dump files during application under high CPU issues.
What happened during high CPU?
Load the dump to
windbg, to inspect what happened during high CPU issue:
.loadby sos clr #load the SOS .runaway #check which thread consume most of CPU time. ~* e !ClrStack #check each call stack for all managed threads.
Most of call stack down to following calling :
System.Collections.Generic.Dictionary`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Insert(System.__Canon, System.__Canon, Boolean) --- or --- System.Collections.Generic.Dictionary`2[[System.Int32, mscorlib],[System.__Canon, mscorlib]].FindEntry(System.Int32)
Then, issue the
~* e ! ClrStack -p to check each parameter of these call stack, the first parameter to above two methods was
this which refer to dictionary itself. All of threads are accessing the same dictionary instance.
Got it, it was thread-safe issue to the dictionary.
How to fix
ConcurrentDictionary to replace the
Dictionary. There are some of tricky parts on
- Microsoft Docs do not guarantee the thread-safe for those interfaces implement.
- Actually most of internal interface implements are thread-safe. eg,
Anyway, just do not depends on these internal implement, just do what’s the docs recommend. Depending on internal implement may lead to subtle issue, eg. using
IDictionary to refer to an
IDictionary implement which may or may not thread-safe.
If you know the reason you can have keyword to get bunch of references.