Understanding AppDomain Unloads & Old Lessons, Learned Anew
I couldn’t believe it. Glimpse knows nothing about OleDb or Access, but alas, the bug was reported and I was able to reproduce it.
With reproduction in hand, I quickly figured out what was really happening. Opening the connection to Access caused the current AppDomain to unload, which in turn tore down Glimpse’s internal data stores resulting in the undesired behavior.
But why was the AppDomain being unloaded? I know tons of applications out there, running on little web servers underneath people’s cubicle desks, leverage Access as a data backend. Surely they don’t recycle the AppDomain after every database connection, do they?
Turns out, this problem would have been very easy to solve if I hadn’t oversimplified ASP.NET’s behavior in my head. I’ve known for a long time that changing a site’s web.config causes the AppDomain to recycle, but that’s only a partial truth now-a-days. With the help of a post by Tess Ferrandez I found souring the internet, I learned that as of ASP.NET 2.0 the file system watcher watches much more than just web.config for changes – including everything in the bin and its subdirectories. (I found out about several other reasons why an AppDomain might recycle from her excellent post too.)
Once I correctly understood modern ASP.NET’s behavior, I realized that connecting to the .MDB Access file in my bin directory was causing the file system watcher to kick off an application domain unload. Awesome! I moved the database out of the bin and the problem was solved.
Of course, after wasting more than a day of tinkering with this, I thought “There must be a way to figure this out without Google’ing high and wide”.
Turns out, there is! Unfortunately it’s not very obvious and requires some private reflection. Scott Gu covers the technique in a blog post from late 2005, before I was even involved in .NET.
The (simplified!) code, which you’d place in the AppDomain.DomainUnload event handler, looks like this:
var httpRuntimeType = typeof(HttpRuntime); var httpRuntime = httpRuntimeType.InvokeMember( "_theRuntime", BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.GetField, null, null, null) as HttpRuntime; var shutDownMessage = httpRuntimeType.InvokeMember( "_shutDownMessage", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.GetField, null, httpRuntime, null) as string;
The shutDownMessage will be a string describing the exact reason why the AppDomain is being recycled – which is great to log and saves lots of guess work.
Now that I know this technique, I have now begun logging all AppDomain recycles within Glimpse’s internal log in an effort to aid diagnosis of future problems like this.
It’s amazing just how easy it is to take for granted the mountain of code we build on top of everyday. This year I’m really going to try and not gloss over the lessons learned by those who came before us – just a mere 8 years ago.
- By Nik Molnar