Got more questions? Find advice on: ASP | SQL | Regular Expressions | Windows
in Search
Welcome to XmlAdvice Sign in | Join | Help
HomeBlogsForumsPhotosDownloads

Kirk Allen Evans' XML Blog

.NET From a Markup Perspective

Assembly Caching and XmlSerializer

Paul Wilson emailed me with some questions regarding perceived memory leaks related to System.Xml.Serialization.XmlSerializer.  I questioned whether it would be a “memory leak”, but pinged Christoph Schittko for some backup evidence.  He said he doubted it as well, else web services everywhere would be falling on their faces.  I pointed him to Doug Purdy's post on inspecting theXmlSerializer's interim assembly, and Chris Sells' utility for precompiling XmlSerializer types.

Paul emailed back that, in fact, they pinpointed the leak, and it was related to the XmlSerializer.  But it is not a memory leak... it is a dynamically created assembly “leak“, and is not by accident, but rather by design.

XmlSerializer manages the generation and execution of per-type reader/writer pairs. These pairs are dynamically generated types that extend the internal types XmlSerializationReader and XmlSerializationWriter, respectively. When instantiating an XmlSerializer, you must provide a System.Type object that represents the type the new serializer object will be used with. To avoid generating redundant assemblies, the XmlSerializer constructor looks in an AppDomain-wide cache of generated reader/writer assemblies and reuses the cached assembly if one is found. If one is not found, the XmlSerializer constructor generates a new dynamic assembly by reflecting against the presented System.Type object. This new assembly is added to the cache so subsequent XmlSerializer objects can reuse it, reducing overall code size and codegen overhead. Unfortunately, the cache only works for some constructors of XMLSerializer, since the lookup index gets more complex with more overrides. Users can cache the serializer themselves—it's freethreaded so you can serialize across threads.

[Don Box, MSDN Magazine, House of Web Services, November 2001]

Christoph followed up with (via IM):

I got used to the pattern of having a static serializer instance in my apps ... Just instantiate it once, assign it to a private or internal static property somewhere in the assembly.  Then you don't even need to hit the cache ...

It's also interesting to note, that System.Messaging does not rely on the cache and keeps its own cache around.

For those that haven't read it, you really need to read Christoph's blog entry on troubleshooting the XmlSerializer.

Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!
Published Wednesday, February 11, 2004 4:11 PM by kaevans
Filed under: ,

Comments

 

kaevans said:

I would still call this a bug personally. Several of the constructor overloads do reuse the dynamically created assembly, and there's no intrinsic reason the others could not do so as well. Maybe someone ran out of time, or patience, in creating the appropriate caching algorithms, and just decided these other cases weren't important enough. But anytime you run a loop over a few identical statements and "leak" more memory every time -- that's a bug! And yes, while the leak is actually a dynamically created assembly, the net result is that it still profoundly affects memory, so it is also a memory leak.

By the way, we had a rather interesting twist to this leak. We are using XmlSerialization to save user actions as reusable tasks. We also track their history to allow undos, and we save a portion of their history to allow them to get back to the same setup, kind of like a macro. OK, so lets say they performed a good number of tasks, each of which was serialized as part of their history. Each one of these tasks ends up being associated with a memory leak, but of course eventually they close the app and get the memory back. Oops! The gotcha for us was that the next time they can start from where they last left off, by having us reapply all of the tasks, which requires all the deserializations. The net result was that we instantly had a situation where we were actually recreating the very memory leaks from the last session!
February 11, 2004 8:17 PM
 

kaevans said:

Paul,

Are you sure the memory really leaks, i.e. it's NEVER claimed by the garbage collection? Have you observed that the GC ran for all generations while you monitored memory usage? The fact that some assemblies are not cached doesn't mean they are leaked.

What exactly are you observing to determine that memory is leaking?

Also, the article Kirk quoted states that only some of more complex constructors of the XmlSerializer cause the XmlSerializer to not cache the temp assemblies. Is there a way to switch to simpler constructs?

Christoph
February 11, 2004 10:36 PM
 

kaevans said:

Yes its a memory leak -- its NEVER reclaimed by the GC. Its the actual dynamic assembly that is "lost", which isn't managed memory. Some of the .net memory counters do increase a little, but its the private bytes of the process that go through the roof. Run this code snippet as the only file in a console app with performance monitor. With the leakMemory variable set to true, then run it again with leakMemory set to false. The only difference is whether or not the XmlSerializer is created each time, or only once. My last test had a fatal memory crash after leaking almost 500MB, which had long since pushed me far into the page file. All memory was reclaimed when the process was killed, or ended if you make it that far, but the GC never manages it or reclaims it. Can we switch to a simpler constructor? No. We could possibly use a different one with more work, but the other links I provided on my blog indicate that most of the constructors, except the very simplest, leak dynamic assemblies.

using System;
using System.IO;
using System.Text;
using System.Xml.Serialization;

namespace XmlLeak
{
class Global
{
private static bool leakMemory = true;
private static XmlSerializer serial = null;

[STAThread]
private static void Main(string[] args) {
for (int index = 0; index < 1000000; index++) {
Test test = new Test();
test.Id = index;
test.Time = DateTime.Now;
StringBuilder builder = new StringBuilder();
StringWriter writer = new StringWriter(builder);
if (leakMemory || serial == null) {
serial = new XmlSerializer(typeof(Base), new Type[] {typeof(Test)});
}
serial.Serialize(writer, test);
string xml = builder.ToString();
}
}
}

[Serializable()]
public class Test : Base
{
private DateTime time;
public DateTime Time {
get { return time; }
set { time = value; }
}
}

[Serializable()]
public class Base
{
private int id;
public int Id {
get { return id; }
set { id = value; }
}
}
}
February 12, 2004 8:53 AM
 

kaevans said:

Any ways to get around the problem then?
September 12, 2004 7:02 PM
 

kaevans said:

I share with you a workaround I found when serializing an ArrayList (when all the items are the same Type, that is "MyClass")

You'll see 3 examples:
1- Serializing the ArrayList "as is"
2- Caching the XmlSerlializer instance
3- Converting the ArrayList to a typed array

http://www.singletonsoftware.com.ar/XmlSerializer/SerTest.cs

Hope you find this useful.
Regards

Alejandro


July 22, 2005 3:30 PM
 

TrackBack said:

February 11, 2004 10:39 PM
 

TrackBack said:

February 12, 2004 11:39 AM
 

Dan's Archive said:

May 24, 2006 11:56 AM
 

Dan's Archive said:

August 4, 2006 5:11 PM
 

ASP.NET Debugging said:

So we didn't get much more on this one, so I'll go ahead and show how we find out what is going on here.&#160;

March 31, 2008 1:56 PM
Anonymous comments are disabled

This Blog

Syndication

Recent Posts

Tags

News

Looking for a place to talk about XML? Tired of the "main feed police" cracking about your interests in football and politics? Sign up for a free web log on XMLAdvice.com.

Archives
Got more questions? Find advice on: ASP | SQL | Regular Expressions | Windows | About xmladvice.com | Privacy Policy | ASP.NET Tutorials | Advertise

Related Resources: Web Development Community
Copyright ©2005 DevAdvice.com