Newtonsoft vs System.Text.Json: Memory Allocations using BenchmarkDotNet

Recently, I’ve been investigating some poor memory behaviour in an Azure App Service and had the usual niggle to dig into performance again.
This time, it’s about serialisation performance, specifically regarding memory allocations.

Why am I looking at Serialisation? #

The app in question is performing badly, but there’s nothing obvious in App Insights.
is running alongside a load of others and the tier is just too low, and Azure calls out one or two of the apps for “high memory” at around 300mb…

We should/will upgrade it’s tier, but first I wanted to see if there was something simple I could do.
That escalated into investigating replacing Newtonsoft with System.Text.Json

There are a number of comparisons between these two online, but they often used toy examples and I wanted to know what a real Enterprise example would look like.
Proper Enterprise, you know, megabytes of json.

Profiling #

Before I go further, yes I did some profiling, and no it didn’t really point at serialisation as an issue.
It did show huge allocations and GC time waiting and I had a thought that it could be the massive objects we’re serialising out to the UI.
I thought “Even if it isn’t that (since it wasn’t really showing in my profiler) I want to see what the difference is.

The profilers call out EFCore as the highest allocator for reading data from DB, but I can’t really do anything about that…

Setup #

I grabbed an example output from my Test environment, and it looks like this:

This is the main page payload, so every user gets this every time they load the site, so I think it’s a reasonable target for investigation.

Also here’s the output from BenchmarkDotNet regarding my machine.

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.23493.1000)
11th Gen Intel Core i7-11800H 2.30GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=8.0.100-preview.5.23303.2
  [Host]     : .NET 7.0.7 (7.0.723.27404), X64 RyuJIT AVX2
  DefaultJob : .NET 7.0.7 (7.0.723.27404), X64 RyuJIT AVX2  

Caveats #

There’s loads of weird things in this app, and I had to do some work to get this to be possible.
As such, I’ve only really done enough for it to be possible, not to make it good and proper.
Doubtless there’s many things I could do to improve this on both sides.

Also, the changes I had to make to get it to work weren’t a 5 minute job.
There’s a bunch of differences between these two implementations, see my older post Migrating Newtonsoft to System.Text.Json.

Benchmarks #

Here’s my code for the benchmarks

Serialisation

Expand/Collapse cs

[MemoryDiagnoser]
public class JsonSerialisationTests
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions()
    {
        // This is important, it apparently slows things down a bit.
        PropertyNameCaseInsensitive = true,
        Converters =
        {
            // ...a bunch of converters, mostly for enums
        }
    };
    private string jsonString = HugeJsonString.JsonString;
    private DTO obj = null;

    private JsonSerializerSettings newtonsoftOpts = new JsonSerializerSettings()
    {
        Converters =
        {
            //... converters
        }
    };

    [GlobalSetup]
    public void Setup()
    {
        obj = JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }

    [Benchmark]
    public string SystemTextJsonSerialise()
    {
        return System.Text.Json.JsonSerializer.Serialize(obj);
    }

    [Benchmark(Baseline = true)]
    public string NewtonsoftSerialise()
    {
        return JsonConvert.SerializeObject(obj);
    }
}
[MemoryDiagnoser]
public class JsonSerialisationTests
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions()
    {
        // This is important, it apparently slows things down a bit.
        PropertyNameCaseInsensitive = true,
        Converters =
        {
            // ...a bunch of converters, mostly for enums
        }
    };
    private string jsonString = HugeJsonString.JsonString;
    private DTO obj = null;

    private JsonSerializerSettings newtonsoftOpts = new JsonSerializerSettings()
    {
        Converters =
        {
            //... converters
        }
    };

    [GlobalSetup]
    public void Setup()
    {
        obj = JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }

    [Benchmark]
    public string SystemTextJsonSerialise()
    {
        return System.Text.Json.JsonSerializer.Serialize(obj);
    }

    [Benchmark(Baseline = true)]
    public string NewtonsoftSerialise()
    {
        return JsonConvert.SerializeObject(obj);
    }
}

Deserialisation

Expand/Collapse cs

[MemoryDiagnoser]
public class JsonDeserialisationTests
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions()
    {
        PropertyNameCaseInsensitive = true,
        Converters =
        {
            //...converters
        }
    };
    private string jsonString = HugeJsonString.JsonString;
    private DTO obj = null;

    private JsonSerializerSettings newtonsoftOpts = new JsonSerializerSettings()
    {
        Converters =
        {
            // ...converters
        }
    };

    [GlobalSetup]
    public void Setup()
    {
        obj = JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }
    
    [Benchmark]
    public DTO SystemTextJsonDeserialise()
    {
        return System.Text.Json.JsonSerializer.Deserialize<DTO>(jsonString, options);
    }

    [Benchmark(Baseline = true)]
    public DTO NewtonsoftDeserialise(JsonSerializerSettings newtonsoftOpts)
    {
        return JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }
}
[MemoryDiagnoser]
public class JsonDeserialisationTests
{
    private readonly JsonSerializerOptions options = new JsonSerializerOptions()
    {
        PropertyNameCaseInsensitive = true,
        Converters =
        {
            //...converters
        }
    };
    private string jsonString = HugeJsonString.JsonString;
    private DTO obj = null;

    private JsonSerializerSettings newtonsoftOpts = new JsonSerializerSettings()
    {
        Converters =
        {
            // ...converters
        }
    };

    [GlobalSetup]
    public void Setup()
    {
        obj = JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }
    
    [Benchmark]
    public DTO SystemTextJsonDeserialise()
    {
        return System.Text.Json.JsonSerializer.Deserialize<DTO>(jsonString, options);
    }

    [Benchmark(Baseline = true)]
    public DTO NewtonsoftDeserialise(JsonSerializerSettings newtonsoftOpts)
    {
        return JsonConvert.DeserializeObject<DTO>(jsonString, newtonsoftOpts);
    }
}

Source generated System.Text.Json!! #

I’ve just learned that System.Text.Json has Source Generation to improve performance!
It requires some setup, but it was just following the docs up there.

Results #

And drumroll 🥁🥁🥁

Serialisation Benchmark #

MethodMeanErrorStdDevMedianRatioGen0Gen1Gen2AllocatedAlloc Ratio
SystemTextJsonSerialise12.306 ms0.2318 ms0.6266 ms12.093 ms0.40296.8750234.3750234.37508.69 MB0.45
SystemTextJsonSerialise_SourceGen9.052 ms0.1591 ms0.1893 ms8.999 ms0.27265.6250265.6250265.62507.44 MB0.38
NewtonsoftSerialise33.071 ms0.5385 ms0.4774 ms33.225 ms1.001250.00001125.0000312.500019.46 MB1.00

Deserialisation Benchmark #

MethodMeanErrorStdDevRatioRatioSDGen0Gen1Gen2AllocatedAlloc Ratio
SystemTextJsonDeserialise46.99 ms0.929 ms2.171 ms0.610.041000.0000666.6667250.000016.68 MB0.57
SystemTextJsonDeserialise_SourceGen45.60 ms0.904 ms1.268 ms0.590.021000.0000666.6667250.000016.67 MB0.57
NewtonsoftDeserialise77.68 ms1.551 ms2.414 ms1.000.002714.28571142.8571285.714329.26 MB1.00

Conclusions #

The benchmarks are pretty clear: System.Text.Json is faster and more memory efficient.
This does come with the caveat that it’s also a needy little thing as well, and I had a lot of converters and reworking constructors as per my other post on Migrating Newtonsoft to System.Text.Json

The source generated version is even faster to serialise, but essentially the same to deserialise.
I might be using it wrong though.

I won’t be rushing out to move all the things away from Newtonsoft, but I won’t be reaching for it either.