Friday, March 18, 2022

Several ways to reduce JSON response size from Web API

If you have Web API which returns some data in JSON format at some point you may need to optimize its performance by reducing response size. It doesn't important on which technology Web API is implemented (ASP.Net Web API, .Net Azure Functions or something else). For this article the only important thing is that it is implemented on .Net stack.

Let's assume that we have an endpoint which returns array of objects of the following class which have many public properties:

public class Foo
{
    public string PropA { get; set; }
    public string PropB { get; set; }
    public string PropC { get; set; }
    public string PropD { get; set; }
    ...
}

By default all these properties will be returned from our API (including those properties which contain nulls):

[{
        "PropA": "test1",
        "PropB": null,
        "PropC": null,
        "PropD": null,
        ...
    }, {
        "PropA": "test2",
        "PropB": null,
        "PropC": null,
        "PropD": null,
        ...
    },
    ...
]

As you can see these properties with nulls still add many bytes to the response. If we want to exclude properties which contain nulls from response (which in turn may significantly reduce response size) we may add special class-level and property-level attributes to our class. Note however that it will help only if you serialize response with Json.Net lib (Newtonsoft.Json):

[JsonObject(MemberSerialization.OptIn)]
public class Foo
{
    [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
    public string PropA { get; set; }
    [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
    public string PropB { get; set; }
    [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
    public string PropC { get; set; }
    [JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
    public string PropD { get; set; }
}

Attribute JsonObject(MemberSerialization.OptIn) means that it should serialize only those properties which are explicitly decorated by JsonProperty attribute. After that response will look like that:

[{
        "PropA": "test1",
    }, {
        "PropA": "test2",
    },
	...
]

i.e. it will only contain those properties which have not-null value.

As you can see response size got significantly reduced. The drawback of this approach is that it will work only with Json.Net lib (JsonObject and JsonProperty attributes classes are defined in Newtonsoft.Json assembly). If we don't use Newtonsoft.Json we need to use another approach which is based on .Net reflection and on the fact that Dictionary is serialized to JSON in the same way as object with the properties. Here is the code which does it:

// get array of items of Foo class
var items = ...;

// map function for removing null values
var propertiesMembers = new List<PropertyInfo>(typeof(Foo).GetProperties(BindingFlags.Instance | BindingFlags.Public));
Func<Foo, Dictionary<string, object>> map = (item) =>
{
    if (item == null)
    {
        return new Dictionary<string, object>();
    }
    var dict = new Dictionary<string, object>();
    foreach (string prop in selectProperties)
    {
        if (string.IsNullOrEmpty(prop) || !propertiesMembers.Any(p => p.Name == prop))
        {
            continue;
        }
		
        var val = item.GetPublicInstancePropertyValue(prop);
        if (val == null)
        {
            // skip properties with null
            continue;
        }
        dict.Add(prop, val);
    }
    return dict;
};

// convert original array to array of Dictionary objects each of which contains only not null properties
return JsonConvert.SerializeObject(items.Select(i => map(i)));

At first we get original array of items of Foo class. Then we define map function which creates Dictionary from Foo object and this dictionary contains only those properties which don't contain null value (it is done via reflection). Here I used helper method GetPublicInstancePropertyValue from PnP.Framework but it is quite easy to implement by yourself also:

public static Object GetPublicInstancePropertyValue(this object source, string propertyName)
{
    return (source?.GetType()?.GetProperty(propertyName,
            System.Reflection.BindingFlags.Instance |
            System.Reflection.BindingFlags.Public |
            System.Reflection.BindingFlags.IgnoreCase)?
        .GetValue(source));
}

With this approach we will also get reduced response which will contain only not-null properties and it will work also without Newtonsoft.Json:

[{
        "PropA": "test1",
    }, {
        "PropA": "test2",
    },
	...
]


No comments:

Post a Comment