Streaming APIs In ASP.NET Core

The topics this week continue to be stream related.

When building an API, some teams have a strict rule to always use ToList in a LINQ query and materialize the query results before sending the results back to an HTTP client. There are some advantages to this eager approach:

1) You’ll fail fast, compared to deferred execution.

2) You’ll never worry about controller code using IQueryable operators that can destroy a query plan.

However, every application may require a different set of rules. For APIs that move heavy amounts of data, chewing up memory to materialize results can lead to high memory usage, paging, and sluggish performance. Let’s look at an example using the following, simple model.

public class Widget
{
    public int Id { get; set; }
    public string Name { get; set; }
}

Imagine we have 1 million widgets to move across the Internet. We’ll use the following controller action to respond to a request for all the widgets.

[HttpGet]
public List<Widget> Get()
{
    var model = _widgetData.GetAll();
    return model;
}

Notice the return type of List<Widget>, meaning the GetAll method has already placed every widget into memory. ASP.NET Core handles the request well. For the client, the 35 megabytes of JSON streams out of the server with a chunked encoding, which is ideal.

HTTP/1.1 200 OK 
Transfer-Encoding: chunked 
Content-Type: application/json; charset=utf-8 
Server: Kestrel

The downside is looking at the memory consumed on the server. After just three requests, the dotnet server process is using over 650MB of memory without serving any other requests. Move 4 million widgets at a time and the process is over 1.3GB.

Using IEnumerable

Assuming your data source can stream results (like the firehose cursor of SQL Server), you can keep more memory available on the server by moving data with IEnumerable or IQueryable all the way to the client.

[HttpGet]
public IEnumerable<Widget> Get()
{
    // GetAll now returns IEnumerable, 
    // not using ToList, but perhaps using
    // AsEnumerable
    var model = _widgetData.GetAll();            
    return model;
}

With multiple inbound requests, moving even 4 million Widgets requires less than 250MB of heap space over time.

Streaming yields a considerable savings in a world where memory is still relatively precious. Don’t let development rules overrule your context or take away your precious resources, and always use benchmarks and tests to determine how your specific application will behave.