ASP.NET Core and the Enterprise Part 2: Hosting

Tuesday, October 25, 2016

Kestrel the bird, not the serverThe hosting model for ASP.NET Core is dramatically different from previous versions of ASP.NET. This is also one area where I’ve seen a fair amount of misunderstanding.

ASP.NET Core is a set of libraries you can install into a project using the NuGet package manager. One of the packages you might install for HTTP message processing is a package named Microsoft.AspNetCore.Server.Kestrel. The word server is in the name because this new version of ASP.NET includes its own web servers, and the featured server has the name Kestrel. 

In the animal kingdom, a Kestrel is a bird of prey in the falcon family, but in the world of ASP.NET, Kestrel is a cross-platform web server. Kestrel builds on top of libuv, a cross-platform library for asynchronous I/O. libuv gives Kestrel a consistent streaming API to use across Windows and Linux. You also have the option of plugging in a server based on the Windows HTTP Server API (Web Listener), or writing your own IServer implementation. Without good reason, you’ll want to use Kestrel by default.

An Overview of How It Works

You can configure the server for your application in the entry point of the application. There is no Application_Start event in this new version of ASP.NET, nor is there any default XML configuration files. Instead, the start of the application is a static Main method, and configuration lives in the code.

public class Program
{
    public static void Main(string[] args)
    {
        var host = new WebHostBuilder()
            .UseKestrel()
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseIISIntegration()
            .UseStartup<Startup>()
            .Build();

        host.Run();
    }
}

If you are looking at the above Program class with a static Main method and thinking the code looks like what you would see in a .NET console mode application, then you are thinking correctly. Compiling an ASP.NET project still produces a .dll file, but with .NET Core we launch the web server from the command line with the dotnet command line interface. The dotnet host will ultimately call into the Main method. In this way of working, .NET Core resembles environments like Java, Ruby, and Python.

dotnet run

 

If you are working on ASP.NET Core from Visual Studio, then you might never see the command line. Visual Studio continues to do a job it has always done, which is to hide some of the lower level details. With Visual Studio, you can set the application to run with Kestrel as a direct host, or to run the application in IIS Express (the default setting). In both cases, the dotnet host and Kestrel server are in play, even when using IIS Express. This brings us to the topic of running applications in production.  

ASP.NET Core Applications in Production

One you realize that ASP.NET includes a cross-platform host and web server, you might think you have all the pieces you need to push to production. There is some truth to this line of thought. Once you’ve invoked the Run method on the WebHost object in the above code, you have a running web server that will listen to HTTP requests and can work on everything from a 32 core Linux server to a Raspberry Pi. However, Microsoft strongly suggests using a hardened reverse proxy in front of your Kestrel server in production. The proxy could be IIS on Windows, or Apache or NGINX.

ASP.NET Core In Production

 

 

Why the reverse proxy? In short because technologies like IIS and Apache have been around for over 20 years and have seen all the evils the Internet can deliver to a network socket. Kestrel, on the other hand, is still a newborn babe. Also, reliable servers require additional infrastructure like careful process management to restart failed applications. Outside of ASP.NET, in the world of Java, Python, Ruby, and NodeJS web apps, you’ll see tools like Phusion Passenger and PM2 work in combination with the reverse proxy. These types of tools provide the watchdog monitoring, logging, balancing, and overall process management needed for a robust server. With ASP.NET Core on Windows you can use IIS to achieve the same goals. HTTP requests will still arrive at IIS first, and IIS can forward requests to the Kestrel application. You can have multiple applications deployed behind a single instance of IIS, and IIS will manage the application and provide logging, request filtering, URL rewrites, and many other useful features. In a way, this isn’t much different than what we’ve done in the past with ASP.NET, but once you dig behind the architectural diagrams, you’ll see the details are very different.

What’s Different?

Deploying ASP.NET Core applications to IIS requires a web.config file. ASP.NET Core knows nothing about web.config files. The web.config file only exists to configure IIS in a reverse proxy role. A typical web.config file will look like the following.

<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
    </handlers>
    <aspNetCore processPath="dotnet" arguments=".\TheWebApp.dll" stdoutLogEnabled="false" 
                   stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false" />
  </system.webServer>
</configuration>

The web.config file instructs IIS to send requests for all paths and verbs to a new HTTP handler named aspNetCore. This ASP.NET Core Module for IIS is a piece of software you’ll need to install on an IIS server to run ASP.NET Core applications. The ASP.NET Core Module is available with the .NET Core SDK install, or with a special Windows Server Hosting .NET Core installer.

The second bit of the web.config file configures the ASP.NET Core module with instructions on how to start your application, which is to use the same dotnet command we saw earlier. Now, instead of our ASP.NET application running inside of a w3wp.exe IIS worker process, the application will execute inside of a dotnet.exe process.

With a better idea of how ASP.NET Core runs in production, let’s talk about benefits and risks.

Benefits

One benefit to Kestrel is the ability to execute across different platforms. You can author an ASP.NET application on Windows using Visual Studio and IIS Express, but deploy the application on Linux with Apache in front. In all scenarios, the server is always Kestrel and your application code doesn’t need to change.

Another benefit to Kestrel is the incredible work Microsoft has put into making a blazing fast web server with managed code. Inside the readme file for the ASP.NET Benchmarks repository, you’ll currently find benchmarks showing ASP.NET Core serving five times the number of requests per second as ASP.NET 4.6 on the same hardware. 313,001 requests per second compared to 57, 843.

benchmarks

Of course, benchmark code and benchmark results don’t always reflect how a specific business application will behave. It’s like seeing an F1 racing car manufactured by Toyota on the television and then thinking you’ll find a car that goes 220 mph at the local Toyota dealer. What you will find at the dealer are cars that indirectly benefit from the millions of dollars that Toyota puts into researching new technologies for their F1 cars. The benefits trickle down.

I decided to try some comparative benchmarks of my own. I created three applications with ASP.NET WebForms, ASP.NET MVC 5, and ASP.NET MVC Core. Each application delivers 3kb of HTML to the client using the typical patterns you would find in business applications. For example, using a master page in WebForms and using a Layout page in MVC. For ASP.NET MVC Core, I ran tests against a naked Kestrel server as well as a Kestrel server proxied by IIS.

Benchmarking ASP.NET

In my tests, the naked Kestrel server delivered 5 times the throughput of WebForms, and over twice the throughput of MVC 5. Once I moved the Core application behind IIS however, throughput dropped below the level of MVC 5. I know these results might surprise many people who believe that ASP.NET Core is inherently faster and lighter than its predecessors. However, ASP.NET Core’s predecessors were deeply integrated into IIS and could execute inside the same worker process where sockets were open. The current recommended setup adds an additional hop.

Another surprise – switching the ASP.NET Core application to run on the full .NET framework instead of .NET Core resulted in very little change in the throughput numbers. As developers, we want to believe .NET Core is also inherently faster and lighter than the full .NET framework, but for runtime performance in this specific application, the difference appears to be negligible.  

I do think we can reasonably expect the performance of ASP.NET Core with IIS to improve in the future, so we’ll leave performance as a benefit.

Risks

The biggest risk I’ve seen in the new hosting model is the confusion that results in using IIS with ASP.NET Core. There are ASP.NET related settings in IIS that have no impact on a .NET Core application. These are settings like the pipeline mode (integrated or classic), and the setting to select a version of the .NET framework for a specific AppPool.  

IIS Settings

Developers and IT operations will need to understand the new deployment strategy and understand which IIS settings are significant and which settings to ignore. Both parties will also need to learn new troubleshooting techniques and diagnose a new set of common errors. 502.5 – “Failed to start process” is a terrifying new common error. One reason for this error is that the web.config file specifies incorrect arguments for the dotnet host.

Summary

Currently, I feel the hosting area is one of the bigger areas of risk in ASP.NET Core. However, I think we will be able to mitigate the risks over time through a combination of experience, education, experimentation, and improvements from Microsoft based on customer feedback. It is important for developers and IT operations to look at how to host an ASP.NET core application early and not treat deployment as a known procedure that can wait till the end of the project.

Also In This Series

ASP.NET Core and the Enterprise Part 1: Frameworks

ASP.NET Core and the Enterprise Part 2: Hosting (current)

ASP.NET Core and the Enterprise Part 3: Middleware


Comments
gravatar Graham Clark Tuesday, October 25, 2016
Thanks! I guess your graph begs the question, what is the story for Kestrel behind Apache and/or NGINX? Is there any reason to think it would be any closer to the "naked Kestrel" performance?
gravatar Nikolay Nikov Tuesday, October 25, 2016
Ever since you comment on the previous article, I expected some disappointing real-world performance test. But the star of the show with Kestrel are the multi-platform capabilities. So I'm curious to see the performance when Kestrel is behind nginx. Plus, for small websites on a budget, Kestrel might still be the better option performance-wise even today, given the licensing costs of Windows Server and its appetite for memory.
gravatar jdan Tuesday, October 25, 2016
It will be nice when they get the publishing down as well. As of now, you get file locking and have to basically shut things down to publish updates.
gravatar Matt Brooks Tuesday, October 25, 2016
"The web.config file instructs IIS to send requests for all paths and verbs to a new HTTP handler named aspNetCore. This ASP.NET Core Module for IIS is a piece of software you’ll need to install on an IIS server to run ASP.NET Core applications." So is this piece of software both a handler and a module, in IIS terms?
gravatar Robert Wednesday, October 26, 2016
Both are really good write ups. Thanks for the more realistic benchmarks.
gravatar Scott Wednesday, October 26, 2016
@Graham: I'd like to know, too. Considering setting up a machine to dual boot. @Matt: It is a module, you'll see it in the XML config for IIS under system.webServer -> modules
gravatar Joe Wednesday, October 26, 2016
Will it be possible to run your tests using WebListener (now that it officially released)?
gravatar Scott Allen Friday, October 28, 2016
@Joe: I'd like to see that, too. Is on my TODO list.
gravatar Nick Craver Saturday, November 5, 2016
Thanks Scott, your notes on confusion are why I'd really want to see tighter integration into IIS. What I've seen so far says that Windows 2016 will not have support...it's still a module to install. That means it's a higher bar, an additional level of confusion, and causes more deployment pain overall. I'm hesitant to port some open source applications we have over (like Opserver) because the deployment story is such a bigger ask. There's also the problem of approval in some environments. Adding an application is typically a low ask and common operation, but deploying anything that requires an installer to IIS machines is way harder. It likely, at the very least, involves more people in the process. I really hope Microsoft reconsiders this, and the module is included in Windows 2016+ at least (as a Windows Feature). I'm not sure if they don't see this as an issue, or if the timelines just aren't lining up for this release. Either way it's very unfortunate. If less of my users are likely to be able to run the application on .NET Core (especially existing users), I either have to maintain 2 VERY different branches in an open source project (no, that's almost never happening), or break existing users, making me the jerk. So what do we do? Do we just assume ASP.NET Core is for new things, and porting isn't really a main use case? My read on all the .NET Core 2.0 compatibility says it IS a major use case, so huge as to cause a major direction overhaul. But the IIS hosting story doesn't line up. I hope the scope of this is realized sooner than later.
gravatar Warren Postma Saturday, November 5, 2016
My feelings on Microsoft's Web Story in 2016 is that it's chaos, and hurt, but that most everything is being worked on right now and is evolving in a good direction. When the current dust and debris settles I will probably be happy, except that I have no idea what the plan is. But we are at a point of inflection where almost everything to do with IIS+web+microsoft sucks at this exact moment. So forgive me for whinging a bit here, here's a list of my current IIS and .net and web and windows platform gripes: 1. IIS falls over in production with .net core. https://github.com/aspnet/IISIntegration/issues/245 2. ANCM doesn't ship in windows, as discussed since May. Still true with Windows Server 2016 RTM because ANCM is best described as preview technology or slightly worse. 3. Web Deploy in IIS remains a train wreck. Installed separately (just like ANCM). Quirky as hell. 4. URL Rewrite in IIS is a train wreck. Installed separately and infrequently updated and thus I wonder if it's even tested. 5. WebDAV in IIS is a train wreck. Turn feature on and by default it inserts itself aggressively deep into your root IIS stack and breaks the entire instance so everything returns a 502 error. Googling this shows that it appears to be by design. Who designs features like this? They compose like alkali metals compose with water. 6. There is no unified model for remote management of an on-premises production Microsoft web server stack. Compare to a Linux managed system with a HAPROXY and a bunch of apache or nginx web nodes managed by Salt Stack. Where's that? Someone finally figured out that a remote RESTful web remote management service makes sense in 2016. I hope by 2018 that will be equal to what I can do today on unix/linux platforms. 7. Where is the unified tool servicing and software install and windows component install for the Windows platform? There's powershell cmdlets to add IIS components, then there are bits which you download and click next next to install. There is the increasingly weird and rotten Microsoft Web Platform Installer and the various confusing conflicting stuff in there. Where is the vision? Make my Windows server web-facing platform as simple and easy to work with as my Linux ones.
gravatar Warren Postma Saturday, November 5, 2016
One way that .net core currently falls over is that there's something bad that happens deep in the IIS stack that causes the current working directory to be incorrect in the process context where the dotnet executable gets spawned. Sometimes, when .net core processes are being killed mercifully and respawned on a periodic timed basis the way that .net core processes like to be killed mercifully on a timer, because MICROSOFT, your processes will fail to start, and all your .net core apps will error 502.5 or something similar. Restarting IIS will clear the condition which persists until you stop W3SVC and start it again. I don't like calling bugs like that "race conditions" because getting into the failure mode is somewhat like hitting a race condition, but unlike a race condition, this one latches and stays latched in failure mode until you restart IIS. I hope this clarification makes my reservations about using IIS and dotnet core together in production clear. In Nov 2016 do NOT do it. Thanks. Warren
gravatar Frank Thursday, November 10, 2016
I would also be interested to see benchmarks of kestrel behind nginx instead of iis. What did you use to benchmark? There's also an issue that's still open regarding the performance in the IISIntegration repo on github (https://github.com/aspnet/IISIntegration/issues/245)
gravatar Andrew Lock Thursday, November 10, 2016
I would love to see those stats on Kestrel behind Nginx too. I'm really hoping it matches closer to the raw Kestrel performance than IIS achieves, otherwise it feels a bit like all the performance claims are somewhat moot, or even false, for the majority of users.
gravatar Richard Borrie Thursday, November 24, 2016
I have to agree with other posts here, .Net Core might be technically superior to .Net 4.6 but it is just too big a leap for too little benefit. If it needs its own webserver instead of IIS, we can just about live with that, but that webserver needs to be an option as part of the vanilla Windows Server installation, not some command line afterthought. And it needs to support the .Net languages, OK just C# at first, but the sooner it runs VB the better. Something has gone badly wrong here. The jump from .Net 4.6 to .Net Core is far higher than the jump from .Net 4.6 to PHP / Apache, and if Microsoft doesn't bring .Net Core into line with 4.6 pretty quick then developers will start to desert.
Comments are closed.

My Pluralsight Courses

K.Scott Allen OdeToCode by K. Scott Allen
What JavaScript Developers Should Know About ECMAScript 2015
The Podcast!