OdeToCode IC Logo

.NET Core Opinion 12 – Use Your ASP.NET Core App as an Executable

Wednesday, March 27, 2019 by K. Scott Allen

There is a special moment of time in the life of every ASP.NET Core process when the application has all the services registered and all the configuration sources in place, but has not yet started to listen for HTTP messages. This is a moment in time you can seize and use to your advantage.

I tell people to think of their ASP.NET Core application as a console application that happens to listen for network connections, because there are many useful tasks the application can perform at development time and in production. Examples would be tasks like migrating database schemas, clearing a distributed cache, and creating storage containers.

public static void Main(string[] args)
{
    var host = CreateWebHostBuilder(args).Build();

    // the moment in time when magic can happen
    ProcessCommands(args, host);

    host.Run();
}

Let’s drill into database migrations. Many people will say we need to keep automatic migrations out of web applications because the results in a server farm are unpredictable. Good advice! I’m not suggesting you implicitly run migrations as a side-effect of the first HTTP request. I’m promoting the use of the application to explicitly run migrations when the app receives the right command line parameter. Think of the parameter as an automation point, and you can now migrate a database from a deployment script, or from a development shell.

Another objection is how executing utility tasks in the application feels like a violation of the single responsibility principle at the application level. Why not build a separate console application to execute utility tasks and leave the main app for processing web requests? This is possible, but you’ll find yourself needing to reuse or duplicate configuration code from the web application. Using database migrations as a concrete example again, there is a lot of work that goes into building the right services and configuration for a specific environment, including non-trivial steps like connection string decryption and network communication with key storage services. The app, assuming it works, is setup perfectly to have everything in place for a specific environment like production, staging, or development.

Here’s another example of something I’ve done in the past. On this project I could run the following command to drop the development database, recreate the database using migrations, seed the database with data, and then exit instead of entering into web server mode.

dotnet run  dropdb migratedb seeddb stop

The overall theme here is not about database specific tasks, but about automating common tasks to make development, testing, and deployments faster and more repeatable. The perfect tool for these jobs might just be the application you are already have!

.NET Core Opinion 11 – Keep Configure Methods Clean Using Extension Methods

Monday, March 25, 2019 by K. Scott Allen

The Configure and ConfigureServices methods in ASP.NET Core tend to become bloated in larger projects. The problem is that larger projects can require lots of configuration, and the configuration requires lots of options. You open a project and there are dozens, even hundreds of lines of code, to configure OIDC, Swagger, authorization policies, data services, claims transformations, cookie options, CORS policies, domain services, and the list goes on and on.

I encourage team members to use extension methods so each line of code in the Configure methods is simple and the configuration details are hidden (yet easy to find by navigating with F12).

For example, instead of using lambdas and overwhelming the reader with lots of configuration details ...

public void ConfigureServices(IServiceCollection services)
{

    services.AddMvc(options =>
    {
        options.Conventions.Add(new FeatureControllerModelConvention());
    })
    .AddRazorOptions(options =>
    {
        options.ViewLocationFormats.Clear();
        options.ViewLocationFormats.Add(@"{3}\{0}.cshtml");
        options.ViewLocationFormats.Add(@"Features\Shared\{0}.cshtml");
        options.ViewLocationExpanders.Add(expander);
    });

    // and more code and more code and more code
}

... Provide a high-level overview of the app configuration, but keep the details encapsulated.

public void ConfigureServices(IServiceCollection services)
{
    services.AddCustomMvc();
    services.AddSecurity();
    services.AddCustomMediator();
    services.AddDataStores(Configuration.GetConnectionString(nameof(LeagueDb)));
}

Hide the configuration details in extension methods which are easy to find.

public static IServiceCollection AddCustomMvc(this IServiceCollection services)
{
    var expander = new FeatureViewLocationExpander();

    services.AddMvc(options =>
     {
         options.Conventions.Add(new FeatureControllerModelConvention());
     })
    .AddRazorOptions(options =>
    {
        options.ViewLocationFormats.Clear();
        options.ViewLocationFormats.Add(@"{3}\{0}.cshtml");
        options.ViewLocationFormats.Add(@"Features\Shared\{0}.cshtml");
        options.ViewLocationExpanders.Add(expander);
    });

    return services;
}

There’s also a good chance you can re-use some of the extension methods, particularly in a micro-services environment. However, the primary focus is to make the startup code usable and readable, not re-usable.

Microsoft Watches GitHub Commits for Azure Secrets

Thursday, March 21, 2019 by K. Scott Allen

One morning last month I woke up, looked at my inbox, and saw:

Cyber Defense Opereations Center

I must admit, my first instinct when seeing any email with the word cyber in a 24 point font is to banish the message into a spam bucket, but something kept me around long enough to follow a link to GitHub. Lo and behold! Here was a connection string with the username and password to an Azure SQL instance using a familiar naming convention. To make a long story short, yes, someone outside the company accidentally checked in some test code with a connection string into a public repository.

There are many stories to tell here, but one story is another angle on the evolution of Microsoft. There was a time when the company shipped the world’s largest target for viruses and malware with no protection from viruses and malware installed. We were left to protect ourselves. The plan worked well for developers and other software savvy individuals, but not so well for the average user. Microsoft is now more aggressive and proactive about security, which is good for everyone.

One additional piece of information. Alerts like the one shown here are obviously important. But, by default the alert only goes to the subscription owner. You can change this setting in the portal by going into Security Center and opening the Security Policy blade. In the policy blade you'll see a list of subscriptions each with a link to configure settings. Click on the link and you can then go to Email notifications and configure a group email address.

Cyber Defense Opereations Center

Hope that helps!

The Lost Art of Right Aligning Code

Tuesday, March 19, 2019 by K. Scott Allen

When I started working with the Go language using Visual Studio Code a few months ago, the Go extension for VS Code auto-formatted one type definition like so:

type ResourceInfo struct {
    ResourceID         string
    SubscriptionID     string
    GroupName          string
    Name               string
    Type               string
    RunID              string
    DocumentType       string
    Location           string
    LocationMisAligned bool
}

When I was a C / C++ developer I created and read right-aligned code all the time. The alignment makes patterns in code easier to find. For another example in Go, it’s easier to see which entries point to a "no-operation" function in the following hash map, because you can scan vertically down the file.

var resourceMap = map[string]func(*ResourceInfo) {
    "Microsoft.Sql/servers":           visitSQLServer,
    "Microsoft.Sql/servers/databases": noop,
    "Microsoft.Cache/Redis":           visitRedisCache,
    "Microsoft.Web/sites":             visitWebSite,
    "Microsoft.Portal/dashboards":     noop,
    // ...
}

Keeping code aligned requires some work, and without tool support it is rare to see the convention in .NET languages. The only tool I've found that comes close to supporting this style is the Always Algined extension for Visual Studio.

.NET Core Opinion 10 – Move More Code Out of Startup

Thursday, March 7, 2019 by K. Scott Allen

In a previous opinion, we looked at the responsibilities of the Startup class in ASP.NET Core. A consistent problem I run across in code reviews is the amount of code that gravitates into the Startup class, making the entire file cluttered and not well factored.

We’ll look at a variety of solutions to declutter Startup.cs, but let’s start with a simple task. Once we agree that the Startup class is only responsible for service configuration and middleware, we should move other code out of Startup and into Program.cs where the Main entry point lives. Ironically, it is the .NET Core templates that I believe encourage cluttered and mismatched code inside of Startup.cs, but Microsoft has been improving these templates with each release. For example, here’s the startup code generated by dotnet new in 1.1.

public Startup(IHostingEnvironment env)
{
    var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile("appsettings.json")
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
            .AddEnvironmentVariables();
    Configuration = builder.Build();
}

// ...

public void Configure(IApplicationBuilder app, IHostingEnvironment env, 
                               ILoggerFactory loggerFactory)
{

    loggerFactory.AddConsole(Configuration.GetSection("Logging"));
    loggerFactory.AddDebug();

         // ...
}

Notice how the code is configuring the configuration system in the constructor, and is configuring the logging settings in the method responsible for installing middleware. New templates in 2.1 remove these bits of code. Some of the code has moved into WebHost.CreateDefaultBuilder, but even if you need custom settings for configuration or logging, I’d suggest keeping the code out of Startup, and place the code into Program.cs instead.

As an example, here’s what Program.cs could look like, if it took on configuration and logging settings.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost
                .CreateDefaultBuilder(args)
                .ConfigureAppConfiguration((hostingContext, config) =>
                {
                    var env = hostingContext.HostingEnvironment;
                    config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true);
                    // ...
                })
                .ConfigureLogging((hostingContext, logging) =>
                {
                    logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
                    logging.AddConsole();
                    // ...
                })
                .UseStartup<Startup>();

Look at extension methods for IWebHostBuilder and see what else you can move into Program.cs. In the next opinion post, we’ll talk about techniques for getting rid of the other clutter in the Configure* methods of Startup.

Debugging .NET Builds with the MSBuild Structured Log Viewer

Tuesday, March 5, 2019 by K. Scott Allen

I was trying to debug a dotnet publish operation on an ASP.NET Core project recently and becoming irrational after staring at MSBuild log files for too long. MSBuild does all the heavy lifting in dotnet publish and build operations. That’s when I discovered the MSBuild Structured Log Viewer from Kirill Osenkov. You can install the viewer from http://msbuildlog.com/.

In the end I was fighting a battle over $(DefaultItemExcludes) during publish and trying to find the right target to override and "fix" the property. Even the simplest builds are complicated these days, with implicit imports for .prop files, hundreds of properties, and dozens of tasks. Trying to figure the order of execution by looking at individual .csproj and .targets files is impossible.

MSBuildLog provides a tree view of all the targets that are executing, and provides the properties and items in effect for each step. The search feature is excellent and usually the best way to navigate into the tree, if you have an idea of what you are looking for. MSBuildLog Tree View

The tool also provides a list of the slowest build tasks, which these days always has Exec tasks calling out to Node or npm at the top of the list.

MSBuildLog slowest tasks

await the async Letdown

Monday, March 4, 2019 by K. Scott Allen

Microsoft added async features to the C# language with more than the usual fanfare. We were told async and await would fundamentally change how .NET developers write software, and future development would be async by default.

After awaiting the future for 7 years, I still brace myself anytime I start working with an async codebase. It is not a question of if, but a question of when. When will async let me down and give me a headache?

The headaches usually fall into one of these categories.

Old Code Is Still Sync Code

Last fall I was working on a project where I needed to execute code during a DocumentProcessed event. Inside the event, I could only use async APIs to read from one stream and write into a second stream. The delegate for the event looked like:

public delegate void ProcessDocumentDelegate(MarkdownDocument document);

If you’ve worked with async in C#, you’ll know that the void return type kills asynchrony. The event handler cannot return a Task, so there is nothing the method can return to encapsulate the work in progress.

The bigger problem here is that even if the event handler could return a Task, the code raising the event is old code with no knowledge of Task objects. I could not allow control to leave the event handler until my async work completed. Thus, I was facing the dreaded "sync over async" obstacle.

There is no getting around or going over this obstacle without feeling dirty. You have to hope you are writing code in a .NET Core console application where you can hold your nose and use .Result without fear of deadlocking. If the code is intended for a library with the possibility of execution in different environments, then the saying abandon hope all ye who enter here comes to mind.

New Code Is Sometimes Still Sync Code

When working with an old code base you can assume you’ll run into problems where async code needs to interact with sync code. But, the situation can happen in new code, and with new frameworks, too. For example, the ConfigureServices method of ASP.NET Core.

public void ConfigureServices(IServiceCollection services)
{
    // ...
}

There are many reasons why you might need async method calls inside ConfigureServices. You might need to obtain an access token or fetch a key over HTTPs from a service like Key Vault. Fortunately, there are a few different solutions for this scenario, and all of them we move the async calls out of ConfigureServices.

The easiest way out is to hope you are using a library designed like the KeyVault library, which moves async code into a callback, and invokes the callback later in an async context.

AuthenticationCallback callback = async (authority,resource,scope) =>
{
    // ...

    var authResult = await authContext.AcquireTokenAsync(resource, credential);
    return authResult.AccessToken;
};
 
var client = new KeyVaultClient(callback);

Another approach is to move the code into Program.cs, where we finally (after waiting for 2.1 releases of C#) have an async Main method to start the process. Finally, you can use one of the approaches described in Andrew Lock’s three part series on Running async tasks on app startup in ASP.NET Core.

The Task API is a Minefield

The Task class first appeared in .NET 4.0 ten years ago as part of the Task Parallel Library. Task was an abstraction with a focus on parallelism, not asynchrony. Because of this early focus, the Task API has morphed and changed over the years as Microsoft tries to push developers into the pit of async success. Unfortunately, the Task abstraction has left behind a trail of public APIs and code samples that funnel innocent developers into a pit of weeping and despair.

Example - which of the following can execute compute-bound work independently for several minutes? (Choose all that apply)

new Task(work);

Task.Run(work);

Task.Factory.StartNew(work);

Task.Factory.StartNew(work).ConfigureAwait(false);

Task.Factory.StartNew(work, TaskCreationOptions.LongRunning);

Answer: all of the above, but some of them better than others, depending on your context.

Good Code Turns Bad

Another disappointing feature of async is how good code can turn bad when using the code in an async context. For example, what sort of problem can the following code create?

using (var streamWriter = new StreamWriter(context.Response.Body))
{
    await streamWriter.WriteAsync("Hello World");
}

Answer: in a system trying to keep threads as busy as possible, the above code blocks a thread when flushing the writer during Dispose. Thanks to David Fowler’s Async Guidance for pointing out this problem, and other subtleties.

Awaiting the End

After all these years with tasks and async, there are still too many traps to catch developers. There is no single pattern to follow for common edge cases like sync over async, yet so many places in C# demand synchronous code (constructors, dispose, and iteration come to mind). Yes, new language features, like async iterators, might shorten this list, but I’m not convinced the pitfalls will disappear. I can only hope that, like the ConfigureAwait disaster, we don’t have to live with the work arounds sprinkled all through our code.