Serialization Options With Azure DocumentDB

Tuesday, April 28, 2015 by K. Scott Allen
0 comments

DocumentDBBehind the scenes, Azure’s DocumentDB uses Json.NET to serialize objects. Json.NET offers some flexibility in how serialization behaves through a number of serialization settings, but the DocumentDB SDK doesn’t expose these settings.

What if we want to change, say, the  NullValueHandling behavior to ignore null values?

Michalis Zervos offers one solution. Serialize objects using your own code then save the result as a document with the stream based APIs of DocumentDB. This approach gives you the ultimate control over when and how to serialize each object.

Another approach is to use the global default settings of Json.NET.

JsonConvert.DefaultSettings = () =>
{
    return new JsonSerializerSettings
    {
        NullValueHandling = NullValueHandling.Ignore
    };
};

Being global, you’ll want these settings to apply to all the documents saved into Azure, as well as any other serialization that might be happening in the same application. but it can turn this document:

{
  "Name": "Scott",
  "Medications": null,
  "Procedures": null,
  "id": "c15a4b48-bc7b-4440-a32b-88a9c345f705"
}

.. into this one:

{
  "Name": "Scott",
  "id": "e388eb16-de6f-4de6-9b11-851c2a67ef9e"
}

DocumentDb Limits and Statistical Outliers

Monday, April 27, 2015 by K. Scott Allen
2 comments

Azure’s DocumentDB has an appealing scalability model, but you must pay attention to the limits and quotas from the start. Of particular interest to me is the maximum request size for a document, which is currently 512kb. When DocumentDB first appeared the limit was a paltry 16KB, so 512kb feels roomy, but how much real application data can that hold?

Let’s say you need to store a collection of addresses for a hospital patient.

public class Patient
{
    public string Id { get; set; }
    public IList<Address> Addresses { get; set; }
}

public class Address
{
    public string Description { get; set; }
    public string City { get; set; }
    public string Country { get; set; }
}

In theory the list of address objects is an unbounded collection and could exceed the maximum request size and generate runtime errors. But in practice, how many addresses could a single person associate with? There is the home address, the business address, perhaps a temporary vacation address. You don’t want to complicate the design of the application to support unlimited addresses, so instead you might enforce a reasonable limit in the application logic and tell customers that having more than 5 addresses on file is not supported.

A Harder Problem

Here’s a slightly trickier scenario.

public class Patient
{
    public string Id { get; set; }
    public IList<Medication> Medications { get; set; }
}

public class Medication
{
    public string Code { get; set; }
    public DateTime Ordered { get; set; }
    public DateTime Administered { get; set; }
}

Each medication entry consists of an 8 character code and two DateTime properties, which gives us a fixed size for every medication a patient receives, but again the potential problem is the total number of medications a patient might receive.

The first question then, is how many Medication objects can a 512kb request support?

The answer, estimated with a calculator and verified with code, is just over 6,000.

The second question then, is 6,000 a safe number?

To answer the second question I found it useful to analyze some real data and find that the odds of busting the request size are roughly 1 in 100,000, which is just over 4 standard deviations. Generally a 4 sigma number is good enough to say “it won’t happen”, but what’s interesting when operating at scale, is that with 1 million patients you’ll observe the 4 sigma event not once, but 10 times.

From the business perspective, the result is unacceptable, so back to the drawing board.

We use to say that you spend 80% of your time on 20% of the problem. At scale there is the possibility of spending 80% of your time on 0.000007% of the problem.

Aurelia Hello World with ASP.NET 5

Tuesday, April 7, 2015 by K. Scott Allen
31 comments

The purpose of this post isn’t to give an exhaustive tour of Aurelia features, but to show a simple Aurelia application running in ASP.NET 5 and demystify some of the tools you’ll commonly see used with Aurelia. This post makes use of jspm and the SystemJS module loader. These tools and libraries aren’t required to use Aurelia, but they sure can make life easier.

1. Create a new, empty ASP.NET 5 web application with Visual Studio 2015.

2. Install and initialize jspm. In a previous post we looked at using jspm with the latest pre-release of ASP.NET. You’ll want to follow all those same steps, particularly the initialize step telling jspm to use ./wwwroot as the public folder path. We want Aurelia installed and served from a jspm_packages folder inside this wwwroot folder, since ASP.NET 5 serves static assets from wwwroot by default.

3. Use jspm to install some of the core Aurelia pieces. From the command line:

jspm install aurelia-framework
jspm install aurelia-bootstrapper

jspm will download all the required files and place them in the jspm_packages folder of wwwroot.

4. We are now ready to create an index.html file in wwwroot. There are 4 significant instructions in the following markup.

<html>
<head>
    <title>Hello from Aurelia</title>
</head>
<body aurelia-app>
    <script src="jspm_packages/system.js"></script>
    <script src="config.js"></script>
    <script>
        System.import("aurelia-bootstrapper");        
    </script>
</body>
</html>

The aurelia-app attribute tells Aurelia where it will take control of the DOM and inject our applications’ first view and viewmodel.

The script tag to load system.js brings glorious magic to the page. SystemJS is a module loader, much like  Require JS, but SystemJS knows how to load different types of modules – AMD, CommonJS, and ES6 modules. SystemJS also contains the ability to dynamically compile ES6 code into ES5 code on the fly. We’ll take advantage of the dynamic compilation features to use Aurelia without setting up any build task for our script code. jspm installs SystemJS by default.

The script tag to load config.js brings in configuration data to tell the module loader where to find script files, like scripts for Aurelia as well as our own application. The configuration also tells the loader how to behave. Fortunately for us, jspm creates and maintains the config.js file, so the only work we have to do is include the file on the page. 

Finally, we will use the module loader’s API to import the aurelia-bootstrapper. The module loader will find and load the bootstrapper and the bootstrapper’s dependencies into the browser.  Once the bootstrapper has control, it will begin to spin up Aurelia using some default configuration options, and go looking for our application, which we will build in the next two steps.

5. By default, Aurelia will go looking for an app.js script in the same folder as our index.html file. Let’s fill app.js with the following code:

export class App {
    
    constructor() {
        this.message = "";
    }

    activate() {
        this.message = "Hello, World!";
    }

    changeMessage() {
        this.message = "Goodbye!";
    }

}

View models in Aurelia are plain, vanilla JavaScript.  There is no magic or registration code required. The framework will automatically invoke the activate method, which can return a promise if long running work needs done.

6. Once Aurelia finds and activates app.js, the framework will try to load an app.html file to function as the view for the model. We’ll create app.html in the wwwroot folder, too.

<template>
    <div>
        <div>${message}</div>
        <button click.trigger="changeMessage()">Say Goodbye</button>
    </div>
</template>

Binding model content into the DOM is as easy as using the string interpolation ${ } syntax, while events are directed to model methods using trigger, delegate, and call expressions.  

image

And that’s it! Once you learn just a few simple conventions, Aurelia makes applications easy. We haven’t touched on routing, dependency injection, custom elements, or any of the other features that make Aurelia great. Those are all topics for the future… 

Working With Aurelia

Monday, April 6, 2015 by K. Scott Allen
14 comments

Aurelia

I’ve been chipping away and contributing to the Aurelia project the last couple months.

When Rob introduced Aurelia earlier this year, I took one look at the code and had a moment of déjà-vu. The Aurelia code base was the code base I saw in a feverish dream one night in early 2014 when I first started digging into the ES6 specification. It’s a beautiful code base building on the newest features of the JavaScript language, features like module and classes, and the code is a dramatic demonstration of just how different and wonderful the new world and the new paradigms can be.

On top of ES6, Rob has added the “Eisenberg Effect”,which is to say the framework is modular, extensible, testable, and favors convention over configuration. You’ll recognize many patterns if you’ve used Durandal or Caliburn Micro in the past.

Best of all, the APIs are crafted to make life easy for the developer instead of the framework authors.

Expect more Aurelia coverage in the future. Next up will be “Hello, World!” with Aurelia, ASP.NET 5, and Visual Studio 2015.

Entry Points for ASP.NET 5 Commands

Monday, March 30, 2015 by K. Scott Allen
3 comments

Inside the project.json file for an ASP.NET 5 application, you can find a commands section:

"commands": {
    "gen": "Microsoft.Framework.CodeGeneration",
    "kestrel": "Microsoft.AspNet.Hosting --server Kestrel --server.urls http://localhost:5004",
    "web": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.WebListener --server.urls http://localhost:5002"
},

You can execute these commands from a command prompt with the .NET Execution Environment (dnx, formerly named k), which will hunt for a standard entry point in the assembly referenced by the command value, and also pass along any parameters in the command value.

In other words, “dnx web” will look in the Microsoft.AspNet.Hosting assembly for a main entry point, and then pass the –server and –server.urls parameters into the entry point.

Typical commands you’ll see in new projects include commands to spin up the application in a web server, or execute database migrations.

You can also create your own command hosts using the “ASP.NET 5 Console Application” template, or, you can add a Program class with a Main method inside an existing web application.

public class Program
{
    public Program(ILibraryManager libraryManager)
    {
        foreach(var lib in libraryManager.GetLibraries())
        {
            Console.WriteLine(lib.Name);
        }
    }

    public void Main(string[] args)
    {
        var config = new Configuration()
                        .AddJsonFile("config.json")
                        .AddCommandLine(args);

        Console.WriteLine(config.Get("message"));

        foreach (var arg in args)
        {
            Console.WriteLine(arg);
        }

        Console.ReadLine();
    }
}

Notice how the Main method doesn’t require the static keyword, in fact the dnx will instantiate the Program class itself if possible, and even inject dependencies. You can ask for IServiceManifest to see all the services available at startup. In the above code, I’ve asked for an ILibraryManager, and I can use the manager to list out all the dependencies for the application.

The ASP.NET 5 configuration system works for this scenario, too. The above code sets up config.json as the first configuration source. If we set up the configuration file with a message property:

{
    "message":  "Hello!"
}

… then running the application will display the message “Hello!”.

We can override the message by specifying a command line value in the project.json file.

"commands": {
    "me" : "TestApp --message=Goodbye"
}

In this case, “me” is the name of the command and “TestApp” is the name of the web application. Running dnx me from the command line will display the string “Goodbye”. You can also override the command line parameters specified in the project commands by passing parameters to dnx. In other words, “dnx me --message=Allo!” will print the string “Allo!”

Although not the most exciting feature of ASP.NET 5, commands do open up some interesting possibilities for hosting and executing utility functions, data migrations, and scheduled tasks.

Pluralsight - First Look At The Next ASP.NET

Monday, March 16, 2015 by K. Scott Allen
12 comments

My latest Plurasight release is an early look at the next version of ASP.NET, including:

- The unification of MVC 6 and WebAPI

- Startup and configuration of an ASP.NET 5 application

- Working with wwwroot and the new ASP.NET project structure

- Using command line tools to manage runtime versions, project commands, and deployment

- Integration with NodeJs and Grunt.

ASP vNext First Look

I hope you like it!

Lazy Evaluation of Generators in ES6

Monday, March 9, 2015 by K. Scott Allen
5 comments

In a previous post I introduced generators in ES6.

Generators in most languages do the least amount of work possible, and generators in JavaScript are no different. When a program invokes a normal function, you can imagine the program jumping into the function and executing code inside the function until the function completes. But, the behavior of a generator function is entirely different from the behavior of a non-generator function.

As an example, let’s use the following generator function.

let random = function*() {

    while(true) {
        console.log("make random")
        yield Math.random();
    }
}

The random function looks like it will burn CPU cycles in an infinite loop. However, if we invoke the function and look for output in the console – there will be no output!

When you invoke a generator function, you can think of the runtime setting up a data structure or a state machine behind the scenes. The state machine knows how to take the code inside the generator function and allow pieces to execute on demand.

What creates the demand?

Since generators produce iterators, then each call to the next method of the iterator will force the state machine to execute just enough code to produce the next value – and no more. For example, if we execute the following code:

let iterator = random();
iterator.next();

Then we will see “make random” appear exactly once in the console. Asking the iterator for the next value forced the state machine to execute the code inside the function until a yield statement manufactured another value. Each time there is a call to next on the iterator, the flow of the program jumps back into the generator function at the point the program last left the function, and execution continues until the program reaches another yield.

In computer science terminology, a generator function offers what we call lazy evaluation. In order to make the function perform real work, we need to pull values out of the function by asking for those values through the iterator API. The random function will only create an infinite loop if we ask for an infinite number of values.

Let’s imagine we wanted a finite number of random numbers, and we only want random numbers less than 0.5. First, we could write a filtering function that will take an iterable object and a predicate function as arguments. The filtering function will only produce the items for which the predicate function returns true.

let filter = function*(items, predicate) {
    for(let item of items){
        console.log("filter", item);
        if(predicate(item)){
            yield item;
        }
    }
};

To make sure we ask for a finite number of items, we can also write a take function, to take a specific number of items.

let take = function*(items, number) {
    let count = 0;
    if(number < 1) return;

    for(let item of items){
        console.log("take", item);
        yield item;
        
        count += 1;
        if(count >= number) {
            return;
        }
    }
};

Note how we can stop iteration using a return statement in the take function.

With the above functions, all of which are generators, we can create two random numbers less than 0.5 with the following code.

let result = take(filter(random(), n => n < 0.5), 2);

Except, we have not created any numbers with the above code! All we’ve created is an iterator, and no output will appear on the console. We could force the functions to execute code using a for of loop, or use Array.from, which is a new API we can use to convert any iterable object to a concrete array.

expect(Array.from(result).length).toBe(2);

Console output of the above code might look like the following.

make random

filter 0.24152887403033674

take 0.24152887403033674

make random

filter 0.6618692250922322

make random

filter 0.49661534116603434

take 0.49661534116603434

Notice how the execution of the different functions interleave. You can imagine the iterators pulling values from each function on an as-needed basis.

Lazy evaluation allows for all sorts of interesting scenarios with control flow and data structures. We’ll look at some of these scenarios in a future post, but coming next: delegating generators.

by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!