Generators in ECMAScript 6

Monday, February 16, 2015 by K. Scott Allen

You’ll know you are looking at a generator, or more properly, a generator function, because a generator function contains an asterisk in the declaration.

function*() {

}

Once you have a generator function, you can use the yield keyword inside to return multiple values to the caller.

let numbers = function*() {

    yield 1;
    yield 2;
    yield 3;
};

But, the numbers don’t come back to the caller all at once. Internally, the runtime builds an iterator for you, so a caller can iterate through the results one by one. You can write a low level iterator and call next to move through each value, or use a for of loop.

let sum = 0;

for(let n of numbers()) {
    sum += n;
}

expect(sum).toBe(6);

Going back to the classroom class we used in the last ES6 post, we can now make a classroom an iterable simply using yield.

class Classroom {

    constructor(...students) {
        this.students = students;
    }

    *[Symbol.iterator]() {
        for(let s of this.students) yield s;
    }
}

var scienceClass = new Classroom("Tim", "Sue", "Joy");

var students = [];
for(let student of scienceClass){
    students.push(student);
}

expect(students).toEqual(["Tim", "Sue", "Joy"])

Notice how the iterator method needs an asterisk before the opening square bracket. The syntax is quirky, but an asterisk is always required when defining a generator function, because generator functions need to behave differently than normal functions right from the start. That’s the topic for the next post in this series.

Roslyn Code Gems - Performance Goals

Friday, February 13, 2015 by K. Scott Allen

Ever since the .NET compiler platform became open source, I’ve been poking around the Roslyn source code. It’s not often you get to look at the internals of a product with a large code base, and not surprisingly there are some gems inside. I have a collection of the gems and each gem falls into one of three categories.

1. Ideas to borrow

2. Trivia

3. Bizarre and outstanding

Today’s gem falls into category two – trivia.

Have you ever wondered what makes for a “captive” audience? According to the PerformanceGoals class, this would be an operation using from one to ten seconds of time.

static PerformanceGoals()
{
    // An interaction class defines how much time is expected to reach a time point, the response 
    // time point being the most commonly used. The interaction classes correspond to human perception,
    // so, for example, all interactions in the Fast class are perceived as fast and roughly feel like 
    // they have the same performance. By defining these interaction classes, we can describe 
    // performance using adjectives that have a precise, consistent meaning.
    //
    // Name             Target (ms)     Upper Bound (ms)        UX / Feedback
    // Instant          <=50            100                     No noticeable delay
    // Fast             50-100          200                     Minimally noticeable delay
    // Typical          100-300         500                     Slower, but still no feedback necessary
    // Responsive       300-500         1,000                   Slower yet, potentially show Wait cursor
    // Captive          >500            10,000                  Long, show Progress Dialog w/Cancel
    // Extended         >500            >10,000                 Long enough for the user to switch to something else

    // Used for throughput scenarios like parser bytes per second.
    const string Throughput_100 = "Throughput_100";

    Goals = new string[(int)FunctionId.Count];
    Goals[(int)FunctionId.CSharp_SyntaxTree_FullParse] = Throughput_100;
    Goals[(int)FunctionId.VisualBasic_SyntaxTree_FullParse] = Throughput_100;
}

Unlike the code in this post, the next entry in this series will feature a gem with potentially useful code – an optimized bit counter.

Conditional Access Operator in C# 6.0

Thursday, February 12, 2015 by K. Scott Allen

My series of posts on C# 6.0 has been on a long hiatus due to changes in the language feature set announced last year. Now with spring just around the corner in the northern hemisphere, Visual Studio 2015 reaching a 5th CTP status, and features hopefully stabilizing, it might be time to start up again.

As of C# version 5, there are many techniques you can use to avoid null reference exceptions (NREs). You can use brute force if else checks, the Null Object design pattern, or even extension methods, since extension method invocations will work on a null reference.

The conditional access operator, ?.,  adds one more option to the list for v6 of the language.

Imagine you are writing a method to log the execution of some code represented by an Action delegate. You aren’t sure if the caller will pass you a proper delegate, or if that delegate will have a method and a name.

The following code uses conditional access to retrieve the method name of an action, or “no name” if the code encounters a null value along the line.

public async Task LogOperation(Action operation)
{
    var name = operation?.Method?.Name ?? "no name";

    operation();
    await _logWriter.WriteAsync(name + " executed");
    
    // ... exception handling to come soon ...          
}

By dotting into the operation parameter using ?., you’ll be able to avoid NREs. ?. will only dereference a pointer if the pointer is non null. Otherwise, the operator will yield a null value.

By combining ?. with the null coalescing operator ??, you can easily specify defaults and write code that must be read using uptalk

Next up in the series: await and exceptions.

Debugging Map Reduce in MongoDB

Wednesday, February 11, 2015 by K. Scott Allen

There isn’t much insight into the execution of a map reduce script in MongoDB, but I’ve found three techniques to help. Of course the preferred technique for map reduce is to use declarative aggregation operators, but there are some problems that naturally lend themselves to copious amounts of imperative code. That’s the kind of debugging i needed to do recently.

Log File Debugging

In a Mongo script you can use print and printjson to send strings and objects into standard output. During a map reduce these functions don’t produce output on stdout, unfortunately, but the output will appear in the log file if the verbosity is set high enough. Starting mongod with a –vvvv flag works for me.

Log file output can be useful in some situations, but in general, digging through a log file created in high verbosity mode is difficult.

Attached To The Output Debugging

The best way I’ve found to debug map reduce scripts running inside Mongo is to attach logging data directly to the output of the map and reduce functions.

To debug map functions, this means you’ll emit an object that might have an array attached, like the following.

{
  "name": "Scott",
  "total" : 15,
  "events" : [
    "Step A worked",
    "Flag B is false",
    "More debugging here"
  ]
}

Inside the map function you can push debugging strings and objects into the events array. Of course the reduce function will have to preserve this debugging information, possibly by aggregating the arrays. However, if you are debugging the map function I’d suggest simplifying the process by not reducing and simply letting emitted objects pass through to the output collection. Another technique to do this is to emit using a key of new ObjectId, so each emitted object is in its own bucket.

As an aside, my favorite tool for poking around in Mongo data is Robomongo (works on OSX and Windows). Robomongo is shell oriented, so you can use all the Mongo commands you already know and love. 

robomongo for mongodb

Robomongo’s one shortcoming is in trying to edit system stored JavaScript. For that task I use MongoVue (Windows only, requires a  license to unlock some features).

Browser Debugging

By far the best debugging experience is to move a map or reduce function, along with some data, into a browser. The browser has extremely capable debugging tools where you can step through code and inspect variables, but there are a few things you’ll need to do in preparation.

1. Define any globals that the map function needs. At a minimum, this would be an emit function, which might be as simple as the following.

var result = null;

window.emit = function(id, value) {
    result = value;
};

2. Have a plan to manage ObjectId types on the client. With the C# driver I use the following ActionResult derived class to get raw documents to the browser with custom JSON settings to transform ObjectId fields into legal JSON. 

public class BsonResult : ActionResult
{
    private readonly BsonDocument _document;

    public BsonResult(BsonDocument document)
    {
        _document = document;
    }

    public override void ExecuteResult(ControllerContext context)
    {
        var settings = new JsonWriterSettings();
        settings.OutputMode = JsonOutputMode.Strict;

        var response = context.HttpContext.Response;
        response.ContentType = "application/json";                      
        response.Write(_document.ToJson(settings));
    }        
}

Note that using JsonOutputMode.Strict will give you a string that a browser can parse using JSON.parse, but it will change fields of type ObjectId into full fledged objects ({ “$oid”: “5fac…ffff”}). This behavior will create a problem if the map script ever tries to compare ObjectId fields by value (object1.id === object2.id will always be false). If the ObjectId creates a problem, the best plan, I think, is to walk through the document using reflection in the browser and change the fields into simple strings with the value of the ID.

Hope that helps!

Thoughts On End to End Testing of Browser Apps

Tuesday, February 10, 2015 by K. Scott Allen

In a previous post on using the PageObject pattern with Protractor, Martin asked how much time is wasted writing tests, and who pays for the wasted time?

To answer that question I want to think about the costs and benefits of end to end testing. I believe the cost benefit for true end to end testing looks like the following.

cost benefit of e2e testing

There are two significant slopes in the graph. First is the “getting started” slope, which ramps up positively, but slowly. Yes, there are some technical challenges in e2e testing, like learning how to write and debug tests with a new tool or framework. But, what typically slows the benefit growth is organizational. Unlike unit tests, which a developer might write on her own, e2e tests requires coordination and planning across teams both technical and non-technical. You need to plan the provisioning and automation of test environments, and have people create databases with data representative of production data, but scrubbed of protected personal data. Business involvement is crucial, too, as you need to make sure the testing effort is testing the right acceptance criteria for stakeholders.

All of the coordination and work required for e2e testing on a real application is a bit of a hurdle and is sure to build resistance to the effort in many projects. However, the sweet spot at the top of the graph, where the benefit reaches a maximum, is a nice place to be. The tests give everyone confidence that the application is working correctly, and allows teams to create features and deploy at a faster pace. There is a positive return in the investment made in e2e tests. Sure, the test suite might take one hour to run, but it can run any time of the day or night, and every run might save 40 hours of human drudgery.

There is also the ugly side to e2e testing where the benefit starts to slope downward. Although the slope might not always be negative, I do believe the law of diminishing returns is always in play. e2e tests can be amazingly brittle and fail with the slightest change in the system or environment. The failures lead to frustration and it is easy for a test suite to become the villain that everyone despises. I’ve seen this scenario play out when the test strategy is driven with mindless metrics, like when there is a goal to reach 90% code coverage.

In short, every application needs testing before release, and automated e2e tests can speed the process. Making a good test suite that doesn't become a detriment to the project is difficult, unfortunately, due to the complex nature of both software and the human mind. I encourage everyone to write high value tests for the riskier pieces of the application so the tests can catch real errors and build confidence.

Creating Iterables In ECMAScript 6

Monday, February 9, 2015 by K. Scott Allen

Imagine you are building an abstraction for a classroom, and one responsibility of a classroom object is to encapsulate the names of the students in the classroom. You might build a class like the following.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }
}

On occasion, the outside world needs to know the names of the students inside the classroom. Perhaps the names will be shown on the screen, or saved in a database. In either case, how should a classroom object give the student names to a caller? One option is to return a reference to the students array, in which case the caller might change the array by adding or removing items. This option isn’t the best option if the classroom want to protect the array and validate the addition or removal of a student.

Another option is to make a copy of the array. Then, even if someone changes the copy, the original student array remains safe. However, copy operations can be expensive.

With ES6 and iterators we now have the ability to provide access to a collection without giving someone access to a mutable array. All we need is to make the classroom iterable by adding a Symbol.iterator method.

First, let’s make the classroom iterable by building our own iterator objects. This is the hard way to solve the problem.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }

    [Symbol.iterator]() {
         var index = 0;
         return {
            next: () => {
                if(index < this.students.length){
                     index += 1;
                     return {done: false, value: this.students[index-1]};
                }
                return { value: undefined, done: true };
            }
         }
    }
}

The above code adds the iterator method to the class. Notice how Symbol.iterator needs square brackets to surround the method “name”. Inside the method is the code to return objects with a next method, which in turn returns objects with done and value properties. A classroom should now behave just like an array and allow developers to use for of loops.

let scienceClass = new Classroom();
let result = [];
for(let name of scienceClass) {
    result.push(name);
}

expect(result).toEqual(["Tim", "Joy", "Sue"]);

One might notice that the code inside the classroom iterator method is the type of code that works with any array. Let’s extract the code into a new class.

class ArrayIterator {
    constructor(array) {
         this.array = array;
         this.index = 0;
     }

    next() {
        let result = { value: undefined, done: true};
        if(this.index < this.array.length) {
            result.value = this.array[this.index];
            result.done = false;
            this.index += 1;
        }
        return result;
    }
}

Now we can reuse ArrayIterator from several places in an application.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }

    [Symbol.iterator]() {
        return new ArrayIterator(this.students);
    }
}

Even though the ArrayIterator class is helpful, we will see an even easier implementation of the iterator method when we get to generators.

Static Members in ES6

Monday, February 2, 2015 by K. Scott Allen

In previous posts, we looked at classes in ES6, but I never gave an example using the static keyword.

A class may contain one or more methods defined with the keyword static.

class Employee {
    constructor(name) {
        this._name = name;
    }

    static convert(thing) {
        if(thing.name) {
            return new Employee(thing.name);
        }
    }
}

Like other languages with static class members, the static keyword will create a method associated with the class, and not with an instance of the class. In other words, you can only reach a static method using the name of the class.

expect(Employee.convert).toBeDefined();
expect(new Employee().convert).toBeUndefined();

Static methods have no access to the fields, properties, and methods defined on an instance of the class using this. For example, the convert method would not be able to use this._name to get to the _name field of an Employee object, as the this pointer inside of convert will never reference an Employee object. However, static methods are often useful and can play the roles of factory methods, conversion methods, or general class helper methods.

let person = { name: "Scott" };
let newHire = Employee.convert(person);
expect(newHire.name).toBe("Scott");

The behavior of static methods is explained by what happens behind the scenes. A static method of a class is associated with the constructor function and not the prototype object like the instance members we saw in the last section.

var Employee = function(name) {
    // ...
};

Employee.convert = function(thing) {
    // ...
};
Pluralsight Courses
What JavaScript Developers Should Know About ECMAScript 2015
The Podcast!