Thoughts On End to End Testing of Browser Apps

Tuesday, February 10, 2015 by K. Scott Allen
5 comments

In a previous post on using the PageObject pattern with Protractor, Martin asked how much time is wasted writing tests, and who pays for the wasted time?

To answer that question I want to think about the costs and benefits of end to end testing. I believe the cost benefit for true end to end testing looks like the following.

cost benefit of e2e testing

There are two significant slopes in the graph. First is the “getting started” slope, which ramps up positively, but slowly. Yes, there are some technical challenges in e2e testing, like learning how to write and debug tests with a new tool or framework. But, what typically slows the benefit growth is organizational. Unlike unit tests, which a developer might write on her own, e2e tests requires coordination and planning across teams both technical and non-technical. You need to plan the provisioning and automation of test environments, and have people create databases with data representative of production data, but scrubbed of protected personal data. Business involvement is crucial, too, as you need to make sure the testing effort is testing the right acceptance criteria for stakeholders.

All of the coordination and work required for e2e testing on a real application is a bit of a hurdle and is sure to build resistance to the effort in many projects. However, the sweet spot at the top of the graph, where the benefit reaches a maximum, is a nice place to be. The tests give everyone confidence that the application is working correctly, and allows teams to create features and deploy at a faster pace. There is a positive return in the investment made in e2e tests. Sure, the test suite might take one hour to run, but it can run any time of the day or night, and every run might save 40 hours of human drudgery.

There is also the ugly side to e2e testing where the benefit starts to slope downward. Although the slope might not always be negative, I do believe the law of diminishing returns is always in play. e2e tests can be amazingly brittle and fail with the slightest change in the system or environment. The failures lead to frustration and it is easy for a test suite to become the villain that everyone despises. I’ve seen this scenario play out when the test strategy is driven with mindless metrics, like when there is a goal to reach 90% code coverage.

In short, every application needs testing before release, and automated e2e tests can speed the process. Making a good test suite that doesn't become a detriment to the project is difficult, unfortunately, due to the complex nature of both software and the human mind. I encourage everyone to write high value tests for the riskier pieces of the application so the tests can catch real errors and build confidence.

Creating Iterables In ECMAScript 6

Monday, February 9, 2015 by K. Scott Allen
0 comments

Imagine you are building an abstraction for a classroom, and one responsibility of a classroom object is to encapsulate the names of the students in the classroom. You might build a class like the following.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }
}

On occasion, the outside world needs to know the names of the students inside the classroom. Perhaps the names will be shown on the screen, or saved in a database. In either case, how should a classroom object give the student names to a caller? One option is to return a reference to the students array, in which case the caller might change the array by adding or removing items. This option isn’t the best option if the classroom want to protect the array and validate the addition or removal of a student.

Another option is to make a copy of the array. Then, even if someone changes the copy, the original student array remains safe. However, copy operations can be expensive.

With ES6 and iterators we now have the ability to provide access to a collection without giving someone access to a mutable array. All we need is to make the classroom iterable by adding a Symbol.iterator method.

First, let’s make the classroom iterable by building our own iterator objects. This is the hard way to solve the problem.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }

    [Symbol.iterator]() {
         var index = 0;
         return {
            next: () => {
                if(index < this.students.length){
                     index += 1;
                     return {done: false, value: this.students[index-1]};
                }
                return { value: undefined, done: true };
            }
         }
    }
}

The above code adds the iterator method to the class. Notice how Symbol.iterator needs square brackets to surround the method “name”. Inside the method is the code to return objects with a next method, which in turn returns objects with done and value properties. A classroom should now behave just like an array and allow developers to use for of loops.

let scienceClass = new Classroom();
let result = [];
for(let name of scienceClass) {
    result.push(name);
}

expect(result).toEqual(["Tim", "Joy", "Sue"]);

One might notice that the code inside the classroom iterator method is the type of code that works with any array. Let’s extract the code into a new class.

class ArrayIterator {
    constructor(array) {
         this.array = array;
         this.index = 0;
     }

    next() {
        let result = { value: undefined, done: true};
        if(this.index < this.array.length) {
            result.value = this.array[this.index];
            result.done = false;
            this.index += 1;
        }
        return result;
    }
}

Now we can reuse ArrayIterator from several places in an application.

class Classroom {
    constructor() {
        this.students = ["Tim", "Joy", "Sue"];
    }

    [Symbol.iterator]() {
        return new ArrayIterator(this.students);
    }
}

Even though the ArrayIterator class is helpful, we will see an even easier implementation of the iterator method when we get to generators.

Static Members in ES6

Monday, February 2, 2015 by K. Scott Allen
2 comments

In previous posts, we looked at classes in ES6, but I never gave an example using the static keyword.

A class may contain one or more methods defined with the keyword static.

class Employee {
    constructor(name) {
        this._name = name;
    }

    static convert(thing) {
        if(thing.name) {
            return new Employee(thing.name);
        }
    }
}

Like other languages with static class members, the static keyword will create a method associated with the class, and not with an instance of the class. In other words, you can only reach a static method using the name of the class.

expect(Employee.convert).toBeDefined();
expect(new Employee().convert).toBeUndefined();

Static methods have no access to the fields, properties, and methods defined on an instance of the class using this. For example, the convert method would not be able to use this._name to get to the _name field of an Employee object, as the this pointer inside of convert will never reference an Employee object. However, static methods are often useful and can play the roles of factory methods, conversion methods, or general class helper methods.

let person = { name: "Scott" };
let newHire = Employee.convert(person);
expect(newHire.name).toBe("Scott");

The behavior of static methods is explained by what happens behind the scenes. A static method of a class is associated with the constructor function and not the prototype object like the instance members we saw in the last section.

var Employee = function(name) {
    // ...
};

Employee.convert = function(thing) {
    // ...
};

Symbols in ES6

Monday, January 26, 2015 by K. Scott Allen
4 comments

In a previous post we looked at the for of loop in ES6, but left an open question. What magic allows for of to find an iterator? Alternatively, what makes an object iterable? To answer either question requires some knowledge of symbols.

A symbol is a new, primitive data type in ES6. There are two key points to understand about symbols.

  1. A symbol is unique and immutable.
  2. You can use symbols as identifiers when adding properties to an object.

To understand the implications of these two statements, and how they relate to iterable objects, let’s start by creating a new symbol. Creating a new symbol is as easy as invoking Symbol as a function.

let s = Symbol();
expect(typeof s).toBe("symbol");

Note that you do not use the new operator when creating a symbol, you only invoke the function. Otherwise, the runtime will raise an exception.

let makeSymbol = () => new Symbol(); 
expect(makeSymbol).toThrow();

What’s important to remember is that each symbol you create is unique and unalterable.

let s1 = Symbol(); 
let s2 = Symbol(); 

expect(s1).toEqual(s1); 
expect(s1).not.toEqual(s2); 

When you create a symbol, you can add a description by passing a string to the Symbol function, but symbols created with the same description will be unique (even if the stringified versions look the same).

let s1 = Symbol("some description"); 
let s2 = Symbol("some description"); 

expect(s1).toEqual(s1); 
expect(s1).not.toEqual(s2); 
expect(s1.toString()).toBe("Symbol(some description)"); 
expect(s2.toString()).toBe("Symbol(some description)");

The Important Part

You can use symbols as keys into an object. In the following code we create one property using a string ("lastName"), and one with a symbol (the firstName symbol).

let firstName = Symbol(); 

let person = { 
    lastName: "Allen", 
    [firstName]: "Scott", 
}; 

expect(person.lastName).toBe("Allen"); 
expect(person[firstName]).toBe("Scott");

Using symbols as keys is where symbols become interesting. Because symbols are unique, a symbol can create a unique property inside an object. You never have to worry about conflicting with existing methods, or being accidently overwritten. ES5 libraries would use timestamps or random numbers to achieve this same effect, but symbols are provided for this exact purpose.

Members of an object created using a symbol will not appear as part of the object when using a for in loop.

let firstName = Symbol(); 

let person = { 
    lastName: "Allen", 
    [firstName]: "Scott", 
}; 

let names = []; 
for(var p in person) { 
    names.push(p); 
} 

expect(names.length).toBe(1); 
expect(names[0]).toBe("lastName"); 

Symbols also won’t appear when using Object.getOwnPropertyNames.

expect(Object.getOwnPropertyNames(person)).toEqual(["lastName"]);

Nor will they appear when serializing an object to JSON.

expect(JSON.stringify(person)).toBe('{"lastName":"Allen"}');

For these reasons, you might think symbols are useful for hiding information in an object. Although using symbols to hide information is useful, there is a new object method named getOwnPropertySymbols that will allow programmers to reflect into symbol properties that are otherwise hidden.

expect(Object.getOwnPropertySymbols(person)).toEqual([firstName]); 

let symbol0 = Object.getOwnPropertySymbols(person)[0]; 
expect(person[symbol0]).toBe("Scott"); 

Back to the question at hand. How do symbols relate to iterators and iterable objects?

Iterables and @@iterator

The JavaScript specification defines special methods to use in specific circumstances. The @@iterator method is one such example. If an object has an @@iterator method (which will return an iterator object), the object is iterable and will work as the source object in a for of loop.

How do we know if an object has an @@iterator method?

By checking to see if an object has a member at Symbol.iterator.

The iterator property attached to the Symbol function is what’s called a well-known symbol. The following code will check to see if strings, arrays, and numbers are iterable by looking for this well-known iterator method.

let site = "OdeToCode.com"; 
let values = [1,2,3,4]; 
let number = 45; 

expect(site[Symbol.iterator]).toBeDefined(); 
expect(values[Symbol.iterator]).toBeDefined(); 
expect(number[Symbol.iterator]).toBeUndefined(); 

The tests will show that strings and arrays are iterable, but numbers are not.

Now that we understand symbols, and the well-known iterator method, we can rewrite the following code.

let sum = 0; 
let numbers = [1,2,3,4]; 

for(let n of numbers) { 
    sum += n; 
} 

expect(sum).toBe(10); 

We will rewrite the code to represent what is really happening behind the scenes.

let sum = 0; 
let numbers = [1,2,3,4]; 

let iterator = numbers[Symbol.iterator](); 
let next = iterator.next(); 

while(!next.done) { 
    sum += next.value; 
    next = iterator.next(); 
} 

expect(sum).toBe(10); 

Using your knowledge of symbols to write low-level iterator code probably won’t be the most exciting application of ES6 symbols, but using symbols to make new iterable objects is exciting, and the topic for a future post.

The for of Loop in ES6

Tuesday, January 20, 2015 by K. Scott Allen
2 comments

In a previous post we looked at using iterators directly. Working with iterators at a low level will sometimes be useful, as we’ll see in future posts, but ES6 also provides a new looping syntax to work with iterators at a higher level – the for of loop.

let sum = 0;
let numbers = [1,2,3,4];

for(let n of numbers) {
    sum += n;
}

expect(sum).toBe(10);

There are a couple observations to make about the above code.

First, the new for of loop looks much like the for in loop, which has always been around, but the two are subtly different. The for in loop works with the enumerable properties of any object, thus when working with an array, for in will iterate over the available indexes of the array, not the available values. To compute the same sum as the above code with for in, we’d need to index into the array with each iteration:

let sum = 0; 
let numbers = [1,2,3,4]; 

for(let i in numbers) { 
    sum += numbers[i]; // notice the indexer 
} 

expect(sum).toBe(10);

The for of loop uses an iterator to visit the values in an array, not the available keys.

for(let n of numbers) { 
    sum += n; 
} 

This brings up the second subtle difference with for of, the for of loop exists not to work with just any type of object, but with iterable objects. Iterable objects are objects with a special method returning an iterator. We’ll look at the special method in a future post, but notice how the above code doesn’t need to use the values method of the array like the low level iterator code we wrote in the last post. The for of loop uses the numbers array directly.

There are many objects in the world of ES6 that will be iterable objects, and we will see more in future posts. For now, we’ve already seen how arrays are iterable. Strings are also iterable (you can iterate over the sequence of characters inside).

let result = ""; 
let message = "Hello"; 

for(let c of message) { 
    result += c; 
} 

expect(result).toBe("Hello"); 

How does does for of retrieve an iterator? And what makes an object iterable? To answer these questions we have to take a brief detour into another new feature of ES6 – the Symbol.

Using JSON Web Tokens with Katana and WebAPI

Thursday, January 15, 2015 by K. Scott Allen
4 comments

A common question I’ve been getting is how to use tokens with ASP.NET, specifically JSON Web Tokens (JWT) with ASP.NET WebAPI where the OAuth server and the resource server are the same. In other words, you have a single web site that wants to both issue tokens to authenticated clients and verify the same tokens on incoming requests.

Setup

To pull this off with Microsoft’s OWIN based components you’ll need the Microsoft.Owin.Security.Jwt package from NuGet, which brings in a few dependencies, including System.IdentityModel.Tokens.Jwt.

The app building setup code is simple (it is the details that are a bit trickier). We need:

  1. OAuth middleware to issue a token
  2. Authentication middleware to validate a token and set the user identity for a request.

OWIN startup code can register both of these middleware pieces using app builder extension methods.

[assembly: OwinStartup(typeof(OwinStartup))]
namespace NgPlaybook.Server.Startup
{
   
    public class OwinStartup
    {
        public void Configuration(IAppBuilder app)
        {
            app.UseOAuthAuthorizationServer(new MyOAuthOptions());            
            app.UseJwtBearerAuthentication(new MyJwtOptions());
        }
    }
}

 

Let’s talk about the two custom classes in the startup code – MyOAuthOptions (which helps in issuing tokens) and MyJwtOptions (which helps validate tokens).

OAuthOptions

The devil in the details starts with MyOAuthOptions, which is a wrapper for the OAuth server options.

public class MyOAuthOptions : OAuthAuthorizationServerOptions
{
    public MyOAuthOptions()
    {
        TokenEndpointPath = "/token";
        AccessTokenExpireTimeSpan = TimeSpan.FromMinutes(60);
        AccessTokenFormat = new MyJwtFormat();
        Provider = new MyOAuthProvider();
        #if DEBUG
            AllowInsecureHttp = true;
        #endif
    }
}

Here we setup the server to listen at /token, just like the token middleware in an out of the box Web API project with authentication.

The AccessTokenFormat property references an object implementing ISecureDataFormat<AuthenticationTicket>, which in this case is our own MyJwtFormat. The purpose of this class is to encode and sign information about an authenticated user into a string.

JwtFormat

An ISecureDataFormat<T> object has a Protect method which does all the heavy lifting. The AuthenticationTicket object given to this method is constructed from claims put together by the server when a user successfully authenticates, we’ll look at this later.

public class MyJwtFormat: ISecureDataFormat<AuthenticationTicket>
{
    private readonly OAuthAuthorizationServerOptions _options;

    public MyJwtFormat(OAuthAuthorizationServerOptions options)
    {
        _options = options;
    }

    public string SignatureAlgorithm
    {
        get { return "http://www.w3.org/2001/04/xmldsig-more#hmac-sha256"; }
    }

    public string DigestAlgorithm
    {
        get { return "http://www.w3.org/2001/04/xmlenc#sha256"; }
    }

    public string Protect(AuthenticationTicket data)
    {
        if (data == null) throw new ArgumentNullException("data");

        var issuer = "localhost";
        var audience = "all";
        var key = Convert.FromBase64String("UHxNtYMRYwvfpO1dS5pWLKL0M2DgOj40EbN4SoBWgfc");
        var now = DateTime.UtcNow;
        var expires = now.AddMinutes(_options.AccessTokenExpireTimeSpan.TotalMinutes);
        var signingCredentials = new SigningCredentials(
                                    new InMemorySymmetricSecurityKey(key), 
                                    SignatureAlgorithm,
                                    DigestAlgorithm);
        var token = new JwtSecurityToken(issuer, audience, data.Identity.Claims, 
                                         now, expires, signingCredentials);

        return new JwtSecurityTokenHandler().WriteToken(token);
    }

    public AuthenticationTicket Unprotect(string protectedText)
    {
        throw new NotImplementedException();
    }
}

Of course many of these details, like the audience, issuer, expiration time, and the key, you’ll want to keep in a configuration file. Also, instead of using InMemorySymmetricSecurityKey, you might want to use a certificate and X509SecurityKey.

Note the OAuth server never uses the Unprotect method. You’d think Unprotect might come in useful when verifying  a token, but that’s just not how these components work.

Now, back to how a user is actually authenticated and given claims that are encrypted into the token. That’s the responsibility of the OAuth provider that plugs into the OAuth server.

OAuthProvider

public class MyOAuthProvider : OAuthAuthorizationServerProvider
{
    public override Task GrantResourceOwnerCredentials(OAuthGrantResourceOwnerCredentialsContext context)
    {
        var identity = new ClaimsIdentity("otc");
        var username = context.OwinContext.Get<string>("otc:username");
        identity.AddClaim(new Claim("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name", username));
        identity.AddClaim(new Claim("http://schemas.microsoft.com/ws/2008/06/identity/claims/role", "user"));
        context.Validated(identity);
        return Task.FromResult(0);
    }

    public override Task ValidateClientAuthentication(OAuthValidateClientAuthenticationContext context)
    {
        try
        {
            var username = context.Parameters["username"];
            var password = context.Parameters["password"];
            
            if (username == password)
            {
                context.OwinContext.Set("otc:username", username);
                context.Validated();
            }
            else
            {
                context.SetError("Invalid credentials");
                context.Rejected();
            }
        }
        catch
        {
            context.SetError("Server error");
            context.Rejected();
        }
        return Task.FromResult(0);
    }
}

The ValidateClientAuthentication method is the place where you’ll make calls to a database or membership system to determine if a user is providing the correct credentials. If so, the method will add data into the OWIN request context for GrantResourceOwnerCredentials to pick up and place into claims (which ultimately become part of the auth ticket and serialized into the JWT). The code above only simulates validation by granting access to any user whose password is the same as their username.

The above code is all you’ll need to grant tokens. We also wanted middleware in the application to verify incoming tokens, and back in the beginning of the post we setup UseJwtBearerAuthentication using the following options class.

JwtBearerAuthenticationOptions

public class MyJwtOptions : JwtBearerAuthenticationOptions
{
    public MyJwtOptions()
    {      
        var issuer = "localhost";
        var audience = "all"; 
        var key = Convert.FromBase64String("UHxNtYMRYwvfpO1dS5pWLKL0M2DgOj40EbN4SoBWgfc");;

        AllowedAudiences = new[] { audience };
        IssuerSecurityTokenProviders = new[]
        {
            new SymmetricKeyIssuerSecurityTokenProvider(issuer, key)
        };
    }
}

You’ll need the issuer, audience, and most importantly the key to matchup with the values provided by the JWT formatter we looked at earlier.

JWT In Action

To get a token with the above code in place, a client needs to POST form encoded credentials to the /token endpoint. Be sure to include a grant_type of password.

POST /login HTTP/1.1
Host: localhost:17648
Content-Length: 51
Accept: application/json, text/plain, */*
Origin: http://localhost:17648
User-Agent: Mozilla/5.0 ...
Content-Type: application/x-www-form-urlencoded
username=sallen&password=sallen&grant_type=password

If the credentials are correct, the server will respond with a JSON payload that includes the access_token.

{"access_token":"eiJ9............AkUg","token_type":"bearer","expires_in":119}

Subsequent requests just need to include the token in an Authorization header.

GET /api/secret HTTP/1.1
Host: localhost:17648
Accept: application/json, text/plain, */*
Authorization: Bearer eiJ9............AkUg

And now you can protect API and MVC controllers with the usual Authorize attribute.

[Authorize]
public class SecretController : ApiController
{
    public IHttpActionResult Get()
    {
        return Ok(new ModelThing());
    }
}

If you were to decode the token, you’d see it consist of a header and a payload that look like the following.

{"typ":"JWT","alg":"HS256"}

{"unique_name":"sallen","role":"user","iss":"localhost",
 "aud":"all","exp":1421127827,"nbf":1421127707}

Which demonstrates one of the features of JWT – all the claims are in the payload. Once the server decodes and verifies the information inside the token, the server has all the information it needs about a user.

Summary

There is a fair amount of code required to setup a JWT OAuth server with the ASP.NET middleware components, but I hope this code might give you a starting point to experiment with.

Iterators in ES6

Tuesday, January 13, 2015 by K. Scott Allen
1 comment

The addition of iterators in ES6 opens up a new world of algorithms and abilities for JavaScript. An iterator is an object that allows code to traverse a sequence of objects. The sequence of objects could be the values inside an array, or values in a more sophisticated data structure like a tree or map. One of the many beautiful characteristics of iterators is how the iterator doesn’t need to know any details about the underlying collection. All an iterator can do is provide the ability to move through a sequence  of items one by one and visit each item inside.

In this post we will look at some of the basic characteristics of an iterator, but a series of future posts will build on this to show both high-level features and some advanced techniques you can use with iterators, or to build your own iterators.

In ES6, array objects will have a few new methods available to work with iterators, and one of these methods is the values method. The values method returns an iterator object that can visit each value in an array.

let sum = 0;
let numbers = [1,2,3,4];

let iterator = numbers.values();
let next = iterator.next();

while(!next.done) {
    sum += next.value;
    next = iterator.next();
}

expect(sum).toBe(10);

An iterator is an object with a next method. Each time the program invokes the next method, the method returns an object with two properties: value and done. The value property represents the next item in the sequence being iterated. The done property is a flag holding the value false if there are more items to iterate through, or the value true if iteration has passed the final item and is complete.

Iterators exist in many other languages, and in some environments, making changes to a sequence during iteration will result in a runtime exception. In JavaScript, changes do not guarantee an error or exception, but you’d still want to exercise caution, because the changes can certainly create confusion. The following code will change the underlying array in the 2nd pass of the while loop. The changes include pushing the value 5 to the end of the array, and unshifting the value 1 to the beginning of the array. The iterator will see the values 1, 2, 2, 3, 4, 5.

let count = 0; 
let sum = 0; 
let numbers = [1,2,3,4]; 

let iterator = numbers.values(); 
let next = iterator.next(); 

while(!next.done) { 
    if(++count == 2) { 
        numbers.push(5); 
        numbers.unshift(1); 
    } 

    sum += next.value; 
    next = iterator.next(); 
} 

expect(sum).toBe(17);

Next up: using iterators the easy way with the new for of.

by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!