A Simpler MapReduce with MongoDB and C#

Thursday, March 29, 2012 by K. Scott Allen
2 comments

Continuing from the previous post about MapReduce with MongoDB, we can clean up the code a bit using an extension method that takes a single string parameter.

var db = server.GetDatabase("mytest");
var collection = db.GetCollection("movies");
var results = collection.MapReduce("CategorySummary");

foreach (var result in results.GetResults())
{
    Console.WriteLine(result.ToJson());
}

The code no longer needs to embed JavaScript into the C# code, or specify the map reduce options. Instead, the "CategorySummary" parameter tells the extension method everything it needs to know.

imageFirst, it can load the map, reduce, and finalize functions from JavaScript files on the file system using the parameter to build the path to the files. Keeping the script code in .js files makes the functions easier to author, maintain, and test.

Secondly, it can automatically build output options and will assume you want to send the MapReduce results to an output collection using the same name. The code looks a bit like the following:

public static class MapReduceExtensions
{
    public static MapReduceResult MapReduce(
        this MongoCollection collection, 
        string name)
    {
        var options = new MapReduceOptionsBuilder();
        options.SetOutput(name);            
        if(FinalizeExists(name))
        {
            options.SetFinalize(ReadFinalizeFile(name));
        }
        return collection.MapReduce(ReadMapFile(name), 
                                    ReadReduceFile(name), 
                                    options);
    }

    static string ReadMapFile(string name)
    {
        return File.ReadAllText(Path.Combine(ROOTDIR, name, "map.js"));
    }

    // ......

    static bool FinalizeExists(string name)
    {
        return File.Exists(Path.Combine(ROOTDIR, name, "finalize.js"));
    }

    private static string ROOTDIR = "mapreduce";
}

The idea is to make things like MapReduce as easy as possible by hiding some infrastructure, and applying easy to learn conventions.

Whiteboard Architecture

Wednesday, March 28, 2012 by K. Scott Allen
11 comments

Laboratory whiteboard, featuring metaheuristicsIt's been my observation that the worst architectural decisions are made when technical people meet by themselves in a room with a whiteboard. I'm not saying the decisions are always bad, or wrong, but the plans and rules that come back to bite you consistently seem to incubate under these conditions.

I first witnessed this early in my career when "the architect" would fly into town for two days every few weeks and diagram the architecture for the upcoming application on a whiteboard. Then the architect would fly out of town and developers would fire up IDEs to implement the architecture. I'm ashamed to say that I had no idea at the time just how utterly insane the process was that we were involved with.

I'm happy to say I've never seen malpractice on such a grand scale in many, many years. However, from time to time I still find myself in a room full of developers and a whiteboard. It's sometimes hard to keep the conversation grounded and not solve problems that don't exist, or devise rules that offer no practical benefit. For example, a rule about how to divide the solution for a new application into umpteen projects before the first line of code is checked in.

I think there are two strategies to avoid architectural flights of fancy. One strategy is to keep a business person in the room. The business person can help a team focus on the real problem to solve, and avoid trying to solve problems that don't exist. The second strategy is bring up application code, and even do some group programming to flesh out details. If the problem doesn't lend itself to group programming, it might be time to let one or two people go off on a spike and bring back some code to evaluate.

Without something tangible and real in the room, it's easy to succumb to the dark side.

Custom Slider Value Display with jQuery UI

Tuesday, March 27, 2012 by K. Scott Allen
4 comments

The jQuery UI slider widget doesn't display it's current value by default. However, you can tap into the slide event and grab the current value from the second parameter passed to the event handler. The second parameter in a jQuery UI event handler is typically named ui, and always contains some useful information about the state of the widget.

$(function () {
    var ageInput = $("#ageInput");
    var ageDisplay = $("#ageInput > div");

    var updateSliderValue = function (event, ui) {
        ageDisplay.text(ui.value);
    };

    ageInput.slider({
        min: 0, max: 122,
        slide: updateSliderValue
    });        
});

You can also position the display of the value above the slider handle / thumb. The DOM element representing the slider thumb is passed in the ui parameter as the handle property.

var updateSliderValue = function (event, ui) {
    ageDisplay.text(ui.value)
              .css($(ui.handle).position());
};

Having the value display immediately (before the user starts dragging and the slide event fires) is a bit trickier. A complete example with one possible solution is available on jsFiddle.

Readable DOM Ready Event Handlers

Monday, March 26, 2012 by K. Scott Allen
9 comments

One reason some people don't like JavaScript, I think, is because idiomatic JavaScript favors fewer keystrokes and CPU cycles over readability. There is a large amount of script code that is hard to read, and even harder to maintain and change.

Take, for example, the following snippet. I think this represents a typical DOM ready event handler with jQuery.

$(function () {
    $("#warning").dialog({ autoOpen: true });
    $("#openDialog").click(function () {
        $("#warning").dialog("open");
    });
    $("#submit").click(function () {
        $("#warning").dialog("close");
    });
    $("#names").autocomplete({
        source: ["Foo", "Bar", "Baz"] 
    });
});

It's a wall of text written in the get in, get done, and get out style. The code requires some concentration to see what is happening inside.

I think the code improves after lifting out the nested, anonymous functions.

$(function () {
    var warning = $("#warning");

    var openDialog = function () {
        warning.dialog("open");
    };

    var closeDialog = function () {
        warning.dialog("close");
    };

    warning.dialog({ autoOpen: false });   
    $("#openDialog").click(openDialog);    
    $("#submit").click(closeDialog);       
    $("#names").autocomplete({
        source: ["Foo", "Bar", "Baz"] 
    });            
});

And to take the idea one step further, here is a pattern I'm going to try and follow over the next few weeks: initialize, implement, act

$(function () {

    var warningDialog = $("#warning");
    var openButton = $("#openDialog");
    var submitButton = $("#submit");
    var nameInput = $("#names");
    var inputValues = ["Foo", "Bar", "Baz"];

    var openDialog = function () {
        warningDialog.dialog("open");
    };

    var closeDialog = function () {
        warningDialog.dialog("close");
    };

    warningDialog.dialog({
        autoOpen: false
    });
    nameInput.autocomplete({
        source: inputValues
    });
    openButton.click(openDialog);
    submitButton.click(closeDialog);
    
});

 

The initialize section initializes variables, and tries to lift as many selectors as possible out of the code that follows. The implement section contains functions and other implementation details for the behavior of a page. The act section is the place for wiring up event handlers, creating widgets, and kickoff any data binding for the page.

The end result is verbose, but I've already found it easier to read and change as the requirements shift.

Thoughts?

Cool Down with Canvas and JavaScript

Thursday, March 22, 2012 by K. Scott Allen
4 comments

The idea is to use an image as a "command button". When the user clicks on the image, the image greys out and acts disabled for a period of time. The image slowly transitions back to an enabled state by letting color sweep around itself in a clock-like manner. 

There are hundreds of approaches you could use to implement the effect. My initial attempt used a single image and pixel manipulation in a canvas.  I need to give credit to David Catuhe (blog, @deltakosh) for giving me the idea of using two images and a canvas. One image has color, and one image is in black and white. Selectively drawing the images into a canvas element can achieve the smooth "World of Warcraft cool down" effect, and the code is simple. David, by the way, is one of the many people I met last year who work for Microsoft France and who were all wonderfully friendly and smart. Fond memories!

Try it out on jsFiddle.

Cool down with Canvas and JavaScript

Yet Another Bundling Approach for MVC 4

Wednesday, March 21, 2012 by K. Scott Allen
5 comments

ASP.NET MVC 4 allows you to bundle multiple JavaScript or CSS files into a single "bundled" download, and optionally minify the bundle to reduce the download size. John Petersen has a good introduction to the feature.

I've been experimenting with an approach that lets me use the following code during application startup.

BundleTable.Bundles.Add(new RGraphBundle());

The RGraphBundle class lists all the JavaScript files needed for a certain feature, and sets the virtual path to reach the bundle.

public class RGraphBundle : JsBundle
{
    public RGraphBundle() : base("~/Rgraph")
    {
        AddFiles(
            "~/Scripts/Rgraph/RGraph.common.core.js",
            "~/Scripts/Rgraph/RGraph.common.context.js",
            "~/Scripts/Rgraph/RGraph.common.zoom.js",
            "~/Scripts/Rgraph/RGraph.common.effects.js",
            "~/Scripts/Rgraph/RGraph.line.js"
        );
    }
}

Everything else is taken care of by base classes.

public class CustomBundle : Bundle
{
    public CustomBundle(string virtualPath) 
        : base(virtualPath)
    {
        
    }

    public void AddFiles(params string[] files)
    {
        foreach (var file in files)
        {
            AddFile(file);
        }            
    }

    public void SetTransform<T>() where T: IBundleTransform
    {
        if(HttpContext.Current.IsDebuggingEnabled)
        {
            Transform = new NoTransform();
        }
        else
        {
            Transform = Activator.CreateInstance<T>();
        }
    }        
}    

public class JsBundle : CustomBundle
{
    public JsBundle(string virtualPath) : base(virtualPath)
    {                        
        SetTransform<JsMinify>();
    }        
}

public class CssBundle : CustomBundle
{
    public CssBundle(string virtualPath) : base(virtualPath)
    {            
        SetTransform<CssMinify>();
    }
}

Checking the IsDebuggingEnabled flag lets me turn minification on and off by toggling the debug setting in web.config, just like the ScriptManager would do in that other ASP.NET web framework.

Avoiding NotSupportedException with IQueryable

Tuesday, March 20, 2012 by K. Scott Allen
11 comments

Most remote LINQ providers can handle simple projections. For example, given a Movie class with lots of properties, and a MovieSummary class with a subset of those Movie properties, you can write a LINQ query like the following:

var summaries = db.Movies.Select(m => new MovieSummary {   
       Title = m.Title,
       Length = m.Length 
});

But it all falls apart if you try to offload some of the work to a MovieSummary constructor.

var db = new MovieDataStore();
var summaries = db.Movies.Select(m => new MovieSummary(m));

If you give the above query to the the Entity Framework, for example, it would throw a NotSupportedException.

Unhandled Exception: System.NotSupportedException: Only parameterless constructors and initializers are supported in LINQ to Entities.

A LINQ provider will not know what code is inside the MovieSummary constructor, because the constructor code isn't captured in the expression tree generated by the query. The Entity Framework tries to translate everything in the LINQ query into T-SQL, but since it can't tell exactly what is happening inside the constructor call it has to stop and throw an exception.

One solution is to move the entire projection out of the expression tree by switching from IQueryable to IEnumerable (using the AsEnumerable LINQ operator).

var summaries = db.Movies.AsEnumerable()
                  .Select(m => new MovieSummary(m));

With this query, however, a LINQ provider won't know you only need two properties from every movie. In the case of EF it will now bring back every column from the Movie table. If you need better performance and readability, it might be better to hide the projection in an extension method instead, and make sure the extension method extends IQueryable to keep the projection in an expression tree.

public static IQueryable<MovieSummary> ToMovieSummary(
    this IQueryable<Movie> source) 
{
    return source.Select(m => 
        new MovieSummary
            {
                Title = m.Title,
                Length = m.Length
            }                    
        );
}

// and in the query ...

var summaries = db.Movies.ToMovieSummary();

With EF, the above code will only select two columns from the database to create the movie summaries.

by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!