Evolution of the Entity Framework: Lesson 3

Wednesday, June 27, 2012 by K. Scott Allen
0 comments

To combat the negativity surrounding the first release of the Entity Framework, Microsoft launched a messaging campaign and repeatedly told us that the EF is not an ORM.

Often, people categorize Entity Framework as an Object/Relational Mapper and try to compare it to other O/R Mappers. I dare say that’s an apples-to-oranges kind of comparison. While Entity Framework does have ORM capabilities, that is only a fraction of its functionality, and more importantly, those ORM capabilities are achieved through a fundamentally different approach compared to typical O/R Mappers … Entity Framework is much bigger than a mere O/R Mapper. It is a conceptual-level development platform, and its ORM capabilities are an interface for applications to interact with conceptual models.

As part of the effort, Danny Simmons wrote a post titled “Why Use The Entity Framework”.

The big difference between the EF and nHibernate is around the Entity Data Model (EDM) and the long-term vision for the data platform we are building around it. The EF was specifically structured to separate the process of mapping queries/shaping results from building objects and tracking changes. This makes it easier to create a conceptual model which is how you want to think about your data and then reuse that conceptual model for a number of other services besides just building objects.

Instead of propelling the Entity Framework forward, the messaging spread more confusion. The first version of the Entity Framework could only do one thing: reverse engineer a database and generate code to access the database. Yet, we were supposed to think of the Entity Framework as more than an ORM and see capabilities that didn’t exist. The Entity Framework walked like a duck, and talked like a duck, but we were told to think of the framework as a golden goose.

Golden Goose

Marketing can change the perception of a product but developers can always detect missing features. No one was sure when the features would arrive. Given the recent history at the time, we didn’t expect a quick turn around for the next release. Promising features in the far off future is like asking developers to take on a debt. “Use our framework today, and we’ll pay you back in two years”. This is a tough promise to accept given the fast changing technology landscape and how quickly Microsoft abandons frameworks.

Lesson 3: Don’t Ask Your Customer For A Long Term Loan

 

The second release of the Entity Framework bumped the version number to 4.0 and arrived with Visual Studio 2010. Although many new features were added, some were superficial and in hindsight, offered another lesson.

To be continued…

Trouble, Trouble, A Quintuple of Double

Tuesday, June 26, 2012 by K. Scott Allen
4 comments
Func<double, double, double, double, double> distance = 
(x1, y1, x2, y2) =>
Math.Sqrt(Math.Pow(x2 - x1, 2) + Math.Pow(y2 - y1, 2));

A lady once asked me if this code was perfectible
if the Quintuple Of Double was somehow susceptible
to replacing with code still likeminded yet loveable
for the Quintuple Of Double offered feelings of trouble.

delegate double TwoPointOperation(double x1, double y1, 
double x2, double y2);

TwoPointOperation distance =
(x1, x2, y1, y2) =>
Math.Sqrt(Math.Pow(x2 - x1, 2) + Math.Pow(y2 - y1, 2));
I told her the code was still quite correctable 
a delegate type would make her worries reversible
and the one thing worse than a Quintuple Of Double
is the Sextuplet Of Object, then you know you’re in trouble.

Parallel Work in Async MVC Actions

Wednesday, June 20, 2012 by K. Scott Allen
5 comments

One of the samples in the ASP.NET MVC 4 release notes is an example of using an async controller action.

public async Task<ActionResult> Index(string city)
{
var newsService = new NewsService();
var sportsService = new SportsService();

return View("Common",
new PortalViewModel
{
NewsHeadlines = await newsService.GetHeadlinesAsync(),
SportsScores = await sportsService.GetScoresAsync()
});
}

At first glance, it might seem like getting the headlines and getting the sports scores are two operations that happen in parallel, but the way the code is structured this can’t happen. It’s easier to see if you add some async methods to the controller and watch the different threads at work in the debugger.

public async Task<ActionResult> Index(string city)
{
return View("Common",
new PortalViewModel
{
NewsHeadlines = await GetHeadlinesAsync(),
SportsScores = await GetScoresAsync()
});
}
async Task<IEnumerable<Score>> GetScoresAsync()
{
await Task.Delay(3000);
// return some scores ...
}

async Task<IEnumerable<Headline>> GetHeadlinesAsync()
{
await Task.Delay(3000);
// return some news
}

In the methods I’ve introduced a delay using Task.Delay (which returns a Task you can await and thereby free the calling thread, unlike Thread.Sleep which will block). The total time to render the view will be at least 6,000 milliseconds, because the Index action awaits the result of GetHeadlinesAsync. Awating will suspend execution before GetScoresAsync has a chance to start.
If you want the headlines and scores to work in parallel, you can kick off both async calls before awaiting the first result.
public async Task<ActionResult> Index(string city)
{
var newsService = new NewsService();
var sportsService = new SportsService();

var newsTask = newsService.GetHeadlinesAsync();
var sportsTask = sportsService.GetScoresAsync();

return View("Common",
new PortalViewModel
{

NewsHeadlines = await newsTask,
SportsScores = await sportsTask
});
}

Evolution of the Entity Framework: Lesson 2

Tuesday, June 19, 2012 by K. Scott Allen
10 comments

The first version of the Entity Framework appeared in the second half of 2008 with an API derived from WinFS and a heavy theoretical focus on entity-relationship modeling. While other object relational mapping frameworks viewed mapping as a (sometimes) necessary evil, entity mapping was a centerpiece of the Entity Framework.

The Entity Framework vision was to create a conceptual data model representing entities and their relationships. The conceptual model would be the ideal model for application developers. You didn’t have to think about database tables or normalization when building the conceptual model. Instead, you were supposed to focus on the important business objects and concepts. Once the model was in place, the Entity Framework could use the model to generate code for programming against the conceptual model inside an application, as well as generate database structures to persist data represented in the model into a relational database. The theory and math behind was the mapping was spelled out in an impressive piece of academic work titled “Compiling Mappings to Bridge Applications and Databases”.

But the Entity Framework vision went beyond just traditional application development and relational databases. It was thought that the conceptual entity data model could be the canonical data model for an entire enterprise, and drive not only line of business applications but also reporting, synchronization services, web services, and data analysis.

Entity Framework Vision

Unfortunately, the Entity Framework, like its predecessor, set out to solve a broad set of problems in enterprise IT and missed the opportunity to solve a specific, common problem in a way that would make developers happy. By the time the framework officially shipped with a Visual Studio service pack, the developer community had already highlighted a number of shortcomings for accessing a relational database from the Entity Framework, which included (but was not limited to):

  • · Unimplemented LINQ operators
  • · No capability for model-first design
  • · No support for complex types
  • · No support for enum types or unsigned integers
  • · No implicit loading of relationships
  • · Limited stored procedure support
  • · Unnecessarily complexity in generated SQL code
  • · Unnecessary complexity and dependencies in generated C# code
  • · Performance

Since developers love to benchmark software, it was the last bullet, performance, that generated many blog posts comparing the Entity Framework to other object relational mapping frameworks like nHibernate and LINQ to SQL. LINQ to SQL ironically was never intended to see the light of day but did ship earlier than the Entity Framework and was gaining in popularity because it was simple to understand and solved the ORM problem in a straightforward fashion. Because LINQ to SQL has fewer architectural layers, it outperformed the Entity Framework in almost every scenario.

Entity Framework Benchmarks

Developers looked to the Entity Framework to solve one specific problem, but the framework lagged other frameworks in almost every area. When you look around at successful products, you’ll typically find they solve at least one problem extremely well. Dropbox, for example, has a minimalistic feature set compared to other file synchronization applications. But, Dropbox is hugely successful because Dropbox does file synchronization extremely well. In fact, the success of Dropbox was the topic for a question on Quora.

Well, let's take a step back and think about the sync problem and what the ideal solution for it would do:

· There would be a folder.

· You'd put your stuff in it.

· It would sync.

They built that.

Why didn't anyone else build that? I have no idea.

"But," you may ask, "so much more you could do! What about task management, calendaring, customized dashboards, virtual white boarding. More than just folders and files!"

No, shut up. People don't use that crap. They just want a folder. A folder that syncs.

Lesson 2: Solve At Least One Customer Problem Well

Early frustrations around the Entity Framework primarily arose because the framework didn’t solve a specific problem well. In turn, this led to negative reviews.

As the pressure mounted on the Entity Framework, another learning opportunity arose which we’ll look at in a future post.

Geolocation, Geocoding, and jQuery Promises

Monday, June 18, 2012 by K. Scott Allen
0 comments

If you want to use the customer’s hardware to find their exact address, one approach is to combine the HTML 5 Geolocation APIs with a Geocoding web service (like Google).

For Google, you can still get in without an API key (for a limited number of calls) using the Google Maps JavaScript library (just reference http://maps.google.com/maps/api/js in a script tag).

With the library in place, the code is straightforward (particularly the following code, which doesn’t have any error handling, but is a good skeleton of the calls you’ll need to make).

(function () {

    var getPosition = function (options) {
        navigator.geolocation.getCurrentPosition(
            lookupCountry,
            null,
            options);
    };

    var lookupCountry = function (position) {
        console.log(position);
        var latlng = new google.maps.LatLng(
                            position.coords.latitude,
                            position.coords.longitude);
        
        var geoCoder = new google.maps.Geocoder();
        geoCoder.geocode({ location: latlng }, displayResults);
    };

    var displayResults = function (results, status) {
        // here you can look through results ...
        $("body").append("<div>").text(results[0].formatted_address);      
    };

    $(function () {
        getPosition();
    });

} ());

 

Making Promises

Adding some jQuery deferred objects makes the code a little longer, but also a little more robust, as the individual pieces of work are no longer responsible for knowing what to do next and we can invert control of the execution flow. In other words, if you return promises from getPosition and lookupCountry:

var getPosition = function (options) {
    var deferred = $.Deferred();

    navigator.geolocation.getCurrentPosition(
        deferred.resolve,
        deferred.reject,
        options);

    return deferred.promise();
};

var lookupCountry = function (position) {
    var deferred = $.Deferred();

    var latlng = new google.maps.LatLng(
                        position.coords.latitude,
                        position.coords.longitude);
    var geoCoder = new google.maps.Geocoder();
    geoCoder.geocode({ location: latlng }, deferred.resolve);

    return deferred.promise();
};

Then the control logic reads pretty well:

$(function () {
    $.when(getPosition())
     .pipe(lookupCountry)
     .then(displayResults);
});

Note that pipe is different than then because pipe gives back a new promise.

Try it out on jsFiddle.

Evolution of the Entity Framework: Lesson 1

Thursday, June 14, 2012 by K. Scott Allen
6 comments

Looking back over the history of the Entity Framework provides some interesting lessons.

Microsoft released the first version of the Entity Framework in August of 2008. One month later, the investment bank of Lehman Brothers filed the largest bankruptcy case in the history of the United States, and thus began the worst global economic downturn since the great depression.

Dow Jones Industrial Average 2007-2009

Coincidence?

Probably.

Although the Entity Framework first appeared in 2008, the story begins much earlier. Microsoft has a long history of inventing and re-inventing data-oriented developer tools and frameworks. Applications like FoxPro and Access are long-standing examples, while Visual Studio LightSwitch is a modern specimen. FoxPro and Access were both successful products, and LightSwitch, while still new, shares the same practical attitude to working with data. All these products allow you to build a data centric application with little architectural fuss.

For .NET developers working in C# and Visual Basic, there were several times when we thought we would also see a focused, practical, no-nonsense approach to working with data. While the .NET framework has always provided abstractions like the DataSet and DataTable, these abstractions essentially represented in-memory databases complete with rows, relationships, and views. However, since C# and Visual Basic are both object-oriented programming languages, it would seem natural to take data from a database and place the data into strongly-typed objects with both properties (state) and methods (behavior). One solution we thought was coming from Microsoft was a framework named ObjectSpaces.

ObjectSpaces

ObjectSpaces was described as an object/relational mapping framework, and like all ORM frameworks ObjectSpaces focused on two goals. Goal #1 was to take data retrieved from SQL Server and map the data into objects. Goal #2 was to track changes on objects in memory so when the application asked to save all the changes, the framework could take data from the objects and insert, update, or delete data in the database. In “A First Look at ObjectSpaces in Visual Studio 2005”, Dino Esposito used the above diagram to illustrate these capabilities.

Unfortunately, ObjectSpaces never saw the light of day.

ORMs in the Stratosphere

In the spring of 2004, Microsoft announced a delay for ObjectSpaces in order to roll the technology into a larger framework named WinFS. WinFS was one of the original pillars of Longhorn (the now infamous codename for Window Vista), and WinFS promised a much larger feature set compared to ObjectSpaces.

The idea behind WinFS was to provide an abstraction over all sorts of data – relational data, structured data, and semi-structured data. WinFS would allow you to search data inside of Microsoft Money just as easily as tables in SQL Server, calendars in Exchange Server or images on the file system, as well as provide notification, synchronization, and access control services for all data sources, everywhere.

Although some of these ideas had been around since the early 1990s (the Object File System of Microsoft’s Cairo operating system), WinFS was hoping to deliver the vision and offer a quantum leap forward in the way you develop with information.

At the 2005 Professional Developers Conference, Microsoft used the following diagram in WinFS talks. When compared to the previous ObjectSpaces diagram, this diagram is more theoretical and focuses on concepts (Schemas and Services) instead of actions (moving objects from SQL Server and back).

WinFS

Despite all the talks about quantum leaps in working with data, Microsoft decided it would not ship WinFS as part of Windows Vista, and on June 23, 2006, Microsoft announced that WinFS would not be delivered as a product. Perhaps this was due to the overly ambitious goals of being all things to anything data related, or perhaps it was due to a heavy focus on architectural issues and not enough focus on the practical and mundane. Matt Warren provides an insider’s view in his post “The Origin of LINQ to SQL”.

We on the outside used to refer to WinFS as the black hole, forever growing, sucking up all available resources, letting nothing escape and in the end producing nothing except possibly a gateway to an alternate reality.

From the outside it looks like WinFS was a classic case of taking a specific problem (I need to access a data) and over-generalizing to the point where the original problem gets lost in the solution. Joel Spolsky wrote about this problem in a post entitled “Don’t Let Architecture Astronauts Scare You”.

The Architecture Astronauts will say things like: "Can you imagine a program like Napster where you can download anything, not just songs?" Then they'll build applications like Groove that they think are more general than Napster, but which seem to have neglected that wee little feature that lets you type the name of a song and then listen to it -- the feature we wanted in the first place. Talk about missing the point. If Napster wasn't peer-to-peer but it did let you type the name of a song and then listen to it, it would have been just as popular.

In retrospect, this was a dark time for Microsoft systems and frameworks. Windows Vista delayed shipping until 2007. WPF (aka Avalon) is today not capable of writing genuine Windows 8 applications. WCF (aka Indigo) is under pressure from lightweight frameworks that embrace HTTP, like the ASP.NET Web API. Windows Workflow was subsequently rewritten from scratch after its initial release, and Windows CardSpace is now defunct.

Lesson 1: Don’t Let The Astronauts Solve The Customer’s Problem

ObjectSpaces failed because it was hauled into a larger vision that was excessively ambitious and over generalized. We’ve seen over the years that the best frameworks start by solving a specific problem, solving it well, and then evolving into something bigger (the shining example being Ruby on Rails). The spirit of agile and lean software development has taught us to continuously deliver something of value, and apply YAGNI ruthlessly. Iteration and shipping bits were not a priority for the WinFS project.

However, you can’t fault Microsoft for thinking big and trying to innovate. Innovation often involves stepping away, trying something different, trying something big, and sometimes failing. Failure, and learning from failure, can lead to new directions and better inventions.

When Microsoft announced the end of WinFS as a product, it also promised to deliver some of the features and ideas encompassed in WinFS through new and different products. One of these products would be the Entity Framework. Would the Entity Framework learn from past mistakes? We’ll take a look in the next post.

Many to Many Relationships with EF

Wednesday, June 13, 2012 by K. Scott Allen
3 comments

A many to many relationship is easy to setup with code first EF. For example, an Author can write many books.

public class Author
{
    public virtual int Id { get; set; }
    public virtual string Name { get; set; }
    public virtual ICollection<Book> Books { get; set; }
}

And a book can have many authors.

public class Book
{
    public virtual int Id { get; set; }
    public virtual string Title { get; set; }
    public virtual ICollection<Author> Authors { get; set; }
}

To start a library, all you need is a book set or author set (or both) in a DbContext derived class.

public class LibraryDb : DbContext
{        
    public DbSet<Book> Books { get; set; }
}

Then the framework will create a database schema with the join table you might expect.

Books and Authors Schema

If you want to have a sequence of objects representing every book and author combination, you can use the SelectMany operator to “flatten” a sequence of book authors.

var authorsBooks = db.Books
                     .SelectMany(
                        book => book.Authors,
                        (book, author) => new {
                            AuthorName = author.Name,
                            BookTitle = book.Title
                     });            

And the above query generates the following SQL:

SELECT 
  [Extent1].[Id] AS [Id], 
  [Join1].[Name] AS [Name], 
  [Extent1].[Title] AS [Title]
FROM  [Books] AS [Extent1]
  INNER JOIN 
  (
      SELECT 
        [Extent2].[Author_Id] AS [Author_Id], 
        [Extent2].[Book_Id] AS [Book_Id], 
        [Extent3].[Id] AS [Id], 
        [Extent3].[Name] AS [Name]
      FROM  [AuthorBooks] AS [Extent2]
        INNER JOIN [Authors] AS [Extent3] ON 
                   [Extent3].[Id] = [Extent2].[Author_Id]
   ) AS [Join1] ON [Extent1].[Id] = [Join1].[Book_Id]
by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!