This version of WWWTC is dedicated to closures. We can use closures and functional programming in general to create elegant, succinct solutions, but sometimes we can also create subtle bugs. 

Exhibit A. The C# developer who wrote this code expected DoSomething to be invoked with the values 0 through 9, but something is terribly wrong.

for (int counter = 0; counter < 10; counter++)
{
    ThreadPool.QueueUserWorkItem(
        state => DoSomething(counter)
    );
}

What’s the problem?

Bonus Round

A JavaScript developer is working with the following markup.

<div class="header">
    <div class="detail">Detail 1 …</div>
</div>
<div class="header">
    <div class="detail">Detail 2 …</div>
</div>
<!-- and so on-->

When the user clicks inside a detail section, the developer wants to change the background color of all the other header sections. Thinking of one way to achieve this behavior, the developer first tests the jQuery not method.

$(function() {
    $(".header").each(function() {
        alert($(".header").not(this).length);
        return false;
    });
});

This code correctly alerts the user once and displays the total number of headers – 1.Emboldened with this proof of concept, the developer puts in the required CSS manipulation and embeds the code into a click event handler.

$(function() {
    $(".header").each(function() {
        $(".detail", this).click(function() {                    
            $(".header").not(this).css("background-color", "red");
        });
    });
});    

Suddenly, things aren’t working and every header section turns red. What went wrong?

In my role as amateur financial analyst I like to read the publications taking an in depth look at the current financial turmoil. I then try to imagine the software tool that could have prevented the turmoil and saved the world millions billions trillions gazillions of dollars. This seems like no small feat for software considering the intricate world built by modern financial engineering - a world where terms like “collateralized debt obligation” are tossed around like dice at a craps table. However, I never imagined the answer could be so simple …

First, some background. I find the current situation interesting because ten years ago I joined an Internet startup by the name of Ultraprise.com. At Ultraprise, we tried to create an online exchange for the secondary mortgage market. Let’s say you have a bit of money saved and want to buy a house, so you find a bank that will loan you the rest of the money you need to buy the house.The bank could then wait for your to repay your loan over the next 30 years and slowly make money on the interest they charge you, but many banks just want to sell your loan to someone else and receive cash in return. Wikipedia has solid coverage of this process in the Securitization entry. The fresh cash received from the sale of your loan allows the bank to stay liquid and fund more loans. But, to sell the loans the bank needs to find a buyer, and this is where Ultraprise came in.

Fannie and Freddie pick up roughly half of the mortgages in the U.S – that is their job, but the rest of the pie is still a substantial amount of debt for sale. At Ultraprise we built a web application for banks to upload pools of loans they wanted to sell. Buyers could then login and bid on the pools. The idea was to cut out the brokers who typically sit in the middle of these transactions and take a cut. Economies of scale meant our fees would be substantially less than the typical fees.

Fail

Ultraprise was out of business after three years, despite having a substantial number of loans posted in the system. We couldn’t sell loans. Part of the problem was, I think, that the people who buy loans really like the human touch that a broker adds. Rib-eye steaks and martinis sell more products than a dull web page full of numbers.

Another problem was getting mortgage data in and out of the system. There was no standard format for pushing mortgage data. Every bank with a loan for sale required a custom import, and every bank looking to buy loans wanted to download data into their custom software for analysis and due diligence.

My experience at Ultraprise led me to believe that banks were exceedingly risk averse and highly analytical, and that they required lots of data before making decisions involving millions of dollars. Thus, I was quite surprised to learn that today’s banks have no clue about the cards they are holding.

“ … a major stumbling block for banks is having the right data on hand …”  - American Banker

“…the reason that banks don’t want to lend to each other anymore is that they don’t trust that the other banks really know the value of their mortgage-backed securities … because they themselves don’t trust the value of their own.” – Aster Data

All this secrecy comes in the wake of an economic crisis brought about in part by a proliferation of financial instruments so opaque that virtually no one understood the risks. – USA Today

Epic Fail

There have been documented changes in the mortgage industry since Ultraprise closed. For example - the lowering of lending standards allowed banks to give more money to people with lower credit scores. However, the outstanding change from my perspective was how those loans were sold to investors. What follows is an extremely gross simplification.

As subprime lending increased, the banks found it harder to sell pools of loans. The risks were too high for the big-money conservative investors like pension funds, and the possible rewards were too little for the hedge funds addicted to double digit growth. Thus, the rise of the aforementioned collateralized debt obligation, a.k.a the CDO.

The investment banks use CDOs to package mortgages, subprime loans, home equity loans, automobile loans, credit card debt, corporate debt, and used cat litter into products they slice up and sell to investors in private offerings (with the help of strippers, martinis, and appalling judgments by our trusted credit rating agencies). 

However, not all of the slices from a CDO are an easy sell (particularly the ones filled with risky loans and cat liter). So … firms will create a CDO squared (CDO^2) from the undesirable slices of multiple CDOs – essentially dressing up pigs with lipstick for the next big investors ball. You can be assured that a CDO cubed (CDO^4) will then arise from the dumping grounds of multiple CDO^2s, and then … well … CDO^N should give you an appreciation of how deep into the rabbit hole an investor can fall.

Investment banks issued hundreds of billions of dollars in CDOs over the last 5 years. Why did the risk averse investors, like the banks and the pension funds, stand in line to buy these CDOs and the other credit derivatives that Warren Buffet labeled “financial weapons of mass destruction”?

In part because a CDO does a good job of obscuring it’s underlying qualities– the loans, the assets, but most of all the risk. It was all a sales job, and something that an unknown software company could never pull off. 

Then How Could Software Help?

The $765 million “Mantoloking CDO 2006-1” was underwritten by Merrill Lynch and is the prime example of a “toxic asset”. The Mantoloking, a CDO squared, was built from the unwanted slices of 126 other CDOs, and is perhaps the most infamous CDO because its spectacular losses facilitated the disappearance of at least two hedge funds. You can find the 200 page prospectus online. It’s a lot to digest. In his paper “The Credit Crunch of 2007: What went wrong? Why? What lessons can be learned?”, esteemed financial engineer John C. Hull says transparency is needed.

[CDOs] … are arguably the most complex credit derivatives that are traded. Lawyers should move with the times and define these instruments using software rather than words. 

Sound familiar? To me it sounds like the software practice of using executable specifications - TDD and BDD for the financial world. No one can hide behind wordy documents and inflated credit ratings. They have to look at real numbers. The only question is - do we write these specifications in C#? Ruby? Haskell?

It turns out that Mr. Hull already had a language in mind. You can find it in his paper as footnote #8.

Given its widespread use in the financial community VBA is a natural choice for the programming language. 

What? A Microsoft Excel spreadsheet with macros might have saved our banks, our brokerage accounts, and our retirement funds?

It’s a stretch, but with the proper models in place it’s certainly a step in the right direction. Mr. Hull’s paper has other prescriptions for the finance world, too, as software is only part of the solution. You can never stop anyone who wants to skip due diligence and go directly to short-sighted greed, but I’d like to think that if our industry could make good software more readily available to the business world, the problems we are experiencing today wouldn’t be quite so bad.

ASP.NET 3.5 includes a URL routing engine and IIS 7.0 can use a URL re-writing engine (x86) (x64). Routing and rewriting sound very similar, but so do robbing and rewarding – you can’t judge features using phonetics.

Ruslan Yakushev wrote a great article on this very topic last year: “IIS URL Rewriting and ASP.NET routing”. Ruslan outlines the conceptual differences between the two URL hacking approaches as:

1. URL rewriting is used to manipulate URL paths before the request is handled by the Web server. The URL-rewriting module does not know anything about what handler will eventually process the rewritten URL. In addition, the actual request handler might not know that the URL has been rewritten.
2. ASP.NET routing is used to dispatch a request to a handler based on the requested URL path. As opposed to URL rewriting, the routing component knows about handlers and selects the handler that should generate a response for the requested URL. You can think of ASP.NET routing as an advanced handler-mapping mechanism.

Read the full article and I’ll think you’ll find that routing and rewriting turn out to be complementary pieces. For example, you might use rewriting for request blocking and enforcement of canonical URLs, while at the same time using ASP.NET routing for friendly URLs in your application. One advantage of using the ASP.NET routing engine in that scenario is that you can use all the routing rules to not only route requests, but also generate URLs when you put links on a page. The rewriting rules are opaque to an application.

The “Principle of Least Astonishment” (a.k.a the principle of least surprise (POLS)) is a guiding design principal for UIs, APIs, and interfaces of all types. Peter Seebach says:

Throughout the history of engineering, one usability principle seems to me to have risen high above all others. It's called the Principle of Least Astonishment -- the assertion that the most usable system is the one that least often leaves users astonished.

POLS plays a role in framework design, too. One framework behavior that I’ve always felt violated POLS was the ToString implementation of the System.Xml.XmlDocument class.

XmlDocument document = CreateSomeXml();
string result = document.ToString();

You’d think that calling ToString on an XmlDocument instance would give you a hunk of XML. But - it doesn’t matter how many nodes you have in the document, the ToString method will happily produce the following string every single time:

System.Xml.XmlDocument

This string represents the fully qualified type name of the object. Thanks, .NET framework! I wasn’t sure I was working with an XmlDocument. Now tell me, how do I get the XML out of this thing?

Take 2

The new XML API in .NET revolves around XElement and friends in the System.Xml.Linq namespace. This API fixes the astonishment factor of the ToString method. 

XDocument document = CreateSomeXml();
string result = document.ToString();

That above code yields real XML – angle brackets and all. In fact, LINQ to XML tends to “do the right thing” in many scenarios. For example, extracting an integer out of an attribute is just one cast operator away:

XDocument document = CreateSomeXml();       
int i = (int)document.Root.Attribute("bar");

Now that we have all the surprises out of the way, we can all get back to work and make the world better.

Wait? What’s that?

My Expectation – Your Heartache

Of course there are a plethora of options you might want to specify when producing a textual representation of an XDocument. There are encoding options, indentation options, new line options – the list goes on and on. You can get to these options with the WriteTo method of an XDocument, but not with ToString. ToString has to pick some defaults. One of the defaults in the ToString implementation is to omit any XML declaration, and this causes some people grief. 

Just recently I came across a post in a Ruby form. The topic was a debate over the naming of mathematical functions, and Matz says:

Ah, please stop mentioning POLS whenever your personal expectation is
not satisfied. I'm sick of them.

Trying to follow the principle of least surprise is one thing, but never surprising anyone is impossible.

What has surprised you recently?

Jeffrey “Party With” Palermo recently posted on “Separating configuration from data lowers total cost of ownership”:

My recommended progressions of configuration locations are as follows.  Specific requirements should be the driving factor that cause you to move configuration from one medium to the next costly alternative.

1. In the code
2. In a file
3. In a database
4. Some other system-specific external configuration store

I wanted to call out option #3, which is to put configuration information in the database. Over the years, I’ve heard numerous proposals for putting configuration information in the database. Nearly every proposal has a good counterargument.

Putting configuration information in the database is easier.

Easier than checking in code? If so, then you might have a problem with your source code repository. More often than not, “easier” means someone is trying to circumvent testing, check-in policies, revision control, or code reviews.These are all things you shouldn't avoid.

Our configuration information is very complicated.

Complexity isn’t something a database can fix. The configuration will still be complicated, and to make matters worse - it’s now in the database!

We need to access configuration values from everywhere.

If this is a highly distributed system, then this might be a good argument, but sometimes “everywhere” means “the business logic inside of stored procedures”. There is no reason to make it easier for business logic to sneak into the database.

The database should be one of your last choices as a repository for configuration information. Configuration in code is easily tested, versioned, changed, and refactored. As Jeff said – only a specific requirement should drive you to a more costly alternative like the database.