Scalability Is Passé

Five years ago, everyone working server-side was obsessed with building scalable software. Some said we could never be scalable with Microsoft technologies.

Fast forward to 2004. Everyone is obsessed with building “secure” software. Some say we can never be secure with Microsoft technologies.

If we look at what has happened over the last five years, perhaps we can see what might happen in the next five.

1) For scalability, expectations have leveled. In 1999 everyone believed they were building the next eBay and Amazon. Today, nobody is over-hyping the ability to go from 0 to huge overnight. I don’t foresee the same leveling off of expectations for secure software. If anything, the expectation is just starting to grow.

2) Security, interoperability, and flexibility have pushed scalability off the radar screen. On Aaron Skonnard's list, scalability is not even in the top 5 qualities to look for in enterprise software. Most shops with limited resources have to carefully select the qualities to focus on - we can’t have it all. Looking ahead, security and interoperability are here to stay for the next five years.

3) Scalability is easier today because of the platform and runtime. We moved from building web pages with loosely-typed interpreted pages to strongly-typed compiled pages. In some ways, the underlying platform lost features that were abused (think of ADO and server side cursors). Security in the next five years has to get easier too. It is currently hard to run as a non-admin. It is currently hard to run assemblies without full-trust. The platform and runtime can get better in this area, but not make them completely painless.

4) Scalability is easier because of the tools. It’s easy to set performance benchmarks and measure scalability with software. It’s easy to catch possible bottlenecks with syntactic analysis of an application: a cursor declaration in a SQL query, a static constructor in a class. Many security problems require the tougher task of semantic analysis. Where is the application saving a plaintext password? Is this string concatenation putting user input into a SQL string? These will be tough problems to uncover. Additional expertise and heuristics will need to exist in security tools.

5) Scalability education has been constant over the last five years. Follow some guidelines and work with the framework instead of against it, and you’ll have a decently scalable system. Don’t write your own connection pooling and thread pooling libraries – chances are the ones in place will work for 95% of the applications being developed. Security education, on the other hand, is still in infancy. You’ll find a good example on any given day in the newsgroups. Someone will post a solution which opens and disposes a database connection in an efficient manner, but somewhere in between concatenates a textbox value into a SQL string. This education takes some time.

In five more years I think we will be better off, but not completely there. We moved from being obsessed with scalability to expecting it out of the box in five years. However, after five more years we will still be obsessed with writing secure software, because security will still be difficult. As they say on Wall Street, past performance is not indicative of future returns.