Scarcity In Software Development

In software development we face many constraints, and we usually think of constraints as bad things that make our jobs miserable. If we had no constraints, we’d build beautiful software with impeccable error handling because there would be no errors.

In one of my first jobs I wrote firmware for lab devices. Each device had a 32kb ROM for program storage, and those 32kb of memory constrained the type of software I could create, and the tools I could use. As an aside, the limited memory did not constrain me from destroying small thermal printers attached to each device because 32kb is still big enough to hide the machine code that writes to memory-mapped IO in an accidentally infinite loop. 

Constraints don’t always have to be a negative, however, and I was reminded of this when reading Scarcity: Why Having Too Little Means So Much. Half the book is dedicated to the destructive impact of scarcity. 

Scarcity captures our minds automatically. And when it does, we do not make trade-offs using a careful cost benefit analysis.

But the other half of the book highlights the unintuitive benefits of scarcity. The authors pointed to one study where two groups of college undergrads were paid to proofread three essays. One group was given a single deadline three weeks out, and the other group was given three one week deadlines. The group with tighter, more frequent deadlines achieved higher productivity. 

They were late less often (although they had more deadlines to miss), they found more typos, and they earned more money.

I know some of you are thinking that the first group was waterfall and the second group was agile and of course the 2nd group performed better because agile iterations are always better and everyone is doing agile, even the people who define requirements during an 8 hour scrum at an off site location where they won’t get interrupted by customers who might pepper them with mundane questions.

The book does a good job of pointing out how a scarcity of any resource (time, money, or food, as some examples), will make our minds focus on the immediate problem, and this focus can yield positive results. However, the focus can also lead to long term problems.

A Scarcity Of Time

I've never seen a software project that was developed too fast. An application that will take one year of work will be even better if it only takes 6 months, and to make everyone happy the app would have to be done by yesterday. We are always working under time constraints. 

It’s not surprising then, that most tools, methodologies, languages, frameworks, and snake oils in this industry try to solve, either directly or indirectly, the time constraint. It doesn’t matter if we talk about sprints, Rails, code generators, or Erlang, they all at one point or another pledge to be more productive or otherwise allow us to deliver more features in less time inside a specific context. 

We’ve also developed some safe guards against being too quick, like unit testing. And often times these safe guards are in direct conflict with the tools and frameworks we use to boost productivity. A framework might allow us to crank out features quickly, but the code is untestable, so maintenance might be a problem in the future.

The days of me programming in a memory constrained environment are long gone, but there are always scarce resources in any software project. The Scarcity book meandered at times, but it is an interesting read for anyone in software because the psychological impacts of scarcity are valuable to understand in this industry.