The Problem with Test Driven Development

Wednesday, July 27, 2005

Unit testing fanatics might find the content of this post disturbing. I’m going to point out a fundamental problem with the TDD approach, and I’m not one to just point out problems, so I also offer a solution.

TDD fanatics often wonder aloud why the weary masses have not thrown off the shackles of hack and slash coding, why they have not come running to the glow of green lights and the feeling of confidence the lights imbue. The merits of TDD are so obvious, the fanatics think, that any software developer should instantly evolve into an enlightened one – a test driven developer. Yet, many are left unconverted.

The adoption of TDD is hampered by a glaring problem. The problem begins in the opening description of the methodology, which typically sounds like the following.

The first step is to quickly add a test, basically just enough code to fail.

Do you see the problem?

Let’s take another example. Here is a shot of the NUnit status bar after test execution.

There is the problem again – do you see it yet?

The problem is no ambitious person will wake up in the morning, walk into the shower, and plan ahead for a day of failure. If they did, they wouldn’t be getting into the shower. People who think that way stay lying in bed, daydreaming about the beach and seeing other people in their wet cotton swimwear. Nobody wants to start with a failure.

The word fail is a dirty, dirty word.

You know what we do with dirty, dirty words, don’t you?

We shun dirty, dirty words. We replace them. We turn them into gentle, obfuscated phrases.

Euphemisms In Software Development

As software developers, we are familiar with many euphemisms. For example, instead of saying software has “defects”, we say software has “issues”. Nowadays, no self respecting company will wait until V 1.0 to release a product. A beta release is as good as gold - it just has a some “issues” is all.  

Likewise, AJAX is a new euphemism for “I wish this wasn’t a browser based application”. The A in AJAX stands for asynchronous. Asychronicity sells - not that asynchronicity is a real word, but it sounds sexy and could be the title of a hip dance song.

I could go on and on with these examples, but I hope you, as the reader, have seen the solution I am driving towards.

Failure Is Not An Option

At this time I’m going to borrow an idea from the British education system – the idea of “deferred success”. Instead of saying that a child has “failed” a class, British schoolteachers want to say the child has merely “deferred success”. The word “fail” could scare a child away from future education. A brilliant idea from you bloody Brits.

Let’s apply the concept of deferred success to test driven development.

The first step is to quickly add a test, basically just enough code to defer success.

Doesn’t that statement make you feel success is just around the corner? Why, the word success even appears at the end of the sentence! No nasty f*** words are jumping off the screen, filling developers with a sense of fear and foreboding.

Even NUnit looks better after two changes to StatusBar.cs and a recompile. I need to get in touch with the PM for Team System unit testing ASAP.

Now the choice is up to the test driven development community. They can continue to force failure after failure on software developers, or they can choose the kindler, gentler path to test driven development.

I hope my posting will not defer it’s success in convincing them to make the right choice.


Comments
John Wednesday, July 27, 2005
...I've noticed a similar thing with my 'design by contract' code. Nearly every time I write this,

throw Contract.GetFailure(...);

I wonder if I should be worried about the subliminal effects that might be having... and flirt with the idea of changing it, just in case. :|

On the bright side, I'm typing,

Contract.RequireInstance(...);

far more frequently. :P

optionsScalper Wednesday, July 27, 2005
Scott,

Asychronicity isn't listed on www.dictionary.com, but synchronicity is. Synchronicity is also an album by "The Police" and can be found here.

www.amazon.com/.../102-4234300-6069745

Obviously, "The Police" were successful and didn't do any TDD. If you search for "The Firemen" on amazon music, you probably won't find much because those guys were busy doing TDD instead of music.

I even mentioned my position on TDD here:

www.jjbresearch.org/...

If it is not obvious, I'm a product of too much TDD. TDD was forced on me in recent years. My bottom status bar message for a simple method that contained one foreach loop in VB.NET usually read something like "Failures: 2.445093639987832969273016*10^17". Yeah, it had to be reported in scientific notation because I failed too much (and it reported more than 17 significant digits, so I guess that the unit testing software was even reporting partial errors). I apparently had failure coming out of every orifice. It has left a long trail of broken vodka bottles from here (somewhere west of Milwaukee, WI) to Boston to New York to San Francisco and other destinations.

Now I've got to get back to IM'ing. I've got at least two conversations waiting (and some more of that research you mentioned a while back). Oh yeah, I have to resolve a few more "issues" on GAI.NET before I release it.

I would also like to congratulate you on your "deferred success" in making your point.

---O

p.s. I heard that TDD was not compatible with the NET 2.0 Framework (this is how rumors get started, right?).
Kenny Kerr Wednesday, July 27, 2005
Nice one Scott. I knew there was a reason I write my production class followed by my test class.

:)
Steve Hall Wednesday, July 27, 2005
Of course, taking this further..."bugs" should really be known as "deferred fixes" (and "fixes" are really just "deferred side-effects", eh?), "design flaws" as "deferred enhancements", and "unfullfilled functional requirements" are merely a "deferred release".

I'm getting to like this idea that my self-esteem is actually worth the time to obsfucate the true units of measurement! These new terms will certainly make my next performance review sound a lot better...
Thomas Eyde Wednesday, July 27, 2005
But then, after a while, "deferred" start to sound scary. When everyone is shipping deferred this and deferred that, then we need a new euphemism (I've learned a new word today, how do you pronounce that?).

Because by then, nobody plan ahead for a deferred day. If they did, they wouldn’t be getting into the shower, would they?
KevinT Wednesday, July 27, 2005
Bwaahahahahaaaaaa. Only in america, i tell you!

Seriously though, you were taking the piss, right? (to borrow another phrase from the poms)
Haacked Wednesday, July 27, 2005
So is that all it takes to get people on board? And I thought they were just lazy.
Jeremy Brayton Wednesday, July 27, 2005
I think you stumbled onto something without knowing it.

Why make failure tests? Why not make success tests?

You know more about how a routine should function than you'll ever know about potential failures. Yes those failures (if known) will help you produce better code but you're throwing code out into the wild with the hope that it isn't buggy and you only fix the problem in the future if you're fortunate (or unfortunate) to locate the exact point of failure.

I don't know about you, but I've had enough "ghost" failures with no reproducable root cause that it's more than annoying. If I can somehow reverse TDD to where I test for passing and all failure is somehow lumped into categories automagically or even just thrown into some big failure bin, I could probably be more productive.

I wouldn't wake up thinking how I could fail today, but how I can succeed. Since in life I tend to fail more than I succeed, it seems to make more sense to work with this than against it. Maybe I'm just blowing smoke but I don't quite grasp the whole "lemme find a bug then put it in our database so that in the future I never make that exact mistake in roughly the same code in other projects". Then again that idea never really stuck because while I do copy/paste quite a bit, context changes those bugs or "deferred fixes" so much that it's almost impossible to make up a test that you can use for everything.
Steven Padfield Wednesday, July 27, 2005
Sorry about my bogus trackback posts... I have no idea what happened. Anyway, I posted about this post in my blog, here:

http://righteousindignation.gotdns.org/blog
scott Wednesday, July 27, 2005
Not a problem Steven, I can clean those up. Good article you have. Of course I was being toungue in cheek, or "taking the piss" as Kevin puts it. I think. I'm not familar with that phrase but it sounds about right.
Jeremy Brayton Saturday, August 6, 2005
I like to pretend I'm not an idiot, but we see that doesn't get me very far.

I never quite understood the "test driven" part of things, though I apparently had a good handle on the "when you see a bug, make a test for it, then test for that bug every other place you can" part of it.

While I don't understand the whole "write for failure first to give yourself a sanity check", I suppose it has some degree of reason. Then again all I have to do to make a test fail is write up the harness with 0 code, run it and I'll get the lovely redness. All that tells me is I've added a test and I should be working towards a pass, which makes putting any logic in the test before doing this seem like a huge waste of time.

So I suppose you want at least one failure when writing your tests first. That failure should be at least the very first time you run the test and every other subsequent run until you get a pass. Get a pass first by cheating then refactor your test to be the smallest algorithm possible. Once your pass becomes a "pass with logic" you're on to the next test.
Comments are now closed.
by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!