Find the evidence you want

I recent discovered a fascinating podcast called Hardcore History hosted by Dan Carlin.  While bouncing around through the backlogs, I came across an episode with historian and author Gwynne Dyer who had a bevy of interesting things to say, but one in particular struck me as being relevant to the software development process.

Mr. Dyer was talking about whether World War I had been inevitable.  Some people think that the events that led to the first World War were merely a pretense and that the war would have happened even if Archduke Ferdinand had not been assassinated, it just may have happened a year or two later.  I’m not going to compare software development to World War (although it’s a bit tempting sometimes, I just can’t trivialize a real actual war while I sit in a Herman Miller Aeron).  Instead I’ll say the thing that jumped out at me was instead his analysis of how historians analyzed that claim.

He said that historians looked back on the evidence and found the particular things they wanted to in order to support their claim.  This is, on the surface, a very simple statement, but the implications are far reaching.  In general, people look at the evidence and find ways to make it support the viewpoint they already hold.  Couple this with findings that throw into question whether facts actually help us change our minds and you should start to be worried.

Let me illustrate with a story from my development past:

Imagine a project that is in as much trouble as a project can be.  It’s overtime, over budget and the original scope has been shredded beyond recognition.  How did it get there?  What went wrong?  It depends who you ask.

The testers say there wasn’t enough testing.  The developers say there wasn’t good project planning.  The project managers say the developers went off course.  Management says people didn’t work hard enough.  Analysts say there wasn’t enough analysis done.

Who was right?

In this case, it doesn’t matter (it was me.) The important point is that everyone found the evidence that supported their particular viewpoint and anything else was ignored or rationalized away.  Would it have been possible to figure out where things really went wrong?  Probably if there was a frank discussion and analysis, but that is far harder to do and requires fighting our tendencies.

It seems so easy to spot this when other people do it, yet we seem blind to it from ourselves.  Can you challenge a conclusion you came to recently?

Add comment