Is technical debt the Pinocchio’s nose of software engineering?

Technical debt seems like the Pinocchio's nose of software engineering; the ‘lie’ that something is finished perpetuates and causes it to multiply in size. Matthew Heusser in his article[1] says in other professional bodies people are referred to as ‘shysters’ or ‘hacks’ if they breach the confidence placed in them. As an example of good professional practice, Matthew describes a lawyer who would drop a case rather than compromise their counsel.

Whether the pressure to ‘get things done’ is internal or external to our engineer; let’s assume for now that they want to do the right thing.Our hypothetical engineer may choose to leave as in Matthews’s example of a lawyer but he then faces is the ‘better the devil you know’ dilemma (fear of the unknown). Another solution may be to find technically competent teams in their organisation who create a professional environment.

The strange thing about these questions of practice is that we all know that time invested in testing and other code hygiene issues pays off even in the relatively short term. Take for example the data from InfoQ that tells us "defects per thousand lines of code, decreased between 40% and 90% relative to the projects that did not use TDD"[6].

In his article Matthew goes to to say "the solution is not to try to measure the debt. If you do that, you’ll take a proxy measurement, which the team can exploit. More time spent measuring more things quickly becomes a death spiral" [1]. Intuitively the only way to avoid an ever growing debt is to avoid it in the first place; this seems like your mom telling you to save for things rather than taking a loan, you know you should, but few do. Another perspective from Ben Scofield is that "adequacy is very interesting. For one thing, adequate performers win more often than they lose" [5]. Given how seductive the allure of ‘winning’ is, the problem becomes how to avoid needing a hero complex in order to do the right thing.

Matthew’s reference to the ‘zero sum gain’ principle evoked another old favourite: the no win scenario. The answer could be the ‘change the rules of the game’. This is mentioned as a solution to the ‘no win situation’ [2] popularised by (among others) the character James.T. Kirk in the fictional Kobayashi Maru training simulation [3]. Try imagining a ship full of software engineers in the same simulator saying ‘sorry guys - we have a deadline so you’re just going to have to sort out your own life support problems’.

To understand how to change the rules of the game first we must change ourselves and our comprehension of the situation. One technique that I found useful is described by Andrew Hunt and David Thomas in ‘The Pragmatic Programmer’ as ‘Tracer Code’. The idea is that you change the assumptions of what you have to deliver in order to learn more about what it is you are trying to build [4].

Ultimately the answer has to be to find a way to avoid stepping on the slippery slope of saying something is ‘done’ when it is not finished. As Matthew says at that point you are a few measure-and-manage steps away from a death spiral.

[1]November/December 2011 Better Software, P18: 10 Thoughts on Technical Debt by Matthew Heusser.
[2] http://en.wikipedia.org/wiki/No-win_situation
[3] http://en.wikipedia.org/wiki/Kobayashi_Maru
[4] http://pragprog.com/the-pragmatic-programmer/extracts/tips
[5] http://pragprog.com/magazines/2010-08/in-defense-of-mediocrity
[6] http://www.infoq.com/news/2009/03/TDD-Improves-Quality

Thanks to Julian Harty for his editorial input.