Posts Tagged ‘Code Coverage’
Round Two: Can Technical Debt Constitute a Breach of Implied Warranties?
In a previous post I discussed whether technical debt could under some conditions constitute a breach of implied warranties. Examining the subject with respect to intent, I made the following observation:
It is a little tricky (though not impossible – see Using Credit limits to Constrain Development on Margin) to define the precise point where technical debt becomes “unmanaged.” One needs to walk a fine line between technical/methodical incompetence and resource availability to determine technical fraud. For example, if your code has 35% coverage, is it or is not unmanaged? Does the answer to this question change if your cyclomatic complexity per class exceeds 30? I would think the courts might be divided for a very long time on the question when does hidden technical debt represent a fraudulent misrepresentation.
One component of technical debt deserves special attention in the context of this post. I am referring to the conscious decision not to do unit testing at all… Such a conscious decision IMHO indicates no intention to pay back this category of technical debt – unit test coverage. It is therefore quite incompatible with the nature of an implied warranty:
Responses to my post were mixed. Various readers who are much more knowledgeable in the law than I am pointed out various legal defenses a software vendor could use. Click here for a deeper understanding of these subtle legal points.
Imagine my delight reading yesterday’s uTest interview with Cem Kaner. Cem makes the following statement in his interview:
ALI [American Law Institute] … started writing the Principles of the Law of Software Contracts. One of its most important rules is one that I advocated: a seller of software who knows about a defect of the software but does not disclose the defect to the customer will be held liable for damages caused to that customer by that defect. Note that this does not apply to free software (not sold). And if the seller discloses the defect, it becomes part of the product’s specification (it’s a feature). And if the seller doesn’t know about the defect there is no liability (once customers tell you about a defect, you have knowledge, so you cannot avoid knowledge for long by not testing). ALI adopted it unanimously last year. This is not law, but until the legislatures pass statutes, the Principles will be an important guide for judges. Even though I am a minor contributor to this work, I think the defect-disclosure requirement might be my career’s most important contribution to software quality. [Highlights by IG].
While not (yet?) a law, the Principles could indeed lead us where IMHO we as an industry should be. Technical debt manifests itself as bugs. By making the specific bugs part of the spec the software evolves from a quality standpoint. Moreover, the defect-disclosure requirement practically force software vendors to address their bugs within a “reasonable” amount of time. Customer bugs become an antidote to the relentless pressure to add functions and features without paying due attention to software quality.
I guess I missed the opportunity to include technical debt in the 2010 revision of the Principles. As we always do in software, I will target the next rev…
Technical Debt Meets Continuous Deployment
As you would expect in a conference entitled velocity, and in a follow-on devops day, speeding up things was an overarching theme. In the context of devops, the theme primarily manifested itself in lively discussions about the number of deploys per day. Comments such as the following reply to my post Ops Driven Dev were typical:
Conceptually, I move the whole business application configuration into the source code…
The theme that was missing for me in many of the presentations and discussions on the subject was the striking of a balance between velocity and quality. The classical trade-off in process control is between production rate and product quality (and safety, but that aspect [safety] is beyond the scope of this post). IMHO this trade-off applies to software just as it applies to mechanical or chemical processes.
The heart of the “deploy early and often” strategy hailed by advocates of continuous deployment is known deployment state to known deployment state. You don’t let the deployment evolve from one state to another before it has stabilized to a robust state. The power of this incremental deployment is in dealing with single-piece (or as small number of pieces as possible) flow rather than dealing with the effects of multiple-piece flow. When the deployment increments are small enough, rollback, root cause analysis and recovery are relatively straightforward if a deployment turns sour. It is a similar concept to Agile development, extending continuous integration to continuous deployment.
While I am wholeheartedly behind this devops strategy, I believe it needs to be reinforced through rigorous quality criteria the code must satisfy prior to deployment. The most straightforward way for so doing is through embedding technical debt criteria in the release/deploy process. For example:
- The code will not be deployed unless the overall technical debt per line of code is lower than $2.
- To qualify for deployment, code duplication levels must be kept under 8%.
- Code whose Cyclomatic complexity per Java class is higher than 15 will not be accepted for deployment.
- 50% unit test coverage is the minimal level required for deployment.
- Many others…
I have no doubt whatsoever that code which does not satisfy these criteria might be successfully deployed in a short-term manner. The problem, however, is the accumulative effect over the long haul of successive deployments of code increments of inadequate quality. As Figure 1 demonstrates, a Java file with Cyclomatic complexity of 38 has a probability of 50% to be error-prone. If you do not stop it prior to deployment through technical debt criteria, it is likely to affect your customers and play havoc with your deployment quite a few times in the future. The fact that it did not do so during the first hour of deployment does not guarantee that such a file will be “well-behaved” in the future.
Figure 1: Error-proneness as a Function of Cyclomatic Complexity (Source: http://www.enerjy.com/blog/?p=198)
To attain satisfactory long-term quality and stability, you need both the right process and the right code. Continuous deployment is the “right process” if you have developed the deployment infrastructure to support it. The “right code” in this context is code whose technical debt levels are quantified and governed prior to deployment.
Can Technical Debt Constitute a Breach of Implied Warranties?
Photo credit: Dancing Lemur (Flickr)
Cunningham’s quip “A little debt speeds development so long as it is paid back promptly with a rewrite” is intuitively very clear. We are talking about short-term debt which will be reduced, and hopefully eliminated in entirety, at the earliest possible time.
The question this post addresses is what happens when the expected short-term technical debt becomes a significant long-term debt? Specifically, can technical debt under some conditions constitute a breach of implied warranties?
In his InformIT article Don’t “Enron” Your Software Project, Aaron Erickson coined the term “Technical Fraud” and connected it to Lemmon Laws:
As a reaction to seeing this condition and its deleterious effects, I coined the term technical fraud to refer to the practice of incurring unmanaged and hidden technical debt. Many U.S. states have “lemon laws” that make it illegal to knowingly sell someone a car that has undisclosed maintenance problems. Selling a “lemon” is a fraudulent practice in the world of cars, and it should be considered as such in the world of software.
It is a little tricky (though not impossible – see Using Credit limits to Constrain Development on Margin) to define the precise point where technical debt becomes “unmanaged.” One needs to walk a fine line between technical/methodical incompetence and resource availability to determine technical fraud. For example, if your code has 35% coverage, is it or is not unmanaged? Does the answer to this question change if your cyclomatic complexity per class exceeds 30? I would think the courts might be divided for a very long time on the question when does hidden technical debt represent a fraudulent misrepresentation.
One component of technical debt deserves special attention in the context of this post. I am referring to the conscious decision not to do unit testing at all.
Best I understand it, the rationale for not “bothering” with unit testing is a variant of the old ploy “we do not have time for testing here.” It is a resource allocation strategy that bets on the code being miraculously bug-free. Some amount of functional testing is done out of necessity – the code in customers hands needs to function as proclaimed. But, the pieces of code from which functionality is constructed are not subject to direct rigorous testing. The individual units of code will be indirectly exercised in some manner through functional testing, but not in a systemic manner to verify and validate correctness of the units of code per se.
Such a conscious decision IMHO indicates no intention to pay back this category of technical debt – unit test coverage. It is therefore quite incompatible with the nature of an implied warranty:
An implied warranty is as an unstated promise, assumed by the law in most sales transactions, that the product will be of at least average quality and will do what the average customer would expect it to do [The Reader’s Digest Legal Questions & Answers Book]
To #1 defense open to a software vendor who gets sued over lack of unit testing is that a fair average quality of software can be attained without any unit testing. As a programmer, I would think such defense would fly at the teeth of the availability since 1987 of the IEEE Standard for Software Unit Testing.
It is fascinating to note the duality between contracts and programming. For the programmer who follows the tenets of design by contract, “a unit test provides a strict, written contract that the piece of code must satisfy…”
Disclaimer: I am not an expert in the law. The opinion expressed in this post merely represents my layman’s understanding of principles of contract law that might be applicable to technical debt situations.
How to Use Observations From Outside the Agile Process
Photo credit: tengtan (Flickr)
Most posts on technical debt in this blog emphasize the use of technical debt for strategic decision-making. In this post we will point out the use of technical debt in Agile teams at the tactical level. Specifically:
- Every two weeks; and/or,
- With every build.
Taking a close look at the various components of technical debt during the bi-weekly iteration review meeting provides plenty of useful information to the process. For example, you might look for insights to explain the following:
- Why is the unit test coverage figure going down?
- Any particular reason the cyclomatic complexity figure has gone up?
- Why is the figure of merit for design lower than the figure indicated in the previous iteration review meeting?
- Many others…
The emphasis in this mode of operation is on guiding the retrospection. Plenty of good and valid reasons might exist for any of the trends mentioned above. However, observing the trends helps you ask the right questions, focusing on what happened during the iteration just completed. In conjunction with technical debt data from previous iteration review meetings, trends that characterize your software development project become visible. You may or may not need to change anything you are doing, but you become very conscious of any “let’s not change” decision.
An intriguing practice suggested by colleague and friend Erik Huddleston is to make technical debt a criterion for the build to pass. The build automatically fails if the technical debt figure has gone up. Or, if you are very focused on a specific aspect of technical debt such as complexity, you fail the build whenever the complexity figure of merit rises above a certain pre-determined threshold. For example, you might fail a build in which the cyclomatic complexity per method has exceeded 4.
The power of failing a build whenever the technical debt arises is in utilizing the build as an exceptionally effective influence point. You instill the discipline of reducing technical debt one build at a time. If your team aggressively practices continuous integration, it will address technical debt issues multiple times a day. Instead of staring at a “mountain” of technical debt towards the release of a product, you chunk it to really small increments that get addressed “real-time.” For instance, a build that failed due to lack of comments can usually be fixed very quickly by the developer who “upset the apple cart” while the logic embedded in the code is fresh on his/her mind.
A good insight to the way the tactical use of technical debt techniques adds value is provided by the following observation: the technical debt data is observed from outside the Agile process. Hence, technical debt data is nicely suited to guiding the process. If you think of the software engineering fabric as a virtual stack, the technical debt “layer” could be considered a layer above the Agile process.
Wrestling with Scaling Software Agility
Software Development Times has just published Guest View: Wrestling with Scaling Software Agility by Ryan Martens and me. This Guest View is a little unique in that Ryan and I actually try to wrestle each other to the ground… Here is why we try to do so:
Agile champions spend a lot of time trying to communicate the agile premise to the executives in their organization. The difference in context between the champion and the executive often makes it a difficult conversation. A Scrum Master versed in behavior-driven design is not always able to relate to the frustrations of a sales executive who gets free advice on how to sell from everyone and his grandmother.
Conversely, a CFO does not necessarily understand why unit test coverage on the company’s legacy code is still inadequate after a full year of investment in agile methods that embrace refactoring as a core practice.
To bridge the chasm through this article, we resort to role-playing. Ryan Martens plays the Agile Champion; Israel Gat plays the Skeptical Executive. Metaphorically speaking, each one tries to wrestle the other to the ground.
Before you get into this Guest View, I would like to reinforce an important disclaimer:
A note of caution before Ryan and Israel make irreparable damage to their long-standing relationship: The two actually understand each other extremely well and rarely are they of different opinions on the fundamentals of agile in real life…
Enjoy the article!
The Agile Executive 