Archive for March 2010
Should You Ship This Code Before Reducing Technical Debt?!
Source: JulesH, Wikipedia, A control flow graph of a simple function
Technical debt is usually perceived as a measure of expediency. You borrow a little (time) with the intent of paying it back as soon as possible. To quote Ward Cunnigham:
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.
As is often the case with financial debt, technical debt accrues with compound interest. Once it reaches a certain level (e.g. $1 per line of code) you stare at a difficult question:
Should I ship this code before reducing the accrued technical debt?!
The Figure below, taken from An Objective Measure of Code Quality by Mark Dixon, answers the question with respect to one important component of technical debt – cyclomatic complexity. Once complexity per source code file exceeds 74, the file is for most practical purposes guaranteed to contain errors. Some of the errors in such a file might be trivial. However, a 2007 study by Capers Jones indicates about a third of the errors found in released code are likely to be serious enough to stop an application from running or create erroneous outputs.
To answer the question cited above – Should You Ship This Software Before Reducing Technical Debt?! – examine both cost and risk for the number of error-prone files you are about to unleash:
- The economics of defect removal clearly favor early defect removal over late defect removal. The cost of removal grows exponentially as function of time.
- Brand risk should be first and foremost on your mind. If complexity figures higher than 74 per file are more of the norm than the exception, you are quite likely to tarnish your image due to poor quality.
If you decide to postpone the release date until the technical debt has been reduced, you can apply yourself to technical debt reduction in a biggest-bang-for-the-buck manner. The analysis of complexity can identify the hot spots in your code, giving you a de-facto roadmap you would be wise to follow.
Conversely, if you opt to ship the code without reducing technical debt, you might lose this degree of freedom to prioritize your “fix it” work. Customer situations and pressures might force you to attend to fixing modules that do not necessarily provide as much bang for the buck.
Postscript: Please note that the discussion in this post is strictly limited to intrinsic quality. It does not address at all extrinsic quality. In other words, reducing/eliminating technical debt does not guarantee that the customer will find the code valuable. I would suggest reading Beyond Scope, Schedule and Cost: Measuring Agile Performance in the Cutter Blog for a more detailed analysis of the distinction between the two.
Erratum: The figure above is actually taken from a blog post on the Mark Dixon paper cited in my post. See McCabe Cyclomatic Complexity: the proof is in the pudding. My apology for the error.
How to Initiate a DevOps Project
Source: 17th/21st Lancers c. 1922-1929 “THE FIGHTING SPIRIT!”
Agile consultants on a development project often start by helping the team construct a backlog. The task is sufficiently concrete to get all stakeholders (product management, project management, development, test, any others) on a collaborative track through the creation of a key artifact. The backlog establishes a base line for the tasks to be carried out in the project.
For a DevOps project, start by establishing the technical debt of the software to be released to operations. By so doing you build the foundations for collaboration between development and operations through shared data. In the DevOps context, the technical debt data form the basis for the creation and grooming of a unified backlog which includes various user stories from operations.
Apply the same approach when you are fortunate to be able to include folks from operations in the Agile team from the very beginning. You start with zero technical debt, but you track it on an ongoing basis and include the corresponding “fix-it” stories in the backlog as you accrue the debt. Running technical debt analytics on the source code every two weeks is a good practice to follow.
As the head of development, you might not be comfortable sharing technical debt data. This being the case, you are not ready for DevOps.
How to Construct a Great Information Radiator
The information radiator above is used by Panic, a Portland, OR software company. It shows the following items:
- E-Mail Queue — number of messages / number of days.
- Project Status — sorry for the heavy censorship — you know how it is!
- Important Countdowns
- Revenue — comparing yesterday to the day before, not so insightful (yet).
- Live Tri-Met Bus Arrivals — when it’s time to go home!
- The Panic Calendar
- Employee Twitter Messages
- Any @Panic Twitter Messages — i.e., be nice! They go on our screen!
Click here for detailed description, including an easy to follow “Implementation Notes” section. The writer provides the following perspective on the effort to construct the board:
From start to finish, this was about a three-week project.
And no, it didn’t slow down development on [insert the app you want the most here]. Check the board!
Measuring Agile Success Rate the Right Way
Much has been said recently about the success/failure rate of Agile projects. In particular, a debate arose around the success rate of Scrum vis-a-vis Kanban. For example, in a post entitled Some Day Kanban will fail 75% of the Time, colleague Jurgen Appelo states as follows:
Unfortunately, some people arguing against Scrum include these ScrumBut teams in their evaluations of the “high failure rate” of Scrum. They love quoting that “at least 75 percent of Scrum implementations fail.” And I think “Yes of course, 75% fails when that includes the teams that don’t understand what they’re doing.”
I would like to add one other “dimension” to the discussion: boundary conditions.
Any Agile initiative – Crystal, Scrum, Kanban, etc. – typically starts from a certain state of affairs of the code that has already been developed using a Waterfall method or no method at all. Even brand new projects produce code that invariably interacts with other software components that are already deployed, warts and everything. Pristine environments with no technical debt for the Agile initiative to deal with are rare.
Like it or not, the Agile initiative is saddled from the outset with a certain amount of technical debt. Code has been duplicated, rules violated, complexity ran amuck, etc. A typical enterprise software team starts with hundreds of thousands $$ in technical debt, if not millions. This debt needs to be “paid back.” Probably not over night, but certainly over a period of time. As illustrated by the following figure from Jim Highsmith, things get ugly if the debt is not paid back over an extended period of time.
The evaluation of success or failure of the Agile initiative needs to take technical debt into account. A team of 50 with an accrued technical debt of $100,000 has a much easier job on its hands transitioning to Agile than a similar size team starting with $1M in technical debt on its hands.
Whatever criteria you use to determine whether an Agile initiative has been successful, I would suggest the following boundary condition needs to be satisfied:
Technical debt at the end of the project/initiative must be significantly lower than technical debt at the start of the project.
Use the techniques outlined in Using Credit Limits to Constrain Development on Margin to calculate technical debt before and after. In addition to qualifying your Agile success, quantifying technical debt will do a lot towards improving the quality of your software.
How to Combine Development Productivity Data with Software Quality Metrics
Consider the situation described in Should You Invest in This Software:
- One of your portfolio companies expects to ship 500K lines of code in 6 months.
- The company asks for additional $2M to complete development and bring the product to market.
- Using technical debt quantification techniques you find the technical debt amounts to $1M.
You are not at all comfortable “paying back” the technical debt in addition to funding the requested $2M. You wonder whether you should start afresh instead of trying to complete and fix the code.
Photo credit: @muntz (Flickr)
A good starting point for assessing the fresh start option is Michael Mah‘s studies of software productivity. Based on the QSMA SLIM metrics database of more than 8,000 projects, Michael will probably bracket the productivity per person in a team consisting of product management, development and test at 10-15K lines of code per year. If you use the 15K lines of code per year figure for the purposes of the analysis, 500K lines of code could theoretically be delivered with an investment of about 33.3 (500/15) man years. Assuming average loaded cost of $99,000 per man-year, the software represents a programming effort of $3.3M. Not much is left if you deduct $3M ($2M+1M) from $3.3M…
Five considerations are of paramount importance in evaluating the start afresh option:
- The comparison above ($3.3M versus $3.0M) is timeless. It is a snapshot at a certain point in time which does not take into account the value of time. To factor in the time dimension, the analysis needs to get into value (as distinct from cost) considerations. See the note on Intrinsic Quality v. Extrinsic Quality at the bottom of this post.
- Your “mileage” may vary. For example, best in class teams in large software projects have reported productivity of 20K lines of code per team member per year. As another example, productivity in business applications is very different from productivity in real-time software.
- If you decide to start with a brand new team, remember Napoleon’s quip: “Soldiers have to eat soup together for a long time before they are ready to fight.”
- If you decide to start afresh with the same team plus some enhancements to the headcount, be mindful of ‘Mythical Man-Month‘ effects. Michael Mah’s studies of the BMC BPM projects indicate that such effects might not hold for proficient Agile teams. Hence, you might opt to go Agile if you plan to enhanced the team in an aggressive manner.
- Starting afresh is not an antidote to accruing technical debt (yet again…) over time. But, it gives you the opportunity to aggressively curtail technical debt by applying the techniques described in Using Credit Limits to Constrain Development on Margin. For example, you might run source code analytics every two weeks and go over the results in the bi-weekly demo.
As long as you are mindful of these five aspects (timeless analysis, your mileage may vary, Napoleon’s quip, mythical man-month effects and credit limits on technical debt), combining technical debt figures with productivity data is an effective way to consider the pros and cons of “fix it” versus starting afresh. The combination of the two simplifies a complex investment decision by reducing all considerations to a single common denominator – $$.
Note: This is not a discussion from a value perspective. The software, warts and everything, might (or might not) be valuable to the target customers. The reader is referred to Jim Highsmith‘s analysis of Intrinsic quality versus Extrinsic Quality in Agile Project Management: Creating Innovative Products. See the Cutter Blog post entitled Beyond Scope, Schedule and Cost: Measuring Agile Performance for a short summary of the distinction between the two.
Turning the “Law of Software Physics” Upside Down
Next week, Michael Mah will be presenting new quantitative data on software productivity, cost, time-to-market and quality. Here is an excerpt from the announcement of his talk:
Many companies are adopting Agile practices in an effort to increase project throughput, reduce cost, and improve quality. But are they working? Drawing from industry statistics, Michael answers vital questions about Agile’s effectiveness, which may be turning the “law of software physics” upside down. Until now, there have been predictable relationships among schedule, staffing, and quality; industry data indicates Agile may be changing all this. See productivity findings at 5 Agile companies, and the results for time-to-market, productivity, and quality. Learn the right practices for your environment, including characteristics of successful measurement. See how metrics reveal insights into Agile approaches that are becoming mainstream.
Knowing Michael (and, well, knowing a bit about his latest and greatest findings…) I have every reason to believe Michael will be breaking new grounds in his presentation. Click here and here for details about this exciting presentation.
Should You Invest in This Software?!
Source: martinfowler.com/bliki/TechnicalDebtQuadrant.html
Consider the following scenario: You are a venture capitalist. One of your portfolio companies has been working for a few years on a promising software application. Various surprises with respect to schedule and functionality have been sprung on you along the way. The company now asks for one last shot-in-the-arm in order to get the product out the door, market and sell it. Should you open your wallet one more time to fund this alleged last push?
It is a familiar scenario not only for venture capitalists, but for CEOs, CFOs, general managers and M&A executives. A renowned CEO once told me the following when I pushed my luck with respect to project funding:
Israel, I have a warehouse of software products that never generated a dime for me.
Believe me, this CEO was neither amused nor philosophical…
Code analysis techniques have progressed to the point that the answer to the software investment question for object-oriented code can to a certain extent be determined through quantifying technical debt. For example, assume the following circumstances:
- A company expects to ship 500K lines of code in 6 months.
- The company asks for additional $2M to complete development and make a significant resound in the market.
To assess the investment decision, apply the code analysis techniques described in Using Credit Limits to Constrain Development on Margin to quantify the technical debt. Assuming a debt of $2 per line of code has been identified, the overall technical debt amounts to $1M (2X500K).
The investment decision then is not an incremental $2M decision. It is actually a $3M ($2M+1M) investment decision when the technical debt is taken into account. The technical debt might not need to be paid overnight, but it will have to be paid back over a period of time. The team might not hire additional resources to reduce/eliminate the technical debt, but the team resources dedicated to reducing technical debt will not be available to carry out other assignments. Hence, the opportunity cost ($1M) is real, relevant and should be taken into account.
If you are hesitant to continue investing in this software/team, you stare at a tricky question:
- What will it take to start afresh?
If you decide to make the $3M investment, two operational questions pose themselves:
- How should work on reducing/eliminating technical debt be interleaved with other pressing work such as new functions and features?
- Given a $1M debt on 500K lines of code, can the company indeed ship as expected in 6 months?
We will address these three questions in forthcoming posts in The Agile Executive.
Using Credit Limits to Constrain “Development on Margin”
Buying (stocks) on margin is broadly recognized as a risky investment strategy. Funding long-term investments with short-term debt exposes the investor to margin calls as he/she might not be able to secure more financing when needed. The resultant margin call is never pleasant.
The accrual of technical debt in the course of aggressively developing functions and features is quite a similar phenomenon. The CTO is betting the functionality he/she is developing will pay off before the need to “pay back” the technical debt becomes imperative. The temptation to do so is particularly strong due to the lack of credit limits on technical debt. For all practical purposes the CTO is “developing on margin.”
In his comprehensive studies of the economics of software, Capers Jones has actually put a 3-5 year ceiling on the economical viability of developing on margin:
Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.
As the CEO leading a company, or the venture capitalist funding it, you can restrain development on margin by establishing credit limits. Use a combination of static code analysis with dynamic program analysis to calculate the amount of accrued technical debt in $$ terms. (An illustration of such calculation as well as a breakdown of the technical debt is given in the Sonar chart above). Set a limit (say $0.25 per line of code) on the amount of permitted technical debt. Once the limit is reached, developers are not allowed to continue developing new functionality – they have to first reduce (and hopefully eliminate) their technical debt.
A very simple “Lacmus test” is available to the CEO/VC until the code is instrumented and the analytics illustrated above generated. Ask your CTO about unit test coverage. If the coverage is low (say <30%), chances are the technical debt is high. Whether the CTO realizes it or not, low unit test coverage is a good indicator of technical debt of all kinds. Moreover, the investment required to develop a full-fledged suite of unit tests is often the largest component of the technical debt to be paid back.
The Agile Executive 