Archive for the ‘Software Costs’ Category
How Technical Debt Ties to Cloud, Mobile and Social
http://en.wikipedia.org/wiki/File:West_Side_Highway_collapsed_at_14th.jpg
Many years ago, when I came to the US, I was shocked to the core seeing the collapsed West Side Highway in New York City. I simply could not believe that a highway would be neglected to that extent amidst all the affluence of the city. The contrast was too much for me.
Nowadays I often have a deja vu sensation in various technical debt engagements in which I find the code crumbling. This sensation is not so much about what happened (see The Real Cost of a One Trillion Dollars in IT Debt: Part II – The Performance Paradox for an explanation of the economics of the neglect of software maintenance during the past decade), but about the company for which I do the assessment giving up on immense forthcoming opportunities.
Whether you do or do not fully subscribe to the vision of the Internet-of-Things depicted in the figure below, it is fairly safe to assume that your business in the years to come will be much more connected to the outside world than it is now. The enhanced connectivity might come through mobile applications, through social networks or through the cloud. As a matter of fact, it is quite likely to come through a confluence of the three: Cloud, Mobile and Social.
http://en.wikipedia.org/wiki/File:Internet_of_Things.png
In the context of current trends in cloud, mobile and social, your legacy software is like the West Side Highway in New York City. If you maintain it to an acceptable level, it can become the core of two major benefits of much higher connectivity and connectedness in the not-too-far future:
- Through mobile and social your legacy software will enable you to flexibly produce, market and distribute small quantities of whatever your products might need to be in niche markets.
- Through cloud it will enable you to offer these very same products and many others as services.
Conversely, if you consistently neglect to pay back your technical debt, your legacy code is likely to collapse due to the effects of software decay. You certainly will not be able to get it to interoperate with mobile and social networking applications, let alone offer it in the form of cloud services. Nor would you be able to wrap additional services around decaying legacy code. Take a look at the warehousing and distribution services offered by Amazon to get a sense of what this kind of additional services could do for your core business: they will enable you to transform your current business design by adding an Online-to-Offline (O2O) component to it.
What is the fine line differentiating “acceptably maintained” code from toxic code? I don’t think I have conducted a large enough sample of technical debt assessments to provide a statistically significant answer. My hunch is that the magic ceiling for software development in the US is somewhere around $10 per line of code in technical debt. As long as you are under this ceiling you could still pay back your technical debt (or a significant portion of it) in an economically viable manner. Beyond $10 per line of code the decay might prove too high to fix.
Why $10 and not $1 or $100 per line of code? It is a matter of balancing investment versus debt. An average programmer (in the US) with a $100,000 salary would probably be able to produce about 10K lines of Java code per year. The cost of a line of code under these simplistic assumptions is $10. Something is terribly wrong if the technical debt exceeds the cost per line. They call it living on margin.
Action item: CIOs should conduct a technical debt assessment on a representative sample of their legacy code. A board level discussion on the strategic implications for the company is called for if technical debt per line of code exceeds $10. The board discussion should focus on the ability of the company (or lack thereof) to participate in the business tsunami that cloud, mobile and social are likely to unleash.
____________________________________________________________________________________________________
Considering modernization of your legacy code? Let me know if you would like assistance in monetizing your technical debt, devising plans to reduce it and governing the debt reduction process. Click Services for details.
____________________________________________________________________________________________________
Should You Ship This Code Before Reducing Technical Debt?!
Source: JulesH, Wikipedia, A control flow graph of a simple function
Technical debt is usually perceived as a measure of expediency. You borrow a little (time) with the intent of paying it back as soon as possible. To quote Ward Cunnigham:
Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.
As is often the case with financial debt, technical debt accrues with compound interest. Once it reaches a certain level (e.g. $1 per line of code) you stare at a difficult question:
Should I ship this code before reducing the accrued technical debt?!
The Figure below, taken from An Objective Measure of Code Quality by Mark Dixon, answers the question with respect to one important component of technical debt – cyclomatic complexity. Once complexity per source code file exceeds 74, the file is for most practical purposes guaranteed to contain errors. Some of the errors in such a file might be trivial. However, a 2007 study by Capers Jones indicates about a third of the errors found in released code are likely to be serious enough to stop an application from running or create erroneous outputs.
To answer the question cited above – Should You Ship This Software Before Reducing Technical Debt?! – examine both cost and risk for the number of error-prone files you are about to unleash:
- The economics of defect removal clearly favor early defect removal over late defect removal. The cost of removal grows exponentially as function of time.
- Brand risk should be first and foremost on your mind. If complexity figures higher than 74 per file are more of the norm than the exception, you are quite likely to tarnish your image due to poor quality.
If you decide to postpone the release date until the technical debt has been reduced, you can apply yourself to technical debt reduction in a biggest-bang-for-the-buck manner. The analysis of complexity can identify the hot spots in your code, giving you a de-facto roadmap you would be wise to follow.
Conversely, if you opt to ship the code without reducing technical debt, you might lose this degree of freedom to prioritize your “fix it” work. Customer situations and pressures might force you to attend to fixing modules that do not necessarily provide as much bang for the buck.
Postscript: Please note that the discussion in this post is strictly limited to intrinsic quality. It does not address at all extrinsic quality. In other words, reducing/eliminating technical debt does not guarantee that the customer will find the code valuable. I would suggest reading Beyond Scope, Schedule and Cost: Measuring Agile Performance in the Cutter Blog for a more detailed analysis of the distinction between the two.
Erratum: The figure above is actually taken from a blog post on the Mark Dixon paper cited in my post. See McCabe Cyclomatic Complexity: the proof is in the pudding. My apology for the error.
Turning the “Law of Software Physics” Upside Down
Next week, Michael Mah will be presenting new quantitative data on software productivity, cost, time-to-market and quality. Here is an excerpt from the announcement of his talk:
Many companies are adopting Agile practices in an effort to increase project throughput, reduce cost, and improve quality. But are they working? Drawing from industry statistics, Michael answers vital questions about Agile’s effectiveness, which may be turning the “law of software physics” upside down. Until now, there have been predictable relationships among schedule, staffing, and quality; industry data indicates Agile may be changing all this. See productivity findings at 5 Agile companies, and the results for time-to-market, productivity, and quality. Learn the right practices for your environment, including characteristics of successful measurement. See how metrics reveal insights into Agile approaches that are becoming mainstream.
Knowing Michael (and, well, knowing a bit about his latest and greatest findings…) I have every reason to believe Michael will be breaking new grounds in his presentation. Click here and here for details about this exciting presentation.
Should You Invest in This Software?!
Source: martinfowler.com/bliki/TechnicalDebtQuadrant.html
Consider the following scenario: You are a venture capitalist. One of your portfolio companies has been working for a few years on a promising software application. Various surprises with respect to schedule and functionality have been sprung on you along the way. The company now asks for one last shot-in-the-arm in order to get the product out the door, market and sell it. Should you open your wallet one more time to fund this alleged last push?
It is a familiar scenario not only for venture capitalists, but for CEOs, CFOs, general managers and M&A executives. A renowned CEO once told me the following when I pushed my luck with respect to project funding:
Israel, I have a warehouse of software products that never generated a dime for me.
Believe me, this CEO was neither amused nor philosophical…
Code analysis techniques have progressed to the point that the answer to the software investment question for object-oriented code can to a certain extent be determined through quantifying technical debt. For example, assume the following circumstances:
- A company expects to ship 500K lines of code in 6 months.
- The company asks for additional $2M to complete development and make a significant resound in the market.
To assess the investment decision, apply the code analysis techniques described in Using Credit Limits to Constrain Development on Margin to quantify the technical debt. Assuming a debt of $2 per line of code has been identified, the overall technical debt amounts to $1M (2X500K).
The investment decision then is not an incremental $2M decision. It is actually a $3M ($2M+1M) investment decision when the technical debt is taken into account. The technical debt might not need to be paid overnight, but it will have to be paid back over a period of time. The team might not hire additional resources to reduce/eliminate the technical debt, but the team resources dedicated to reducing technical debt will not be available to carry out other assignments. Hence, the opportunity cost ($1M) is real, relevant and should be taken into account.
If you are hesitant to continue investing in this software/team, you stare at a tricky question:
- What will it take to start afresh?
If you decide to make the $3M investment, two operational questions pose themselves:
- How should work on reducing/eliminating technical debt be interleaved with other pressing work such as new functions and features?
- Given a $1M debt on 500K lines of code, can the company indeed ship as expected in 6 months?
We will address these three questions in forthcoming posts in The Agile Executive.
Using Credit Limits to Constrain “Development on Margin”
Buying (stocks) on margin is broadly recognized as a risky investment strategy. Funding long-term investments with short-term debt exposes the investor to margin calls as he/she might not be able to secure more financing when needed. The resultant margin call is never pleasant.
The accrual of technical debt in the course of aggressively developing functions and features is quite a similar phenomenon. The CTO is betting the functionality he/she is developing will pay off before the need to “pay back” the technical debt becomes imperative. The temptation to do so is particularly strong due to the lack of credit limits on technical debt. For all practical purposes the CTO is “developing on margin.”
In his comprehensive studies of the economics of software, Capers Jones has actually put a 3-5 year ceiling on the economical viability of developing on margin:
Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.
As the CEO leading a company, or the venture capitalist funding it, you can restrain development on margin by establishing credit limits. Use a combination of static code analysis with dynamic program analysis to calculate the amount of accrued technical debt in $$ terms. (An illustration of such calculation as well as a breakdown of the technical debt is given in the Sonar chart above). Set a limit (say $0.25 per line of code) on the amount of permitted technical debt. Once the limit is reached, developers are not allowed to continue developing new functionality – they have to first reduce (and hopefully eliminate) their technical debt.
A very simple “Lacmus test” is available to the CEO/VC until the code is instrumented and the analytics illustrated above generated. Ask your CTO about unit test coverage. If the coverage is low (say <30%), chances are the technical debt is high. Whether the CTO realizes it or not, low unit test coverage is a good indicator of technical debt of all kinds. Moreover, the investment required to develop a full-fledged suite of unit tests is often the largest component of the technical debt to be paid back.
Standish Group Chaos Reports Revisited
The Standish Group “Chaos” reports have been mentioned in various posts in this blog and elsewhere. The following figure from the 2002 study is quite representative of the data provided in the Standish annual surveys of the state of software projects:
The January/February 2010 issue of IEEE Software features an article entitled The Rise and Fall of the Chaos Report Figures. The authors – J. Laurenz Eveleens and Chris Verhoef of the VU University, Amsterdam – give the following summary of their findings:
In 1994, Standish published the Chaos report that showed a shocking 16 percent project success. This and renewed figures by Standish are often used to indicate that project management of application software development is in trouble. However, Standish’s definitions have four major problems. First, they’re misleading because they’re based solely on estimation accuracy of cost, time, and functionality. Second, their estimation accuracy measure is one-sided, leading to unrealistic success rates. Third, steering on their definitions perverts good estimation practice. Fourth, the resulting figures are meaningless because they average numbers with an unknown bias, numbers that are introduced by different underlying estimation processes. The authors of this article applied Standish’s definitions to their own extensive data consisting of 5,457 forecasts of 1,211 real-world projects, totaling hundreds of millions of Euros. The Standish figures didn’t reflect the reality of the case studies at all.
I will leave it to the reader to draw his/her conclusion with respect to the differences between the Standish Group and the authors. I would, however, quote Jim Highsmith‘s deep insight on the value system within its context we measure performance. Following excerpt is from Agile Project Management: Creating Innovative Products:
It we are ultimately to gain the full range of benefits of agile methods, if we are ultimately to grow truly agile, innovative organizations, then, as these stories show, we will have to alter our performance management systems…. We have to be as innovative with our measurement systems as we are with our development methodology.
See pp. 335-358 of Jim’s book for details on transforming performance management systems. His bottom line is elusively simple:
The Standish data are NOT a good indicator of poor software development performance. However, they ARE an indicator of systemic failure of our planning and measurement processes.
Jim is referring to the standard definition of project “success” – on time, on budget, all specified features.
I will be working with a client to carry out the performance management ideas articulated by Jim later this month. Jim indicated he has a customer engagement in February where he expects to learn about interesting ways in which the client is using the Agile Triangle (which is conceptually quite related to the fundamental question what to measure). Client confidentiality permitting, I am confident we will soon be able to brief readers of The Agile Executive on our progress.
Is There Something Inherently un-Agile About ERP Software?
A reader of the post Make the Hairs on the Back of Your Neck Stand Up posed the following question:
I wonder if there’s something inherently un-Agile (and thus, unable to change fast enough to meet new business demands) about older enterprise software, or just ERP software?
The answer IMHO is size. To quote Capers Jones:
Since defect potentials tend to rise with the overall size of the application, and since defect removal efficiency levels tend to decline with the overall size of the application, the overall volume of latent defects delivered with the application rises with size. This explains why super-large applications in the range of 100,000 function points, such as Microsoft Windows and many enterprise resource planning (ERP) applications may require years to reach a point of relative stability. These large systems are delivered with thousands of latent bugs or defects.
The phenomenon described by Jones is often exacerbated through the “ship more infrequently” syndrome. IMVU’s Timothy Fritz describes it as follows:
While this may decrease downtime (things break and you roll back), the cost on development time from work and rework will be large, and mistakes will continue to slip through. The natural tendency will be to ship even more infrequently, until you aren’t shipping at all. Then you’ve gone and forced yourself into a total rewrite. Which will also be doomed.
You might choose to reduce your technical debt instead of trying total rewrite. Chances are you will struggle to find Elbow Room for Handling Technical Debt. My hunch is that once >50% of development resources are assigned to maintaining the software on an on-going basis, it is time to get into refactoring big time. If you don’t, sooner or later you are likely to find you can’t afford the software you developed.
Make the Hairs on the Back of Your Neck Stand Up
Cote sent me the recent CIO Magazine article entitled ERP’s Paralysis Problem and the Repercussions for Business Everywhere. The article discusses the findings from a December 2009 study conducted by IDC and sponsored by ERP vendor Agresso, as follows:
A couple of verbatim responses from respondents should make the hairs on the back of your neck stand up: “Capital expenditure priorities are shifted into IT from other high-payback projects” just to perform necessary ERP changes, noted one respondent. Said another: “Change to ERP paralyzes the entire organization in moving forward in other areas that can bring more value.”
To make doubly certain the message gets across, the article finishes with the following “nocturnal” paragraph:
As the sun finally sets on the first decade in the new millennium, it’s high time we say good night to ERP. A new day will be starting soon, and the blemished legacy and failings of ERP’s nearly four-decade-long reign will be a distant memory.
Maybe. While ERP systems no doubt have their own particular twists, the sorry state of affairs described above is true of various industries that have developed complex software systems over prolonged periods of time. Just in the past few months I have witnessed such situations in banking and health care. In previous life I had been exposed to more of the same in other industries. The software decayed and decayed but technical debt had never been reduced. Consequently, the cost of change, any change, today is horrendous. As Jim Highsmith‘s chart below indicates, “once on the right of the curve, all choices are hard.”
In Estimating Software Costs, author Capers Jones quantifies five-year cost of software application ownership (for the vendor). He examines three similar applications, each of nominal size of 1000 function points, as a function of the sophistication of the corresponding projects. The respective life cycle costs are as follows:
- Lagging projects: $2,316,000
- Average projects: $1,860,000
- Leading projects: $1,312,000
Jones goes on to issue the following stern warning:
All known compound objects decay and become more complex with the passage of time unless effort is exerted to keep them repaired and updated. Software is no exception… Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.
Enough, indeed, to make the hairs on the back of your neck stand up…
The Agile Executive 
Cutter’s Technical Debt Assessment and Valuation Service
with 3 comments
The Cutter Consortium has announced the availability of the Technical Debt Assessment and Valuation Service. The service combines static code analytics with dynamic program analytics to give the client “x-rays” of the software being examined at any desired granularity – from the whole project portfolio to a single instruction. It breaks down technical debt into the areas of coverage, complexity, duplication, violations and comments. Clients get an aggregate dollar figure for “paying back” debt that they can then plug into their financial models to objectively analyze their critical software assets. Based on these metrics, they can make the best decisions about their ongoing strategy for the software development effort under scrutiny.
This new service is an important addition to the enlightened software governance framework that Jim Highsmith, Michael Mah and I have been thinking about and contributing to for sometime now (see Beyond Scope, Schedule and Cost: Measuring Agile Performance and Quantifying the Start Afresh Option). The heart of both the technical debt service and the enlightened governance framework is captured by the following words from the press release:
By boiling down technical debt to dollars and tying it to cost and value, the service enables a metrics-driven governance framework for the use of five major constituencies, as follows:
It should finally be pointed out that the technical debt assessment service and the governance framework it enables are applicable to any software method. They can be used to:
Forthcoming Cutter Executive Reports, Executive Updates and Email Advisors on the technical debt service are restricted to Cutter clients. As appropriate, I will publish the latest and greatest news on the subject in the Cutter Blog (which is an open forum I highly recommend).
Acknowledgements: I would like to wholeheartedly thank the following colleagues for inspiring, enlightening and supporting me during the preparation of the service:
Written by israelgat
May 5, 2010 at 4:40 am
Posted in Agile Performance Management, Companies, Events, Performance Measurement, Software Costs
Tagged with Anne Mullaney, CEO, Chris Sterling, Cindy Swain, CIO, Comments, Complexity, Coverage, CTO, Cutter Consortium, Duplication, Governance Framework, Industry Norms, Israel Gat, IT Operations, Jennifer Flaxman, Jim Highsmith, John Heintz, Jonathon Golden, Kara Letourneau, Karen Coburn, Ken Collier, Kim Leonard, M&A, Michael Mah, NPV, Paying Back, Software Method, Technical Debt, Valuation, Venture Capitalist, Violation