The Agile Executive

Making Agile Work

Posts Tagged ‘ROI

Technical Debt: Assessment and Reduction

with one comment

Below is the detailed outline for my August 8, 1:30-5:00PM Technical Debt Workshop in Agile 2011. I look forward to meeting you and interacting with you in the conference before, during and after this workshop!

Best,

Israel

Technical Debt: Assessment and Reduction

Part I: Technical Debt in the Overall Context of the Software Process

  • A Holistic Model of the Software Process
  • Two Aspects of Output
  • Three Aspects of Technical Debt
  • Six Aspects of Software

Part II: What Really is Technical Debt?

  • What’s in a Metaphor?
  • Code Analysis
  • Time is Money
  • Monetizing Technical Debt
  • Typical Stakeholder Dialog Around Technical Debt
  • Analysis of the Cassandra Code
  • Project Dashboard

Part III : Case Study – NotMyCompany, Inc.

  • NotMyCompany Highlights
  • Modernizing Legacy Code
  • Error Proneness

Part IV: The Tricky Nature of Technical Debt

  • The Explicit Form of Technical Debt
  • The Implicit Form of Technical Debt
  • The Strategic Impact of Technical Debt
  • No Good Strategy Following Prolonged Neglect

Part V: Unified Governance

  • How We View Success
  • Three Core Metrics
  • Productivity, Affordability, Risk
  • What is the Real ROI?

Part VI: Process Control Models

  • A Typical Technical Debt Pattern
  • Process Control View of Scrum
  • Integration of Technical Debt in the Agile Process
  • Using Statistical Process Control Methods

Part VII: Reducing Technical Debt

  • A Framework for Thinking about and Acting on Technical Debt Issues
  • Portfolio Governance

Part VIII: Takeaways

  • Nine Simple Takeaway
  • Connecting the dots
Advertisements

Using Credit Limits to Constrain “Development on Margin”

with 9 comments

Buying (stocks) on margin is broadly recognized as a risky investment strategy. Funding long-term investments with short-term debt exposes the investor to margin calls as he/she might not be able to secure more financing when needed. The resultant margin call is never pleasant.

The accrual of technical debt in the course of aggressively developing functions and features is quite a similar phenomenon. The CTO is betting the functionality he/she is developing will pay off before the need to “pay back” the technical debt becomes imperative. The temptation to do so is particularly strong due to the lack of credit limits on technical debt. For all practical purposes the CTO is “developing on margin.”

In his comprehensive studies of the economics of software, Capers Jones has actually put a 3-5 year ceiling on the economical viability of developing on margin:

Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.

As the CEO leading a company, or the venture capitalist funding it, you can restrain development on margin by establishing credit limits. Use a combination of static code analysis with dynamic program analysis to calculate the amount of accrued technical debt in $$ terms. (An illustration of such calculation as well as a breakdown of the technical debt is given in the Sonar chart above). Set a limit (say $0.25 per line of code) on the amount of permitted technical debt. Once the limit is reached, developers are not allowed to continue developing new functionality – they have to first reduce (and hopefully eliminate) their technical debt.

A very simple “Lacmus test” is available to the CEO/VC until the code is instrumented and the analytics illustrated above generated. Ask your CTO about unit test coverage. If the coverage is low (say <30%), chances are the technical debt is high. Whether the CTO realizes it or not, low unit test coverage is a good indicator of technical debt of all kinds. Moreover, the investment required to develop a full-fledged suite of unit tests is often the largest component of the technical debt to be paid back.

Make the Hairs on the Back of Your Neck Stand Up

with 9 comments

Cote sent me the recent CIO Magazine article entitled ERP’s Paralysis Problem and the Repercussions for Business Everywhere. The article discusses the findings from a December 2009 study conducted by IDC and sponsored by ERP vendor Agresso, as follows:

A couple of verbatim responses from respondents should make the hairs on the back of your neck stand up: “Capital expenditure priorities are shifted into IT from other high-payback projects” just to perform necessary ERP changes, noted one respondent. Said another: “Change to ERP paralyzes the entire organization in moving forward in other areas that can bring more value.”

To make doubly certain the message gets across, the article finishes with the following “nocturnal” paragraph:

As the sun finally sets on the first decade in the new millennium, it’s high time we say good night to ERP. A new day will be starting soon, and the blemished legacy and failings of ERP’s nearly four-decade-long reign will be a distant memory.

Maybe. While ERP systems no doubt have their own particular twists, the sorry state of affairs described above is true of various industries that have developed complex software systems over prolonged periods of time. Just in the past few months I have witnessed such situations in banking and health care. In previous life I had been exposed to more of the same in other industries. The software decayed and decayed but technical debt had never been reduced. Consequently, the cost of change, any change, today is horrendous. As Jim Highsmith‘s chart below indicates, “once on the right of the curve, all choices are hard.”

in-can-you-afford-the-software-you-are-developing

In Estimating Software Costs, author Capers Jones quantifies five-year cost of software application ownership (for the vendor). He examines three similar applications, each of nominal size of 1000 function points, as a function of the sophistication of the corresponding projects. The respective life cycle costs are as follows:

  • Lagging projects: $2,316,000
  • Average projects: $1,860,000
  • Leading projects: $1,312,000

Jones goes on to issue the following stern warning:

All known compound objects decay and become more complex with the passage of time unless effort is exerted to keep them repaired and updated. Software is no exception… Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.

Enough, indeed, to make the hairs on the back of your neck stand up…

The Business Value of Agile Software Methods

with 3 comments

I conducted 10 sessions this year on the topic Socializing Agile with Your Executives. The various Agile champions that attended these sessions identified two major obstacles to successful vetting of the topic:

  1. Lack of hard quantitative data.
  2. The “It won’t work here” syndrome.

This post is about the first of the two – lack of hard quantitative data. A follow-on post will deal with the second obstacle.

Michael Mah‘s landmark study How Agile Projects Measure Up, and What This Means to You has been my recommendation for the Agile champion who needs to elevate his/her Agile pitch from qualitative to quantitative. This excellent study in nicely supplemented now by The Business Value of Agile Software Methods: Maximizing ROI with Just-in-Time Processes and Documentation by Rico, Sayani and Sone. It is an excellent fit for the champion promoting Agile for the following reasons:

  1. The book captures, analyzes and synthesizes the results of hundreds of systemic research studies.
  2. It provides data on the various Agile methods without favoring one over another. Furthermore, the authors are quite explicit in stating that it not the method itself but the fit of a method to a company/culture/environment that counts.
  3. It places equal weight on costs and benefits of Agile, thereby giving the reader a good grasp on trade-offs. This grasp can be enhanced through free downloads of cost and benefit spreadsheets from the corresponding Download Resource Center.
  4. A very impressive aspect of this new book is the broad spectrum of the metrics it provides. Just about any business metric your CIO/CFO/CXO might use as the basis for his/her decision-making process, including Real Options Analysis (ROA), is provided. Moreover, the book encourages the use of multiple metrics, clearly indicating the pro and cons of individual metrics. For example:

The business value of Agile methods may be as much as 90% higher than NPV using ROA under extreme market conditions, including high inflation, risk change, and amount of time.

Readers of this blog are familiar with my quip “Don’t take you boss to lunch; take him/her to the daily stand-up meeting.” I would suggest you give The Business Value of Agile Software Methods to your boss at the end of his/her first stand-up meeting. This recommendation is nicely seconded by the following excerpt from Sanjiv Augustine‘s review of the book:

… those looking to build a bullet proof case for agile methods based on solid data and comprehensive research and analysis will find this an invaluable work.

 

Disclosure: Colleague David F. Rico has kindly sent me a free copy of The Business Value of Agile Software Methods.

Cloud Computing Meet the Iterative Requirements of Agile

leave a comment »

It so happened that a key sentence fell between my editing fingers while publishing Annie Shum‘s splendid post Cloud Computing: Agile Deployment for Agile QA Testing. Here is the corrected paragraph with the missing sentence highlighted:

By providing virtually unlimited computing resources on-demand and without up-front CapEx or long-term commitment, QA/load stress and scalability testing in the Cloud is a good starting point. Especially, the flexibility and on-demand elasticity of the Cloud Computing meet the iterative requirements of Agile on an on-going basis. More than likely it will turn out to be one of the least risky but quick ROI pilot Cloud projects for enterprise IT. Case in point, Franz Inc, opted for the Cloud solution when confronted with the dilemma of either abandoning their critical software product testing plan across dozens of machines and databases or procuring new hardware and software that would have been cost-prohibitive. Staging the stress testing study in Amazon’s S3, Franz completed its mission within a few days. Instead of the $100K capital expense for new hardware as well as additional soft costs (such as IT staff and other maintenance costs), the cost of the Amazon’s Cloud services was under $200 and without the penalty of delays in acquisition and configuration.

Reading the whole post with this sentence in mind makes a big difference… And, it is is a little different from my partner Cote‘s perspective on the subject

My apology for the inconvenience.

Israel

Cloud Computing: Agile Deployment for Agile QA Testing

with 9 comments

Annie Shum‘s original thinking has often been quoted in this blog. Her insights are always characterized by seeing the world through the prism of fractals principles.  And, she always relentlessly pursues the connecting of the dots. In this guest post, she examines in an intriguing manner both the tactical and the strategic aspects of large scale testing in the cloud.

Here is Annie:

Cloud Computing: Agile Deployment for Agile QA Testing
Annie Shum twitter@insightspedia
Invariably, the underlying questions at the heart of every technology or business initiative are less about technology but more about the people (generally referred to as the users and consumers in the IT industry). For example, “How does this technology/initiative impact the lives and productivity of people?” or “What happens to the uses/consumers when they are offered new power or a new vehicle of empowerment?” Remarkably, very often the answers to these questions will directly as well as indirectly influence whether the technology/initiative will succeed or fail; whether its impact will be lasting or fleeting ; and whether it will be a strategic game-changer (and transform society) or a tactical short-term opportunity.
One can approach some of the Cloud-friendly applications, e.g. large scale QA and load stress testing in the Cloud, either from a tactical or from a strategic perspective. As aforementioned, the answer to the question “What happens to the uses/consumers when they are offered new power or a new vehicle of empowerment?” can influence whether a new technology initiative will be a strategic game-changer (and transform society) or a tactical short-term opportunity. In other words, think about the bacon-and-eggs analogy where the chicken is involved but the pig is committed. Look for new business models and innovation opportunities by leveraging Cloud Computing that go beyond addressing tactical issues (in particular, trading CapEx for OpEx). One example would be to explore transformative business possibilities stemming from Cloud Computing’s flexible, service-based delivery and deployment options.
Approaching Large-scale QA and Load Stress Testing in the Cloud from a Tactical Perspective
Nowadays, an enterprise organization is constantly under pressure to demonstrate ROI of IT projects. Moreover, they must be able to do this quickly and repeatedly. So as they plan for the transition to the Cloud, it is only prudent that they start small and focus on a target area that can readily showcase the Cloud potential. One of the oft-touted low hanging fruit of Cloud Computing is large scale QA (usability and functionality) testing and application load stress testing in the Cloud. Traditionally, one of the top barriers and major obstacles to comprehensive and high quality (iterative) QA testing is the lack of adequate computing resources. Not only is the shortfall due to budget constraint but also staff scheduling conflicts and the long lead time to procure new hardware/software. This can cause significant product release delays, particularly problematic with new application development under Scrum. An iterative incremental development/management framework commonly used with Agile software development, Scrum requires rapid successive releases in chunks, commonly referred to as splints. Sophisticated Agile users leverage this chunking technique as an affordable experimentation vehicle that can lead to innovationi. However, the downside is each iteration can lead to new testing needs and further compounding the QA woes.
By providing virtually unlimited computing resources on-demand and without up-front CapEx or long-term commitment, QA/load stress testing in the Cloud is a good starting point. More than likely it will turn out to be one of the least risky but quick ROI pilot Cloud projects for enterprise IT. In addition, the flexibility and on-demand elasticity of Cloud Computing meet the iterative nature of Agile on an on-going basis. Case in point, Franz Inc, opted for the Cloud solution when confronted with the dilemma of either abandoning their critical software product testing plan across dozens of machines and databases or procuring new hardware and software that would have been cost-prohibitive. Staging the stress testing study in Amazon’s S3, Franz completed its mission within a few days. Instead of the $100K capital expense for new hardware as well as additional soft costs (such as IT staff and other maintenance costs), the cost of the Amazon’s Cloud services was under $200 and without the penalty of delays in acquisition and configuration.
Approaching Large-scale QA and Load Stress Testing in the Cloud from a Strategic Perspective
While Franz Inc. leverages the granular utility payment model, the avoidance of upfront CapEx and long-term commitment for a one-off project, other entrepreneurs have decided to harness the power of on-demand QA testing in the Cloud as a new business model. Several companies, e.g. SOASTA, LoadStorm and Browsermob are now offering “Testing as a Service” also known as “Reliability as a Service” to enable businesses to test the real-world performance of their Web applications based on a utility-based, on-demand Cloud deployment model. Compared to traditional on-premises enterprise testing tool such as LoadRunner, the Cloud offerings promise to reduce complexity without any software download and up-front licensing cost. In addition, unlike conventional outsourcing models, enterprise IT can retain control of their testing scenarios. This is important because comprehensive QA testing typically requires an iterative process of test-analyze-fix-test cycle that spans weeks if not months.
Notably, all three organizations built their service offerings on Amazon EC2 infrastructure. LoadStorm launched in January 2009 and Browsermob (open source) currently in beta, each enable users to run iterative and parallel load tests directly from its Website. SOASTA, more established than the aforementioned two startups, recently showcases the viability of “Testing as a Service” business model by spawning 650 EC2 Servers to simulate load from two different availability zones to stress test a music-sharing website QTRAX. As reported by Amazon, after a 3-month iterative process of test-analyze-fix-test cycle, QTRAX can now serve 10M hits/hour and handle 500K concurrent users.
The bottom line is there are effectively two different perspectives: tactical (“involved”) versus the strategic (“committed”) and both can be successful. Moreover, the consideration of tactical versus strategic is not a discrete binary choice but a granularity spectrum that accommodates amalgamations of short term and long-term thinking. Every business must decide the best course to meet its goals.
i A shout out to Israel Gat for his insightful comment on chunking as a vehicle for innovation.

Invariably, the underlying questions at the heart of every technology or business initiative are less about technology but, as Clive Thompson of Wired Magazine observed, more about the people (generally referred to as the users and consumers in the IT industry). For example, “How does this technology/initiative impact the lives and productivity of people?” or “What happens to the uses/consumers when they are offered new power or a new vehicle of empowerment?” Remarkably, very often the answers to these questions will directly as well as indirectly influence whether the technology/initiative will succeed or fail; whether its impact will be lasting or fleeting ; and whether it will be a strategic game-changer (and transform society) or a tactical short-term opportunity.

One can approach some of the Cloud-friendly applications, e.g. large scale QA and load stress testing in the Cloud, either from a tactical or from a strategic perspective. As aforementioned, the answer to the question “What happens to the uses/consumers when they are offered new power or a new vehicle of empowerment?” can influence whether a new technology initiative will be a strategic or tactical. In other words, think about the bacon-and-eggs analogy where the chicken is involved but the pig is committed. Look for new business models and innovation opportunities by leveraging Cloud Computing that go beyond addressing tactical issues (in particular, trading CapEx for OpEx). One example would be to explore transformative business possibilities stemming from Cloud Computing’s flexible, service-based delivery and deployment options.

Approaching Large-scale QA and Load Stress Testing in the Cloud from a Tactical Perspective

Nowadays, an enterprise organization is constantly under pressure to demonstrate ROI of IT projects. Moreover, they must be able to do this quickly and repeatedly. So as they plan for the transition to the Cloud, it is only prudent that they start small and focus on a target area that can readily showcase the Cloud potential. One of the oft-touted low hanging fruit of Cloud Computing is large scale QA (usability and functionality) testing and application load stress testing in the Cloud. Traditionally, one of the top barriers and major obstacles to conducting comprehensive, iterative and massively parallel QA test cases is the lack of adequate computing resources. Not only is the shortfall due to budget constraint but also staff scheduling conflicts and the long lead time to procure new hardware/software. This can cause significant product release delays, particularly problematic with new application development under Scrum. An iterative incremental development/management framework commonly used with Agile software development, Scrum requires rapid successive releases in chunks, commonly referred to as splints. Advanced Agile users leverage this chunking technique as an affordable experimentation vehicle that can lead to innovation. However, the downside is the rapid accumulation of new testing needs.

By providing virtually unlimited computing resources on-demand and without up-front CapEx or long-term commitment, QA/load stress and scalability testing in the Cloud is a good starting point. Especially, the flexibility and on-demand elasticity of the Cloud Computing meet the iterative requirements of Agile on an on-going basis. More than likely it will turn out to be one of the least risky but quick ROI pilot Cloud projects for enterprise IT. Case in point, Franz Inc, opted for the Cloud solution when confronted with the dilemma of either abandoning their critical software product testing plan across dozens of machines and databases or procuring new hardware and software that would have been cost-prohibitive. Staging the stress testing study in Amazon’s S3, Franz completed its mission within a few days. Instead of the $100K capital expense for new hardware as well as additional soft costs (such as IT staff and other maintenance costs), the cost of the Amazon’s Cloud services was under $200 and without the penalty of delays in acquisition and configuration.

Approaching Large-scale QA and Load Stress Testing in the Cloud from a Strategic Perspective

While Franz Inc. leverages the granular utility payment model, the avoidance of upfront CapEx and long-term commitment for a one-off project, other entrepreneurs have decided to harness the power of on-demand QA testing in the Cloud as a new business model. Several companies, e.g. SOASTA, LoadStorm and Browsermob are now offering “Testing as a Service” also known as “Reliability as a Service” to enable businesses to test the real-world performance of their Web applications based on a utility-based, on-demand Cloud deployment model. Compared to traditional on-premises enterprise testing tool such as LoadRunner, the Cloud offerings promise to reduce complexity without any software download and up-front licensing cost. In addition, unlike conventional outsourcing models, enterprise IT can retain control of their testing scenarios. This is important because comprehensive QA testing typically requires an iterative process of test-analyze-fix-test cycle that spans weeks if not months.

Notably, all three organizations built their service offerings on Amazon EC2 infrastructure. LoadStorm launched in January 2009 and Browsermob (open source) currently in beta, each enable users to run iterative and parallel load tests directly from its Website. SOASTA, more established than the aforementioned two startups, recently showcases the viability of “Testing as a Service” business model by spawning 650 EC2 Servers to simulate load from two different availability zones to stress test a music-sharing website QTRAX. As reported by Amazon, after a 3-month iterative process of test-analyze-fix-test cycle, QTRAX can now serve 10M hits/hour and handle 500K concurrent users.

The bottom line is there are effectively two different perspectives: tactical (“involved”) versus the strategic (“committed”) and both can be successful. Moreover, the consideration of tactical versus strategic is not a discrete binary choice but a granularity spectrum that accommodates amalgamations of short term and long-term thinking. Every business must decide the best course to meet its goals.

P.S.  A shout out to Israel Gat for not only allowing me to post my piece today but for his always insightful comments in our daily email exchanges.