The Agile Executive

Making Agile Work

Posts Tagged ‘Metrics

Technical Debt at Cutter

leave a comment »

No, this post is not about technical debt we identified in the software systems used by the Cutter Consortium to drive numerous publications, events and engagements. Rather, it is about various activities carried out at Cutter to enhance the state of the art and make the know-how available to a broad spectrum of IT professionals who can use technical debt engagements to pursue technical and business opportunities.

The recently announced Cutter Technical Debt Assessment and Valuation service is quite unique IMHO:

  1. It is rooted in Agile principles and theory but applicable to any software method.
  2. It combines the passion, empowerment and collaboration of Agile with the rigor of quantified performance measures, process control techniques and strategic portfolio management.
  3. It is focused on enlightened governance through three simple metrics: net present value, cost and technical debt.

Here are some details on our current technical debt activities:

  1. John Heintz joined the Cutter Consortium and will be devoting a significant part of his time to technical debt work. I was privileged and honored to collaborate with colleagues Ken Collier, Jonathon Golden and Chris Sterling in various technical debt engagements. I can’t wait to work with them, John and other Cutter consultants on forthcoming engagements.
  2. John and I will be jointly presenting on the subject Toxic Code in the Agile Roots conference next week. In this presentation we will demonstrate how the hard lesson learned during the sub-prime loans crisis apply to software development. For example, we will be discussing development on margin…
  3. My Executive Report entitled Revolution in Software: Using Technical Debt Techniques to Govern the Software Development Process will be sent to Cutter clients in the late June/early July time-frame. I don’t think I had ever worked so hard on a paper. The best part is it was labor of love….
  4. The main exercise in my Agile 2010 workshop How We Do Things Around Here in Order to Succeed is about applying Agile governance through technical debt techniques across organizations and cultures. Expect a lot of fun in this exercise no matter what your corporate culture might be – Control, Competence, Cultivation or Collaboration.
  5. John and I will be doing a Cutter webinar on Reining in Technical Debt on Thursday, August 19 at 12 noon EDT. Click here for details.
  6. A Cutter IT Journal (CITJ) on the subject of technical debt will be published in the September-October time-frame. I am the guest editor for this issue of the CITJ. We have nine great contributors who will examine technical debt from just about every possible perspective. I doubt that we have the ‘real estate’ for additional contributions, but do drop me a note if you have intriguing ideas about technical debt. I will do my best to incorporate your thoughts with proper attribution in my editorial preamble for this issue of the CITJ.
  7. Jim Highsmith and I will jointly deliver a seminar entitled Technical Debt Assessment: The Science of Software Development Governance in the forthcoming Cutter Summit. This is really a wonderful ‘closing of the loop’ for me: my interest in technical debt was triggered by Jim’s presentation How to Be an Agile Leader in the Agile 2006 conference.

Standing back to reflect on where we are with respect to technical debt at Cutter, I see a lot of things coming nicely together: Agile, technical debt, governance, risk management, devops, etc. I am not certain where the confluence of all these threads, and possibly others, might lead us. However, I already enjoy the adrenaline rush this confluence evokes in me…

How Many Metrics do You Need to Effectively Govern the Software Process?

with 4 comments

A Simple Metrics-Driven Software Governance Framework Based on Jim Highsmith’s Agile Triangle Framework

In my recent Cutter Blog post entitled Three Governance Metrics I recommended using just three metrics:

  • Value
  • Cost
  • Technical debt

The heart pf this recommendation is that all three can be expressed in dollar terms as depicted in the figure above. An apples-to-apples comparison is made through the common denominator – $$. For example, something is likely to be either technically, methodically or governance-wise wrong if the technical debt figure exceeds the cost figure for a prolonged period of time. One can actually characterize such a situation as accruing debt faster than building equity.

I am often asked about adding metrics to this simple governance framework. For example, should not productivity be included in the framework?

‘Less is more’ is my usual response to such questions. IMHO value, cost and technical debt address the most important high level governance considerations:

  • Value –> Why are we doing the project?
  • Cost –> Can we afford the project?
  • Technical debt –> Is the execution risk acceptable?

Please pay special attention to the unit of measure of any metric you might add  to this simple governance framework. As long as the metric is a dollar-based metric, the cohesion of the governance framework can be maintained. However, metrics which are not expressed in dollars will probably superimpose other frameworks on top of the simple governance framework. For example, you introduce a programming framework if you add a productivity metric which is measured in function points per man month. Sponsors who govern using value, cost, technical debt and productivity will need to mentally alternate between the simple governance framework and the programming framework whenever they try to combine the productivity metric with any of the other three metrics.

Toxic Code

with 2 comments

“[The 100% loan-to-value subprime loan is] the most dangerous product in existence and there can be nothing more toxic…”

This quip by Angelo Mozilo, founder of Countrywide Financial, led me to coin the term “toxic code.” The code stops being an asset when it accumulates technical debt equal to the value it is expected to generate. As matter of fact, the code could become a liability when the cost to “pay back” the technical debt exceeds the revenues the code would generate. Such situations have been known to happen more often than we might realize. They occur, for example, when numerous customers develop elaborate business processes around poor quality enterprise software.

I will be exploring the topic in my May 27, 2010 presentation at The Path to Agility conference in Columbus, OH. Here is the abstract of my talk:

Technical debt had originally been conceived as an expediency measure – “a little debt speeds development so long as it is paid back promptly with a rewrite.” However, like financial debt, unrestrained borrowing can lead to a broad spectrum of difficulties, from collapsed roadmaps to inability to respond to customer problems in a timely manner, and anything in between.

Recent advances in source code analysis enable us to quantify technical debt and express it in $$ terms. By so doing, the software development process can be governed with unprecedented effectiveness. It is possible to constrain the “development on margin” mal-practice and avoid the toxic code phenomenon: technical-debt-to-value ratio of 100%. Moreover, even toxic code can ultimately be “marked-to-market” by reducing/eliminating technical debt.

The combination of Agile refactoring practices with technical debt analytics brings rigor to the software development process through:

  • Providing quantifiable data about the overall state of the code and its value
  • Highlighting error-prone modules in the code and offering guidance to fixing them in a biggest-bang for-the-buck manner
  • Pinpointing technical issues all the way down to the individual line of code level
  • Balancing the technical debt work stream vis-a-vis other work streams

In the course of managing software development in this manner, your team will improve its design, coding, testing and project management skills. Ultimately, these improvements are the best antidote against accrual of technical debt in the future.

Written by israelgat

April 19, 2010 at 4:45 am

Can Technical Debt Constitute a Breach of Implied Warranties?

with 12 comments

POGO_film_diffs by Dancing Lemur.

Photo credit: Dancing Lemur (Flickr)

Cunningham’s quip “A little debt speeds development so long as it is paid back promptly with a rewrite” is intuitively very clear. We are talking about short-term debt which will be reduced, and hopefully eliminated in entirety, at the earliest possible time.

The question this post addresses is what happens when the expected short-term technical debt becomes a significant long-term debt? Specifically, can technical debt under some conditions constitute a breach of implied warranties?

In his InformIT article Don’t “Enron” Your Software Project, Aaron Erickson coined the term “Technical Fraud” and connected it to Lemmon Laws:

As a reaction to seeing this condition and its deleterious effects, I coined the term technical fraud to refer to the practice of incurring unmanaged and hidden technical debt. Many U.S. states have “lemon laws” that make it illegal to knowingly sell someone a car that has undisclosed maintenance problems. Selling a “lemon” is a fraudulent practice in the world of cars, and it should be considered as such in the world of software.

It is a little tricky (though not impossible – see Using Credit limits to Constrain Development on Margin) to define the precise point where technical debt becomes “unmanaged.” One needs to walk a fine line between technical/methodical incompetence and resource availability to determine technical fraud. For example, if your code has 35% coverage, is it or is not unmanaged? Does the answer to this question change if your cyclomatic complexity per class exceeds 30? I would think the courts might be divided for a very long time on the question when does hidden technical debt represent a fraudulent misrepresentation.

One component  of technical debt deserves special attention in the context of this post. I am referring to the conscious decision not to do unit testing at all.

Best I understand it, the rationale for not “bothering” with unit testing is a variant of the old ploy “we do not have time for testing here.” It is a resource allocation strategy that bets on the code being miraculously bug-free. Some amount of functional testing is done out of necessity – the code in customers hands needs to function as proclaimed.  But, the pieces of code  from which functionality is constructed are not subject to direct rigorous testing. The individual units of code will be indirectly exercised in some manner through functional testing, but not in a systemic manner to verify and validate correctness of the units of code per se.

Such a conscious decision IMHO indicates no intention to pay back this category of technical debt – unit test coverage. It is therefore quite incompatible with the nature of an implied warranty:

An implied warranty is as an unstated promise, assumed by the law in most sales transactions, that the product will be of at least average quality and will do what the average customer would expect it to do  [The Reader’s Digest Legal Questions & Answers Book]

To #1 defense open to a software vendor who gets sued over lack of unit testing is that a fair average quality of software can be attained without any unit testing. As a programmer, I would think such defense would fly at the teeth of the availability since 1987 of the IEEE Standard for Software Unit Testing.

It is fascinating to note the duality between contracts and programming.  For the programmer who follows the tenets of design by contract, “a unit test provides a strict, written contract that the piece of code must satisfy…”

Disclaimer: I am not an expert in the law. The opinion expressed in this post merely represents my layman’s understanding of  principles of contract law that might be applicable to technical debt situations.

Should You Ship This Code Before Reducing Technical Debt?!

with 8 comments

File:Control flow graph of function with loop and an if statement without loop back.svg

Source: JulesH, Wikipedia, A control flow graph of a simple function

Technical debt is usually perceived as a measure of expediency. You borrow a little (time) with the intent of paying it back as soon as possible. To quote Ward Cunnigham:

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.

As is often the case with financial debt, technical debt accrues with compound interest. Once it reaches a certain level (e.g. $1 per line of code) you stare at a difficult question:

Should I ship this code before reducing the accrued technical debt?!

The Figure below, taken from An Objective Measure of Code Quality by Mark Dixon, answers the question with respect to one important component of technical debt – cyclomatic complexity. Once complexity per source code file exceeds 74, the file is for most practical purposes guaranteed to contain errors. Some of the errors in such a file might be trivial. However, a 2007 study by Capers Jones indicates about a third of the errors found in released code are likely to be serious enough to stop an application from running or create erroneous outputs.


To answer the question cited above – Should You Ship This Software Before Reducing Technical Debt?! –  examine both cost and risk for the number of error-prone files you are about to unleash:

  • The economics of defect removal clearly favor early defect removal over late defect removal. The cost of removal grows exponentially as function of time.
  • Brand risk should be first and foremost on your mind. If complexity figures higher than 74 per file are more of the norm than the exception, you are quite likely to tarnish your image due to poor quality.

If you decide to postpone the release date until the technical debt has been reduced, you can apply yourself to technical debt reduction in a biggest-bang-for-the-buck manner. The analysis of complexity can identify the hot spots in your code, giving you a de-facto roadmap you would be wise to follow.

Conversely, if you opt to ship the code without reducing technical debt, you might lose this degree of freedom to prioritize your “fix it” work.  Customer situations and pressures might force you to attend to fixing modules that do not necessarily provide as much bang for the buck.

Postscript: Please note that the discussion in this post is strictly limited to intrinsic quality. It does not address at all extrinsic quality. In other words, reducing/eliminating technical debt does not guarantee that the customer will find the code valuable. I would suggest reading Beyond Scope, Schedule and Cost: Measuring Agile Performance in the Cutter Blog for a more detailed analysis of the distinction between the two.

Erratum: The figure above is actually taken from a blog post on the Mark Dixon paper cited in my post. See McCabe Cyclomatic Complexity: the proof is in the pudding. My apology for the error.

How to Initiate a DevOps Project

with 4 comments

17th/21st Lancers c. 1922-1929 "THE FIGHTING SPIRIT!" by sunnybrook100 - One Million Views!.

Source: 17th/21st Lancers c. 1922-1929 “THE FIGHTING SPIRIT!”

Agile consultants on a development project often start by helping the team construct a backlog. The task is sufficiently concrete to get all stakeholders (product management, project management, development, test, any others) on a collaborative track through the creation of a key artifact. The backlog establishes a base line for the tasks to be carried out in the project.

For a DevOps project, start by establishing the technical debt of the software to be released to operations. By so doing you build the foundations for collaboration between development and operations through shared data. In the DevOps context, the technical debt data form the basis for the creation and grooming of  a unified backlog which includes various user stories from operations.

Apply the same approach when you are fortunate to be able to include folks from operations in the Agile team from the very beginning. You start with zero technical debt, but you track it on an ongoing basis and include the corresponding “fix-it” stories in the backlog as you accrue the debt. Running technical debt analytics on the source code every two weeks is a good practice to follow.

As the head of development, you might not be comfortable sharing technical debt data. This being the case, you are not ready for DevOps.

Using Credit Limits to Constrain “Development on Margin”

with 9 comments

Buying (stocks) on margin is broadly recognized as a risky investment strategy. Funding long-term investments with short-term debt exposes the investor to margin calls as he/she might not be able to secure more financing when needed. The resultant margin call is never pleasant.

The accrual of technical debt in the course of aggressively developing functions and features is quite a similar phenomenon. The CTO is betting the functionality he/she is developing will pay off before the need to “pay back” the technical debt becomes imperative. The temptation to do so is particularly strong due to the lack of credit limits on technical debt. For all practical purposes the CTO is “developing on margin.”

In his comprehensive studies of the economics of software, Capers Jones has actually put a 3-5 year ceiling on the economical viability of developing on margin:

Indeed, the economic value of lagging applications is questionable after about three to five years. The degradation of initial structure and the increasing difficulty of making updates without “bad fixes” tends towards negative returns on investment (ROI) within a few years.

As the CEO leading a company, or the venture capitalist funding it, you can restrain development on margin by establishing credit limits. Use a combination of static code analysis with dynamic program analysis to calculate the amount of accrued technical debt in $$ terms. (An illustration of such calculation as well as a breakdown of the technical debt is given in the Sonar chart above). Set a limit (say $0.25 per line of code) on the amount of permitted technical debt. Once the limit is reached, developers are not allowed to continue developing new functionality – they have to first reduce (and hopefully eliminate) their technical debt.

A very simple “Lacmus test” is available to the CEO/VC until the code is instrumented and the analytics illustrated above generated. Ask your CTO about unit test coverage. If the coverage is low (say <30%), chances are the technical debt is high. Whether the CTO realizes it or not, low unit test coverage is a good indicator of technical debt of all kinds. Moreover, the investment required to develop a full-fledged suite of unit tests is often the largest component of the technical debt to be paid back.

Use the Agile Triangle Instead of the Balanced Scorecard

with one comment

As the name implies, the Balanced Scorecard strives to strike a balance between various performance measures.  When Financial, CustomerBusiness Processes and Learning and Growth measures are presented together, as in Figure 1 below, the Balanced Scorecard allows managers to view the company from several perspectives at once.

Figure 1 – The Balanced Scorecard (source:  Trump University)

Likewise, the Agile Triangle depicted in Figure 2,  presents in a single “dashboard” the three dimensions critical to Agile performance measurement – Value, Quality and Constraints. Just as in the Balanced Scorecard, it is easy to see imbalances between the three, to respond to them and to restore balance. For example, the tendency to produce more and more lines of code is held in check through the quality metrics.

Figure 2 – The Agile Triangle  (based on Figure 1-3 in Jim Highsmith‘s Agile Project Management: Creating Innovative Products.)

My recommendation to clients who do Agile as a strategic initiative is to drop the Balanced Scorecard and use the Agile Triangle instead. There is precious little, if any, to be gained by using the two in parallel. As a matter of fact, one could easily interfere with the other.

The Learning and Growth dimension of the Balanced Scorecard, which does not explicitly show in the Agile Triangle, is, of course, important. As part of an Agile initiative I would expect Agile proficiency to be closely observed. However, I would not include it explicitly in a system based on the Agile Triangle. Agile proficiency is not and end to itself. If the outputs and outcomes we measure through the Agile Triangle are unsatisfactory over a prolonged period of time, a close examination of the way Agile is practiced is called for.

The Business Value of Agile Software Methods

with 3 comments

I conducted 10 sessions this year on the topic Socializing Agile with Your Executives. The various Agile champions that attended these sessions identified two major obstacles to successful vetting of the topic:

  1. Lack of hard quantitative data.
  2. The “It won’t work here” syndrome.

This post is about the first of the two – lack of hard quantitative data. A follow-on post will deal with the second obstacle.

Michael Mah‘s landmark study How Agile Projects Measure Up, and What This Means to You has been my recommendation for the Agile champion who needs to elevate his/her Agile pitch from qualitative to quantitative. This excellent study in nicely supplemented now by The Business Value of Agile Software Methods: Maximizing ROI with Just-in-Time Processes and Documentation by Rico, Sayani and Sone. It is an excellent fit for the champion promoting Agile for the following reasons:

  1. The book captures, analyzes and synthesizes the results of hundreds of systemic research studies.
  2. It provides data on the various Agile methods without favoring one over another. Furthermore, the authors are quite explicit in stating that it not the method itself but the fit of a method to a company/culture/environment that counts.
  3. It places equal weight on costs and benefits of Agile, thereby giving the reader a good grasp on trade-offs. This grasp can be enhanced through free downloads of cost and benefit spreadsheets from the corresponding Download Resource Center.
  4. A very impressive aspect of this new book is the broad spectrum of the metrics it provides. Just about any business metric your CIO/CFO/CXO might use as the basis for his/her decision-making process, including Real Options Analysis (ROA), is provided. Moreover, the book encourages the use of multiple metrics, clearly indicating the pro and cons of individual metrics. For example:

The business value of Agile methods may be as much as 90% higher than NPV using ROA under extreme market conditions, including high inflation, risk change, and amount of time.

Readers of this blog are familiar with my quip “Don’t take you boss to lunch; take him/her to the daily stand-up meeting.” I would suggest you give The Business Value of Agile Software Methods to your boss at the end of his/her first stand-up meeting. This recommendation is nicely seconded by the following excerpt from Sanjiv Augustine‘s review of the book:

… those looking to build a bullet proof case for agile methods based on solid data and comprehensive research and analysis will find this an invaluable work.


Disclosure: Colleague David F. Rico has kindly sent me a free copy of The Business Value of Agile Software Methods.

The Executive’s Workshop for Scaling Agile to The Enterprise

with one comment

Readers of this blog are well aware of my keen interest in enterprise level Agile. I am now offering a specialized workshop for executives on this topic.

The Executive’s Workshop for Scaling Agile to The Enterprise

This one day workshop and free follow-on coaching service prepares executives for their roles in large-scale Agile implementation.

This workshop is ideal for building a shared understanding of your company’s Agile goals and practices amongst members of the leadership team. It illustrates how executives could/should engage in the Agile process in a meaningful manner, and includes strategies for addressing common challenges.  Your team will see how to govern Agile effectively, and most importantly, you’ll learn proven practices for attaining the operational, financial and business benefits of a successful enterprise-level Agile implementation.


Agile is shown to cut the cost, improve the flexibility and shorten time-to-market of software-driven projects. Upon completion of this service, executive teams will be able to:

  • Scale Agile to the enterprise level
  • Minimize risks associated with large-scale Agile rollout
  • Apply Agile practices in development and beyond
  • Galvanize the team around a shared, cross-functional Agile vision


The Executive’s Workshop for Scaling Agile to The Enterprise service is divided into three parts, each designed to help company leaders accelerate their adoption of Agile.

Part I: Preparation via phone interviews and web-based coaching. The workshop leader works with your executive team to gather context, discuss logistics and focus the on-site workshop on your needs.

Part II: One day on-site workshop is delivered through combination of presentation, examples, exercises and participant discussion.

Part III: Free telephone coaching and mentoring with the Workshop Leader for six months after the workshop. The objective is to help executives respond effectively to the challenges they encounter in the course of implementing Agile.

On-Site Workshop Details

Leading an enterprise adoption of Agile requires that you understand the key concepts, principles and practices of Agile without getting bogged down in technical details.  You must learn techniques for handling the expected “noise” associated with organizational change while identifying the critical tasks needing your attention and leadership to succeed. The workshop is designed to address these challenges with a minimal investment of time.

Here is an overview of the key topics you will add to your experience set:

Explaining the Rationale for Agile to Your Company

  • Why now?
  • What is the state of the art in Agile and what is our goal
  • Expected return on our Agile investment

How The Agile Process Fits into Your Company:

  • Understanding Agile as an example of  other common,  iterative, quality-oriented processes
  • How Agile works with other software development life cycle processes
  • How to run a heterogeneous software development environment that mixes Waterfall, Agile and other methods
  • Connecting Agile to your budgeting process
  • How to perform governance and portfolio management with Agile

How to Implement Agile:

  • Choosing suitable projects for Agile methods
  • Rollout strategies that mitigate risks
  • Keeping departments aligned during the Agile rollout
  • Defining the social contract for Agile
  • How to make Agile sustainable in your particular culture
  • How Agile impacts your partner eco-system
  • Succeeding with off-shoring and outsourcing

Setting Up The Agile Enterprise:

  • Determining your performance metrics for Agile
  • How to negotiate Agile contracts
  • Achieving breakthrough innovation through Agile
  • Business designs that utilize the power of Agile

Price and Availability