The Agile Executive

Making Agile Work

Posts Tagged ‘Michael Mah

This is Not a “Me Too” Product

leave a comment »

I had the distinct pleasure to discuss AgileEVM over dinner yesterday with its two “fathers” – Chris Sterling and Brent Barton. I was much impressed by the originality of the concept, the clear differentiation of the product and the laser focus on Earned Value Management (EVM) aspects.

Governance of the software development process is a topic that has been much discussed in various Cutter publications – see for example recent Cutter blog posts by Stephen AndrioleRobert Charette, Jim Highsmith, Vince Kellen, Masa Maeda, Michael Mah and me. Chris and Brent add a layer of EVM – both in terms of theoretical foundations and actual tool – to our spectrum of thoughts on governance. IMHO AgileEVM has the potential to induce a disruption in the way companies go about the business of governing software.

AgileEVM is particularly intriguing as a {product + professional services} combo. Chris and Brent are well-known as awesome Agile consultants. The opportunity to pick their brains twice – once through the product, a second time through a related consulting engagement with them – is most attractive.

The two “pages” below give a sense of the power of the product. Even better, simply download the product through the AgileEVM website.

Grab Chris or Brent for a 1-1 demo if you are at Agile 2010 this week. You will not be disappointed…

Using 3σ Control Limits in Software Engineering

with 2 comments

Source: Wikipedia; Control Chart

The  July/August 2010 issue of IEEE Software features an article entitled “Monitoring Software Quality Evolution for Defects” by Hongyu Zhang and Sunghun Kim. The article is of interest to the software developer/tester/manager in quite a few ways. In particular, the authors report on their successful use of 3σ control limits in c-charts used to plot defects in software projects.

To put things in perspective, consider my recent assessment of the results accomplished by Quick Solutions (QSI) in two of their projects:

One to one-and-a-half standard deviation better than the mean might not seem like much to six-sigma black belts. However, in the context of typical results we see in the software industry the QSI results are outstanding.  I have not done the exact math whether those results are superior to 95%, 97% or 98% of software projects in Michael Mah‘s QSMA database as the very exact figure almost does not matter when you achieve this level of excellence.

A complementary perspective is provided by Capers Jones in Estimating Software Costs: Bringing Realism to Estimating:

Another way of looking at six-sigma in a software context would be to achieve a defect-removal efficiency level of about 99.9999 percent. Since the average defect-removal efficiency level in the United States is only about 85 percent, and less than one project in 1000 has ever topped 98 percent,  it can be seen that actual six-sigma results are beyond the current state of the art.

The setting of control limits is, of course, quite a different thing from the actual defect-removal efficiency numbers reported by Jones for the US and the very low number of defects reported by Mah for QSI. Having said that, driving a continuous improvement process through using 3σ control limits is the best recipe toward eventually reaching six-sigma results. For example, one could drive the development process by using Cyclomatic complexity per Java class as the quality characteristic in the figure at the top of this post. In this figure, a Cyclomatic complexity reading higher than 10.860 (the Upper Control Limit) will indicate a need to “stop the line” and attend to reducing complexity before resuming work on functions and features.

Coming on the heels of the impressive results reported by David Joyce on the use of statistical process control (SPC) techniques by the BBC, the article by Zhang and Kim is another encouraging report on the successful application of manufacturing techniques to software (and to knowledge work in general). I am not at liberty to quote from this just published IEEE article, but here is the abstract:

Quality control charts, especially c-charts, can help monitor software quality evolution for defects over time. c-charts of the Eclipse and Gnome systems showed that for systems experiencing active maintenance and updates, quality evolution is complicated and dynamic. The authors identify six quality evolution patterns and describe their implications. Quality assurance teams can use c-charts and patterns to monitor quality evolution and prioritize their efforts.

Beautiful Quality

leave a comment »

Figure 1: Agile Assessment – Quality (Source: QSMA)

Colleague and friend Michael Mah has kindly shared with me the figure above – quality assessment for a sample of Agile projects in the QSMA metrics database of more than 8000 software projects. The two red squares in this figure represent the recent results Mah measured on two projects carried out by Quick Solutions (QSI) – a Westerville, OH company offering a broad range IT services.

One to one-and-a-half standard deviation better than the mean might not seem like much to six sigma black belts. However, in the context of typical results we see in the software industry the QSI results are outstanding.  I have not done the exact math whether those results are superior to 95%, 97% or 98% of software projects in the QSMA database as the very exact figure almost does not matter when you achieve this level of excellence.

I asked Bart Murphy – QSI’s Vice President of Delivery and Operations – for the ‘secret sauce.’ Here is the QSI recipe:

For the projects referenced in Michael’s evaluation, our primary focus was quality…  Our team was tasked with not only a significant Innovation effort, but we also managed all aspects of supporting and stabilizing the application (production support, triage, infrastructure, database support, etc.).  Our ‘secret sauce’ was assembling a world class team and executing our Agile methodology.  We organized the team efforts to focus on the three major initiatives; Innovation, Stabilization (Technical Debt), and Production Support.  We developed a release plan and coordinated efforts to deploy releases that resolved a significant number of defects, introduced market differentiating features, and addressed massive amount of technical debt.  The team was able to accomplish this without introducing additional defects into the production system.   Our success can be attributed to the commitment from the business to understand our Agile methodology and being highly engaged throughout the project (co-located with the team).  In addition, the focus of quality that is integrated into our process through the use of Test Driven Development, Continuous Integration, Test Automation, Quality Assurance, and Show & Tells.  Lastly, this would not have been possible without the co-location of the entire team given the significant issues and time constraints for delivery.

Bart also provided me with a table  ‘a la Capers Jones’ in which he elaborates on the factors that helped QSI achieve these results and those that stood in the way. I will publish and discuss Bart’s table in a forthcoming post. As part of this post I will also compare the factors identified by Bart with those reported by Capers Jones.

Cutter’s Technical Debt Assessment and Valuation Service

with 3 comments



Source: Cutter Technical Debt and Valuation Service

The Cutter Consortium has announced the availability of the Technical Debt Assessment and Valuation Service. The service combines static code analytics with dynamic program analytics to give the client “x-rays” of the software being examined at any desired granularity – from the whole project portfolio to a single instruction. It breaks down technical debt into the areas of coverage, complexity, duplication, violations and comments. Clients get an aggregate dollar figure for “paying back” debt that they can then plug into their financial models to objectively analyze their critical software assets. Based on these metrics, they can make the best decisions about their ongoing strategy for the software development effort under scrutiny.

This new service is an important addition to the enlightened software governance framework that Jim Highsmith, Michael Mah and I have been thinking about and contributing to for sometime now (see Beyond Scope, Schedule and Cost: Measuring Agile Performance and Quantifying the Start Afresh Option). The heart of both the technical debt service and the enlightened governance framework is captured by the following words from the press release:

Executives in charge of software governance have long dealt with two kinds of dollar figures: One, the cost of producing and maintaining the software; and two, the value of the software, which is usually expressed in terms of the net present value associated with the expected value stream the product will generate. Now we can deal with technical debt in the same quantitative manner, regardless of the software methods a company uses.

When expressed in terms of dollars, technical debt ties neatly into value vis-à-vis cost considerations. For a “well behaved” software project, three factors — value, cost, and technical debt — have to satisfy the equation Value >> Cost > Technical Debt. Monitoring the balance between value, cost, and technical debt on an ongoing basis is an effective way for organizations to stay on top of their real progress, and for stakeholders and investors to ensure their investment is sound.

By boiling down technical debt to dollars and tying it to cost and value, the service enables a metrics-driven governance framework for the use of five major constituencies, as follows:

Technical debt assessments and valuation can specifically help CIOs ensure alignment of software development with IT Operations; give CTOs early warning signs of impending project trouble; assure those involved in due diligence for M&A activity that the code being acquired will adapt to meet future needs; enables CEOs to effectively govern the software development process; and, it provides critical information as to whether software under consideration constitutes an asset or a liability for venture capitalists who need to make informed investment decisions.

It should finally be pointed out that the technical debt assessment service and the governance framework it enables are applicable to any software method. They can be used to:

  • Govern a heterogeneous environment in which multiple software methods are used
  • Make apples-to-apples comparisons between disparate software projects
  • Assess project performance vis-a-vis industry norms

Forthcoming Cutter Executive Reports, Executive Updates and Email Advisors on the technical debt service are restricted to Cutter clients. As appropriate, I will publish the latest and greatest news on the subject in the Cutter Blog (which is an open forum I highly recommend).

Acknowledgements: I would like to wholeheartedly thank the following colleagues for inspiring, enlightening and supporting me during the preparation of the service:

  • Karen Coburn
  • Jennifer Flaxman
  • Jonathon Golden
  • John Heintz
  • Jim Highsmith
  • Ken Collier
  • Kim Leonard
  • Kara Letourneau
  • Michal Mah
  • Anne Mullaney
  • Chris Sterling
  • Cindy Swain
  • Sarah Wiesbrock

The Concise Executive Guide to Agile

with 3 comments



The IEEE Computer Society Press published today a ReadyNote[1] that I authored entitled The Concise Executive Guide to Agile. It is available through the IEEE Computer Society Store. A Kindle version will be published in June.

I had two main objectives in writing the guide:

  1. Provide the know-how for approaching Agile in a concise manner that requires minimal investment of time and effort by the reader. The ReadyNote does so by summarizing most Agile topics in a page or two. Detailed coverage of a topic is left for follow-on reading in the selected references that accompany each topic and in the Further Reading appendix.
  2. Be accessible to any executive  — R&D, Marketing, Sales, Program Management, Professional Services, Customer Support, Finance, or IT.  The only assumption I make is that the reader has a conceptual grasp of software and software engineering as well as an interest in learning about Agile. No deep knowledge, let alone technical knowledge, about software engineering is required for comprehending the guide.

There is no fluff in the guide. Every paragraph has been written to satisfy the “And therefore what?!” criterion. It is my intent to drive a point home and make it clear to the reader what he/she could do with the information in as few words as possible.

A simple acid test for the guide is your successful assimilation of it in entirety during a flight in the continental US. Something has not quite worked if you need to fly all the way from the US to Europe or vice versa in order to comprehend the guide…

I would like to express my sincere thanks to Michael Cote, Michael Mah, Hubert Smits and to the fourth reviewer (whose identity I don’t know) for their many helpful insights and suggestions. I am also deeply indebted to Linda Shafer and Kate Guillemette of the IEEE Computer Society who got me to write the guide and supported the writing and editing process along the way.

Enjoy reading!


  1. “ReadyNotes are short e-books that are tightly focused on specific topics” [IEEE Computer Society Press].

How to Combine Development Productivity Data with Software Quality Metrics

with 2 comments

Consider the situation described in Should You Invest in This Software:

  • One of your portfolio companies expects to ship 500K lines of code in 6 months.
  • The company asks for additional $2M to complete development and bring the product to market.
  • Using technical debt quantification techniques you find the technical debt amounts to $1M.

You are not at all comfortable “paying back” the technical debt in addition to funding the requested $2M. You wonder whether you should start afresh instead of trying to complete and fix the code.

Photo credit: @muntz (Flickr)

A good starting point for assessing the fresh start option is Michael Mah‘s studies of software productivity. Based on the QSMA SLIM metrics database of more than 8,000 projects, Michael will probably bracket the productivity per person in a team consisting of product management, development and test at 10-15K lines of code per year. If you use the 15K lines of code per year figure for the purposes of the analysis, 500K lines of code could theoretically be delivered with an investment of about 33.3 (500/15) man years. Assuming average loaded cost of $99,000 per man-year,  the software represents a programming effort of $3.3M. Not much is left if you deduct $3M ($2M+1M) from $3.3M…

Five considerations are of paramount importance in evaluating the start afresh option:

  • The comparison above ($3.3M versus $3.0M) is timeless. It is a snapshot at a certain point in time which does not take into account the value of time. To factor in the time dimension, the analysis needs to get into value (as distinct from cost) considerations. See the note on Intrinsic Quality v. Extrinsic Quality at the bottom of this post.
  • Your “mileage” may vary. For example, best in class teams in large software projects have reported productivity of 20K lines of code per team member per year. As another example, productivity in business applications is very different from productivity in real-time software.
  • If you decide to start with a brand new team, remember Napoleon’s quip: “Soldiers have to eat soup together for a long time before they are ready to fight.”
  • If you decide to start afresh with the same team plus some enhancements to the headcount, be mindful of  ‘Mythical Man-Month‘ effects. Michael Mah’s studies of the BMC BPM projects indicate that such effects might not hold for proficient Agile teams. Hence, you might opt to go Agile if you plan to enhanced the team in an aggressive manner.
  • Starting afresh is not an antidote to accruing technical debt (yet again…) over time. But, it gives you the opportunity to aggressively curtail technical debt by applying the techniques described in Using Credit Limits to Constrain Development on Margin. For example, you might run source code analytics every two weeks and go over the results in the bi-weekly demo.

As long as you are mindful of these five aspects (timeless analysis, your mileage may vary, Napoleon’s quip, mythical man-month effects and credit limits on technical debt), combining technical debt figures with productivity data is an effective way to consider the pros and cons of “fix it” versus starting afresh. The combination of the two simplifies a complex  investment decision by reducing all considerations to a single common denominator – $$.

Note: This is not a discussion from a value perspective. The software, warts and everything, might (or might not)  be valuable to the target customers. The reader is referred to Jim Highsmith‘s analysis of Intrinsic quality versus Extrinsic Quality in Agile Project Management: Creating Innovative Products. See the Cutter Blog post entitled Beyond Scope, Schedule and Cost: Measuring Agile Performance for a short summary of the distinction between the two.

Turning the “Law of Software Physics” Upside Down

leave a comment »

Agile Event

Next week, Michael Mah will be presenting new quantitative data on software productivity, cost, time-to-market and quality. Here is an excerpt from the announcement of his talk:

Many companies are adopting Agile practices in an effort to increase project throughput, reduce cost, and improve quality. But are they working? Drawing from industry statistics, Michael answers vital questions about Agile’s effectiveness, which may be turning the “law of software physics” upside down. Until now, there have been predictable relationships among schedule, staffing, and quality; industry data indicates Agile may be changing all this. See productivity findings at 5 Agile companies, and the results for time-to-market, productivity, and quality. Learn the right practices for your environment, including characteristics of successful measurement. See how metrics reveal insights into Agile approaches that are becoming mainstream.

Knowing Michael (and, well, knowing a bit about his latest and greatest findings…) I have every reason to believe Michael will be breaking new grounds in his presentation. Click here and here for details about this exciting presentation.

Written by israelgat

March 5, 2010 at 12:34 pm