The Agile Executive

Making Agile Work

Posts Tagged ‘Erik Huddleston

A Special Technical Debt Offer from the Cutter Consortium

leave a comment »

The good folks at Cutter are making the October Issue of the Cutter IT Journal (CITJ) available to anyone who is interested in getting deeper into the intricacies of technical debt. Here is the table of contents for this issue:

  • Opening Statement by Israel Gat
  • Modernizing the DeLorean System: Comparing Actual and Predicted Results of a Technical Debt Reduction Project by John Heintz
  • The Economics of Technical Debt by Stephen Chin, Erik Huddleston, Walter Bodwell, and Israel Gat
  • Technical Debt: Challenging the Metaphor by David Rooney
  • Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process by Brent Barton and Chris Sterling
  • The Risks of Acceptance Test Debt by Ken Pugh
  • Transformation Patterns for Curing the Human Causes of Technical Debt by Jonathon Michael Golden
  • Infrastructure Debt: Revisiting the Foundation by Andrew Clay Shafer

Being the guest editor for this issue, I can attest better than anyone else how much I learned from the various authors, from Karen Pasley (the October issue editor) and Chris Generali (CITJ Editor-in-Chief).

Click here for details of this special offer including downloading instruction.

Fresh Perspectives on Technical Debt

leave a comment »

Update, October 15: The issue has been posted on the Cutter website (Cutter IT Journal subscription privileges required).

Cutter is just about ready to post the October issue of the IT Journal for which I am the guest editor. Print subscribers should receive it by the last week of the month. Jim Highsmith and I will be reflecting on it in our forthcoming seminar on technical debt in the Cutter Summit.

This issue sheds light on three noteworthy aspects of technical debt techniques:

  1. Their pragmatic use as an integral part of Governance, Risk and Compliance (GRC).
  2. Extending the techniques to shed light on various nuances of technical debt that have alluded us so far.
  3. Applying the techniques in new domains such as devops.

Here is the Table of Contents for this exciting issue:

Opening Statement

by Israel Gat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  3

Modernizing the DeLorean System: Comparing Actual and Predicted Results of a Technical Debt Reduction Project

by John Heintz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

The Economics of Technical Debt

by Stephen Chin, Erik Huddleston, Walter Bodwell, and Israel Gat . . . . . . . . . . . . . . . . . 11

Technical Debt: Challenging the Metaphor

by David Rooney . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process

by Brent Barton and Chris Sterling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

The Risks of Acceptance Test Debt

by Ken Pugh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Transformation Patterns for Curing the Human Causes of Technical Debt

by Jonathon Michael Golden . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  30

Infrastructure Debt: Revisiting the Foundation

by Andrew Shafer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

 

Action Item: Apply the techniques recommended in this issue to govern your software assets in an effective manner.

____________________________________________________________________________________________________

Overwhelmed by a “mountain” of technical debt? Let me know if you would like assistance in devising and carrying out plans to reduce the debt in a biggest-bang-for-the-buck manner. Click Services for details.

____________________________________________________________________________________________________

Apropos is Going Places

with one comment

Pictured above is a screen shot from the forthcoming Rally implementation of Apropos – the end-to-end Kanban system unveiled by Erik Huddleston, Stephen Chin, Walter Bodwell and me in the Lean Software and Systems conference last April.

Pictured below is Stephen Chin presenting the forthcoming product in the recent JavaOne conference:

The commercial version by Rally builds on the four pillars of the original implementation of Apropos at Inovis and the subsequent open source version:

  • Stakeholder Based Investment Themes
  • Business Case Management
  • Upstream and Downstream WIP Limits
  • Dynamic Allocations

These four pillars enable Apropos users to dynamically adjust their plans as needed in accord with the realities of end-to-end execution. Agile portfolio planning and actual execution truly run alongside each other as depicted in the following figure:

Adjustments to allocations can take place in either in the plan or in execution. Here are two typical examples of stakeholders’ dialogs:

  • In planning: “In response to the quick growth of the sales funnel, we decide to increase the % of time allotted to tactical sales opportunities from 35% of the total R&D budget to 40%.”
  • In execution: “The introduction of product Pj will be delayed by three months due to lack of qualified professional services resources. During this period, the affected R&D resources will be reassigned to help with multi-tenant aspects of a SaaS version of product Pk.”

Recommendations: Consider using the open source version of Apropos for a small-scale pilot as part of your 2011 planning/budget cycle. If the pilot proves a good fit with your needs,  switch over to the commercial version in the 2012 planning/budget cycle.

____________________________________________________________________________________________________

Considering end-to-end Agile/Kanban roll-out? Let me know if you would like assistance in planning and implementing a roll-out which focuses on continuous value delivery. Click Services for details.

____________________________________________________________________________________________________

The Real Cost of One Trillion Dollars in IT Debt: Part I – Value Generation and Recapture

with 2 comments

In a recent research note and a corresponding press release, Gartner’s Andrew Kyte assessed the current level of world-wide IT Debt [1] at about $500 billion. Andrew actually considers $500 billion a conservative estimate, expecting it to grow to $1 trillion by 2015. As was pointed out by David Nagel, $1 trillion is more than four times total worldwide enterprise software expenditures this year.

I am publishing two posts in order to put these staggering figures in perspective. This post primarily addresses the business design ramifications of the numbers quoted above. A follow-on post in the coming week will explore the dire repercussions of disregarding the proven practice “an ounce of prevention is worth a pound of cure.” It will also offer an explanation rooted in our business context why this tried and true wisdom has so often been disregarded over the past decade.

The first thing to point out about the Kyte/Gartner figures is that these figures are the cost of fixing, not the value that could be lost due to the myriad malfunctions that a $1 trillion worth of software quality deficits can cause. It is like the loss incurred through fixing the truck in Figure 1 below. The cost of fixing might be $10,000. The corresponding loss of value due to the time it took to carry out the fixing of the truck is vividly captured in the quip written on the back of the sleeper: “Day late $100,000 short.”

Figure 1: “Day late $100,000 short”

Source: http://www.flickr.com/photos/bachir/489090063/

If you accept this premise, one real risk of a high level of IT debt is the deterioration of services provided through the software. An even bigger risk, however, is obsolescence of business designs due to the software systems decaying to the point that adding critical services is next to impossible. For example, consider the following B2B eCommerce services for retailers (taken from an unrelated exchange I recently had with my friend Erik Huddleston):

  • Vendor drop ship
  • Catalog/data sync
  • Vendor management
  • Compliance

It is unlikely a 10-year-old eCommerce software system whose upkeep was neglected for the past decade would have enough changeability left in it to enable providing such services. Lacking these services, the business is likely to revert to outdated designs for generating and recapturing value.

The B2B eCommerce situation discussed above is not really different from the classical dynamics of regression in the development of a child. It is, of course, poignant when a child suffers during one phase or another in his/her development. The bigger poignancy, however, is that the struggling child gets stuck. He/she is unable to move on to the next developmental phase(s). Other children surpass him/her.

What it means in less metaphorical terms is that  an incumbent with a significant IT debt might fall behind new entrants who are not (yet?) saddled with such debt. The new entrants can utilize the flexibility of their software to satisfy customer needs in ways that the incumbent’s legacy software will be hard pressed to meet. Moreover, the new entrants can modify their software in response to actual customer feedback in a much faster manner than the ‘neglectful incumbent’ can.

As an incumbent, you need to really start worrying about your IT debt if you accept the inevitability of the transformation driven by the confluence of Cloud, Mobile and Social (see Consumerization of Enterprise Software). No matter what industry you are in, the versatility, modularity, flexibility and mobility of the forthcoming consumerized enterprise software apply to every aspect of your business design. The IT debt you did not ‘pay back’ stands in the way of modernizing your business design.

Footnotes:

[1] Kyte/Gartner define IT Debt as “the costs for bringing all the elements [i.e. business applications] in the [IT] portfolio up to a reasonable standard of engineering integrity, or replace them.” In essence, IT Debt differs from the definition of Technical Debt used in The Agile Executive in that it accounts for the possible costs associated with replacing an application. For example, the technical debt calculated through doing code analysis on a certain application might amount to $500K. In contrast, the cost of replacement might be $250K, $1M or some other figure that is not necessarily related to intrinsic quality defects in the current code base.

Apropos has been Open Sourced

leave a comment »

Erik HuddlestonWalter BodwellStephen Chin and I unveiled Apropos – the Agile Project Portfolio Scheduler – a month ago in the LSSC10 conference in Atlanta, GA. The system is now available as open source. Click here to go to the home page of the project and download the software. It will enable you to:

  • Synergies R&D with downstream organizations such as Operations, Professional Services, and Sales
  • Increase delivery value through organization-wide alignment of priorities
  • Achieve continuous improvement by whole process feedback loops
  • Gain realtime visibility into delivery status and potential blockages

The core concept of Apropos – multiple parallel feedback loops – is  demonstrated by the following process control diagram:

Figure 1: Process Control View of Apropos

Enjoy Apropos, benefit from it and please give us feedback!

Toward An Event Driven Scrum Process

with 10 comments

Recent posts in this blog and elsewhere recommended using technical debt (TD) criteria as an integral part of the build process. One could fail the build under conditions such as:

  • TD(current build) > TD(previous build)
  • TD per line of code > $1
  • TD as % of cost  > 50%
  • Many others…

If you follow this recommendation while integrating continuously, chances are some fixing might need to be done multiple times a day. This being the case, the relationship between the daily stand-up meeting and the build fail event needs to be examined. If the team anyway re-groups whenever the build fails, what is the value add of the daily stand-up meeting?

Figure 1 below, taken from Agile Software Development with Scrum, examines the feedback loop of Scrum from a process control perspective. It shows the daily stand-up meeting providing the process control function.

Figure 1: Process Control View of Scrum

Figure 2 below shows the feedback loop being driven by the build process instead of by the daily stand-up meeting. From a process control standpoint it is essentially identical to Figure 1. Control through the regularly scheduled daily stand-up meeting has simply been replaced by event-driven control. The team re-groups whenever the build fails.

Figure 2: Build Driven Process Control

Some groups, e.g. the Scrum/Kanban teams under Erik Huddleston and Stephen Chin at Inovis have been practicing Event Driven Scrum for some time now. A lot more data is needed in order to validate the premise that Event Driven Scrum is indeed an improvement over vanilla Scrum. Until the necessary data is collected and examined, two intuitive observations are worth mentioning:

  1. Event Driven Scrum preserves the nature of Scrum as an empirical process.
  2. A control unit that is triggered by events (build fail) is in the very spirit of adaptive Agile methods.

Open-Sourcing the Inovis End-to-End Kanban System

leave a comment »



Source: Gat, Huddleson, Bodwell and Chin, “Reformulating the Product Delivery Process

Colleague and “partner in crime” Stephen Chin has published a post on the Inovis End-to-End Kanban System (aka Apropos) we presented at the LSSC10 conference on April 23.  As readers of this blog might recall, the system tracks features through their full life-cycle from proposal to validation, ensuring actionable feedback cycles. By so doing it firmly anchors the software method in the overall business context with special attention to operational aspects such as deployment, monitoring and support.

Stephen outlines details of the forthcoming open-sourcing of Apropos as follows:

The plan for this tool is to do the initial launch of a BSD-licensed open-source version on May 22nd.  This will include support for the Rally Community Edition, which is free for up to 10 users.  In future releases we plan to support other Agile Lifecycle Management tools, both commercial and open-source, but will need assistance from the community to do this.

If you are interested in helping out with this project, please contact me.  I will have limited bandwidth until after the initial launch, but after that would love to scale up this project with interested parties.

I really can’t wait till the 22nd. IMHO Apropos has the potential to become the leading Kanban system by the community for the community.

Apropos – The Inovis End-to-End Kanban System

with 4 comments



Figure 1:  End-to-end flow slide from Reformulating the Product Delivery Process

Erik HuddlestonWalter BodwellStephen Chin and I delivered a presentation at LSSC10 on the design and implementation of the end-to-end Kanban system Apropos at Inovis. The presentation highlights four key ingredients of the ‘secret sauce’ that makes Apropos so powerful:

  • Stakeholder Based Investment Themes
  • Business Case Management
  • Upstream and Downstream WIP Limits
  • Dynamic Allocations

You can read the slides here. A recording of the presentation will soon be posted by InfoQ. A commercial friendly open-source license of the code will be available on May 22, 2010.

Reformulating the Product Delivery Process

leave a comment »

LSSC ATLANTA 2010

Erik Huddleston, Walter Bodwell, Stephen Chin and I will  present and demo an end-to-end Kanban system that addresses the #1 challenge modern software methods pose – reformulation of the product delivery process. We will do so the coming Friday, April 23, 10:45AM at the Lean Software and Systems Conference. Here is the abstract for our presentation/demo:

Software methods can be viewed as the glue that holds the product development process together. With Kanban, the glue is melting on both sides of the process. Traditional portfolio management systems and organizations have difficulty coping with the granularity of Kanban. Likewise, today’s product release and delivery systems and the corresponding organizational constructs are ill-equipped to effectively handle the Kanban flow.

We present a field-tested system for implementing Kanban on an end-to-end basis – from product ideation through continuous delivery. This system reformulates the deconstructed product delivery process to strike an optimal balance between planning, development and operations.

Written by israelgat

April 21, 2010 at 3:25 am

How to Use Observations From Outside the Agile Process

with 2 comments

a9723 The Whorl of Architecture by tengtan.

Photo credit: tengtan (Flickr)

Most posts on technical debt in this blog emphasize the use of technical debt for strategic decision-making. In this post we will point out the use of technical debt in Agile teams at the tactical level. Specifically:

  • Every two weeks; and/or,
  • With every build.

Taking a close look at the various components of technical debt during the  bi-weekly iteration review meeting provides plenty of useful information to the process. For example, you might look for insights to explain the following:

  • Why is the unit test coverage figure going down?
  • Any particular reason the cyclomatic complexity figure has gone up?
  • Why is the figure of merit for design lower than the figure indicated in the previous iteration review meeting?
  • Many others…

The emphasis in this mode of operation is on guiding the retrospection. Plenty of good and valid reasons might exist for any of the trends mentioned above. However, observing the trends helps you ask the right questions, focusing on what happened during the iteration just completed. In conjunction with technical debt data from previous iteration review meetings, trends that characterize your software development project become visible. You may or may not need to change anything you are doing, but you become very conscious of any “let’s not change” decision.

An intriguing practice suggested by colleague and friend Erik Huddleston is to make technical debt a criterion for the build to pass. The build automatically fails if the technical debt figure has gone up. Or, if you are very focused on a specific aspect of technical debt such as complexity, you fail the build whenever the complexity figure of merit rises above  a certain pre-determined threshold. For example, you might fail a build in which the cyclomatic complexity per method has exceeded 4.

The power of failing a build whenever the technical debt arises is in utilizing the build as an exceptionally effective influence point. You instill the discipline of reducing technical debt one build at a time. If your team aggressively practices continuous integration, it will address technical debt issues multiple times a day. Instead of staring at a “mountain” of technical debt towards the release of a product, you chunk it to really small increments that get addressed “real-time.” For instance, a build that failed due to lack of comments can usually be fixed very quickly by the developer who “upset the apple cart” while the logic embedded in the code is fresh on his/her mind.

A good insight to the way the tactical use of technical debt techniques adds value is provided by the following observation: the technical debt data is observed from outside the Agile process. Hence, technical debt data  is nicely suited to guiding the process. If you think of the software engineering fabric as a virtual stack, the technical debt “layer” could be considered a layer above the Agile process.