The Agile Executive

Making Agile Work

Posts Tagged ‘Cutter Consortium

The BMC Agile Transformation: A Seven-Year Perspective

leave a comment »

This Executive Update represents my current understanding of the deeper reasons behind the success of  the BMC rollout. It reviews past decisions in light of  knowledge, experience, and insights that evolved a long  time after the decisions, for better or worse, had been executed. In general, it’s about my making sense of  things and sharing my insights with Cutter clients.

To download the Executive Update, click here and use the promotion code BMCAGILE.

Time-to-Market as a Function of Release Size (Source: QSM sample of ~7,500 projects)

Written by israelgat

November 18, 2011 at 4:07 pm

Late Night Thoughts on Stepping Into Cutter’s Agile Practice Director Role

with one comment

http://www.flickr.com/photos/holia/3204431590/

I have just stepped into the role of Director, Cutter’s Agile Product and Project Management Practice. It has been a long time since I felt so honored. Little had I expected that a friendly 2008 email from Brian Robertson suggesting I write an article for Cutter, and a later invite from Jim Highsmith to join the practice would lead to my heading it now.

My preliminary thoughts about evolving the practice are summarized in the Cutter press release. I view Agility as much more than a ‘mere’ software method. I envision the combination of Agile, Cloud, Mobile and Social as transformative in nature. Specifically:

It is not ‘just’ about doing one thing or another a little faster. Rather, it is about enabling new business designs that utilize the ultra-fast pace and flexibility of multiple links in the company’s value chain. [Excerpt from the Cutter press release].

An example of the transformation I foresee is given in my 2011 prediction:

The paramount need to deliver faster/earlier is, for all practical purposes, dictated by today’s markets becoming hyper-segmented. For example, my (or your) Twitter network today is an evolving market segment. My Twitter network in March 2011 could easily be a different segment than the segment it is today. The only way to penetrate such fluid market segments effectively is by following the classic Agile mantra “Release early and often.”

Viewed from such perspective, Agility is more than a strategic initiative. It actually becomes a philosophy of life in the best sense of the word:

The real challenge, however, lies in how to go about solving problems when you don’t understand them well enough to get to a viable solution … when you don’t have a clear enough understanding of the problem to create clear solutions, you have to iterate. [Interview with Russ Daniels]

I will be the first one to admit that I don’t fully understand various facets of what it will take to make the Agile practice most meaningful to current Cutter clients and highly enticing to future prospects. Just as Russ suggests above, I plan to iterate.

And this, in the final analysis, is all that matters in an Agile practice.

Written by israelgat

December 19, 2010 at 10:35 pm

A Special Technical Debt Offer from the Cutter Consortium

leave a comment »

The good folks at Cutter are making the October Issue of the Cutter IT Journal (CITJ) available to anyone who is interested in getting deeper into the intricacies of technical debt. Here is the table of contents for this issue:

  • Opening Statement by Israel Gat
  • Modernizing the DeLorean System: Comparing Actual and Predicted Results of a Technical Debt Reduction Project by John Heintz
  • The Economics of Technical Debt by Stephen Chin, Erik Huddleston, Walter Bodwell, and Israel Gat
  • Technical Debt: Challenging the Metaphor by David Rooney
  • Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process by Brent Barton and Chris Sterling
  • The Risks of Acceptance Test Debt by Ken Pugh
  • Transformation Patterns for Curing the Human Causes of Technical Debt by Jonathon Michael Golden
  • Infrastructure Debt: Revisiting the Foundation by Andrew Clay Shafer

Being the guest editor for this issue, I can attest better than anyone else how much I learned from the various authors, from Karen Pasley (the October issue editor) and Chris Generali (CITJ Editor-in-Chief).

Click here for details of this special offer including downloading instruction.

Fresh Perspectives on Technical Debt

leave a comment »

Update, October 15: The issue has been posted on the Cutter website (Cutter IT Journal subscription privileges required).

Cutter is just about ready to post the October issue of the IT Journal for which I am the guest editor. Print subscribers should receive it by the last week of the month. Jim Highsmith and I will be reflecting on it in our forthcoming seminar on technical debt in the Cutter Summit.

This issue sheds light on three noteworthy aspects of technical debt techniques:

  1. Their pragmatic use as an integral part of Governance, Risk and Compliance (GRC).
  2. Extending the techniques to shed light on various nuances of technical debt that have alluded us so far.
  3. Applying the techniques in new domains such as devops.

Here is the Table of Contents for this exciting issue:

Opening Statement

by Israel Gat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  3

Modernizing the DeLorean System: Comparing Actual and Predicted Results of a Technical Debt Reduction Project

by John Heintz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

The Economics of Technical Debt

by Stephen Chin, Erik Huddleston, Walter Bodwell, and Israel Gat . . . . . . . . . . . . . . . . . 11

Technical Debt: Challenging the Metaphor

by David Rooney . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Manage Project Portfolios More Effectively by Including Software Debt in the Decision Process

by Brent Barton and Chris Sterling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

The Risks of Acceptance Test Debt

by Ken Pugh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Transformation Patterns for Curing the Human Causes of Technical Debt

by Jonathon Michael Golden . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  30

Infrastructure Debt: Revisiting the Foundation

by Andrew Shafer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

 

Action Item: Apply the techniques recommended in this issue to govern your software assets in an effective manner.

____________________________________________________________________________________________________

Overwhelmed by a “mountain” of technical debt? Let me know if you would like assistance in devising and carrying out plans to reduce the debt in a biggest-bang-for-the-buck manner. Click Services for details.

____________________________________________________________________________________________________

The Real Cost of One Trillion Dollars in IT Debt: Part II – The Performance Paradox

with 7 comments

Some of the business ramifications of the $1 trillion in IT debt have been explored in the first post of this two-part analysis. This second post focuses on “an ounce of prevention is worth a pound of cure” aspects of IT debt. In particular, it proposes an explanation why prevention was often neglected in the US over the past decade and very possibly longer. This explanation is not meant to dwell on the past. Rather, it studies the patterns of the past in order to provide guidance for what you could do and should do in the future to rein in technical debt.

The prevention vis-a-vis cure trade-off  in software was illustrated by colleague and friend Jim Highsmith in the following figure:

Figure 1: The Technical Debt Curve

As Jim astutely points out, “once on far right of curve all choices are hard.” My experience as well as those of various Cutter colleagues have shown it is actually very hard. The reason is simple: on the far right the software controls you more than you control it. The manifestations of technical debt [1] in the form of pressing customer problems in the production environment force you into a largely reactive mode of operation. This reactive mode of operation is prone to a high error injection rate – you introduce new bugs while you fix old ones. Consequently, progress is agonizingly slow and painful. It is often characterized by “never-ending” testing periods.

In Measure and Manage Your IT Debt, Gartner’s Andrew Kyte put his finger on the mechanics that lead to the accumulation of technical debt – “when budget are tight, maintenance gets cut.” While I do not doubt Andrew’s observation, it does not answer a deeper question: why would maintenance get cut in the face of the consequences depicted in Figure 1? Most CFOs and CEOs I know would get quite alarmed by Figure 1. They do not need to be experts in object-oriented programming in order to take steps to mitigate the risks associated with slipping to the far right of the curve.

I believe the deeper answer to the question “why would maintenance get cut in the face of the consequences depicted in Figure 1?” was given by John Seely Brown in his 2009 presentation The Big Shift: The Mutual Decoupling of Two Sets of Disruptions – One in Business and One in IT. Brown points out five alarming facts in his presentation:

  1. The return on assets (ROA) for U.S. firms has steadily fallen to almost one-quarter of 1965 levels.
  2. Similarly, the ROA performance gap between corporate winners and losers has increased over time, with the “winners” barely maintaining previous performance levels while the losers experience rapid performance deterioration.
  3. U.S. competitive intensity has more than doubled during that same time [i.e. the US has become twice as competitive - IG].
  4. Average Lifetime of S&P 500 companies [declined steadily over this period].
  5. However, in those same 40 years, labor productivity has doubled – largely due to advances in technology and business innovation.

Discussion of the full-fledged analysis that Brown derives based on these five facts is beyond the scope of this blog post [2]. However, one of the phenomena he highlights –  “The performance paradox: ROA has dropped in the face of increasing labor productivity” – is IMHO at the roots of the staggering IT debt we are staring at.

Put yourself in the shoes of your CFO or your CEO, weighing the five facts highlighted by Brown in the context of Highsmith’s technical debt curve. Unless you are one of the precious few winner companies, the only viable financial strategy you can follow is a margin strategy. You are very competitive (#3 above). You have already ridden the productivity curve (#5 above). However, growth is not demonstrable or not economically feasible given the investment it takes (#1 & #2 above). Needless to say, just thinking about being dropped out of the S&P 500 index sends cold sweat down your spine. The only way left to you to satisfy the quarterly expectations of Wall Street is to cut, cut and cut again anything that does not immediately contribute to your cashflow. You cut on-going refactoring of code even if your CTO and CIO have explained the technical debt curve to you in no uncertain terms. You are not happy to do so but you are willing to pay the price down the road. You are basically following a “survive to fight another day” strategy.

If you accept this explanation for the level of debt we are staring at, the core issue with respect to IT debt at the individual company level [3] is how “patient” (or “impatient”) investment capital is. Studies by Carlota Perez seem to indicate we are entering a phase of the techno-economic cycle in which investment capital will shift from financial speculation toward (the more “patient”) production capital. While this shift is starting to happens, you have the opportunity to apply “an ounce of prevention is worth a pound of cure” strategy with respect to the new code you will be developing.

My recommendation would be to combine technical debt measurements with software process change. The ability to measure technical debt through code analysis is a necessary but not sufficient condition for changing deep-rooted patterns. Once you institute a process policy like “stop the line whenever the level of technical debt rose,” you combine the “necessary” with the “sufficient” by tying the measurement to human behavior. A possible way to do so through a modified Agile/Scrum process is illustrated in Figure 2:

Figure 2: Process Control Model for Controlling Technical Debt

As you can see in Figure 2, you stop the line and convene an event-driven Agile meeting whenever the technical debt of a certain build exceeds that of the previous build. If ‘stopping the line’ with every such build is “too much of a good thing” for your environment, you can adopt statistical process control methods to gauge when the line should be stopped. (See Using 3σ  Control Limits in Software Engineering for a discussion of the settings appropriate for your environment.)

An absolutely critical question this analysis does not cover is “But how do we pay back our $1 trillion debt?!I will address this most important question in a forthcoming post which draws upon the threads of this post plus those in the preceding Part I.

Footnotes:

[1] Kyte/Gartner define IT Debt as “the costs for bringing all the elements [i.e. business applications] in the [IT] portfolio up to a reasonable standard of engineering integrity, or replace them.” In essence, IT Debt differs from the definition of Technical Debt used in The Agile Executive in that it accounts for the possible costs associated with replacing an application. For example, the technical debt calculated through doing code analysis on a certain application might amount to $500K. In contrast, the cost of replacement might be $250K, $1M or some other figure that is not necessarily related to intrinsic quality defects in the current code base.

[2] See Hagel, Brown and Davison: The Power of Pull: How Small Moves, Smartly Made, Can Set Big Things in Motion.

[3] As distinct from the core issue at the national level.

The Gat/Highsmith Joint Seminar on Technical Debt and Software Governance

leave a comment »

Jim and I have finalized the content and the format for our forthcoming Cutter Summit seminar. The seminar is structured around a case study which includes four exercise. We expect the case study/exercises will take close to two-thirds of the allotted time (the morning of October 27). In the other third we will provide the theory and practices to be used in the seminar exercises and (hopefully) in many future technical debt engagements participants in the workshop will oversee.

The seminar does not require deep technical knowledge. It targets participants who possess conceptual grasp of software development, software governance and IT operations/ITIL. If you feel like reading a little about technical debt prior to the Summit, the various posts on technical debt in this blog will be more than sufficient.

We plan to go with the following agenda (still subject to some minor tweaking):

Agenda for the October 27, 9:30AM to 1:00PM Technical Debt Seminar

  • Setting the Stage: Why Technical Debt is a Strategic Issue
  • Part I: What is Technical Debt?
  • Part II : Case Study – NotMyCompany, Inc.
    • Exercise #1 – Modernizing NotMyCompany’s Legacy Code
  • Part III: The Nature of Technical Debt
  • Part IV: Unified Governance
    • Exercise #2 – The acquisition of SocialAreUs by NotMyCompany
  • Part V: Process Control Models
    • Exercise #3 – How Often Should NotMyCompany Stop the Line?
  • (Time Permitting – Part VI: Using Technical Debt in Devops
    • Exercise #4 – The Agile Versus ITIL Debate at NotMyCompany)

By the end of the seminar you will know how to effectively apply technical debt techniques as an integral part of software governance that is anchored in business realities and imperatives.

Written by israelgat

September 30, 2010 at 3:20 pm

What 108M Lines of Code Tell Us

with 16 comments

Results of the first annual report on application quality have just been released by CAST. The company analyzed 108M lines of code in 288 applications from 75 companies in various industries. In addition to the ‘usual suspects’ –  COBOL, C/C++, Java, .NET – CAST included Oracle 4GL and ABAP in the report.

The CAST report is quite important in shedding light on the code itself. As explained in various posts in this blog, this transition from the process to its output is of paramount importance. Proficiency in the software process is a bit allusive. The ‘proof of the pudding’ is in the output of the software process. The ability to measure code quality enables effective governance of the software process. Moreover, Statistical Process Control methods can be applied to samples of technical debt readings. Such application is most helpful in striking a good balance in ‘stopping the line’ – neither too frequently nor too rarely.

According to CAST’s report, the average technical debt per line of code across all application is $2.82.  This figure, depressing that it might be, is reasonably consistent with quick eyeballing of Nemo. The figure is somewhat lower than the average technical debt figure reported recently by Cutter for a sample of the Cassandra code. (The difference is probably attributable to the differences in sample sizes between the two studies). What the data means is that the average business application in the CAST study is saddled with over $1M in technical debt!

An intriguing finding in the CAST report is the impact of size on the quality of COBOL applications.  This finding is demonstrated in Figure 1. It has been quite a while since I last saw such a dramatic demonstration of the correlation between size and quality (again, for COBOL applications in the CAST study).

Source: First Annual CAST Worldwide Application Software Quality Study – 2010

One other intriguing findings in the CAST study is that “application in government sector show poor changeability.” CAST hypothesizes that the poor changeability might be due to higher level of outsourcing in the government sector compared to the private sector. As pointed out by Amy Thorne in a recent comment posted in The Agile Executive, it might also be attributable to the incentive system:

… since external developers often don’t maintain the code they write, they don’t have incentives to write code that is low in technical debt…

Congratulations to Vincent Delaroche, Dr. Bill Curtis, Lev Lesokhin and the rest of the CAST team. We as an industry need more studies like this!

Why Spend the Afternoon as well on Technical Debt?

with 2 comments

Source: http://www.flickr.com/photos/pinksherbet/233228813/

Yesterday’s post Why Spend a Whole Morning on Technical Debt? listed eight characteristics of the technical debt metric that will be discussed during the morning of October 27 when Jim Highsmith and I deliver our joint Cutter Summit seminar. This posts adds to the previous post by suggesting a related topic for the afternoon.

No, I am not trying to “hijack” the Summit agenda messing with the afternoon sessions by colleagues Claude Baudoin and Mitchell Ummel. I am simply pointing out a corollary to the morning seminar that might be on your mind in the afternoon. Needless to say, thinking about it in the afternoon of the 28th instead of the afternoon of the 27th is quite appropriate…

Yesterday’s post concluded with a “what it all means” statement, as follows:

Technical debt is a meaningful metric at any level of your organization and for any department in it. Moreover, it is applicable to any business process that is not yet taking software quality into account.

If you accept this premise, you can use the technical debt metric to construct boundary objects between various departments in your company/organization. The metric could serve as the heart of boundary objects between dev and IT ops, between dev and customer support, between dev and a company to which some development tasks are outsourced, etc. The point is the enablement of working agreements between multiple stakeholders through the technical debt metric. For example, dev and IT ops might mutually agree that the technical debt in the code to be deployed to the production environment will be less than $3 per line of code. Or, dev and customer support might agree that enhanced refactoring will commence if the code decays over time to more than $4 per line of code.

You can align various departments by by using the technical debt metric. This alignment is particularly important when the operational balance between departments has been disrupted. For example, your developers might be coding faster than your ITIL change managers can process the change requests.

A lot more on the use of the technical debt metric to mitigate cross-organizational dysfunctions, including some Outmodel aspects, will be covered in our seminar in Cambridge, MA on the morning of the 27th. We look forward to discussing this intriguing subject with you there!

Israel

Outline of the Technical Debt Seminar at the Cutter Summit

leave a comment »



Pictured above are speakers of the forthcoming Cutter Summit. Between the seventeen of us we will cover a broad spectrum of IT topics such as Agile, Enterprise Architecture, Business Strategy, Cloud Computing, Collaboration, Governance and Security. Inter-disciplinary seminars, panels and case studies will weave all those threads together to give participants a clear view of the unfolding transformation in IT and of the new way(s) companies are starting to utilize IT. Click here for a details.

As Jim Highsmith and I continue to develop our joint seminar on technical debt for the summit, I would like to give readers of this blog a sense of where we are and ask for feedback. Right now we are considering the following building blocks for the seminar:

  • The Nature of Technical Debt
  • Technical Debt Metrics
  • Monetizing Technical Debt
  • Constructing Roadmaps for Paying Back Technical Debt
  • Risk Assessment and Mitigation
  • A Simple Software Governance Framework
  • Schedule in the Simple Governance Framework
  • Enlightened Governance
  • Baking in Quality One Build at a Time
  • How Often Should the Project Team Regroup?
  • Multi-Level Governance
  • Extending  Technical Debt Techniques to Devops
  • Use of Technical Debt Techniques in Agile Portfolio Management
  • The Start Afresh Option
  • Technical Debt as an Integral Part of a Value Delivery Culture

In the course of going through a subset of these building blocks, we will cover the latest and greatest from the October issue of the Cutter IT Journal on technical debt, present two case studies, and conduct a few group exercises.

From Vivek Kundra to Devops and Compounding Interest: Cutter’s Forthcoming Special Issue on Technical Debt

leave a comment »

Source: http://www.flickr.com/photos/wallyg/152453473/

In a little over a month the Cutter Consortium will publish a special issue of the Cutter IT Journal (CITJ) on Technical Debt. As the guest editor for this issue I had the privilege to set the direction for it and now have early exposure to the latest and greatest in research and field work from the various authors. This short post is intended to share with you some of the more exciting findings you could expect in this issue of the CITJ.

The picture above of the debt clock is a common metaphor that runs through all articles. The various authors are unanimously of the opinion that one must measure his/her technical debt, embed the measurements in the software governance process and relentlessly push hard to reduce technical debt. One can easily extrapolate this common thread to conjecture an initiative by Vivek Kundra to assess technical debt and its ramification at the national level.

Naturally, the specific areas of interest with respect to technical debt vary from one author to another. From the broad spectrum of topics addressed in the journal, I would like to mention two that are quite representative:

  • One of the authors focuses on the difference between the manifestation of technical debt in dev versus its manifestation in devops, reaching the conclusion that the change in context (from dev to devops) makes quite a difference. The author actually doubts that the classical differentiation between “building the right system” and “building the system right” holds in devops.
  • Another author derives formulas for calculating  Recurring Interest and Compounding Interest in technical debt. The author uses these formulas to demonstrate two scenario: Scenario A in which technical debt as % of total product revenue is 12% and Scenario B in which technical debt as % of total product revenue is 280%. The fascinating thing is that this dramatic difference (12% v. 280%) is induced through much smaller variances in the Recurring Interest and the Compounding Interest.

I will blog much more on the subject when the CITJ issue is published in October. In addition, Jim Highsmith and I will discuss the findings of the various authors as part of our joint seminar on the subject in the forthcoming Cutter Summit.

Stay tuned!

Follow

Get every new post delivered to your Inbox.

Join 36 other followers