Archive for the ‘How-to’ Category
As a starting point, please read Implication of Technical Debt Uncertainty for Software Licensing Negotiations. Everything stated there holds for negotiating M&A deals. In particular:
- You (as the buyer) should insist on conducting a Technical Debt Assessment as part of the due diligence process.
- You should be able to deduct the monetized technical debt figure from the price of the acquisition.
- You should be able to quantify the execution risk (as far as software quality is concerned).
An important corollary holds with respect to acquiring a company who is in the business of doing maintenance on an open source project, helping customers deploy it and training them in its use. You can totally eliminate uncertainty about the quality of the open source project without needing to negotiate permission to conduct technical debt assessment. Actually, you will be advised to conduct the assessment of the software prior to approaching the target company. By so doing, you start negotiations from a position of strength, quite possibly having at your disposal (technical debt) data that the company you consider acquiring does not possess.
Action item: Supplement the traditional due diligence process with a technical debt assessment. Use the monetized technical debt figure to assess execution risk and drive the acquisition price down.
Negotiating a major M&A deal? Let me know if you would like assistance in conducting a technical debt assessment and bringing up technical debt issues with the target company. I will help you with negotiating the acquisition price down. Click Services for details and contact information.
A few month ago Chris Sterling and I were carrying out a Cutter Technical Debt Assessment and Valuation engagement for a venture capitalist who was considering a certain company. We discovered various things in the code of this company. More noteworthy, my deep domain expertise led to Chris discovering the great Greek dish Moussaka.
I have eaten a lot of good Moussakas over the years. Even against this solid gastronomic background I can’t forget how the eyes of Chris lit up when he took the first bite. It took him only a tiny little time to get on his iPhone and tweet on the culinary aspects of our engagement. I then knew it was going to be a very successful engagement…
The relationship with Chris deepened since this episode. For example, in collaboration with Brent Barton Chris contributed a great article to the forthcoming issue of the Cutter IT Journal on Technical Debt. In this article Chris and Brent demonstrate how technical debt techniques can be applied at the portfolio level. They make the reader step into the shoes of the project portfolio planner and walk him through their approach to enhancing the decision-making process by using the software debt dashboard.
Chris has just published an excellent post entitled “Using Sonar Metrics to Assess Promotion of Builds to Downstream Environments” in Getting Agile and was kind enough to suggest I cross-post it in The Agile Executive. Here it is (please note that the examples given below by Chris have nothing to do with the engagement described above):
“For those of you that don’t already know about Sonar you are missing an important tool in your quality assessment arsenal. Sonar is an open source tool that is a foundational platform to manage your software’s quality. The image below shows one of the main dashboard views that teams can use to get insights into their software’s health.
The dashboard provides rollup metrics out of the box for:
- Duplication (probably the biggest Design Debt in many software projects)
- Code coverage (amount of code touched by automated unit tests)
- Rules compliance (identifies potential issues in the code such as security concerns)
- Code complexity (an indicator of how easy the software will adapt to meet new needs)
- Size of codebase (lines of code [LOC])
Before going into how to use these metrics to assess whether to promote builds to downstream environments, I want to preface the conversation with the following note:
Code analysis metrics should NOT be used to assess teams and are most useful when considering how they trend over time
Now that we have this important note out-of-the-way and, of course, nobody will ever use these metrics for “evil”, lets discuss pulling data from Sonar to automate assessments of builds for promotion to downstream environments. For those that are unfamiliar with automated promotion, here is a simple, happy example:
A development team makes some changes to the automated tests and implementation code on an application and checks their changes into source control. A continuous integration server finds out that source control artifacts have changed since the last time it ran a build cycle and updates its local artifacts to incorporate the most recent changes. The continuous integration server then runs the build by compiling, executing automated tests, running Sonar code analysis, and deploying the successful deployment artifact to a waiting environment usually called something like “DEV”. Once deployed, a set of automated acceptance tests are executed against the DEV environment to validate that basic aspects of the application are still working from a user perspective. Sometime after all of the acceptance tests pass successfully (this could be twice a day or some other timeline that works for those using downstream environments), the continuous integration server promotes the build from the DEV environment to a TEST environment. Once deployed, the application might be running alongside other dependent or sibling applications and integration tests are run to ensure successful deployment. There could be more downstream environments such as PERF (performance), STAGING, and finally PROD (production).
The tendency for many development teams and organizations is that if the tests pass then it is good enough to move into downstream environments. This is definitely an enormous improvement over extensive manual testing and stabilization periods on traditional projects. An issue that I have still seen is the slow introduction of software debt as an application is developed. Highly disciplined technical practices such as Test-Driven Design (TDD) and Pair Programming can help stave off extreme software debt but these practices are still not common place amongst software development organizations. This is not usually due to lack of clarity about these practices, excessive schedule pressure, legacy code, and the initial hurdle to learning how to do these practices effectively. In the meantime, we need a way to assess the health of our software applications beyond just tests passing and in the internals of the code and tests themselves. Sonar can be easily added into your infrastructure to provide insights into the health of your code but we can go even beyond that.
The Sonar Web Services API is quite simple to work with. The easiest way to pull information from Sonar is to call a URL:
This will return an XML response like the following:
248390 com.adobe:as3corelib AS3 Core Lib AS3 Core Lib PRJ TRK flex 1.0 2010-09-19T01:55:06+0000 technical_debt_ratio 12.4 12.4% Within this XML, there is a section called that includes the value of the metric we requested in the URL, “technical_debt_ratio”. The ratio of technical debt in this Flex codebase is 12.4%. Now with this information we can look for increases over time to identify technical debt earlier in the software development cycle. So, if the ratio to increase beyond 13% after being at 12.4% 1 month earlier, this could tell us that there is some technical issues creeping into the application.
Another way that the Sonar API can be used is from a programming language such as Java. The following Java code will pull the same information through the Java API client:
Sonar sonar = Sonar.create("http://nemo.sonarsource.org"); Resource commons = sonar.find(ResourceQuery.createForMetrics("248390", "technical_debt_ratio")); System.out.println("Technical Debt Ratio: " + commons.getMeasure("technical_debt_ratio").getFormattedValue());
This will print “Technical Debt Ratio: 12.4%” to the console from a Java application. Once we are able to capture these metrics we could save them as data to trend in our automated promotion scripts that deploy builds in downstream environments. Some guidelines we have used in the past for these types of metrics are:
- Small changes in a metric’s trend does not constitute immediate action
- No more than 3 metrics should be trended (the typical 3 I watch for Java projects are duplication, class complexity, and technical debt)
- The development should decide what are reasonable guidelines for indicating problems in the trends (such as technical debt +/- .5%)
In the automated deployment scripts, these trends can be used to stop deployment of the next build that passed all of its tests and emails can be sent to the development team regarding the metric culprit. From there, teams are able to enter the Sonar dashboard and drill down into the metric to see where the software debt is creeping in. Also, a source control diff can be produced to go into the email showing what files were changed between the successful builds that made the trend go haywire. This might be a listing per build and the metric variations for each.
This is a deep topic that this post just barely introduces. If your organization has a separate configuration management or operations group that managed environment promotions beyond the development environment, Sonar and the web services API can help further automate early identification of software debt in your applications before they pollute downstream environments.”
Thank you, Chris!
An outline of my forthcoming Agile 2010 workshop was given in the post “A Recipe for Handling Cultural Conflicts in Devops and Beyond” earlier this week. Here is the case study around which the workshop is structured:
NotHere, Inc. Case Study
NotHere, Inc. is a $500M company based in Jerusalem, Israel. The company developed an eCommerce platform for small to medium retailers. Through a combination of this platform and its hosting data center, NotHere provides online store fronts, shopping carts, order processing, inventory, billing and marketing services to tens of thousands of retailers in a broad spectrum of verticals. For these retailers, NotHere is a one-stop “shopping” for all their online needs. In particular, instead of partnering with multiple companies like Amazon, Ebay, PayPal and Shopzilla, a retailer merely needs to partner with NotHere (who partners with these four companies and many others).
The small to medium retailers that use the good services of NotHere are critically dependent on the availability of its data center. For all practical purposes retailers are (temporarily) dead when the NotHere data center is not available. In recognition of the criticality of this aspect of its IT operations, NotHere invested a lot of effort in maturing its ITIL[i] processes. Its IT department successfully implements the ITIL service support and service delivery functions depicted in the figure below. From an operational perspective, an overall availability level of four nines is consistently attained. The company advertises this availability level as a major market differentiator.
In response to the accelerating pace in its marketplace, NotHere has been quite aggressive and successful in transitioning to Agile in product management, dev and test. Code quality, productivity and time-to-producing-code have been much improved over the past couple of years. The company measures those three metrics (quality, productivity, time-to-producing-code) regularly. The metrics feed into whole-hearted continuous improvement programs in product management, dev and test. They also serve as major components in evaluating the performance of the CTO and of the EVP of marketing.
NotHere has recently been struggling to reconcile velocity in development with availability in IT operations. Numerous attempts to turn speedy code development into fast service delivery have not been successful on two accounts:
- Technical: Early attempts to turn Continuous Integration into Continuous Deployment created numerous “hiccups” in both availability and audit.
- Cultural: Dev is a competence culture; ops is a control culture.
A lot of tension has arisen between dev and ops as a result of the cultural differences compounding the technical differences. The situation deteriorated big time when the “lagging behind” picture below leaked from dev circles to ops.
The CEO of the company is of the opinion NotHere must reach the stage of Delivery over Development. She is not too interested in departmental metrics like the time it takes to develop code or the time it takes to deploy it. From her perspective, overall time-to-delivery (of service to the retailers) is the only meaningful business metric.
To accomplish Delivery over Development, the CEO launched a “Making Cats Work with Dogs[ii]” project. She gave the picture above to the CTO and CIO, making it crystal clear that the picture represents the end-point with respect to the relationship she expects the two of them and their departments to reach. Specifically, the CEO asked the CTO and the CIO to convene their staffs so that each department will:
- Document its Outmodel (in the sense explored in the “How We Do Things Around Here In Order to Succeed” workshop) of the other department.
- Compile a list of requirements it would like to put on the other group “to get its act together.”
The CEO also indicated she will convene and chair a meeting between the two departments. In this meeting she would like each department to present its two deliverables (world view of the other department & and the requirements to be put on it) and listen carefully to reflections and reactions from the other department. She expects the meeting will be the first step toward a mutual agreement between the two departments how to speed up overall service delivery.
[i] “Information Technology Infrastructure library – a set of concepts and practices for Information Technology Services Management (ITSM), Information Technology (IT) development and IT operations” [Wikipedia].
[ii] I am indebted to Patrick DeBois for suggesting this title.
© Copyright 2010 Israel Gat
- The ascendance of Agile portfolio management in a world characterized by loosely coupled processes
- Devops dynamics are becoming more and more characteristic of end-to-end Agile/Kanban patterns
- Viral spread of technical debt metrics in software governance
- Increasing use of boundary objects in the enterprise context
The workshop is structured around three case studies/exercises that will take about two-thirds of the allotted time (the morning of August 9). The other third provides the theory and tools to be used in the three workshop exercises and (hopefully) in many future engagements participants in the workshop will carry out. Deep technical knowledge is not required – the workshop targets any Agile practitioner who has conceptual grasp of culture, software development, IT operations and portfolio management.
The #1 takeaway from the presentation is the details you need to know about creation and capture of lasting value through end-to-end Agile initiatives.
Here is the workshop agenda (still subject to some minor tweaking):
- Introduction to Cultural Framework
- Exercise #1: Strengths and Weaknesses of Your Culture
- Change Behavior, Not Culture
- When Organizations Clash
- Exercise #2: Conflicts in Devops
- The Agile Flywheel
- Exercise #3: Using Technical Debt as a Boundary Object in Devops
- Bringing Organizations Together Through Enlightened Governance Loops
I look forward to meeting you in the workshop and learning from your experiences and insights!
Scrum set the flywheel in motion and caused the rest of the IT process life cycle to respond. ITIL’s processes still form the solid core of service support and we’ve improved the processes’ capability to handle intense work velocity. The organization adapted by developing unprecedented speed in the ability to deliver production fixes and to solve root cause problems with agility.
From what I gleaned yesterday in the O’Reilly Velocity conference I believe the tables are turning. Ops, or at least web ops, will soon drive development.
The reason for my saying so is quite simple: the breadth and depth of forthcoming web analytics unveiled in the conference. This is not “just” about Google making website performance part of their ranking algorithm. Everything related to web performance will soon be analyzed mercilessly under the “make the web faster” mantra. Dev will need to respond to analytics from operations with an unprecedented speed. For most practical purposes analytics run in ops will dictate the speed for dev.
The phenomenon actually goes beyond performance aspects. To be able to implement changes quickly, dev will need to be very good in ensuring the quality of fast changes. While quality has many dimensions to it, the most applicable one is test coverage. There is no way to change the code quickly without a comprehensive automated test suite.
The first step toward dev meeting the required speed is described in the post How to Initiate a Devops Project:
For a devops project, start by establishing the technical debt of the software to be released to operations. By so doing you build the foundations for collaboration between development and operations through shared data. In the devops context, the technical debt data form the basis for the creation and grooming of a unified backlog which includes various user stories from operations.
I would actually go one step further and suggest including technical debt criteria in the release process. The code is not accepted unless the technical debt per line of code is below a certain pre-set level such as $2. The criteria, of course, can be refined to include specific criteria for the various components of technical debt such as coverage, complexity or duplication. For example, unit test coverage in excess of 70% could be established as a technical debt criterion.
Once such release criteria are established, the metaphorical flywheel starts turning in an opposite direction to that described in The Agile Flywheel. With technical debt criteria embedded in the release process, the most straightforward way for dev to meet these criteria is to use the very same criteria as integral part of the build process. The scheme for so doing in given in the following chart:
One last recommendation: don’t wait till Velocity 2011 to start on the path described above. Velocity 2010 already provides plenty of actionable insights to warrant starting now. Just take a look at the web site.
A Simple Metrics-Driven Software Governance Framework Based on Jim Highsmith’s Agile Triangle Framework
- Technical debt
The heart pf this recommendation is that all three can be expressed in dollar terms as depicted in the figure above. An apples-to-apples comparison is made through the common denominator – $$. For example, something is likely to be either technically, methodically or governance-wise wrong if the technical debt figure exceeds the cost figure for a prolonged period of time. One can actually characterize such a situation as accruing debt faster than building equity.
I am often asked about adding metrics to this simple governance framework. For example, should not productivity be included in the framework?
‘Less is more’ is my usual response to such questions. IMHO value, cost and technical debt address the most important high level governance considerations:
- Value –> Why are we doing the project?
- Cost –> Can we afford the project?
- Technical debt –> Is the execution risk acceptable?
Please pay special attention to the unit of measure of any metric you might add to this simple governance framework. As long as the metric is a dollar-based metric, the cohesion of the governance framework can be maintained. However, metrics which are not expressed in dollars will probably superimpose other frameworks on top of the simple governance framework. For example, you introduce a programming framework if you add a productivity metric which is measured in function points per man month. Sponsors who govern using value, cost, technical debt and productivity will need to mentally alternate between the simple governance framework and the programming framework whenever they try to combine the productivity metric with any of the other three metrics.