The Agile Executive

Making Agile Work

Archive for February 2009

iBetaTest

leave a comment »

Colleague and friend Sebastian Hassinger drew my attention to iBetaTest – a new service that brings together application developers and beta testers.  It is a fascinating concept no matter what software development method you use. Once it spreads beyond iPhone applications it could have quite an impact.

Written by israelgat

February 10, 2009 at 9:07 pm

Posted in Testing

Tagged with ,

Active Releases

with one comment

A Continuum

The traditional way of examining software from a life-cycle perspective is phase-by-phase. The software is developed; deployed; monitored; maintained; changed; and, eventually retired.

True though this description is, more and more executives these days actually view it all as one continuum. An application is developed, and  then deployed and maintained as part of some business process. At a certain level it might not really matter to a business executive what the software life-cycle is and which party carries out what phase. The thing that matters is that some service is performed to customer satisfaction. One could actually do complete  Business Process Outsourcing, chartering  a third party to take care of all the “headaches” in the continuum – from coding a critical software component to repairing a delivery truck to answering calls like, “My shipment has not arrived on the promised date.”

This post looks inside the continuum to comments on the implications of increasing the number of releases. Aspects related to, IT operationsIT service management and the customer as a strategic partner will be discussed in subsequent posts.

Number of Active Releases

The delivery of value to the customer is a fundamental tenet of Agile. The whole Agile development process is geared to that end. Customer value, however,  is realized when the customer is able to start using the product . As deployment cycles for enterprise software can be quite long, time gained through Agile development is not necessarily of immediate value to the customer. For customer value to materialize, the deployment cycle needs to be fast as well.

Companies that use Agile successfully can be quite different with respect to deployment practices. Here is a quick comparison of BMC Software, PatientKeeper and Google:

  • The successful implementation of Scrum at BMC Software led to producing 3-4 releases of the BMC Performance Manager per year.  However, time to deployment by various customers varied greatly.  Hence, a relatively small number of releases was active at any point in time.
  • With PatientKeeper,  Sutherland exploited the hyper-productivity of his Agile teams to produce a relatively large number of releases. When a customer needed a “release” it was downloaded via VPN and installed in fairly short order. Some 45 active releases of the software existed at any point in time.
  • Companies such as Google expedite deployment by using Software as a Service (SaaS) as the delivery mechanism. Google has only one actively deployed release at a time, but produces many fast releases.

As one increases the number of releases, a reasonable balance between velocity of development versus velocity of deployment in the continuum must be struck. To streamline end-to-end operations, velocity gains in one should be matched by gains in the other.

It does not Really Matter if you can Tell the Egg from the Chicken

One can speculate on how things evolve between development and deployment. Whether Agile software development leads to improvement in deployment, or Software as a Service “deployment” induces faster development processes. The Agile philosophy is well expressed in either case. In either direction, the slowere speed area is considered an opportunity, not a barrier. A Software as a Service Operations person who pushes for faster development speed lives the Agile philosophy even if he knows nothing about Agile methods.

Written by israelgat

February 10, 2009 at 2:58 pm

Between R&D and Sales

with 3 comments

In Selling is Dead, authors Miller and Sinkovitz describe the ascendancy of a class of professional procurers who are not easily manipulated by the old  sales tricks in the book. The proverbial sales rep with the Rolex watch who takes his customer to lunch is deemed ineffective by the authors. Instead, the sales rep needs to be a business person who sells.

The thinking articulated by Miller and Sinkovitz is quite relevant to Agile methods. In rolling out Agile in R&D, you need to be very mindful of the way your sales force sells your product.

Product Management versus Development

Various Agile case studies point to initial, and sometimes sustained friction between product management and development. Ditto for various Agile forums. For example, the subject was the source of a lively discussion during this week’s Agile Austin presentation. “Best practices” on how Agile developers should handle product managers in the “real world”, and vice versa, were traded…

The development versus product management conflict usually centers around prioritization and delivery in the context of an impossible backlog. And, it can get nasty about misunderstandings as to what was promised or implied versus what was actually delivered. Poorly articulated, poorly understood requirements tend to aggravate the situation, particularly when the product management team is spread too thin between inbound tasks and outbound assignments.

A Matter of Optics

The product management versus development conflict is really a symptom, not a root cause. The real issue is rooted in the philosophy of sales.

The availability of one feature or another is not usually an issue when selling is focused on value. Important though a certain feature may be, the selling is really about how the customer will derive value from the business process in which the software is embedded, not from the software per se. For example, a bank could satisfy a compliance requirement by way of a management application that monitors logs. Monitoring the log to itself has some value from an IT perspective, but the greater value is in the process ensuring regulatory compliance based on the data provided by the log monitoring application.

Failure to elevate a transaction to the value level usually results in client and vendor dealing primarily on the basis of features and functions. The sales process often deteriorates to the bill of goods level. The process focuses on what the software is, as distinct from what the software can do for the customer. The customer tends to hold the feet of the vendor to the fire with respect to each and every feature.

Two Strategies for Agile Roll-Out

The Agile roll-out in R&D needs to take into account the maturity of your sales organization:

  • If the sales organization primarily sells features, you need to focus the Agile roll-out on effective interaction of development with testing. Start testing as early as possible as an integral part of development, and possibly as the driver of development under the test driven development paradigm. Basically, you are betting on improving the engineering process to the point in which you are over-delivering.
  • If the sales organization is largely selling value, focus on the integration of Sales (and customers) in your Agile process. For example, integrate Sales in your release planning, in the bi-weekly demo and in the release retrospective. In certain cases you might even include Sales in some iteration retrospectives. In this scenario you are betting on the core principle behind Agile: Do only the most important things at any point in time.

In either case, extricate your product management team from the “cross fire” between R&D and Sales. Product management is simply the adaptor for the impedance mis-match between Agile principles and sales maturity.

Written by israelgat

February 5, 2009 at 3:37 pm

Scaling Agility: From 0 to 1000

with one comment

Walter Bodwell delivered an excellent presentation on the subject in Agile Austin last night. The presentation is quite unique in seeing both the “forest” and the “trees”. Walter addresses the operational day-to-day aspects of Scrum in the trenches in parallel with providing insights on the roll-out at the executive level. Highly recommended!

Written by israelgat

February 4, 2009 at 9:25 am

What Exactly is a “Release”?

with one comment

Israel has been working on a multi-part series for the Cutter Consortium around the question of what a release actually is in Agile Development. For as long as I’ve known Israel, he’s been obsesses with the idea of moving a release down from a monolithic bomb-drop of code to a more gentle stream of functionality – “value,” really – delivered to the customer. This series of papers wraps text and PDF around that ongoing thought-train of his, and they’re nice reads.

You can get the papers by going over to the Cutter page and using the promo code “RELEASEMYTH” to get them for free.

Here’re the summaries of each section:

In Part I, the author explains why he considers the whole release concept a myth. Like a set of nested Russian dolls, the duality of the term “release” often masks a deeper duality. You can view the end user as the primary target of a release or you can view the release as a vehicle for market development. Philosophical as this differentiation of the release might seem, it actually brings to the forefront essential questions about the nature of software and the nature of the market.

Part II of this series suggests that software that is alive and always evolving poses unique opportunities for custom-tailoring solutions directly from Research and Development. It also describes an effective way to distribute custom-tailored solutions, and it highlights partial disintermediation that transforms the traditional software value chain.

Part III identifies a third “secret sauce” ingredient that further accelerates development and deployment. The critical question then becomes, “What do we do with this ultrafast development and deployment?”

Written by Coté

February 4, 2009 at 9:07 am

Posted in The Agile Life

A Social Contract for Agile

with 10 comments

How do you make Agile work in spite of layoffs? The following war story is pertinent.

During the 1967 war between Israel on the one side and Egypt, Jordan, and Syria on the other, my paratroop battalion carried out a helicopter raid on the Egyptian artillery in Um Katef. The raid was quite successful in neutralizing Egyptian artillery fire which was playing havoc with the Israeli infantry. We incurred, however, quite a few casualties as a result of firing bazookas at Egyptian ammunition trucks. The resultant explosions created an inferno that proved lethal to Egyptians and Israelis alike. We retreated when the Egyptians sent in reinforcements to reign us in. We carried back our wounded in dunes in which you sink to your knees even if you are not carrying anything. Also, we carried back our dead. The dead continued to be part of us in spite of not being with us anymore.

The relevance of a paratroop raid some 40 years ago in the Sinai desert to Agile practices in software circles today might appear to be tenuous.  This post takes the view that the two – paratroop units and Agile teams –  are actually quite similar from a leadership perspective. It is next to impossible to get soldiers over the top if they suspect they might be left behind on the battlefield if they were wounded or killed. Likewise, empowerment, collaboration and the willingness to experiment go down the drain when an Agile team expects “casualties” in the form of forthcoming layoffs.

Flying at the Teeth of Layoffs

The design for the 2005 roll-out of Agile at BMC Software did not put in place any planning in advance on how to deal with the impact of possible layoffs on Agile assimilation. Everyone knew, of course, that layoffs might and do happen in the software industry. However, Agile was kept in a different mental “drawer” from the one in which possible layoffs were kept. Planning-wise the two were not really connected.

When we had to carry out layoffs in 2005, we encountered severe cognitive dissonance:

  • On the one hand, the excitement in the business unit about Agile was tremendous. We were witnessing the psychological aspects of Deming‘s System of Profound Knowledge in an unprecedented manner. Skill, pride, confidence and collaboration were building up faster than you could say “Agile”.
  • On the other hand, the layoffs put a damper on all four elements – skills, pride, confidence and collaboration. Folks who worried whether they might be next in line to be laid off were not able to place the Scrum team’s accomplishments ahead of the credit they sought for themselves. The determination to learn more and deeper about Scrum slackened as employees bided their time in anticipation of the next round of layoffs. Obviously, pride and confidence went down the drain.

Reciprocity

Preserving the Agile momentum in the face of layoffs required striking a new balance between what employees were getting out of Agile versus what the company was expecting to accomplish through Agile. I actually viewed it as a kind of balance of payments problem. It was quite clear what the company was already getting out of Agile, and even clearer that the long-term payoffs from successful implementation of Agile could be very significant indeed. We did not, however, have a good answer then for worried employees who asked: “What is in it for me?” A record breaking Scrum implementation 12 months down the road is not too meaningful for an employee who suspects he might not be with the company in 6 months.

It so happened that in 2005 we started seeing job specifications that either asked for or mandated Agile skills. This gave us the clue to a meaningful reciprocity in the face of potential layoffs. By investing aggressively in Agile training we would not “only” improve productivity, we would also enhance marketability of our employees. We would be providing a better safety net for them.

A Social Contract for Agile

My sense in 2005 was that the social contract between employers and employees in the software industry was broken. Without a working social contract, the friction and antagonism can bring a system down. For example, in 1942 – the turning-point year of WWII – 833,000 days of coal mining were lost due to strikes in the British coal industry.

The preliminary thoughts I had on the subject in 2005 turned into a mini social contract. The mini social contract is an agreement between an executive and his employees on the rules of the game that are mutually satisfactory. The mini social contract I used evolved over the years. Here it is in its latest form:

Team, my overarching organizational objective is to preserve our team and its institutional knowledge for our corporation and its customers for years to come. We will achieve this goal by enhancing our software engineering prowess to the level that the resultant benefits will outweigh the repercussions of the current financial crisis. The state of the Agile art should enable us to attain hyper-productivity which will serve as the best antidote to layoffs. In the event that we fail to accomplish hyper-productivity and our assignments fade away, you will find the Agile skills you developed much in demand in the market. Whether you will or will not be with the company in the future, I acknowledge your need to develop professionally as Agile practitioners and commit to invest in your education/training.

Impact of the Social Contract

Applying the social contract produced four effects:

  1. It legitimized discussions on a taboo topic.
  2. It reduced fear while increasing hope.
  3. It was a hard proof point to the folks in the trenches that I as an executive was one of them. This is a most critical aspect in the executive-employee relationship in software companies.
  4. It led to fruitful exchanges with Agile practitioners in other companies who were wrestling with the very same issue. These exchanges helped us improve the social contract.

Criticism of the Social Contract

I got some flak from colleagues on the commitment to invest in Agile training. The concern was that such a commitment, even if it is given as a gentleman’s word, is too open-ended. Proficiency in Agile, it was argued, is quite relative. Hence, when could one state that the contract was “fulfilled” with respect to commitment to invest in Agile training and coaching?

The Fine Line Between Management and Leadership

The criticism of the social contract cited above should not be rejected out of hand. Obviously, one does not want to create expectations that might not be fulfilled.

Quite simply, committing to Agile training and coaching actually illustrates the fine line between management and leadership. As the quip goes – “Leaders do the right things; managers do things right”.  Committing as an executive to Agile training and coaching across the board might appear to be not doing things “right”.  Making this kind of commitment, however, is in the very nature of leadership. And, frankly, it is also doing things “right” for executives who have the best interest of the company at heart.

Where do You Lead From?

We were taught to lead from the front in the paratroop corp. Over the years, I have learned one other important aspect of leadership: leading from within. To me, the social contract is leading from within.

Written by israelgat

February 3, 2009 at 5:32 pm

Israel Joins the Cutter Consortium

with one comment

I noticed this morning that Israel had signed up with the Cutter Consortium:

Israel Gat, recognized and highly acclaimed as the architect of the Agile transformation at BMC Software, has joined Cutter Consortium’s team of Senior Consultants. Israel is currently focusing on enterprise-level Agile deployments. His activities in this space include consulting on R&D transformation, coaching executives on how to implement Agile, and developing business designs that take full advantage of Agile methods.

Knowing all of the casual consulting and more commentary Israel has been doing around Agile over the past few years, this is a natural progression towards formalizing his experience-driven insights into applying Agile Software Development. The Cutter Consortium has always been at the middle of discussing software development, and with the august members on the Agile Project Management team, Israel is in good company.

Also, this means you can schedule time with Israel and Cutter to discuss on work on applying Agile in your organization: either email press@cutter.com or call Kim Leonard  at +1 781 641 5111.

And don’t worry, the blog-posting work-horse that’s Israel will still be pumping out posts here, at The Agile Executive, for sure. In fact, they posts can only get better as Israel builds up and refines more material, experience, and insights from his work with Cutter 😉

Written by Coté

February 2, 2009 at 9:16 am

Posted in The Agile Life

Tagged with ,

Can You Afford the Software You are Developing?

with 15 comments

A reader of The Agile Executive brought up some questions about product retirement in the context of  project teams that use Agile methods. For example: Should a product backed by a hyper-productive Agile project team be retired at the same point that an aging Waterfall product typically would?

The question is important. Customers can get very upset over the retirement of a product, particularly a mission-critical product. Even if the vendor offers a new product that replaces the one to be retired, the operational disruption associated with migrating to the new product is often troublesome. On the other hand, the cost of maintaining software, let alone keeping it current, could be and is often high for the enterprise software vendor.

The answer to the product retirement question ties Agile methods and practices to the fabric and economics of software engineering.  A good way to address the subject is to ask the following two questions:

  • Can you afford the software you are developing now?
  • Would you be able to continue to adequately invest in the software as it evolves down the road?

Rules of Thumb for Affordability

Affordability is, of course, in the eyes of the beholder. Your CFO might see it in quite differently than your CMO. To bring a discussion between the two, or any other forum of CXOs, to a common denominator, you need to get a handle on two numbers:

  • Development cost (including product management and test costs) per story card
  • Development cost as a percentage of product life-cycle cost

Development costs and life-cycle costs vary greatly from one company to another as well as within your company. For example:

  • Off-shore costs can be quite different from on-shore costs
  • The costs of maintaining high quality code are drastically different from those for average quality code. (See Estimating Software Costs by Capers Jones for a detailed analysis of the subject).
  • Productivity of an Agile team can easily eclipse that of a Waterfall team.

Laborious and time consuming that collecting good cost data across development methods, projects, sites and continents might be, you are essentially flying blindly with respect to affordability unless you have very specific cost data.

Until you gather this data, here are two rules of thumb that can be used to get a rough sense of  affordability:

  • A typical figure for development and test cost per story card for enterprise software project teams is thousands and thousands of dollars. It can exceed $10,000.  This (order of magnitude)  figure is for a contemporary software development and test organization in the US that is “reasonably” balanced between on-shore and off-shore development
  • Development cost is typically less than 50% of the total software life-cycle costs. Again, the assumption of reasonable balance between on-shore and off-shore applies

These rules of thumb should be used prudently. For example, Mens and Demeyer report cases in which software development costs constituted a mere 10% of the total life-cycle cost.

What is your Software Evolution Strategy?

In Program Evolution: Processes of Software Change, authors Lehman and Belady summarized years of research on the subject they and various collaborators carried out. Their bottom line is deceptively simple: software is live and always evolving. Furthermore, software decays.

Jim Highsmith uses the following great graph to demonstrate the effect of accrued technical debt on cost of change and responsiveness to customers:

in-can-you-afford-the-software-you-are-developing

Jim points out that no good option exists once the software has decayed to the point of excessive technical debt. Furthermore, once you are in the far right of curve estimation is next to impossible and afforability calculations become pretty useless. You might think about technical debt like debt on a credit card – you become a slave to servicing the debt instead of paying off the principal.

Affordability Revisited

Between the initial development cost and the cost of evolving and maintaining decaying software, many software development projects find themselves in dire need of higher productivity. Hence, a more precise statement of affordability is as follows:

  • Can you afford the software you are developing given your productivity during and after development of the first release?

The productivity results reported for companies successfully using Agile methods such as  BMC SoftwareSirsiDynix and Xebia indicate productivity gains of at least 2X, and often higher, compared to industry average. Everything else being equal you would be able to retire a product backed by a good Agile team later than a product backed by a Waterfall team.

Many Agile teams tend to be inclined to refactor the code on an on-going basis. For example, Salesforce devotes about 20% of development resources to refactoring. As a result, software decay is slower for such teams. They reach the point of no good options in Jim Highsmith’s graph later than teams who do not refactor the code day in and day out.

Refactoring is like flossing your teeth regularly. The dental tape disconnects your bank account from the dentist’s…

Written by israelgat

February 1, 2009 at 9:46 pm