Archive for January 2009
Pragmatic programmers have been wrestling with Template Zombies– folks for whom form takes precedence over content – since time immemorial. The fight has intensified in the late 1990’s and early 2000’s as Agile methods got traction. This tension can now be seen in Agile contracts: precise and definitive contractual language does not easily lend itself to expressing fluid Agile concepts which rely on “infinite” number of feedback loops. This difficulty manifests itself as transaction costs. In particular, the bargaining costs and policing costs associated with Agile contracts can be significant. The number of contract types summarized in Alistair Cockburn‘s list of Agile contracting ideas is illustrative of how tricky it is to wrestle an Agile contact to the ground.
Building a Contract around Agile Principles
As indicated in the post The Core Principle Behind Agile, the effectiveness and efficiency of Agile is based on doing the most important things at any point in time. Hence, an Agile contract must preserve this principle. After all, what is the point of developing software in an Agile manner if the most fundamental Agile principle cannot be contractually adhered to in an economical manner?!
The key to solving this riddle is understanding the fundamental risk each party is striving to minimize:
- The client needs to minimize market risk – the software is being contracted to accomplish some business or market objective
- The Agile provider must minimize delivery risk
These two perspectives do not actually conflict with respect to changing the software along the way. As a matter of fact, being able to change product definition during the development cycle is in the client’s best interest in a world of constant disruption. For the experienced Agile software provider, changes from one iteration to another, sometimes from one day to another, are anyway a standard operating procedure. As long as the Agile process does not change, the delivery risk need not get out of hand.
It follows that the fundamental question under investigation here is the construction of a suitable vehicle for enabling requirements to change during the development cycle without getting bogged down in tedious inter-company bargaining and policing processes. Change in itself is not the core issue.
Money for Nothing
In an Agile 2008 presentation entitled “Money for Nothing and Your Change for Free”, Jeff Sutherland described a novel Agile contract. It is a fixed price contract that allows the client to terminate it at any point in time. When the client terminates a contract, he is only billed for the remaining 20% of unbilled contract value.
The phenomenon on which the money for nothing contract is based is exponential accumulation of value in highly productive Agile teams versus linear payment terms. In the “50-90” case discussed in The View from the Executive Suite post, the customer would be billed for 60% (50+0.2X50=60) of the contract value. The corresponding accumulated business value for the client at the point of contract termination is 90%.
Your Change for Free
In addition to permitting early termination, Sutherland’s scheme allows substitution. The customer can add or change requirements on the fly as long as he is willing to de-commit an equivalent amount of labor. Upon presenting a new requirement, a new story card (call it X) will be added to the release to implement the new requirement. In return, a couple of less hefty story cards (call them Y and Z) will be moved from the release to the backlog. The contract terms do not change. The contract administrator merely notes that X will be implemented instead of Y and Z.
The Firm, the Market and the Law
It is fascinating to consider the money for nothing scheme in the context of the concepts introduced by Coase in The Firm, the Market and the Law. Using the termination and substitution clauses described above, Sutherland reduces Coase’s costs of the price mechanism to the level that makes the Agile contracts viable in the market. How appropriate it is that the reduction in the cost of the price mechanism is done in the context of Agile methods that strive to reduce cost!
In his classical book The Innovator’s Dilemma, author Clayton M. Christensen introduced a framework for analyzing disruptive technological changes. The framework is based on four principles, as follows:
- Companies Depend on Customers and Investors for Resources.
- Small Markets Don’t Solve the Growth Needs of Large Companies
- Market that Don’t Exist Can’t be Analyzed
- The Technology Supply May not Equal Market Demand
Christensen illustrates how the four principles apply to computers, retailing, pharmaceuticals, automobiles and steel at various critical junctures along the road. In the software industry, Netscape’s web browser technology is an enlightening example of disruptive technology in action.
Enterprise Software Innovator’s Dilemma
The key point this charts gets across is that Open Source Software is becoming “good enough”. It has already met or will soon be meeting the minimum requirements of the enterprise customer. By so doing, Open Source Software will steadily gain ground from traditional enterprise software vendors.
Strategic Implications for the Incumbent
The maturation of Open Source Software forces incumbent software vendors to seek differentiation for their proprietary products. Such differentiation usually attempts to follow one or more of the following three patterns:
- Strategy of fear, uncertainty and doubt (FUD), portraying open source products and the communities behind them as not quite appropriate for mission critical applications. As the open source community has been tackling more complicated problems and producing increasingly sophisticated products, the FUD strategy has lost much of its effectiveness.
- Functional differentiation through features that are not available in open source products. This kind of differentiation loses much of its effectiveness once the “good enough” open source software line in the chart above intersects with the enterprise customer requirements line (the purple line in the chart). The extra features might indeed be included in the proprietary software, but the incremental value to the customer diminishes.
- Customizing proprietary software offerings to the particular requirements of top customers. This differentiation stratergy is discussed in the next section.
Given the ineffectiveness of the first two strategies to deal with Open Source, various incumbent enterprise software vendors try to protect their business by way of customizing their vanilla offerings to the particular environments, needs and use patterns of their top customers. Such customization is typically done through professional services engagements for which the customer usually pays. Under this business design, IT service revenues – professional services as well as customer support – often become more important than product revenues. The attach rate – the ratio of service revenues to product revenues tends to hover in the 100- 400% range. In other words, for every dollar a sales rep brings in, in the form of product revenue, he is expected (and often goaled) to bring as much as one to four dollars in service revenues.
Although a high attach rate might be attractive from a revenue stream perspective, it is counter strategic on three critical accounts:
- It completely distorts the economics of the product – a $1 product could for most practical purposes cost $2-5. Readers of the Customer Intimacy post in this blog might recall the characterization given by Crawford and Mathews to Consumer Underworld relationship between vendor and customer: “Be inconsistent, unclear, or misleading in your pricing”.
- It accentuates the customer frustration with the enterprise software model, tilting the balance further in favor of Open Source Software. IT services are seen for what they often are – complementary business to unsatisfactory enterprise software.
- The attach rate dives like a rock in difficult economic times such as the current macro-economic crisis. The phenomenon is described in the recent MGI Research article on the subject.
Agile Based Market-of-One
Hyper-productive Agile teams are sure-footed in moving requirements back and forth between release and backlog. This flexibility in changing the release contents quickly can be channeled toward customizing software to the needs of top customers straight from the R&D lab. Instead of doing it through professional services, R&D tailors custom releases. In other words, the enterprise software vendor produces markets of one in an efficient manner through Agile methods. If these efficiencies are passed on to the customer, the customer-vendor relation with respect to price could be transformed from Consumer Underworld all the way to Customer seeks the Company.
Although the development of market of one software through Agile methods in the lab can be quite effective, its distribution, deployment and subsequent operation in the customer environment need to be equally efficient. The “containerization”, distribution and provisioning of the output of hyper-productive Agile teams enables the achievement of end-to-end agility. This topic will be explored in forthcoming posts with special emphasis on three important business aspects, as follows:
Talking about Agile to executives can be like feeding turkey to your family on Thanksgiving; it puts everyone into a sleepy stupor.
Jean goes on to recommend Lean as a way to get the message across. To quote her:
Through Lean, I am able to tap into discussions about waste versus value. I can engage the executive team into looking at their entire organization. And, these “seeing the whole” discussions help them then understand why they should care about an engineering groups adoption of Agile.
Working the Lean angle the way Jean recommends could most certainly open the discussion and enrich it. Success, however, depends on a certain kind of mindset of the executive you are talking to. This mindset is nicely described in H. Thomas Johnson‘s article Manage a Living System, Not a Ledger:
…a business organization cannot improve its long-run financial results by working to improve its financial results. But the only way to ensure satisfactory and stable long-term financial results is to work on improving the system from which those results emerge.
If you accept the premise expressed by Johnson, you need to consider two kinds of possible dialogs to get your Agile message across:
- Agile focused dialog about the what, why, how and when of Agile. You do this kind of dialog when your counterpart is already at the point of looking for sustainable value generation, not for a magic bullet.
- Recipe for success dialog. This kind of dialog establishes the foundation required for the first dialog when the executive is not quite ready yet for Agile. Give the executive the opportunity to think deeply on his algorithm for success. It may take a few conversations until the algorithm is spelled out. Once it is, you can start working with the executive on what Agile is and how it might fit into his algorithm for success.
An extremely important point to keep in mind is that mindsets evolve. The set of business circumstances under which an executive is operating can lead to a certain mindset. The mindset can be quite different at another point in time due to change in strategy, priorities and constraints. A good fit for Agile may not exist today, but it might exist tomorrow.
“I don’t care about Agile; I care about money!” These words, said to me not too long ago by an executive with whom I was discussing the roll-out of Agile in his division, are the impetus for this post. Whether you are or are not in R&D, Agile can have a significant impact on you and on your company, including the bottom line.
To keep things simple, listed below are a few “front lobe” considerations for CXOs – CEOs, CFOs, CIOs, CMOs, COOs and CTOs. The listed considerations are neither “good” nor “bad” – they are what they are. They are rendered here for the thinking when an Agile topic comes up in the domain(s) you are responsible for.
Some of the considerations for CXO’s are linked to posts, articles or books that already address the subject to some degree. As additional related posts will get rolled out on this blog, they will be linked to the bullets below. In other words, the Agile Considerations for CXOs post will evolve, serving as a quick index for an eclectic list of Agile CXO considerations.
- Business designs to utilize the power of Agile methods
- Market disruption by competitors who adopt Agile
- The Agile enterprise
- Efficiency; Lean; cost structure of the corporation
- Capitalization of (Agile) software
- Agile contracts
- Agile Business Service Management
- IT modernization
- Bridging the chasm between product development, deployment and operations
- Effective use of Agile in the data center
- Guidelines for BSM Investments
- Reconfiguring the business
- Agile portfolio management
- Off-shoring strategy vis-a-vis near-shoring vis-a-vis captive US teams; Distributed Agile
- Can your company afford the software it is developing?
- How many product generations should the support policy cover?
- Application life-cycle management
- Enterprise software considerations
“50-50” is the rule of a thumb many audiophiles use for configuring a good stereo system. 50% of the budget goes toward the speakers; the other 50% toward everything else in the stereo system. The reason is quite simple: stereo systems succeed or fail on the merits of the speakers.
Whatever your Agile budget might be, a good starting point is to spend about 50% of the budget on training, consulting and coaching during the first year of Agile rollout. This figure will probably go down to about 25% during the second year; precious little thereafter. Such budget allocation was used at BMC Software in its Agile rollout and operation between 2004 and 2008: about 50% of the total Agile budget in 2005; 25% in 2006; single digit percentages in 2007 and 2008.
First Year Agile Budget
Starting with 50% for consulting, training and coaching during the first year is rooted in the software engineer being a craftsman. A craftsman learns and develops through apprenticeship. He learns from the masters. The popular Program With the Stars sessions in various software development conferences recognize the power of the apprecnticeship paradigm. A good review of the power of such a session in the Agile 2008 conference can be found in the Working Together… with Technology post by Andrew Shafer.
If you accept the premise of the software engineer as a craftsman, you need to invest in consulting and coaching more than in training. A two or three day Scrum Training class for numerous product managers, developers and testers in your company is, of course, a good start. However, it is the day-in day-out consulting and coaching that will make the training applicable. There is no substitute to a competent Agile coach saying in the middle of a stand-up meeting “Folks, what happened to our collaboration? We are not getting the benefits of the wisdom of teams.”
In addition to the coaching needs of the teams, you as an Agile executive will probably need some coaching by a sure-footed Agile executive coach. Topics should include the following:
- Your role in the Agile roll-out
- Deliverable you own in the Agile roll-out
- Behaviors that are supportive of Agile
- Identification of promising indicators for Agile roll-out
- Identification of early warning sign for Agile roll-out going the wrong way
- Synchronizing release trains across multiple software development methods
- Impact of Agile on downstream functions
Don’t think about these coaching items as luxury you can’t afford. Rather, it is money well spent. It is the cost you as an executive need to pay for being on the Agile train. The train might leave the station without you if you do not invest in your Agile education.
Second and Third Year Agile Budgets
The rationale for reducing the investment in consulting, training and coaching in the second and third year is simple. Various Agilists in your company would become experts themselves. Needless to say you will continue to invest in their skills by sending them to a conference such as Agile 2009. Much of the consulting and coaching, however, should over time be done by your home-grown Agilists.
Consider it an early warning sign if the assimilation of expertise in your company has not generated Agile experts in your teams within a couple of years. You could, of course, allocate your Agile $$ in the second and third year on the 50-50 basis recommended above for the first year. But, chances are something is not working well with your Agile roll-out. The norm in successful Agile roll-outs is to harvest a whole bunch of quotes like the following quip made in 2006 by a QA Director with BMC Software:
There had never been a thought towards returning to Waterfall. We only think about how to be more Agile, how to do this better. No one wants to go back!
In the Myth of Excellence, authors Crawford and Mathews examine customer-vendor interactions and relations. Interactions are evaluated by the following attributes:
The customer-vendor relation is assessed on the following four level scale:
- Consumer Underworld
- I: Consumer accepts the company (Operate at Par)
- II: Consumer accepts the company (Differentiate)
- III: Consumer seeks the company (Dominate)
Combining these two “dimensions”, Crawford and Mathews present a 4X5 table, as follows:
As can be seen in the table, the customer Experience at the Consumer Underworld level is dehumanize me, disrespect me, ignore my needs. In contrast, the Experience at Level III is establish intimacy with me by doing something no one else can.
Between the Customer and the R&D Lab
Quality of software and responsiveness by customer support are often a matter of furious exchanges between the customer and the software vendor. Escalations to the executive level in order to get better response on various software defects and deficits are quite common. On many occasions, an escalation becomes the battle of the spreadsheets. The customer steps into a meeting armed with a long spreadsheet of defects that have not been fixed in a timely manner. Typically, the customer also presents a long spreadsheet of unfulfilled requests for enhancements (RFE’s) that were promised, implied or expected. The vendor typically comes to such a meeting with as long spreadsheets demonstrating how responsive and diligent the service to the customer has been. Even with good intentions on both sides, such meetings are often loaded with defensive routines. Frequently they become adversarial. Some even conclude with a forceful customer statement such as “I want my money back”.
The mirror image of such customer escalations is the chronic complaint in the R&D lab about lack of good requirements. By the time the needs of multitudes of customers have been translated and aggregated to produce a market requirements document, a lot has been filtered and some has been distorted. Furthermore, as product managers are often stretched very thin, developers and testers interpret requirements only as best as they can. The loss of direct touch between customer and vendor can lead to results such as those reported by the Standish group in their 2002 “Chaos” study:
A Model for Including the Customer in the Agile Process
The state of affairs described above has multiple causes which will be explored in depth in forthcoming posts. Whatever the causes might be, direct connection between the customer and the developer/tester can help reduce mutual pains. The method for so doing is straightforward:
- Customer participates in Release Planning
- Customer participates in the bi-weekly demo.
- Customer participates in Release Retrospective
Regular participation in these three activities amounts to 100-150 hours per year. The advantages to the customer of making this time commitment are significant:
- Influence product direction
- Gain deeper understanding of the product
- Develop relations with members of the vendor’s project team
- Gain experience in Agile methods
For the software vendor, customer participation along the lines recommended above is as beneficial, enabling the vendor to:
- Get over the us versus them mindset
- Raise the probability of doing the right product and doing the product right
- Gain precious insights how the product will be used
- Win over an evangelist: a customer employee working in such a manner with a software vendor’s Agile team tends to be proud of the product and his contributions to it
Back to The Myth of Excellence
Examine the Customer-Vendor Interaction and Relations table pasted above. The inclusion of the customer in the Agile process directly addresses the Experience and Service attributes for Level III:
- Experience: Establish Intimacy
- Service: Customize the product or service to fit my needs
An indirect benefit that is likely to materialize is with respect to the Product attribute. An agile team that experiments on an on-going basis, using the bi-weekly demos as the “firewall” for containing unsuccessful experimentation, is likely to be quite innovative without accruing higher risk than acceptable. Hence, inspire me with an assortment of great products is an outcome that could be expected in the longer term.
As pointed out in the recent Cutter Consortium essay, experienced Agile teams can cater to the needs of their top customers by providing Market-of-One. The very many advantages of so doing will be discussed in forthcoming posts.
The Choosing Between Two Disruptions post highlighted the fundamental predicament one faces with respect to large scale Agile deployment – Is an in-house Agile disruption initiated by you preferable to a market disruption through Agile which you will need to react to? I trust you will decide to give Agile a try even if you are concerned about a possible in-house disruption. This being the case, you will need to make numerous decisions about the way you will roll Agile. This posts deals with the very first decision to make with respect to so doing: Choosing the Agile learning paradigm which will be most effective in your company.
Learning and Thinking
Years ago, when I was a student at the Israel Institute of Technology (Technion), we had a quip about the learning paradigm: “In the second year you understand what you were taught in the first year; in the third year you get what you learned in the second year”. The quip captured the essence of the de facto learning paradigm that largely prevailed at the Technion back then. The Technion probably aspired for a very different learning paradigm, but circumstances in Israel during these years dictated a different pattern. As many of us were called to reserve army service for 30-60-90 days a year, we did not usually have enough time on our hands to think deeply about what we were studying. We absorbed a lot of information quickly in a somewhat mechanical way. Only over an extended period of time were we able to generate sufficient cycles to think deeply about what we were taught. This later thinking eventually led to understanding. We assimilated the curriculum with an average one year lag.
How will your Company Learn?
Business realities these days might force you to introduce Agile in a “Technion style” manner – you introduce Agile as a recipe. You do your best, of course, to explain the deeper thinking behind Agile, but you focus on the “How?” You bet the Agile roll-out on getting results through applying Agile practices. Once you have a couple of Agile success stories under your belt, people will naturally get curious about the underlying principles. They will rediscover them, assimilate them and propagate them.
The other way to go is to start with the deeper truths – software engineering laws, anomalies and trends that over the years led to the rise of Agile. You start with the “Why?”, bootstrapping your way towards the “How?”
The Reengineering Alternative
William E. Schneider’s book The Reengineering Alternative is a must read prior to making the decision how your company will learn Agile.
Schneider characterizes four core organizational cultures and provides the tools to self-determine your corporate culture and to assess the strengths and weaknesses that go with it. Each of the four core cultures – Control, Collaboration, Competence, Cultivation – has its own characteristic response to change. Choosing the learning paradigm best suited to your company is largely a matter of cultural fit.
For example, according to Schneider, the Control culture mandates change. Hence the “Technion style” approach discussed above could be quite appropriate if your company is of the Control ilk.
Time for Thinking
Reducing the pressure on a software team to the point at which thinking can be done in parallel with coding and testing is of immense value. Higher levels of quality and innovation are visible pretty quickly. Longer term, code adaptability and the ability to evolve the code pay off big dividends. These benefits are particularly important for teams who incorporate bleeding edge technologies in their product. You must have the time to think about such technologies and how to utilize them.
The pain of having an Agile project team gated on the availability of some feature from a Waterfall team can be significant. Painful though it may be, such a situation could actually be seen as a blessing in disguise. The team members have time to think. And, they have the time to refactor their code.
The Agile Leader
In addition to helping you make your first decision with respect to the Agile roll-out, Schneider’s book will give you a precious clue as to your role as an Agile leader:
A leader’s effectiveness can be measured by the degree the leader’s approach is integrated with the organization’s core culture.
A fascinating aspect of this sound advice is the culture within a culture situation. As Agile tends to create its own culture, the Agile leader often needs to align the Agile culture with a broader corporate culture. To succeed, the Agile leader needs to create such an alignment while protecting the Agile culture. Various means for so doing are described in the presentation Leading from Within.