The Agile Executive

Making Agile Work

Posts Tagged ‘Highsmith

Questions from “Ask an Expert”

leave a comment »

I had the pleasure of participating in the May 7 and May 14 sessions of the Agile Austin “Ask an Expert” Service. You can read the questions/topics brought up in these sessions by clicking here. Following are two observations from the questions cited so far:

  • Enterprise readiness issues are rarely understood, let alone addressed in advance of an Agile roll-out. The focus on the “hows” of the process seems to consume the energy of the Agile practitioners. Precious little is left for the “whys.”
  • Many questions (and the discussions that follow) are actually about the software engineering fabric, not about Agile per se.

The two are actually related. It is too easy to try to boil the ocean if you do not think of Agile as a single “layer” in the overall software engineering fabric, pretty much along the lines one thinks of a layer such as Transport in the OSI Model. Needless to say, trying to boil the ocean can consume you to the point nothing is left for the deeper understanding of the “whys” of Agile.

The reader is encouraged to take a look at the post entitled The House Jim Built. The two views of Agile given in this post by Cutter consultants Jim Highsmith and David Spann capture the essence of Agile in a lucid manner. You can start with either of the two views, using the one you prefer as a guide to placing Agile in the bigger picture.

(Please note Anne Mullaney‘s kind offer in the thread accompanying the post to send the full copy of Jim’s E-Mail Advisor to readers of the blog. David’s Research Report is in the public domain).

Written by israelgat

May 18, 2009 at 4:51 pm

A Single Point of Accountability

leave a comment »

Lack of executive support is often flagged as a major problem for Agile adoption. Jean discusses “checkbook commitments” from executive management in a recent post. Christophe Louvion has highlighted the issue during the Rally event in Los Angeles. I certainly have for quites some time been (and still am) of the opinion that executive support is critical for Agile success.

In the course of working on my forthcoming presentation at the Agile Roots conference, I took a fresh look at quite a few of the convictions I hold, including lack of executive support. Having such support, of course, is awesome. However, is the difficulty in securing executive support the fundamental problem or is it a symptom of an underlying problem?

A couple of years ago my colleague and friend Yechiam Yemini made a very astute observation on accountability in system management. Yechiam observed that system management applications of the “You got a problem on your hands” variety generally don’t get endorsed by IT executives if they do not indicate clear accountability. The last thing in the world an IT executive wants is finger pointing between his/her network management folks, the storage management team and the help desk expert. A lot of time and effort is wasted in resolving such situations. IT executives hate them with a passion.

I am starting to think a similar phenomenon might be manifesting itself with respect to  Agile adoption. For example, if things do not go well for a Scrum project, is it a matter for the Scrum Master, the Product Owner or the self-organized team? From an enlightened Agile perspective, the whole thing is about the wisdom and commitment of teams. It might however be seen in quite a different light by the executive who has not had the opportunity to immerse himself/herself  in Agile.

“We are all in it together” is a quip frequently used by executives in time of crisis. When the quip is sincere, it can provide the underpinnings on which to develop a deeper understanding of accountability in the Agile context. Part of being in it together is the executive’s accountability to follow Agile values and principles. The house metaphor Jim Highsmith proposed can be used very effectively in the context we are discussing . One starts building a house by laying the foundations – Agile values and principles in Jim’s metaphor. Pillars and roof come later.

Written by israelgat

April 27, 2009 at 12:54 pm

The House Jim Built

with 10 comments

Colleague Jim Highsmith uses an interesting metaphor in a Cutter Advisory published a few days ago:

Visualize a house structure with a roof, a foundation, and three pillars… The roof is business goals — the rationale for implementing agile methods and scaling to larger agile projects. The foundation is agile values or principles — principles that need careful interpretation as to how to apply them to larger teams. And finally, the three pillars: organization, product backlog, and process/practice.

The simplicity of the metaphor makes it quite effective in communicating what Agile is in a concise manner. The need to do a better job conveying the concepts (as distinct from the practices) of Agile was highlighted during Rally’s recent event in Los Angeles. Numerous participants in the event felt they have not managed to get the Agile premise across to their executives.

Jim’s metaphor is nicely supplemented by the following short characterization of Scrum by Cutter Consultant David Spann in a recent Report:

Scrum, as a management methodology, is elegant in its design, requiring only three roles (i.e., product owner, ScrumMaster, and self-organized team), three ceremonies (sprint/iteration planning, daily Scrum/debrief, and sprint review meetings), and three artifacts (product and sprint backlogs and the burndown chart) — just-enough practical advice so agile teams do not overcomplicate the development lifecycle with too much ceremony and documentation.

Between Jim and David, you should have no problem carrying the day in discussing Agile methods with an uninitiated executive.

[David’s Report is available in entirety here. You will need to subscribe to the Cutter service in order to get a copy of Jim’s Advisory. The excerpt cited above, taken from the public summary of the Advisory, is largely self-contained and should suffice for delivering the core message].

Written by israelgat

April 15, 2009 at 7:04 pm

Don’t agile the Agile

with one comment

I have been known to frequently say say don’t agile the Agile. I use this quip in two ways:

  1. To demonstrate the importance of patience in rolling out Agile. I consider patience a critical ingredient in the ‘secret sauce’ required to properly introduce Agile in a company that is new to Agile methods.
  2. To set expectations realistically for executive who wonder about the payoff period for an Agile roll-out.  It is particularly important to do so when an executive has a view of Agile as a silver bullet.

During the Rally Agile Success Tour I realized many Agile champions are under pressure in their companies to promise quick results. No doubt, Agile gives you good forums to demonstrate what has been accomplished -bi-weekly demos and frequent releases. But, Agile also introduces a lot of change:

  • The software changes. You are doing new software, evolving software or fixing software.
  • The software process changes. You are introducing Agile methods.
  • The organization changes. New organizational entities such as Scrum teams get formed and evolve.

These three dimensions of change take place simultaneously. You and your teams need time to assimilate the changes. In particular, techniques to manage inter-dependencies between the three could require a fair amount of time  to master.

Jim Highsmith told me he asks the executive with whom he discusses Agile roll-out to recount an example how his/her organization has recently managed change. I borrowed this good question from Jim and I use it often. The typical response to the question is a minute of silence followed by something like “this is a very good question”. In most cases, the executive “fails” to provide an example that has as many dimension of change as an Agile roll-out would have. When this point is reached, the message don’t agile the Agile is heard.

(Note: Agile roll-out can and often will introduce additional dimensions of change. For example, you might need to revise your customer support policy as well as the way your Support organization functions. I do not mention such secondary changes in this post as the three primary dimensions indicated above are more than plenty).

Written by israelgat

April 14, 2009 at 11:43 am

Why Agile Matters

with 9 comments

Agile matters because it is a critical factor in the techno-economic process. In fact, it fits nicely in both the classical theory of techno-economic cycles, and the latest revisions to the theory. Whichever school of thought you might follow with respect to what the nature of the current macro-economic crisis is and what the implications for your company might be, Agile will serve you well.

Technological revolutions

In Technological Revolutions and Financial Capital, author Carlota Perez views technological revolutions as part of techno-economic cycles. According to Perez, techno-economic cycles are composed of a sequence of events that proceeds in the following way:

  1. A major technological innovation introduces a new infrastructure
  2. The new infrastructure disrupts both industry and commerce (and very possibly society)
  3. Overtime, the new technology gets to be understood; it then gets harnessed
  4. As the new technology gets harnessed, confidence in it grows and it gets used broadly. Consequently, in good time, the new infrastructure itself becomes a stabilizing force
  5. A new technological innovation disrupts the prevailing order and gives rise to start a new techno-economic cycle

Perez identifies five successive technological revolutions between the 1770s and the 2000s:

slide1

To summarize, Perez perceives that technological revolutions are subject to laws of cyclicality. The cardinal law in the cyclicality is that intertia eventually becomes the legacy of successfule innovation.

Has the techno-economic paradigm been disrupted?

In a recent Harvard Business Review article, authors Hagel, Brown and Davison suggest that we are witnessing a profound shift:

The historical pattern – disruption followed by stabilization – has itself been disrupted.

Hagel, Brown and Davison perceive an exponential pace in information and telecommunication. The exponential pace creates a new reality and a new order:

Because the underlying technologies are developing continuously and rapidly, there is no prospect for stabilization. Businesses and social institutions constantly find themselves racing to catch up with and learn the steadily improving foundational technologies…  The core technology infrastructures that once formed the bedrock have turned into plasma.

Choosing between the two theories

The debate about which techno-economic paradigm – the Carlota Perez paradigm or the Hagel, Brown and Davison paradigm – is better suited for the current circumstances is beyond the scope of this blog. Be it as it might, Agile fits well with either paradigm, though in a different manner.

Agile as a low cost input

If you believe the classical techno-economic paradigm a la Perez continues to prevail, you need to think of software produced by hyperproductive Agile team as a low-cost input in the techno-economic system.  If Agile is massively adopted, it will satisfy the four conditions stipulated by Perez for an input to be a key factor in the technological revolution:

  1. Clearly perceived low – and descending – relative cost
  2. Unlimited supply for all practical purposes
  3. Potential all-pervasiveness
  4. Capacity to reduce the cost of capital, labour and products as well as to change them

The potential  implications of Agile software getting to satisfy these four conditions are far reaching. According to Perez, the current technological cycle started in 1971 with the introduction of the microprocessor by Intel. Hence, one might think of Agile software as a “mid-life kicker” to the current techno-economic cycle.

At the micro-economic level, the implications of Agile as a low-cost input for traditional enterprise software vendors are dire. As pointed out in the post Enterprise Software Innovator’s Dilemma, some low-cost input effects are already manifesting themselves through Open Source Software.

 Agile as agile

If you subscribe to the “bedrock into plasma” paradigm advocated by Hagel, Brown and Davison, you need to view Agile from the point of view of, well, agility. Irrespective of whether your business strategy calls for adapting to the market or shaping it, speeding up the business, not “just” R&D, is absolutely of the essence. You need to apply the Agile principles to the way you develop your business and your customers.

The point in nicely illustrated in The Lean Startup presentation by Steve Blank and Eric Ries. The authors are emphatic in their recipe for success:

Winners are those that can move faster than their competition.

To move faster than the competition, Blank and Ries recommend applying Agile principles to customer development. In so doing, they extend Agile beyond the traditional starting point when the (customer) problem is supposed to be known but the (software) solution is unknown. They elevate the paying fields to the level in which customer development is done in  parallel to product development. This kind of uninterrupted flow in the feedback loop between developer and end-user makes Agile extraordinarily powerful. 

For further research

I encouraged you to consider the ramifications of the ideas presented above and to comment on them in this blog and elsewhere. Of the numerous avenues that we can explore in pursuing the boundaries of Agile, the following topics are particularly intriguing:

  • Alan Fusfeld introduced the Technological Progress Function as a tool to quantify technological progress as function of time and scale. The opportunity exists to combine software metrices, the technological progress function, and the techno-economic paradigm in a unified model.
  • As pointed out above, inertia eventually becomes the legacy of successful innovation. Conversely, one needs to overcome inertia in order to introduce successful innovation. Jim Highsmith‘s recommendation to innovate through experimentation a la Agile is very powerful at the project level. It is not fully clear how a similar approach could be facilitated at the policy level.

Beautiful Software

with 3 comments

Peggy Reed shared with me her thoughts about beautiful software. To quote Peggy:

Beautiful software goes beyond elegant GUI and elegant APIs.

Peggy will elaborate on this intriguing subject on March 18 in Denver as part of the Agile Success Tour organized by Rally Software. Until she discloses her “secret sauce”, I would allow myself to share one fascinating linkage: IMHO Peggy’s thoughts about beautiful software are very synergistic with Jim Highsmith‘s ideas about intrinsic quality.

Both the Rally team and I will elaborate on the subject after the March 18 event. Stay tuned…

Written by israelgat

February 26, 2009 at 12:14 pm

Can You Afford the Software You are Developing?

with 15 comments

A reader of The Agile Executive brought up some questions about product retirement in the context of  project teams that use Agile methods. For example: Should a product backed by a hyper-productive Agile project team be retired at the same point that an aging Waterfall product typically would?

The question is important. Customers can get very upset over the retirement of a product, particularly a mission-critical product. Even if the vendor offers a new product that replaces the one to be retired, the operational disruption associated with migrating to the new product is often troublesome. On the other hand, the cost of maintaining software, let alone keeping it current, could be and is often high for the enterprise software vendor.

The answer to the product retirement question ties Agile methods and practices to the fabric and economics of software engineering.  A good way to address the subject is to ask the following two questions:

  • Can you afford the software you are developing now?
  • Would you be able to continue to adequately invest in the software as it evolves down the road?

Rules of Thumb for Affordability

Affordability is, of course, in the eyes of the beholder. Your CFO might see it in quite differently than your CMO. To bring a discussion between the two, or any other forum of CXOs, to a common denominator, you need to get a handle on two numbers:

  • Development cost (including product management and test costs) per story card
  • Development cost as a percentage of product life-cycle cost

Development costs and life-cycle costs vary greatly from one company to another as well as within your company. For example:

  • Off-shore costs can be quite different from on-shore costs
  • The costs of maintaining high quality code are drastically different from those for average quality code. (See Estimating Software Costs by Capers Jones for a detailed analysis of the subject).
  • Productivity of an Agile team can easily eclipse that of a Waterfall team.

Laborious and time consuming that collecting good cost data across development methods, projects, sites and continents might be, you are essentially flying blindly with respect to affordability unless you have very specific cost data.

Until you gather this data, here are two rules of thumb that can be used to get a rough sense of  affordability:

  • A typical figure for development and test cost per story card for enterprise software project teams is thousands and thousands of dollars. It can exceed $10,000.  This (order of magnitude)  figure is for a contemporary software development and test organization in the US that is “reasonably” balanced between on-shore and off-shore development
  • Development cost is typically less than 50% of the total software life-cycle costs. Again, the assumption of reasonable balance between on-shore and off-shore applies

These rules of thumb should be used prudently. For example, Mens and Demeyer report cases in which software development costs constituted a mere 10% of the total life-cycle cost.

What is your Software Evolution Strategy?

In Program Evolution: Processes of Software Change, authors Lehman and Belady summarized years of research on the subject they and various collaborators carried out. Their bottom line is deceptively simple: software is live and always evolving. Furthermore, software decays.

Jim Highsmith uses the following great graph to demonstrate the effect of accrued technical debt on cost of change and responsiveness to customers:

in-can-you-afford-the-software-you-are-developing

Jim points out that no good option exists once the software has decayed to the point of excessive technical debt. Furthermore, once you are in the far right of curve estimation is next to impossible and afforability calculations become pretty useless. You might think about technical debt like debt on a credit card – you become a slave to servicing the debt instead of paying off the principal.

Affordability Revisited

Between the initial development cost and the cost of evolving and maintaining decaying software, many software development projects find themselves in dire need of higher productivity. Hence, a more precise statement of affordability is as follows:

  • Can you afford the software you are developing given your productivity during and after development of the first release?

The productivity results reported for companies successfully using Agile methods such as  BMC SoftwareSirsiDynix and Xebia indicate productivity gains of at least 2X, and often higher, compared to industry average. Everything else being equal you would be able to retire a product backed by a good Agile team later than a product backed by a Waterfall team.

Many Agile teams tend to be inclined to refactor the code on an on-going basis. For example, Salesforce devotes about 20% of development resources to refactoring. As a result, software decay is slower for such teams. They reach the point of no good options in Jim Highsmith’s graph later than teams who do not refactor the code day in and day out.

Refactoring is like flossing your teeth regularly. The dental tape disconnects your bank account from the dentist’s…

Written by israelgat

February 1, 2009 at 9:46 pm

Uncertainty, Complexity, Correctness

with 2 comments

The most frequent misconception I encounter in preliminary stages of Agile adoption is about the exact “pain” Agile addresses. Time and again I witness the surprise of executives, who are not deeply versed in software engineering, when I point out to them that poor technology and packaging choices often manifest themselves as process pains.  In many ways it is like the way pain “travels” in the human body. The back muscle I pulled yesterday during a long flight led to contraction of my neck muscles at night, giving me a headache today. A couple of  Tylenol caplets might help some, but a muscle relaxant is likely to be much more effective.

Three Dimensions to Consider

Following Jim Highsmith’s teachings on coping with uncertainty versus coping with complexity, the conceptual framework I use to frame the subject in the context of Agile engagements has three dimensions, as follows:

  • Uncertainty
  • Complexity
  • Correctness

I literally ask the person with whom I am discussing a project to characterize the nature and status of his project for me, and more importantly for himself, in terms of these three dimensions. Once the project has been characterized in such a manner, our discussion progresses to how each of the three project dimensions could be addressed.

Uncertainty versus Complexity versus Correctness

Agile is all about effectively addressing uncertainty, I say. I stress that Agile does not address complexity per se. It might indirectly help with complexity if it leads you towards deeper thinking about Complex Adaptive Systems. For example, you might consider evolving the product architecture in the course of your Agile project instead of pre-defining it. However, Agile is not a “medicine” for complexity pains.

Nor is Agile about correctness. A hyper-productive Agile team could actually go fast nowhere implementing a poorly conceived product. The “real time” feedback  loops of  the project team might help uncover that a product is mis-conceived. However, independent of the team feedback, you still need to determine what correctness means to you and how you would assess it as the product evolves.

Levels of Correctness

An intriguing question for enterprise level Agile deployment is at what level you should “measure” correctness:

  • Product level?
  • Solution level?
  • Service level?
  • Business process level?
  • Strategy level?
  • Policy level?
  • All of the above?

I will address this important question in length and depth in forthcoming posts on Agile Portfolio Management.

Written by israelgat

January 17, 2009 at 9:46 am