Archive for July 2009
In this episode, Israel and I talk with Jim Highsmith. We center the discusion around the new edition of Jim’s book, Agile Project Management, but this pulls in all sorts of general discussion about Agile:
- How did Jim get into Agile? Going from traditional software development to Agile.
- How does Rapid Development compare to Agile? Tools in RAD vs. tools in Agile.
- Pivoting on a mention of Jim being in China, I ask him about cultural differences of applying Agile, for instance, based on geo-cultural differences.
- What’s new in the new edition that leads into larger applications of Agile? Release planning, “scaling” self-organizing teams, governance issues, and measuring.
- How does Agile work in a systems, or hardware plus software situation.
- Israel asks Jim for some advice on synching up software developed in an Agile fashion with hardware folks. There’s primarily more coordination and dependency management between teams and features.
- Release planning – most Agile teams focus on iteration planning, without peaking up to concerns at the release level, e.g., budgeting, timing, and marketing concerns.
- How can “the business” get involved with the process to make sure focus is kept on the release?
- What’s this “scaling” business? Scaling a team up in size, or scaling a team out in a distributed process.
- Israel and Jim then dig into distributed scaling, adding in off-premise teams and collaboration.
- Tracking and measuring things from a (business) strategic orientation. This hits on keeping track of value (will this software make us money?), quality and other “metrics” over time. Who is that determines this “value” ongoing? Getting people to figure out “value points.”
- Israel then asks Jim for a retrospective on where we’ve been and are after the Agile Manifesto.
Also, see Jim’s sum-up of the new content in the book.
I am running for the Agile Alliance board. Here is my position statement:
Agile to me is about finding my voice. For most of my professional career I was entrusted with developing and bringing to market large scale enterprise software systems. Fulfilling and rewarding that so doing was, rarely had I experienced the great excitement that comes from the pursuit of a bigger purpose. Over the past few years Agile has been giving me this extra gratification. I feel privileged and fortunate to participate in and contribute to a movement that has the potential to transform quite a few industries.
My voice has been expressed in various speaking engagements, research notes by the Cutter Consortium, blog posts in The Agile Executive, and tweets under the handle agile_exec. I am primarily concerned with elevating Agile to the enterprise level, making certain Agile “islands” scale up, scale out and scale downstream. Moreover, I push toward devising business models that utilize the power of Agile instead of shoe horning Agile methods to fit arcane business designs.
As an Agile Alliance board member, I will focus on mainstreaming Agile methods with an eye toward making a significant economic impact. I share the concern Diana Larsen expressed in a recent Agile Roots panel: Agile as a term has crossed the chasm, but Agile as a method might not. The main obstacle IMHO is that our business fabric has not caught up with Agile methods. Software capitalization and Agile contracts are two good examples of areas which are not yet where they need to be. I plan to address both, and then some, if I get elected.
If we as a movement succeed in making Agile cross the chasm, the economics of software, of products in which software is embedded and of business processes that utilize software could change dramatically. As software is becoming pervasive, Agile software has the potential to become a low cost input in our economy. The macro-economic effect of this descending cost of software could be as powerful as that of the prosperity ultra cheap oil (as energy source) produced during the period 1908-1971. I am committed to doing my bit toward this worthy goal through the Agile Alliance.
Rally’s July 23 Agile Success Tour in Washington, DC was somewhat unique demographically. About 50% of participants work for the government. Moreover, many of the commercial enterprises represented in the event derive a significant amount of their revenues from federal government contracts. The Agile challenges encountered by these folks reflect practices that are not necessarily applicable to “pure” commercial environment. For example, one of the participants asked me about Agile for a project of 500 developers/testers in which her company is the prime contractor for 100 subcontractors! (Recommendation: must devise a business design enabling her company to profitably invest in laying a joint Agile infrastructure across all these subcontractors. Such infrastructure leads to standardization of the Agile data. Click here for details).
In spite of the different demographics, most of the Agile issues brought up in DC were quite similar to those expressed in previous Agile Success Tour events. The bureaucracy with which various Agile champions in DC need to deal with might be stricter (due to security/confidentiality aspects of much of the development carried out in DC), but the underlying needs and dynamics are not really different from those in other cities in which Success Tour events were held.
Here is a sample of enlightening threads I listened to or participated in during the event:
- The business fabric has not quite caught up with Agile methods. In particular, Agile contracts are not yet where they need to be. The costs associated with “ECOing the contract” each time a change in requirements is made offset the methodical benefits of Agile. We need to find a way to “encode” Agile principles in contracts.
- In pitching software methods to their executives, Agile champions need to go beyond the benefits of Agile. Risk and risk mitigation are of equal importance. See The View from the Executive Suite for detailed guidance on the subject.
- The benefits of Agile have to be expressed in terms of the business of the business. One has to go beyond capturing “just” the operational benefits and address financial and business benefits. Peter Drucker’s famous quip “Companies make shoes!” applies. Click here for examination of the quip in the Agile context.
- Innovation through affordable experimentation is an Agile benefit that is under-represented in many discussions Agile champions hold. See the new edition of Jim Highsmith‘s Agile Project Management for an excellent discussion of this critical benefit.
- Agile is about uncertainty, not about complexity. To demonstrate the power of Agile, choose a project of high uncertainty. Complexity in such a project depends on your risk tolerance – it could be either low or high. Be aware that various issues related to complexity might manifest themselves on the surface as process issues. See Uncertainty, Complexity, Correctness for an in-depth discussion of the subject.
Next stops of the Agile Success Tour “train” are in Boston, Chicago, Seattle and London. Stay tuned…
Forrester’s Tom Grant posted a podcast with me on Agile adoption. Here is an excerpt from Tom’s preamble to the subject:
Israel Gat illuminates the ways in which Agile adoption depends on organizational and cultural factors. We also muse about Helmut von Moltke, 19th century military Agilist. (Go look it up!)
Click here for Tom’s preamble in entirety, including the link to the podcast.
Rally’s Alan Atlas shares with us his experience as the first full-time Agile trainer/coach with Amazon. His account is both enlightened and enlightening. He connects the “hows”, “whats” and “whys” of Scrum in the Amazon context, making sense for the reader of what took place and what did not at Amazon. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan on what he learned about Scrum transition at Amazon.com:
Agile practices were present at Amazon.com as early as 1999, but it wasn’t until the years 2004 – 2009 that widespread adoption of Scrum occurred throughout Amazon’s development organizations. Amazon.com’s unplanned, decentralized Scrum transformation is of interest because it is different from the current orthodoxy regarding enterprise Scrum transitions, and its strengths and weaknesses reveal some fundamental lessons that can be applied to other enterprise Scrum transitions.
Here are the major forces that played in the transition.
Teams (including local management, of course) at Amazon have historically been given wide latitude to solve their problems (coupled with responsibility to do so without waiting for outside help) and are usually not overburdened with detailed prescriptive practices promulgated by centralized corporate sources. The emphasis is on creating, delivering, and operating excellent software in as streamlined and process-light a way as possible. Teams at Amazon have permission to choose.
The corporate culture at Amazon.com has always been surprisingly consistent with and friendly towards Agile practices. The 2 Pizza Team concept has been written about many times over the years (click here), and a close look shows that a 2 Pizza Team is basically a Scrum team without the Scrum. Teams at Amazon, especially 2 Pizza Teams, are stable and long-lived. Usually a development team reports to a single direct manager.
All it took to light the fire was someone who was willing to spend a little time educating interested parties about Scrum. Teams who learned about Scrum were able to make local decisions to implement it. Results were demonstrated that kindled interest in other teams.
Over time, an email-based Scrum community formed. Scrum Master training was provided on an occasional basis by someone who simply wanted to do so. Basic Scrum education continued on an ad hoc and voluntary basis. Eventually enough teams had adopted Scrum that a need was seen and a position of Scrum Trainer/Coach was created. Having a full-time Trainer and Coach available made adoption easier and improved the quality of scrum implementations. By mid-2008 the community was able to support an Open Space Scrum Gathering within the company.
What was (and one assumes is still) missing was higher level engagement at the organization and enterprise levels. No executive support for Scrum ever emerged, and the transition was therefore limited primarily to the team level, with many organizational impediments still in place.
The success of Scrum at Amazon validates one easy, frictionless way to begin a Scrum transition.
- Establish stable teams
- Make Agile and Scrum information widely and easily available
- Give permission to adopt Scrum
The advantage of this approach is that it requires a minimum of enterprise-wide planning and it allows teams to select Scrum, rather than mandating it. All of the rest of an enterprise Scrum transition can be accomplished by simply responding to impediments as raised by the teams and providing management support for change. Based on experience, the impediments raised will include demand (pull) for coaching, scaling, training, organizational change, a Transition Team, PMO changes, and all of the other aspects of an enterprise transition that many organizations labor so mightily to plan and control. Leadership for this kind of transition can only be Servant Leadership from the C-level, which is exactly the right kind for an Agile initiative, isn’t it?
The only impediment to Scrum adoption at Amazon was lack of knowledge. Teams were in place, and permission was part of the culture. When knowledge was provided, teams adopted Scrum. The strength of this process was based on the fact that only teams that were interested in trying Scrum actually tried it. There was no mandate or plan or schedule for this uptake. Nobody was forced to use Scrum. Teams made an independent, informed decision to try to solve some of their problems. Lean and Agile thinkers will recognize that this as a pull-based incremental approach and not a plan-driven, command and control, push-based approach.
What about the things that didn’t happen at Amazon? The transition stalled at the team level due to failure to engage either middle or upper management in a meaningful way. Both of those groups are required to bring a transition to its full potential. Training for middle managers, in particular, is crucial, but will usually reach them only with executive sponsorship. A Transition Team is crucial when organizational and enterprise-wide impediments begin to be unearthed. Support from a source of advanced knowledge and experience (trainer/coach) is key.
Was Scrum good or bad overall for Amazon? There is only spotty, anecdotal data to report. Certainly there are many stories of teams that used Scrum very successfully. The Amazon S3 project, which not only delivered on time after about 15 months of work, but nearly achieved the unusual result of having the entire development team take a week’s vacation leading up to the launch day. It was not the crunch-time, last minute, panic-drenched effort that is common with projects of this scope and complexity. There was the team that “hasn’t been able to deliver any software for 8 months” that, sure enough, delivered some software a month later at the end of their first sprint. Another team reported that their internal customers came to them some time after the team had adopted Scrum, asking that they implement a whole list of random features. “We know these aren’t your responsibility, but you’re the only team that is able to respond to our requests.” Finally, there was the platform team that had literally dozens of internal customers. When that team adopted Scrum, they organized their customers into a customer council of sorts and let them simply decide each month what the team would work on, for the good of all, in order of value to Amazon overall. But even if none of these anecdotes were available to tell, the mere fact that teams opted on their own to adopt Scrum implies that something about Scrum helped them be more successful and more satisfied professionally. If that were not true, then they would not have stayed with it at all, right?
As Israel alluded to last week, in this episode of the Agile Executive podcast, Israel and I talk with Reductive Lab’s Andrew Shafer. Put broadly, the topic is “Agile Infrastructure,” which kind of boils down to the connection between Agile development and the IT department, esp. in trying to get IT to be Agile itself. Here are some, admittedly, poor notes from the show:
After some brief introductory stuff, Andrew launches off: traditional operations has built up a resistance to change. “Change is what enables the business.” There’s an interesting discussion here of operations and infrastructure concerns being “non-functional requirements,” which are sort of second class citizens in some Agile practice.
I ask Andrew and Israel where, beyond web companies, they see these practices happening or finding interest. Andrew admits that its only web companies that he sees applying this thinking, but analogizes it to early Agile, XP in particular, which had a small, narrow focus at first and then spread over 10 odd years to where it (Agile) is today. Israel weighs in with an example from a few years ago in the financial sector
Having talked about these ideas in abstract, we talk about some of the practices themselves:
- as mentioned about, treating your infrastructure like source code – something you can rebuild on-demand.
- automate your infrastructure – from bare metal to running services.
- capacity planning, but better, management and acquisition – e.g., rebuilding 60 machines from metal to production in a few hours at Digg.com vs. two full days or work.
Israel asks, is ITIL for the data-center like water-fall for development? Both Andrew and I weight in on how much water-fall you can buy into, making analogues to eat-the-whole-pie RUP. This also recalls a conversation I had on another podcast, the IT Management & Cloud podcast with Rob England, aka, The IT Skeptic, on the topic of CMDBs and ITIL.