Archive for the ‘Starting Agile’ Category
Rally’s July 23 Agile Success Tour in Washington, DC was somewhat unique demographically. About 50% of participants work for the government. Moreover, many of the commercial enterprises represented in the event derive a significant amount of their revenues from federal government contracts. The Agile challenges encountered by these folks reflect practices that are not necessarily applicable to “pure” commercial environment. For example, one of the participants asked me about Agile for a project of 500 developers/testers in which her company is the prime contractor for 100 subcontractors! (Recommendation: must devise a business design enabling her company to profitably invest in laying a joint Agile infrastructure across all these subcontractors. Such infrastructure leads to standardization of the Agile data. Click here for details).
In spite of the different demographics, most of the Agile issues brought up in DC were quite similar to those expressed in previous Agile Success Tour events. The bureaucracy with which various Agile champions in DC need to deal with might be stricter (due to security/confidentiality aspects of much of the development carried out in DC), but the underlying needs and dynamics are not really different from those in other cities in which Success Tour events were held.
Here is a sample of enlightening threads I listened to or participated in during the event:
- The business fabric has not quite caught up with Agile methods. In particular, Agile contracts are not yet where they need to be. The costs associated with “ECOing the contract” each time a change in requirements is made offset the methodical benefits of Agile. We need to find a way to “encode” Agile principles in contracts.
- In pitching software methods to their executives, Agile champions need to go beyond the benefits of Agile. Risk and risk mitigation are of equal importance. See The View from the Executive Suite for detailed guidance on the subject.
- The benefits of Agile have to be expressed in terms of the business of the business. One has to go beyond capturing “just” the operational benefits and address financial and business benefits. Peter Drucker’s famous quip “Companies make shoes!” applies. Click here for examination of the quip in the Agile context.
- Innovation through affordable experimentation is an Agile benefit that is under-represented in many discussions Agile champions hold. See the new edition of Jim Highsmith‘s Agile Project Management for an excellent discussion of this critical benefit.
- Agile is about uncertainty, not about complexity. To demonstrate the power of Agile, choose a project of high uncertainty. Complexity in such a project depends on your risk tolerance – it could be either low or high. Be aware that various issues related to complexity might manifest themselves on the surface as process issues. See Uncertainty, Complexity, Correctness for an in-depth discussion of the subject.
Next stops of the Agile Success Tour “train” are in Boston, Chicago, Seattle and London. Stay tuned…
Rally’s Alan Atlas shares with us his experience as the first full-time Agile trainer/coach with Amazon. His account is both enlightened and enlightening. He connects the “hows”, “whats” and “whys” of Scrum in the Amazon context, making sense for the reader of what took place and what did not at Amazon. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan on what he learned about Scrum transition at Amazon.com:
Agile practices were present at Amazon.com as early as 1999, but it wasn’t until the years 2004 – 2009 that widespread adoption of Scrum occurred throughout Amazon’s development organizations. Amazon.com’s unplanned, decentralized Scrum transformation is of interest because it is different from the current orthodoxy regarding enterprise Scrum transitions, and its strengths and weaknesses reveal some fundamental lessons that can be applied to other enterprise Scrum transitions.
Here are the major forces that played in the transition.
Teams (including local management, of course) at Amazon have historically been given wide latitude to solve their problems (coupled with responsibility to do so without waiting for outside help) and are usually not overburdened with detailed prescriptive practices promulgated by centralized corporate sources. The emphasis is on creating, delivering, and operating excellent software in as streamlined and process-light a way as possible. Teams at Amazon have permission to choose.
The corporate culture at Amazon.com has always been surprisingly consistent with and friendly towards Agile practices. The 2 Pizza Team concept has been written about many times over the years (click here), and a close look shows that a 2 Pizza Team is basically a Scrum team without the Scrum. Teams at Amazon, especially 2 Pizza Teams, are stable and long-lived. Usually a development team reports to a single direct manager.
All it took to light the fire was someone who was willing to spend a little time educating interested parties about Scrum. Teams who learned about Scrum were able to make local decisions to implement it. Results were demonstrated that kindled interest in other teams.
Over time, an email-based Scrum community formed. Scrum Master training was provided on an occasional basis by someone who simply wanted to do so. Basic Scrum education continued on an ad hoc and voluntary basis. Eventually enough teams had adopted Scrum that a need was seen and a position of Scrum Trainer/Coach was created. Having a full-time Trainer and Coach available made adoption easier and improved the quality of scrum implementations. By mid-2008 the community was able to support an Open Space Scrum Gathering within the company.
What was (and one assumes is still) missing was higher level engagement at the organization and enterprise levels. No executive support for Scrum ever emerged, and the transition was therefore limited primarily to the team level, with many organizational impediments still in place.
The success of Scrum at Amazon validates one easy, frictionless way to begin a Scrum transition.
- Establish stable teams
- Make Agile and Scrum information widely and easily available
- Give permission to adopt Scrum
The advantage of this approach is that it requires a minimum of enterprise-wide planning and it allows teams to select Scrum, rather than mandating it. All of the rest of an enterprise Scrum transition can be accomplished by simply responding to impediments as raised by the teams and providing management support for change. Based on experience, the impediments raised will include demand (pull) for coaching, scaling, training, organizational change, a Transition Team, PMO changes, and all of the other aspects of an enterprise transition that many organizations labor so mightily to plan and control. Leadership for this kind of transition can only be Servant Leadership from the C-level, which is exactly the right kind for an Agile initiative, isn’t it?
The only impediment to Scrum adoption at Amazon was lack of knowledge. Teams were in place, and permission was part of the culture. When knowledge was provided, teams adopted Scrum. The strength of this process was based on the fact that only teams that were interested in trying Scrum actually tried it. There was no mandate or plan or schedule for this uptake. Nobody was forced to use Scrum. Teams made an independent, informed decision to try to solve some of their problems. Lean and Agile thinkers will recognize that this as a pull-based incremental approach and not a plan-driven, command and control, push-based approach.
What about the things that didn’t happen at Amazon? The transition stalled at the team level due to failure to engage either middle or upper management in a meaningful way. Both of those groups are required to bring a transition to its full potential. Training for middle managers, in particular, is crucial, but will usually reach them only with executive sponsorship. A Transition Team is crucial when organizational and enterprise-wide impediments begin to be unearthed. Support from a source of advanced knowledge and experience (trainer/coach) is key.
Was Scrum good or bad overall for Amazon? There is only spotty, anecdotal data to report. Certainly there are many stories of teams that used Scrum very successfully. The Amazon S3 project, which not only delivered on time after about 15 months of work, but nearly achieved the unusual result of having the entire development team take a week’s vacation leading up to the launch day. It was not the crunch-time, last minute, panic-drenched effort that is common with projects of this scope and complexity. There was the team that “hasn’t been able to deliver any software for 8 months” that, sure enough, delivered some software a month later at the end of their first sprint. Another team reported that their internal customers came to them some time after the team had adopted Scrum, asking that they implement a whole list of random features. “We know these aren’t your responsibility, but you’re the only team that is able to respond to our requests.” Finally, there was the platform team that had literally dozens of internal customers. When that team adopted Scrum, they organized their customers into a customer council of sorts and let them simply decide each month what the team would work on, for the good of all, in order of value to Amazon overall. But even if none of these anecdotes were available to tell, the mere fact that teams opted on their own to adopt Scrum implies that something about Scrum helped them be more successful and more satisfied professionally. If that were not true, then they would not have stayed with it at all, right?
Just about three months ago we started an “Ask an Expert” service for Agile practitioners in Austin. The service was defined as follows:
The objective of the Ask an Expert program is to provide free consultation by experienced Agile Austin coaches to any Austinite that wrestles with issues related to promoting, planning and executing Agile methods. The program will address the needs of practitioners in companies that produce software, embed software, or use software as an integral part of their business processes. In addition to 1-1 consultation, coaches will gladly hold discussions with entire teams.
Ask an Expert sessions should be primarily regarded as a step toward addressing concrete Agile issues that manifest themselves in a specific environment. Coaches might not be able to complete a comprehensive analysis, but will make certain to suggest what the heart of the problem might be and point out Agile resources that practitioners could use on their own.
To ensure available access to experts, consultative session time will be divided between attendees. Team discussions with any Agilists attending the program will be encouraged to maximize the sharing of experience and draw out the wisdom of crowds. One-on-one sessions are available on request, but will be time-limited based on attendance (15 minutes typical).
The program will strive to balance utility with fun. Utility will primarily be delivered through actionable insights; fun will be had through passionate exploration of Agile topics in a friendly and collaborative manner.
Our experience over the past three months indicates:
- A broad spectrum of question/topics has been brought up. Most of the questions revolve around the “hows” of Agile. Some questions address the “whats” of Agile. Precious few get into the “whys” of Agile. Click here for details.
- Majority of questions apply to the project team level. Only a few address enterprise level issues.
- Many questions (and the discussions that follow) are actually about the software engineering fabric, not about Agile per se.
- The “all singing all dancing” format of the sessions seems to work pretty well. It often leads to uncovering questions/issues we had not thought about before.
- Having said that, we do not really know at this point in time whether some of the participants would have preferred a more traditional 1-1 format.
- Most participants seem to have already been sold on the benefits of Agile. We do not usually get folks who are struggling with “Waterfall v. Agile” questions.
Most gratifying, some early “return on investment” indicators have been noticed. For example, one of the participants was so kind to send the following note:
Thank y’all for your help with my presentation about Agile to my VP. The meeting went well and we are moving forward with Agile. I’m going to work on a mock-up of a release and project, to show what Agile release planning and budgeting would look like. I’ll get buy-in based on this mock-up from the directors, then move on to a pilot project.
This is a huge step for… [company name deleted by IG], one I wouldn’t have predicted 6 months ago. The information and resources available through Agile Austin were essential in making this happen. Thank you for your help!
Colleague Annie Shum sent me her thoughts on the e-discussion that evolved around The Tool is the Method and Four Priciples, Four Cultures, One Mirror. As always, her thoughts connect a lot of dots. Here is Annie:
“Change versus shifts. For me, I like to focus today’s discussion on the latter and let me just define a shift as large scale changes; notably transformative cultural changes. Historically, cultural shifts are not top-down engineered by master planners; instead self-organizing emergence is the process by which all shifts happen on this planet. Similar to an ant colony where each ant works individually without pre-planned central orchestrations, our individual actions on a local level can collectively impact the emergence process. Positive as well as negative Feedback Loops are the fuel of the emergence process. Emergence can result in shifts that can both benefit (e.g. WWW, Information Age) as well as harm our society (e.g. wars, riots).
In thinking about tools in the context of impacting large scale cultural changes, one can’t help but immediately think about the computer/digital computing. By all definitions, a computer is a tool – in fact, I would venture to characterize a computer as a “universal tool” that can, in theory, perform almost any task. Moreover, I can’t think of any other tool that can surpass the computer and digital computing as the most disruptive transformation agents of our society and cultural shifts in the 21st Century. According to James Moor, the computer revolution is occurring in two stages. We are now at the cusp of the 2nd stage – “one that the industrialized world has only recently entered — is that of “technological permeation” in which technology gets integrated into everyday human activities and into social institutions, fundamentally changing and transforming society at its very core.” http://plato.stanford.edu/entries/ethics-computer ”
The live recording of my Agile Roots keynote presentation is available here. In addition to threading the slides together, the recording captures the Q&A session that followed. Various questions and observations were brought up by Diana Larsen, Jeff Patton, Andrew Shafer as well as other participants in the conference. In particular, linkages were made during the Q&A session to Appreciative Inquiry and to Serious Games.
Needless to say, comments on the presentation will be much appreciated.
The problem is not transitioning from Waterfall to Agile. The real problem is transitioning from ad hoc to Agile.
This observation by Darren really resonates with me. Nowadays the preferred solution to various software engineering problems seems to be Agile. Latching to Agile, however, does not necessarily indicate that the software method in use has failed. Rather, it often indicates the lack of an appropriate software method/process.
Hence, my invariable counsel to folks who are about to embark on an Agile rollout: start by recording the state of affairs before the Agile rollout. For example, capture your productivity metrics for the period prior to training your teams in Agile. For better or worse, this is your true baseline.
Did you ever notice that our first inclination is to reach for a tool when we want to change something? What we always seem to forget is that web operations, as a discipline, is only partially about technology.
Damon’s states the following view on the right balance to be struck:
The success of your web operations depends more on the processes and culture your people work within than it does on any specific tooling choices… We see this repeatedly in our consulting business. Time after time we are called in to do a specific automation project and wind up spending the bulk of the effort as counselors and coaches helping the organization make the cultural shift that was the real intention of the automation project.
While I am in full agreement with Damon on the phenomenon, I would like to highlight two nuances that in many cases make the tool is the method an effective approach to rolling Agile:
- The rise of professional procurers in Global 2000 companies (see Selling is Dead) changes transactional aspects of Agile engagements. Professional procurers typically focus on negotiating the best possible deal for the tool(s). Moreover, they tend to determine the preferred tool(s) in the early stages of negotiating the engagement.
- Tools might not change culture, but they can and often do change behavior. Think, for example, about the way numerous folks use Twitter. What starts as “having a little bit of fun” often leads to major changes in the way one collects, stores, analyzes and assimilates information. The changes happen not due to an explicit intention to change, but as part of “playing” with Twitter.
Between these two nuances, a typical progression for an enterprise level Agile initiative tends to be as follows:
- A tool is chosen
- Teams start using the tool
- The tool induces behavioral changes
- These behavioral changes prevail, overshadowing cultural change initiatives
Hence, in many circumstances the tool indeed is the method. The chosen tool becomes a factor of the first order in determining not “only” how Agile (or any other software method) will be practiced, but what mindset will evolve in the course of the rollout.
See the presentation entitled Four Principles, Four Cultures, One Mirror for additional details on the subject. A short summary of the presentation is given here. Related views are summarized by InfoQ here.
Yesterday’s Agile Success Tour event in Atlanta was characterized by quite a strong interest of various participants in software capitalization. While the topic came up in previous success tour events, it was more pronounced in Atlanta. A subtle shift seems to be taking place: from technical aspects of rolling out Agile to business aspects the Agile champion needs to be aware of and prepared to address.
In the course of exploring software capitalization, participants in the Atlanta event brought up related questions about “the business of Agile.” For example:
- Cost accounting for Agile
- Return on the Agile investment
- Agile Portfolio Management
- Governance of the Agile project/initiative
Some of the participants were quite thoughtful about the dividing line between project-level Agile implementation and Enterprise-level implementation. They grasp that scaling Agile changes the nature of the questions one wrestles with. A few of them are actually considering top-down approaches to implementing Agile. They do not necessrily subscribe to rolling Agile in a bottom-up manner. They devote quality time to wrestling with Agile scaling issues prior to starting the first Agile project.
I am hesitant to draw general conclusions based on a single event in a particular city. For example, I wonder whether the emphasis on capitalization might be related to the concentration of Supply Chain Management firms in Atlanta.
As the Success Tour “train” continues its journey (next stop: Washington, DC on July 23) we will soon have additional data points. Stay tuned…
- The Agile Manifesto principles are considered timeless.
- Application of Agile can create cultural duality/conflict. The core culture of the organization that rolls out Agile is not necessarily aligned with the Agile culture.
- Successful application of the Manifesto principles needs to build on the strength of the specific core culture – Control, Competence, Cultivation or Collaboration – in the organization rolling out Agile .
- Schwaber’s 75% failure rate estimate corresponds to attempts to change the core culture of an organization as part of the Agile rollout.
- Success does not necessarily beget success in Agile rollouts. The interplay between scale and culture poses serious challenges to scaling Agile successfully.
- The Agile infrastructure places a practical limit on the scope of the Agile rollout. Constituencies that are not able to use a joint Agile infrastructure are not likely to collaborate.
- The fine points of one Agile method versus another are far less important to the success of an Agile implementation than cultural subtleties of the target environment in which Agile is applied.
- Good Agile tools are likely to induce behavioral changes without necessitating major cultural pushes.
The June 18 “Ask and Expert” session of Agile Austin poses a unique opportunity to to discuss the pros and cons of the “All In!” approach with Erik Huddleston- an Agile champion who has successfully implemented Scrum in this manner. Bringing Scum to Inovis in 2007, Erik opted for an “All In!” implementation instead of the more customary team-by-team rollout. The Inovis case study is one of the very few authoritative sources on this gutsy approach.
If you can’t attend the clinic in Austin on the 18th, you might want to watch out for his forthcoming Agile Success Tour panel session in Atlanta, GA on the 25th. Erik’s insights will be posted here and here a few days after the event.