Archive for the ‘Scaling Agile’ Category
The BMC Agile Transformation: A Seven-Year Perspective
This Executive Update represents my current understanding of the deeper reasons behind the success of the BMC rollout. It reviews past decisions in light of knowledge, experience, and insights that evolved a long time after the decisions, for better or worse, had been executed. In general, it’s about my making sense of things and sharing my insights with Cutter clients.
To download the Executive Update, click here and use the promotion code BMCAGILE.
Time-to-Market as a Function of Release Size (Source: QSM sample of ~7,500 projects)
A Seven Year Retrospective
The results measured by Michael reaffirmed for me a core belief that I had developed as a young man in the Israeli army: ordinary people can achieve extraordinary results. We did not have, with all due respect, extraordinary talent at BMC; our development tools were nothing to write home about; the problems of communicating effectively across 10.5 hours of time zone difference from Austin, TX to Pune, India were very real; and, we were subject to repeated layoffs. What we accomplished was primarily a matter of doing the right things by an extremely important stakeholder – the business unit employees.
Click here for a detailed account of the 2004 experience from a 2011 perspective.
The Things They Say
Here is a compilation of summaries of Amazon reviews on the Kindle edition of The Concise Executive Guide to Agile:
If you are such a decision maker, and like most are hard-pressed for time, then you must read this book. Dr. Gat cuts right to the chase and gives pearls of wisdom – that come from having “done it himself” when it comes to the daunting task of TRANSFORMING an organization into successfully applying Agile methods. If you’re a team leader and you need executive support and buy-in, then buy this book for the key people whom you value the most. Give it to them as a gift. They will thank you for it!
“The Concise Executive Guide to Agile” by Israel Gat is a must-read for enthusiasts of agile methods at all levels of the organization. Israel has well over 30 years of industry experience, has his PhD in Computer Science, and has helped manage an agile rollout involving 1,000 software developers. While there are many technical and managerial treatises on agile methods emerging in increasing frequency all of the time, Israel’s new guide is meant to explain the essence of agile methods to R&D, Marketing, Sales, Program Management, Professional Services, Customer Support, Finance, or IT folks.
Dr. Gat goes beyond the traditional software development process literature and broadens the case for the Agile organization, with effectiveness, efficiency, and integrity – much like the Agile development process described by Israel.
A must read for any organization leader dependant on software whether it be for software for internal operations or software features built for customer sales. Israel’s breadth and depth of both knowledge and experience shine through as he provides practical advice to help you understand what Agile can and can not do for your organization.
For me, this book provided something unique, which is a look into the way high level executives think about Agile.
This book rapidly cuts through the layers of confusion surrounding Agile development and focuses on what you “NEED” to know. Dr. Gat’s vast experience shines through as he elegantly simplifies the process.
Israel’s experience and wisdom in transforming a software development culture in a very large multi-site virtual team environment is well captured… Israel has done a great job in transforming his experience into a guide. That is the beauty of the book.
This is the “must-read” book if you’re an executive charged with Agile transformation, product development, or business strategy. The insights from Gat cover all three. Most companies talk a good Agile game. If you want results, however, please pay heed to the advice in this book.
The book will help you to flexibly navigate the hurdles inherent in selling and succeeding with Agile in your business: i.e. existing methodologies, outsourcing, diverse geographies, resistance to change, and the rest. They are all explained here by someone who has apparently already successfully fought the battles. A great read!
He has been there, and this book shares the unique insights of a senior manager who guided a large organization through the change process. If you are in this seat in your organization then this book is for you.
Israel distills his broad experience and many years of wisdom rolling out Agile into a succinct overview that is deep in knowledge, but accessible.
I’ve been a fan of Israel’s “The Agile Executive” blog for some time and had high hopes for this book… and the book didn’t disappoint!
While I was looking for answers; what I found instead was much more interesting. Israel was able to collect all the wisdom of doing large scale agile rollouts into one document, highlighting the geographic features along the way. That is a lot harder to do than simply proclaiming one correct path, and, speaking from experience, more helpful to the reader than they know.
You can read the full reviews here.
The Friction of Agile
Everything is very simple in War, but the simplest thing is difficult. These difficulties accumulate and produce a friction which no man can imagine exactly who has not seen War… This enormous friction, which is not concentrated, as in mechanics, at a few points, is therefore everywhere brought into contact with chance, and thus incidents take place upon which it was impossible to calculate…
IMHO, this excerpt from On War applies exceptionally well to Agile roll-outs these days:
- Simplicity: The four principles of the Agile Manifesto are intuitively compelling. You could (and probably should) use them as the core definition of what Agile is all about. Likewise, you do not need more than a single hand-drawn matrix to illustrate how WIP limits in Kanban work. In contrast to various other terms used in development and IT – e.g. SOA – the conceptual power of Agile methods is easy to grasp.
- Friction: Assume you were building a company from scratch without any pre-conceived notions of the organizational constructs you would put in place. Assume as well that you were developing your organization with end-to-end Agile effectiveness in mind. You would probably devise a holistically integrated organization. For example, you might opt for an organizational design in which each level of the organization will include all functions relevant to Agile – R&D, IT, Marketing, Support, Sales etc. In other words, ideally you will not go for the traditional organizational design: a vertical R&D stove pipe, a vertical Marketing stove pipe, a vertical Sales stove pipe, etc. As in reality you are unlikely to get the charter to start with a clean sheet of paper, the friction arises in each and every point in which your end-to-end organizational design for Agile deviates from the traditional organizational constructs your company uses.
- Not concentrated: As Clausewitz points out, the friction of war is not mechanical friction – you can’t address it by pouring in a ‘organizational lubricant’ in just a few places. Flooding the whole organization with the lubricant is likely to create a morass in which all agility will be lost.
I recommend four principles to minimize the organizational friction of Agile, as follows:
- Acknowledge you accrued organizational debt: It is conceptually quite similar to accruing technical debt – various organizational decisions and compromises taken along the way were rushed to the extent that they leave much to be desired. The organizational constructs and practices that sprang out of these decisions need to be refactored.
- Carry out the organizational refactoring work from the outside to the inside: A truly holistic Agile design will incorporate customers and partners. Start with the way you will integrate them, thence apply this very same way to the integration of the organizational entities within your company.
- Build on the strengths of your core corporate culture: As pointed out by Drucker:
… culture is singularly persistent… changing [organizational] behavior works only if it can be based on the existing ‘culture’… [Drucker, 1991]
- Use modern corporate for the fundamental organizing principles. See Relational Networks, Strategic Advantage: New Challenges for Collaborative Control by Hagel, Brown and Jelinek for a good recent article on the subject.
Since the end of the Cold War, a fair amount of debate has taken place around the applicability of the friction of war principles to armed conflicts in the information age. The conclusion is of interest to both military personnel, Agile practitioners and IT professionals:
… while technological advances might temporarily mitigate general friction, they could neither eliminate it nor substantially reduce its potential magnitude.
Wrestling with Scaling Software Agility
Software Development Times has just published Guest View: Wrestling with Scaling Software Agility by Ryan Martens and me. This Guest View is a little unique in that Ryan and I actually try to wrestle each other to the ground… Here is why we try to do so:
Agile champions spend a lot of time trying to communicate the agile premise to the executives in their organization. The difference in context between the champion and the executive often makes it a difficult conversation. A Scrum Master versed in behavior-driven design is not always able to relate to the frustrations of a sales executive who gets free advice on how to sell from everyone and his grandmother.
Conversely, a CFO does not necessarily understand why unit test coverage on the company’s legacy code is still inadequate after a full year of investment in agile methods that embrace refactoring as a core practice.
To bridge the chasm through this article, we resort to role-playing. Ryan Martens plays the Agile Champion; Israel Gat plays the Skeptical Executive. Metaphorically speaking, each one tries to wrestle the other to the ground.
Before you get into this Guest View, I would like to reinforce an important disclaimer:
A note of caution before Ryan and Israel make irreparable damage to their long-standing relationship: The two actually understand each other extremely well and rarely are they of different opinions on the fundamentals of agile in real life…
Enjoy the article!
How Hard Should the Agile Champion Push?
A question which often comes up in the course of the Rally Agile Success Tour is about the balance to strike between conviction, passion and politics. By virtue of his/her interest in the topic, the Agile champion is often more knowledgeable than his/her superiors on what Agile really is and which strategies are best suited to roll it out. A thoughtful push toward Agile values, principles and practices may lead to major improvements in the way an organization practices Agile. In contrast, an overly hard push might lead to regression in the state of Agile affairs. Moreover, it could easily lead to a breach of the psychoilogical contract between the Agile champion and employer.
Reading The Bureaucratic Phenomenon – a book recommended to me by Forrester’s Tom Grant – I found the following nugget:
The power of decision making within a bureaucratic system of organization is located exactly at the points where the stability of the internal “political” system is preferred to the achievement of the functional goals of the organization.
The specific nature of the bureaucracy with which one has to cope changes, of course, from one company to another. No matter what kind of bureaucracy the Agile champion has to cope with, the observation cited above applies. The Agile champion can successfully push hard as long as he/she does not cross the fine line between achievement of functional goals and “political” stability.
Scrum at Amazon – Guest Post by Alan Atlas
Rally’s Alan Atlas shares with us his experience as the first full-time Agile trainer/coach with Amazon. His account is both enlightened and enlightening. He connects the “hows”, “whats” and “whys” of Scrum in the Amazon context, making sense for the reader of what took place and what did not at Amazon. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan on what he learned about Scrum transition at Amazon.com:
Agile practices were present at Amazon.com as early as 1999, but it wasn’t until the years 2004 – 2009 that widespread adoption of Scrum occurred throughout Amazon’s development organizations. Amazon.com’s unplanned, decentralized Scrum transformation is of interest because it is different from the current orthodoxy regarding enterprise Scrum transitions, and its strengths and weaknesses reveal some fundamental lessons that can be applied to other enterprise Scrum transitions.
Here are the major forces that played in the transition.
Permission
Teams (including local management, of course) at Amazon have historically been given wide latitude to solve their problems (coupled with responsibility to do so without waiting for outside help) and are usually not overburdened with detailed prescriptive practices promulgated by centralized corporate sources. The emphasis is on creating, delivering, and operating excellent software in as streamlined and process-light a way as possible. Teams at Amazon have permission to choose.
Teams
The corporate culture at Amazon.com has always been surprisingly consistent with and friendly towards Agile practices. The 2 Pizza Team concept has been written about many times over the years (click here), and a close look shows that a 2 Pizza Team is basically a Scrum team without the Scrum. Teams at Amazon, especially 2 Pizza Teams, are stable and long-lived. Usually a development team reports to a single direct manager.
Knowledge
All it took to light the fire was someone who was willing to spend a little time educating interested parties about Scrum. Teams who learned about Scrum were able to make local decisions to implement it. Results were demonstrated that kindled interest in other teams.
Impetus
Over time, an email-based Scrum community formed. Scrum Master training was provided on an occasional basis by someone who simply wanted to do so. Basic Scrum education continued on an ad hoc and voluntary basis. Eventually enough teams had adopted Scrum that a need was seen and a position of Scrum Trainer/Coach was created. Having a full-time Trainer and Coach available made adoption easier and improved the quality of scrum implementations. By mid-2008 the community was able to support an Open Space Scrum Gathering within the company.
What was (and one assumes is still) missing was higher level engagement at the organization and enterprise levels. No executive support for Scrum ever emerged, and the transition was therefore limited primarily to the team level, with many organizational impediments still in place.
The success of Scrum at Amazon validates one easy, frictionless way to begin a Scrum transition.
- Establish stable teams
- Make Agile and Scrum information widely and easily available
- Give permission to adopt Scrum
The advantage of this approach is that it requires a minimum of enterprise-wide planning and it allows teams to select Scrum, rather than mandating it. All of the rest of an enterprise Scrum transition can be accomplished by simply responding to impediments as raised by the teams and providing management support for change. Based on experience, the impediments raised will include demand (pull) for coaching, scaling, training, organizational change, a Transition Team, PMO changes, and all of the other aspects of an enterprise transition that many organizations labor so mightily to plan and control. Leadership for this kind of transition can only be Servant Leadership from the C-level, which is exactly the right kind for an Agile initiative, isn’t it?
The only impediment to Scrum adoption at Amazon was lack of knowledge. Teams were in place, and permission was part of the culture. When knowledge was provided, teams adopted Scrum. The strength of this process was based on the fact that only teams that were interested in trying Scrum actually tried it. There was no mandate or plan or schedule for this uptake. Nobody was forced to use Scrum. Teams made an independent, informed decision to try to solve some of their problems. Lean and Agile thinkers will recognize that this as a pull-based incremental approach and not a plan-driven, command and control, push-based approach.
What about the things that didn’t happen at Amazon? The transition stalled at the team level due to failure to engage either middle or upper management in a meaningful way. Both of those groups are required to bring a transition to its full potential. Training for middle managers, in particular, is crucial, but will usually reach them only with executive sponsorship. A Transition Team is crucial when organizational and enterprise-wide impediments begin to be unearthed. Support from a source of advanced knowledge and experience (trainer/coach) is key.
Was Scrum good or bad overall for Amazon? There is only spotty, anecdotal data to report. Certainly there are many stories of teams that used Scrum very successfully. The Amazon S3 project, which not only delivered on time after about 15 months of work, but nearly achieved the unusual result of having the entire development team take a week’s vacation leading up to the launch day. It was not the crunch-time, last minute, panic-drenched effort that is common with projects of this scope and complexity. There was the team that “hasn’t been able to deliver any software for 8 months” that, sure enough, delivered some software a month later at the end of their first sprint. Another team reported that their internal customers came to them some time after the team had adopted Scrum, asking that they implement a whole list of random features. “We know these aren’t your responsibility, but you’re the only team that is able to respond to our requests.” Finally, there was the platform team that had literally dozens of internal customers. When that team adopted Scrum, they organized their customers into a customer council of sorts and let them simply decide each month what the team would work on, for the good of all, in order of value to Amazon overall. But even if none of these anecdotes were available to tell, the mere fact that teams opted on their own to adopt Scrum implies that something about Scrum helped them be more successful and more satisfied professionally. If that were not true, then they would not have stayed with it at all, right?
Tools, Behavior, Culture
Colleague Annie Shum sent me her thoughts on the e-discussion that evolved around The Tool is the Method and Four Priciples, Four Cultures, One Mirror. As always, her thoughts connect a lot of dots. Here is Annie:
“Change versus shifts. For me, I like to focus today’s discussion on the latter and let me just define a shift as large scale changes; notably transformative cultural changes. Historically, cultural shifts are not top-down engineered by master planners; instead self-organizing emergence is the process by which all shifts happen on this planet. Similar to an ant colony where each ant works individually without pre-planned central orchestrations, our individual actions on a local level can collectively impact the emergence process. Positive as well as negative Feedback Loops are the fuel of the emergence process. Emergence can result in shifts that can both benefit (e.g. WWW, Information Age) as well as harm our society (e.g. wars, riots).
In thinking about tools in the context of impacting large scale cultural changes, one can’t help but immediately think about the computer/digital computing. By all definitions, a computer is a tool – in fact, I would venture to characterize a computer as a “universal tool” that can, in theory, perform almost any task. Moreover, I can’t think of any other tool that can surpass the computer and digital computing as the most disruptive transformation agents of our society and cultural shifts in the 21st Century. According to James Moor, the computer revolution is occurring in two stages. We are now at the cusp of the 2nd stage – “one that the industrialized world has only recently entered — is that of “technological permeation” in which technology gets integrated into everyday human activities and into social institutions, fundamentally changing and transforming society at its very core.” http://plato.stanford.edu/entries/ethics-computer ”
Live Recording of Four Principles, Four Cultures, One Mirror
The live recording of my Agile Roots keynote presentation is available here. In addition to threading the slides together, the recording captures the Q&A session that followed. Various questions and observations were brought up by Diana Larsen, Jeff Patton, Andrew Shafer as well as other participants in the conference. In particular, linkages were made during the Q&A session to Appreciative Inquiry and to Serious Games.
Needless to say, comments on the presentation will be much appreciated.
The Tool is the Method
In a recent post in dev2ops, Damon Edwards examines the role tools often play in the context of a desired cultural change. To quote Damon:
Did you ever notice that our first inclination is to reach for a tool when we want to change something? What we always seem to forget is that web operations, as a discipline, is only partially about technology.
Damon’s states the following view on the right balance to be struck:
The success of your web operations depends more on the processes and culture your people work within than it does on any specific tooling choices… We see this repeatedly in our consulting business. Time after time we are called in to do a specific automation project and wind up spending the bulk of the effort as counselors and coaches helping the organization make the cultural shift that was the real intention of the automation project.
While I am in full agreement with Damon on the phenomenon, I would like to highlight two nuances that in many cases make the tool is the method an effective approach to rolling Agile:
- The rise of professional procurers in Global 2000 companies (see Selling is Dead) changes transactional aspects of Agile engagements. Professional procurers typically focus on negotiating the best possible deal for the tool(s). Moreover, they tend to determine the preferred tool(s) in the early stages of negotiating the engagement.
- Tools might not change culture, but they can and often do change behavior. Think, for example, about the way numerous folks use Twitter. What starts as “having a little bit of fun” often leads to major changes in the way one collects, stores, analyzes and assimilates information. The changes happen not due to an explicit intention to change, but as part of “playing” with Twitter.
Between these two nuances, a typical progression for an enterprise level Agile initiative tends to be as follows:
- A tool is chosen
- Teams start using the tool
- The tool induces behavioral changes
- These behavioral changes prevail, overshadowing cultural change initiatives
Hence, in many circumstances the tool indeed is the method. The chosen tool becomes a factor of the first order in determining not “only” how Agile (or any other software method) will be practiced, but what mindset will evolve in the course of the rollout.
See the presentation entitled Four Principles, Four Cultures, One Mirror for additional details on the subject. A short summary of the presentation is given here. Related views are summarized by InfoQ here.
The Agile Executive 