Archive for the ‘Cultural Aspects’ Category
A Devops Case Study
An outline of my forthcoming Agile 2010 workshop was given in the post “A Recipe for Handling Cultural Conflicts in Devops and Beyond” earlier this week. Here is the case study around which the workshop is structured:
NotHere, Inc. Case Study
NotHere, Inc. is a $500M company based in Jerusalem, Israel. The company developed an eCommerce platform for small to medium retailers. Through a combination of this platform and its hosting data center, NotHere provides online store fronts, shopping carts, order processing, inventory, billing and marketing services to tens of thousands of retailers in a broad spectrum of verticals. For these retailers, NotHere is a one-stop “shopping” for all their online needs. In particular, instead of partnering with multiple companies like Amazon, Ebay, PayPal and Shopzilla, a retailer merely needs to partner with NotHere (who partners with these four companies and many others).
The small to medium retailers that use the good services of NotHere are critically dependent on the availability of its data center. For all practical purposes retailers are (temporarily) dead when the NotHere data center is not available. In recognition of the criticality of this aspect of its IT operations, NotHere invested a lot of effort in maturing its ITIL[i] processes. Its IT department successfully implements the ITIL service support and service delivery functions depicted in the figure below. From an operational perspective, an overall availability level of four nines is consistently attained. The company advertises this availability level as a major market differentiator.
In response to the accelerating pace in its marketplace, NotHere has been quite aggressive and successful in transitioning to Agile in product management, dev and test. Code quality, productivity and time-to-producing-code have been much improved over the past couple of years. The company measures those three metrics (quality, productivity, time-to-producing-code) regularly. The metrics feed into whole-hearted continuous improvement programs in product management, dev and test. They also serve as major components in evaluating the performance of the CTO and of the EVP of marketing.
NotHere has recently been struggling to reconcile velocity in development with availability in IT operations. Numerous attempts to turn speedy code development into fast service delivery have not been successful on two accounts:
- Technical: Early attempts to turn Continuous Integration into Continuous Deployment created numerous “hiccups” in both availability and audit.
- Cultural: Dev is a competence culture; ops is a control culture.
A lot of tension has arisen between dev and ops as a result of the cultural differences compounding the technical differences. The situation deteriorated big time when the “lagging behind” picture below leaked from dev circles to ops.
The CEO of the company is of the opinion NotHere must reach the stage of Delivery over Development. She is not too interested in departmental metrics like the time it takes to develop code or the time it takes to deploy it. From her perspective, overall time-to-delivery (of service to the retailers) is the only meaningful business metric.
To accomplish Delivery over Development, the CEO launched a “Making Cats Work with Dogs[ii]” project. She gave the picture above to the CTO and CIO, making it crystal clear that the picture represents the end-point with respect to the relationship she expects the two of them and their departments to reach. Specifically, the CEO asked the CTO and the CIO to convene their staffs so that each department will:
- Document its Outmodel (in the sense explored in the “How We Do Things Around Here In Order to Succeed” workshop) of the other department.
- Compile a list of requirements it would like to put on the other group “to get its act together.”
The CEO also indicated she will convene and chair a meeting between the two departments. In this meeting she would like each department to present its two deliverables (world view of the other department & and the requirements to be put on it) and listen carefully to reflections and reactions from the other department. She expects the meeting will be the first step toward a mutual agreement between the two departments how to speed up overall service delivery.
[i] “Information Technology Infrastructure library – a set of concepts and practices for Information Technology Services Management (ITSM), Information Technology (IT) development and IT operations” [Wikipedia].[ii] I am indebted to Patrick DeBois for suggesting this title.
© Copyright 2010 Israel Gat
A Recipe for Handling Cultural Conflicts in Devops and Beyond
My Agile 2010 workshop “How We Do Things Around Here In Order To Succeed” will weave together four trends that I am witnessing in my practice:
- The ascendance of Agile portfolio management in a world characterized by loosely coupled processes
- Devops dynamics are becoming more and more characteristic of end-to-end Agile/Kanban patterns
- Viral spread of technical debt metrics in software governance
- Increasing use of boundary objects in the enterprise context
The workshop is structured around three case studies/exercises that will take about two-thirds of the allotted time (the morning of August 9). The other third provides the theory and tools to be used in the three workshop exercises and (hopefully) in many future engagements participants in the workshop will carry out. Deep technical knowledge is not required – the workshop targets any Agile practitioner who has conceptual grasp of culture, software development, IT operations and portfolio management.
The #1 takeaway from the presentation is the details you need to know about creation and capture of lasting value through end-to-end Agile initiatives.
Here is the workshop agenda (still subject to some minor tweaking):
- Introduction to Cultural Framework
- Exercise #1: Strengths and Weaknesses of Your Culture
- Change Behavior, Not Culture
- When Organizations Clash
- Exercise #2: Conflicts in Devops
- The Agile Flywheel
- Exercise #3: Using Technical Debt as a Boundary Object in Devops
- Bringing Organizations Together Through Enlightened Governance Loops
I look forward to meeting you in the workshop and learning from your experiences and insights!
Israel
Predictability is Bad for Your Business
I had the pleasure of meeting some old colleagues a few weeks ago. They work for a software company that pays a lot of attention to software engineering practices and invests heavily in software tools. Financial results, however, have not been great over the past few years.
Obviously, the disparity between the strength of the software engineering discipline and the relative weakness of the financial results is due to more than a single cause. One factor, however, was highlighted time and time again by my colleagues:
Predictability is killing us!
Paradoxical that this observation might seem, it is actually quite straightforward. Senior management in their company is really forceful about predictability. Hence, initiative, (affordable) experimentation and innovation have pretty much faded away. For most practical purposes it has become a check-the-box culture. All attempts to substitute reliable delivery for predictability seem to have failed so far.
One last “ingredient” to add to the story. This company is rich in talent. Generally speaking, the folks in the engineering trenches are gifted, knowledgeable, capable and dedicated.
How predictably poignant!
Role of the Agile Leader in Reconfiguring the Business
Click here for the slide deck from my Agile 2009 presentation.
Abstract: The presentation applies Agile thinking to critical aspects of strategy and execution at a time of uncertainty and disruption. The essential point is simple and logical: Agile values and principles are indivisible. To succeed, they must be applied not just to R&D, but also to customer and company, simultaneously. This requires reconfiguration of customer relationships, employee policy, software development, and the relationship that binds the three. The resulting paradigm shift could lower the cost of software and produce prosperity similar to the one induced by ultra-cheap oil in the 50’s.
Perspective: In addition to being a ‘think-piece,’ the presentation offers pragmatic recommendations for the Agile champion in three critical areas:
- It explains how the Agile champion can cross three chasms that tend to form in the course of large scale Agile rollouts.
- It explores how to apply Agile priciples to software deployment and operations.
- It shows how earned value management can utilize ‘real time’ customer feedback in companies that embrace end-to-end Agility.
Think About Pilot Teams, not Pilot Projects – Guest Post by Alan Atlas
Rally’s Alan Atlas shares with us his insights on picking a pilot project for Agile. His post nicely complements the account Sue McKinney and Pollyanna Pixton gave about their approach to bootstrapping Agile at IBM (click here). It also touches on some of the points made in our post The First Decision to Make. Whether you agree or disagree with Alan, his thoughts are always intriguing. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan:
Picture this: You’re an Agile Coach and you arrive for the first day of your new, monster engagement at a large enterprise that has hired you to help them become Agile. You’re very excited as you walk into your first training session with a select group of employees. As you start the training, you are greeted with questions from your happy, excited audience. “Can we get this over with early?” “I don’t want to be here. My manager said I had to come.” “What is this all about, anyway?” “I have a friend who got fired for advocating Scrum. I don’t want anything to do with it.” “Why am I here?” Is this any way to start an Agile transformation?
A necessary step in Agile adoptions is picking where to start. Consultants often help management or transition teams with the selection of so-called pilot projects. Project teams are then notified that they are the lucky winners in the Scrum Lottery. The result can be a (not entirely incorrect) feeling amongst employees that they are being forced to become Agile, which can lead to the scene I just described.
This command-and-control (C&C) approach can easily erode trust, destroy motivation and handicap an adoption program because it ignores the critical contribution that team buy-in and ownership can make toward successful implementation of the new development process. At best, it leaves a mild bad taste in the mouths of the employees and gives them a good reason to believe that this Scrum thing is just another management flavor of the week. At worst, it removes the single most important success factor in Agile adoption: the active and enthusiastic participation of the team.
Is there a reasonable way to avoid the pitfalls of the C&C approach and still meet the legitimate needs of management? Can we start a transformation with excited, enthusiastic employees instead of sullen ones? Can management make a decision to ‘go Agile’ and still establish a collaborative relationship with employees? Can we take advantage of the inherent appeal of Agile methods to increase our chances of success? I think the answer to all of these is, “Yes!”
Agile’s people-based approach tells us to view the world from a team-centric point of view and not a project-centric point of view. Applying this to our Agile transition itself, we realize that we don’t need to assign Scrum to projects. Instead, we can let teams choose to adopt Scrum (or not to adopt, to be fair). It’s the team that will make the Agile process work and lead to success, not the project itself. This approach will maximize the likelihood of success by finding the teams that want to make it work.
The way to ensure the success of early Agile transformation efforts and simultaneously to align management and employees without coercion is to provide teams with the permission and knowledge to make their own decisions, and then for management to support those decisions. Teams that choose to ‘go Agile’ will make it work. Teams that are told to ‘go Agile’ might or might not make it work.
Implementing this isn’t hard. Start by making introductory Agile training available to the target organization (a few two-hour classes spread throughout a week is a good start for all but the largest organizations). Announce that teams are welcome to try Scrum, and tell them how to request further team training and coaching (don’t neglect to make sure their immediate management is on board). There might be reasons to give priority to certain teams and possibly to delay others, but in general the teams that want to do Agile should be allowed to do it. The company is now supporting and leveraging teams that want to transition to Agile and allowing those that don’t want to change to continue as they are. This in itself is an important message to all that the Agile philosophy is being taken seriously by management.
The result of using this approach is that there is no bad taste among employees, the most enthusiastic teams self-select to participate in the new experiment, nobody is forced, and management demonstrates its willingness to support employees in the new endeavor.
No managers were harmed in the filming of this scenario. 🙂 Seriously, we put the focus on teams without negating any of the needs of the enterprise. The ability of teams to get further training can still be managed. The identities of the teams can be known so that there can be followup assessments, monitoring, and support. In the cases where it is necessary, teams that have interdependencies or other complexities can be staged appropriately. The biggest and most important difference between this scenario and the more traditional C&C scenario is that here the teams that are excited and interested about Agile become the first in the water.
Using this method for starting your transition doesn’t change anything else about your organizational Agile initiative. You still have all of the cultural, technical, engineering, management, organizational, and human issues to deal with. It just gives you a way to pick the starting point in a more positive and Agile way.
A Note on “The Tool is the Method”
The post The Tool is the Method generated a lively discussion. Whether you agree or disagree with the post, you are likely to find the experience reported by Rod McLaren in Jira, Greenhopper, Lighthouse: tools vs practice in Scrum of interest. Here is an excerpt from Rod’s post:
Our biggest problem with Jira/Greenhopper was that it isn’t as well-aligned to our developing practice of scrum as we needed it to be. The tool got in the way of our learning the practice, and started behaviourally skewing our view of scrum (‘The chosen tool becomes a factor of the first order in determining not “only” how Agile (or any other software method) will be practiced, but what mindset will evolve in the course of the rollout’). We were trying to fit scrum to the tool we had, and were talking more about how to use the tool well than we were about whether the sprint was going well.
I don’t know that the debate tool v. method could or should be resolved here. Absent resolution, the perspective on the subject Annie Shum expressed in Tools, Behavior, Culture is quite a good one to keep in mind:
In thinking about tools in the context of impacting large scale cultural changes, one can’t help but immediately think about the computer/digital computing. By all definitions, a computer is a tool – in fact, I would venture to characterize a computer as a “universal tool” that can, in theory, perform almost any task. Moreover, I can’t think of any other tool that can surpass the computer and digital computing as the most disruptive transformation agents of our society and cultural shifts in the 21st Century.
The Heretech Episode 14 – Agile Adoption
Forrester’s Tom Grant posted a podcast with me on Agile adoption. Here is an excerpt from Tom’s preamble to the subject:
Israel Gat illuminates the ways in which Agile adoption depends on organizational and cultural factors. We also muse about Helmut von Moltke, 19th century military Agilist. (Go look it up!)
Click here for Tom’s preamble in entirety, including the link to the podcast.
Scrum at Amazon – Guest Post by Alan Atlas
Rally’s Alan Atlas shares with us his experience as the first full-time Agile trainer/coach with Amazon. His account is both enlightened and enlightening. He connects the “hows”, “whats” and “whys” of Scrum in the Amazon context, making sense for the reader of what took place and what did not at Amazon. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan on what he learned about Scrum transition at Amazon.com:
Agile practices were present at Amazon.com as early as 1999, but it wasn’t until the years 2004 – 2009 that widespread adoption of Scrum occurred throughout Amazon’s development organizations. Amazon.com’s unplanned, decentralized Scrum transformation is of interest because it is different from the current orthodoxy regarding enterprise Scrum transitions, and its strengths and weaknesses reveal some fundamental lessons that can be applied to other enterprise Scrum transitions.
Here are the major forces that played in the transition.
Permission
Teams (including local management, of course) at Amazon have historically been given wide latitude to solve their problems (coupled with responsibility to do so without waiting for outside help) and are usually not overburdened with detailed prescriptive practices promulgated by centralized corporate sources. The emphasis is on creating, delivering, and operating excellent software in as streamlined and process-light a way as possible. Teams at Amazon have permission to choose.
Teams
The corporate culture at Amazon.com has always been surprisingly consistent with and friendly towards Agile practices. The 2 Pizza Team concept has been written about many times over the years (click here), and a close look shows that a 2 Pizza Team is basically a Scrum team without the Scrum. Teams at Amazon, especially 2 Pizza Teams, are stable and long-lived. Usually a development team reports to a single direct manager.
Knowledge
All it took to light the fire was someone who was willing to spend a little time educating interested parties about Scrum. Teams who learned about Scrum were able to make local decisions to implement it. Results were demonstrated that kindled interest in other teams.
Impetus
Over time, an email-based Scrum community formed. Scrum Master training was provided on an occasional basis by someone who simply wanted to do so. Basic Scrum education continued on an ad hoc and voluntary basis. Eventually enough teams had adopted Scrum that a need was seen and a position of Scrum Trainer/Coach was created. Having a full-time Trainer and Coach available made adoption easier and improved the quality of scrum implementations. By mid-2008 the community was able to support an Open Space Scrum Gathering within the company.
What was (and one assumes is still) missing was higher level engagement at the organization and enterprise levels. No executive support for Scrum ever emerged, and the transition was therefore limited primarily to the team level, with many organizational impediments still in place.
The success of Scrum at Amazon validates one easy, frictionless way to begin a Scrum transition.
- Establish stable teams
- Make Agile and Scrum information widely and easily available
- Give permission to adopt Scrum
The advantage of this approach is that it requires a minimum of enterprise-wide planning and it allows teams to select Scrum, rather than mandating it. All of the rest of an enterprise Scrum transition can be accomplished by simply responding to impediments as raised by the teams and providing management support for change. Based on experience, the impediments raised will include demand (pull) for coaching, scaling, training, organizational change, a Transition Team, PMO changes, and all of the other aspects of an enterprise transition that many organizations labor so mightily to plan and control. Leadership for this kind of transition can only be Servant Leadership from the C-level, which is exactly the right kind for an Agile initiative, isn’t it?
The only impediment to Scrum adoption at Amazon was lack of knowledge. Teams were in place, and permission was part of the culture. When knowledge was provided, teams adopted Scrum. The strength of this process was based on the fact that only teams that were interested in trying Scrum actually tried it. There was no mandate or plan or schedule for this uptake. Nobody was forced to use Scrum. Teams made an independent, informed decision to try to solve some of their problems. Lean and Agile thinkers will recognize that this as a pull-based incremental approach and not a plan-driven, command and control, push-based approach.
What about the things that didn’t happen at Amazon? The transition stalled at the team level due to failure to engage either middle or upper management in a meaningful way. Both of those groups are required to bring a transition to its full potential. Training for middle managers, in particular, is crucial, but will usually reach them only with executive sponsorship. A Transition Team is crucial when organizational and enterprise-wide impediments begin to be unearthed. Support from a source of advanced knowledge and experience (trainer/coach) is key.
Was Scrum good or bad overall for Amazon? There is only spotty, anecdotal data to report. Certainly there are many stories of teams that used Scrum very successfully. The Amazon S3 project, which not only delivered on time after about 15 months of work, but nearly achieved the unusual result of having the entire development team take a week’s vacation leading up to the launch day. It was not the crunch-time, last minute, panic-drenched effort that is common with projects of this scope and complexity. There was the team that “hasn’t been able to deliver any software for 8 months” that, sure enough, delivered some software a month later at the end of their first sprint. Another team reported that their internal customers came to them some time after the team had adopted Scrum, asking that they implement a whole list of random features. “We know these aren’t your responsibility, but you’re the only team that is able to respond to our requests.” Finally, there was the platform team that had literally dozens of internal customers. When that team adopted Scrum, they organized their customers into a customer council of sorts and let them simply decide each month what the team would work on, for the good of all, in order of value to Amazon overall. But even if none of these anecdotes were available to tell, the mere fact that teams opted on their own to adopt Scrum implies that something about Scrum helped them be more successful and more satisfied professionally. If that were not true, then they would not have stayed with it at all, right?
