The Agile Executive

Making Agile Work

Archive for the ‘Cultural Aspects’ Category

A Devops Case Study

leave a comment »

An outline of my forthcoming Agile 2010 workshop was given in the post “A Recipe for Handling Cultural Conflicts in Devops and Beyond” earlier this week. Here is the case study around which the workshop is structured:

NotHere, Inc. Case Study

NotHere, Inc. is a $500M company based in Jerusalem, Israel. The company developed an eCommerce platform for small to medium retailers. Through a combination of this platform and its hosting data center, NotHere provides online store fronts, shopping carts, order processing, inventory, billing and marketing services to tens of thousands of retailers in a broad spectrum of verticals. For these retailers, NotHere is a one-stop “shopping” for all their online needs. In particular, instead of partnering with multiple companies like Amazon, Ebay, PayPal and Shopzilla, a retailer merely needs to partner with NotHere (who partners with these four companies and many others).

The small to medium retailers that use the good services of NotHere are critically dependent on the availability of its data center. For all practical purposes retailers are (temporarily) dead when the NotHere data center is not available. In recognition of the criticality of this aspect of its IT operations, NotHere invested a lot of effort in maturing its ITIL[i] processes. Its IT department successfully implements the ITIL service support and service delivery functions depicted in the figure below. From an operational perspective, an overall availability level of four nines is consistently attained. The company advertises this availability level as a major market differentiator.

In response to the accelerating pace in its marketplace, NotHere has been quite aggressive and successful in transitioning to Agile in product management, dev and test. Code quality, productivity and time-to-producing-code have been much improved over the past couple of years. The company measures those three metrics (quality, productivity, time-to-producing-code) regularly. The metrics feed into whole-hearted continuous improvement programs in product management, dev and test. They also serve as major components in evaluating the performance of the CTO and of the EVP of marketing.

NotHere has recently been struggling to reconcile velocity in development with availability in IT operations. Numerous attempts to turn speedy code development into fast service delivery have not been successful on two accounts:

  • Technical:  Early attempts to turn Continuous Integration into Continuous Deployment created numerous “hiccups” in both availability and audit.
  • Cultural: Dev is a competence culture; ops is a control culture.

A lot of tension has arisen between dev and ops as a result of the cultural differences compounding the technical differences. The situation deteriorated big time when the “lagging behind” picture below leaked from dev circles to ops.

The CEO of the company is of the opinion NotHere must reach the stage of Delivery over Development. She is not too interested in departmental metrics like the time it takes to develop code or the time it takes to deploy it. From her perspective, overall time-to-delivery (of service to the retailers) is the only meaningful business metric.

To accomplish Delivery over Development, the CEO launched a “Making Cats Work with Dogs[ii]” project. She gave the picture above to the CTO and CIO, making it crystal clear that the picture represents the end-point with respect to the relationship she expects the two of them and their departments to reach. Specifically, the CEO asked the CTO and the CIO to convene their staffs so that each department will:

  • Document its Outmodel (in the sense explored in the “How We Do Things Around Here In Order to Succeed” workshop) of the other department.
  • Compile a list of requirements it would like to put on the other group “to get its act together.”

The CEO also indicated she will convene and chair a meeting between the two departments. In this meeting she would like each department to present its two deliverables (world view of the other department & and the requirements to be put on it) and listen carefully to reflections and reactions from the other department. She expects the meeting will be the first step toward a mutual agreement between the two departments how to speed up overall service delivery.


[i] “Information Technology Infrastructure library – a set of concepts and practices for Information Technology Services Management (ITSM), Information Technology (IT) development and IT operations” [Wikipedia].

[ii] I am indebted to Patrick DeBois for suggesting this title.

© Copyright 2010 Israel Gat

A Recipe for Handling Cultural Conflicts in Devops and Beyond

with 4 comments

My Agile 2010 workshop “How We Do Things Around Here In Order To Succeed”  will weave together four trends that I am witnessing in my practice:

  • The ascendance of Agile portfolio management in a world characterized by loosely coupled processes
  • Devops dynamics are becoming more and more characteristic of end-to-end Agile/Kanban patterns
  • Viral spread of technical debt metrics in software governance
  • Increasing use of boundary objects in the enterprise context

The workshop is structured around three case studies/exercises that will take about two-thirds of the allotted time (the morning of August 9). The other third provides the theory and tools to be used in the three workshop exercises and (hopefully) in many future engagements participants in the workshop will carry out. Deep technical knowledge is not required – the workshop targets any Agile practitioner who has conceptual grasp of culture, software development, IT operations and portfolio management.

The #1 takeaway from the presentation is the details you need to know about creation and capture of lasting value through end-to-end Agile initiatives.

Here is the workshop agenda (still subject to some minor tweaking):

  • Introduction to Cultural Framework
  • Exercise #1: Strengths and Weaknesses of Your Culture
  • Change Behavior, Not Culture
  • When Organizations Clash
  • Exercise #2: Conflicts in Devops
  • The Agile Flywheel
  • Exercise #3: Using Technical Debt as a Boundary Object in Devops
  • Bringing Organizations Together Through Enlightened Governance Loops

I look forward to meeting you in the workshop and learning from your experiences and insights!

Israel

Predictability is Bad for Your Business

with 2 comments

I had the pleasure of meeting some old colleagues a few weeks ago. They work for a software company that pays a lot of attention to software engineering practices and invests heavily in software tools. Financial results, however, have not been great over the past few years.

Obviously, the disparity between the strength of the software engineering discipline and the relative weakness of the financial results is due to more than a single cause. One factor, however, was highlighted time and time again by my colleagues:

Predictability is killing us!

Paradoxical that this observation might seem, it is actually quite straightforward. Senior management in their company is really forceful about predictability. Hence, initiative, (affordable) experimentation and innovation have pretty much faded away. For most practical purposes it has become a check-the-box culture. All attempts to substitute reliable delivery for predictability seem to have failed so far.

One last “ingredient” to add to the story. This company is rich in talent. Generally speaking, the folks in the engineering trenches are gifted, knowledgeable, capable and dedicated.

How predictably poignant!

Written by israelgat

November 12, 2009 at 4:00 am

Teach Your Boss to be Agile with a Social Contract – Guest Post by Alan Atlas

with 4 comments

When I [Israel] joined BMC Software in Fall 2004, I made a promise to each and every one of the hundreds of employees in my business unit:

I commit to read and respond within 48 hours to any email you send me. My answer is not likely to be long as I am drinking from a hose right now. But, it will be substantive.

Till this very day, various ex-employees of mine tell me that this simple statement was actually the first step toward our adopting Agile – it created mutuality in our relationships. A few months later, when we started discussing the ground rules for Agile team empowerment, I was credible with respect to adhering to voluntary “contracts” due to the mutuality established in the “email policy” cited above (and other “it cuts both ways” steps we as a management team have taken along the way).

Fast forward five years to today. How delightful it was to get the guest post below from Rally coach Alan Atlas. His post has taken me all the way back to the very gratifying experience of starting the enterprise-level Agile “adventure” at BMC. Furthermore, Alan’s post made me realize I need to add mutuality as the fifth ingredient in my ‘secret sauce’ for large-scale Agile implementations.

Here is Alan:

Hey, how’re you doing? How’s the new Agile thing going? What? Oh, yeah. We had that same thing. The manager thing. Yep. It can be a killer! Wanna know what we did?

Dan Rawsthorne first introduced me to social contracts. He had put together an example that was called something like “Contract Between the Team and the Organization”.  Israel Gat’s work on Agile Social Contracts gives perspective on using them at the enterprise level.

I have put social contracts to good use more than once, and I am convinced that they are a tool that has much to offer to coaches, consultants, and maybe most importantly, internal Agile champions at companies around the world.

Most of us are familiar with the idea of Working Agreements. Working Agreements are a form of social contract that is often used to help a self-organizing team to establish behavioral standards without having them imposed from the outside (e.g., by a manager). Working Agreements are:

  • Established and changed by mutual agreement
  • Enforced by mutual agreement
  • Outside of established corporate legal structure, and
  • Made between peers.

A social contract, at least the way I have seen them created and used, is essentially a Working Agreement between non-peers.  In an Agile context, social contracts are written between a team and its management, or a team and its encompassing organization.

Of what use is an agreement in which each clause can be summarized as “I promise to do something that you want until I change my mind”? I think there are two really important benefits to be realized by using social contracts:

  • They codify and externalize the agreement, making the substance of the agreement clear, and
  • They form the basis of an “interaction between people” (i.e., a conversation).

Here’s an extract from a hypothetical Social Contract:

  1. The Organization and the Team agree that:
    1. Customer satisfaction is our ultimate goal
    2. Mutual respect will be the foundation for trust between all parties
    3. The Team promises the Organization that:
      1. It will develop the most valuable software, as defined by the Organization through the Product Owner, at all times
      2. It will provide transparency in all things related to its activities
      3. The Organization promises the Team that:
        1. The Team’s success will be judged by the production of working software
        2. The Team will have a Product Owner and a Scrum Master and a reasonable expectation of team stability

Yes, it does seem a bit idealistic and maybe even unrealistic. Yet, it is invaluable when used in the following way:

“Hey Boss. One part of launching the team on Scrum is to sit down with you and go over our social contract. Let’s take a look and talk about the things that are in here.”

The social contract is, above all things, a means to direct a conversation with your manager about roles and behaviors in the new world of Scrum. If you can all actually agree on one, and even sign it, Wow! But if you can’t, you can still use it to begin to teach your boss (I bet she wasn’t included in team Scrum training, was she?) how to be a good boss in an Agile world. If you are afraid of broaching certain subjects, arrange to have a neutral Agile coach supply you with an Agile contract, in which case you can’t be blamed for the content.

“The Organization promises the Team that impediments raised by the team will receive prompt and thorough attention at all levels.”

“The Team promises the Organization that it will adhere to all corporate quality standards when building software.”

“The Team promises the Organization that it will maintain the highest possible level of technical rigor through design and code reviews.”

And on and on…

Don’t be too disappointed if it never gets completely agreed and signed and put on the wall. Use it to open up a dialog with your management that you can build on over time.

Written by israelgat

November 5, 2009 at 7:46 am

More on the Social Contract

with 3 comments

The posts A Social Contract for Agile and Additions to the Social Contract established the dire need to reconstitute the social contract at a time when software development and test jobs migrate off-shore in an unprecedented manner. As stated in the first of these two posts:

My sense in 2005 was that the social contract between employers and employees in the software industry was broken. Without a working social contract, the friction and antagonism can bring a system down. For example, in 1942 – the turning-point year of WWII – 833,000 days of coal mining were lost due to strikes in the British coal industry.

Colleague and friend Ryan Martens has just published an article on the subject in Dr. Dobb’s. Ryan examines the Agile Social Contact in the context of what it really takes to get Agile rolling. To quote him:

Can you see the simplicity of Agile Adoption when you apply appropriate commitment and structure? A truly effective Agile Social Contract that creates true trust and commitment requires clarity and discipline. With the transparency of a clearly communicated Agile Social Contract, you will establish a strong leadership mechanism that aligns all the stakeholders and teams within your Agile adoption. Of course Enterprise-scale agile adoptions take place in a larger context of the business and market. As Israel Gat stated in his personal Agile Social Contract, we cannot guarantee lifetime employment in this globally competitive world. But, by making a clear commitment to win-win agreements, we can change the conversation into a motivating one versus a de-motivating one. Don’t try to scale Agile without a real and personal commitment or without a clear rollout structure.

The fascinating thing to me is that Rally’s own social contract seems to have developed completely on its own. Best I know there had never been a conscious attempt to develop a social contract. Yet, the company is well-known for the strong affinity of its employees.

I will leave it Ryan to comment on this riddle…

Written by israelgat

October 27, 2009 at 3:26 pm

Role of the Agile Leader in Reconfiguring the Business

with one comment

Click here for the slide deck from my Agile 2009 presentation. 

Abstract: The presentation applies Agile thinking to critical aspects of strategy and execution at a time of uncertainty and disruption. The essential point is simple and logical: Agile values and principles are indivisible. To succeed, they must be applied not just to R&D, but also to customer and company, simultaneously. This requires reconfiguration of customer relationships, employee policy, software development, and the relationship that binds the three. The resulting paradigm shift could lower the cost of software and produce prosperity similar to the one induced by ultra-cheap oil in the 50’s.

Perspective: In addition to being a ‘think-piece,’ the presentation offers pragmatic recommendations for the Agile champion in three critical areas:

  •  It explains how the Agile champion can cross three chasms that tend to form in the course of large scale Agile rollouts.
  •  It explores how to apply Agile priciples to software deployment and operations.
  • It shows how earned value management can utilize ‘real time’ customer feedback in companies that embrace end-to-end Agility.

Think About Pilot Teams, not Pilot Projects – Guest Post by Alan Atlas

with one comment

Rally’s Alan Atlas shares with us his insights on picking a pilot project for Agile. His post nicely complements the account Sue McKinney and Pollyanna Pixton gave about their approach to bootstrapping Agile at IBM (click here). It also touches on some of the points made in our post The First Decision to Make. Whether you agree or disagree with Alan, his thoughts are always intriguing. You will find additional insights by Alan in The Scrum Mechanic.

Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.

Here is Alan:

Picture this: You’re an Agile Coach and you arrive for the first day of your new, monster engagement at a large enterprise that has hired you to help them become Agile. You’re very excited as you walk into your first training session with a select group of employees. As you start the training, you are greeted with questions from your happy, excited audience. “Can we get this over with early?” “I don’t want to be here. My manager said I had to come.” “What is this all about, anyway?” “I have a friend who got fired for advocating Scrum. I don’t want anything to do with it.” “Why am I here?”  Is this any way to start an Agile transformation?

A necessary step in Agile adoptions is picking where to start. Consultants often help management or transition teams with the selection of so-called pilot projects. Project teams are then notified that they are the lucky winners in the Scrum Lottery. The result can be a (not entirely incorrect) feeling amongst employees that they are being forced to become Agile, which can lead to the scene I just described.

This command-and-control (C&C) approach can easily erode trust, destroy motivation and handicap an adoption program because it ignores the critical contribution that team buy-in and ownership can make toward successful implementation of the new development process.  At best, it leaves a mild bad taste in the mouths of the employees and gives them a good reason to believe that this Scrum thing is just another management flavor of the week. At worst, it removes the single most important success factor in Agile adoption: the active and enthusiastic participation of the team.

Is there a reasonable way to avoid the pitfalls of the C&C approach and still meet the legitimate needs of management? Can we start a transformation with excited, enthusiastic employees instead of sullen ones? Can management make a decision to ‘go Agile’ and still establish a collaborative relationship with employees? Can we take advantage of the inherent appeal of Agile methods to increase our chances of success? I think the answer to all of these is, “Yes!”

Agile’s people-based approach tells us to view the world from a team-centric point of view and not a project-centric point of view. Applying this to our Agile transition itself, we realize that we don’t need to assign Scrum to projects. Instead, we can let teams choose to adopt Scrum (or not to adopt, to be fair). It’s the team that will make the Agile process work and lead to success, not the project itself. This approach will maximize the likelihood of success by finding the teams that want to make it work.

The way to ensure the success of early Agile transformation efforts and simultaneously to align management and employees without coercion is to provide teams with the permission and knowledge to make their own decisions, and then for management to support those decisions. Teams that choose to ‘go Agile’ will make it work. Teams that are told to ‘go Agile’ might or might not make it work.

Implementing this isn’t hard. Start by making introductory Agile training available to the target organization (a few two-hour classes spread throughout a week is a good start for all but the largest organizations). Announce that teams are welcome to try Scrum, and tell them how to request further team training and coaching (don’t neglect to make sure their immediate management is on board). There might be reasons to give priority to certain teams and possibly to delay others, but in general the teams that want to do Agile should be allowed to do it. The company is now supporting and leveraging teams that want to transition to Agile and allowing those that don’t want to change to continue as they are. This in itself is an important message to all that the Agile philosophy is being taken seriously by management.

The result of using this approach is that there is no bad taste among employees, the most enthusiastic teams self-select to participate in the new experiment, nobody is forced, and management demonstrates its willingness to support employees in the new endeavor.

No managers were harmed in the filming of this scenario. 🙂  Seriously, we put the focus on teams without negating any of the needs of the enterprise. The ability of teams to get further training can still be managed. The identities of the teams can be known so that there can be followup assessments, monitoring, and support. In the cases where it is necessary, teams that have interdependencies or other complexities can be staged appropriately. The biggest and most important difference between this scenario and the more traditional C&C scenario is that here the teams that are excited and interested about Agile become the first in the water.

Using this method for starting your transition doesn’t change anything else about your organizational Agile initiative. You still have all of the cultural, technical, engineering, management, organizational, and human issues to deal with. It just gives you a way to pick the starting point in a more positive and Agile way.

Written by israelgat

August 19, 2009 at 7:01 am

A Note on “The Tool is the Method”

with one comment

The post The Tool is the Method generated a lively discussion. Whether you agree or disagree with the post, you are likely to find the experience reported by Rod McLaren in Jira, Greenhopper, Lighthouse: tools vs practice in Scrum of interest. Here is an excerpt from Rod’s post:

Our biggest problem with Jira/Greenhopper was that it isn’t as well-aligned to our developing practice of scrum as we needed it to be. The tool got in the way of our learning the practice, and started behaviourally skewing our view of scrum (‘The chosen tool becomes a factor of the first order in determining not “only” how Agile (or any other software method) will be practiced, but what mindset will evolve in the course of the rollout’). We were trying to fit scrum to the tool we had, and were talking more about how to use the tool well than we were about whether the sprint was going well.

I don’t know that the debate tool v. method could or should be resolved here. Absent resolution, the perspective on the subject Annie Shum expressed in Tools, Behavior, Culture is quite a good one to keep in mind:

In thinking about tools in the context of impacting large scale cultural changes, one can’t help but immediately think about the computer/digital computing.  By all definitions, a computer is a tool – in fact, I would venture to characterize a computer as a “universal tool” that can, in theory, perform almost any task.  Moreover, I can’t think of any other tool that can surpass the computer and digital computing as the most disruptive transformation agents of our society and cultural shifts in the 21st Century.

Written by israelgat

August 5, 2009 at 6:55 pm

The Heretech Episode 14 – Agile Adoption

leave a comment »

Forrester’s Tom Grant posted a podcast with me on Agile adoption. Here is an excerpt from Tom’s preamble to the subject:

Israel Gat illuminates the ways in which Agile adoption depends on organizational and cultural factors. We also muse about Helmut von Moltke, 19th century military Agilist. (Go look it up!)

Click here for Tom’s preamble in entirety, including the link to the podcast.

Written by israelgat

July 20, 2009 at 1:31 pm

Scrum at Amazon – Guest Post by Alan Atlas

with 6 comments

Rally’s Alan Atlas shares with us his experience as the first full-time Agile trainer/coach with Amazon. His account is both enlightened and enlightening. He connects the “hows”, “whats” and “whys” of Scrum in the Amazon context, making sense for the reader of what took place and what did not at Amazon. You will find additional insights by Alan in The Scrum Mechanic.

Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.

Here is Alan on what he learned about Scrum transition at Amazon.com:

Agile practices were present at Amazon.com as early as 1999, but it wasn’t until the years 2004 – 2009 that widespread adoption of Scrum occurred throughout Amazon’s development organizations. Amazon.com’s unplanned, decentralized Scrum transformation is of interest because it is different from the current orthodoxy regarding enterprise Scrum transitions, and its strengths and weaknesses reveal some fundamental lessons that can be applied to other enterprise Scrum transitions.

Here are the major forces that played in the transition.

Permission

Teams (including local management, of course) at Amazon have historically been given wide latitude to solve their problems (coupled with responsibility to do so without waiting for outside help) and are usually not overburdened with detailed prescriptive practices promulgated by centralized corporate sources. The emphasis is on creating, delivering, and operating excellent software in as streamlined and process-light a way as possible. Teams at Amazon have permission to choose.

Teams

The corporate culture at Amazon.com has always been surprisingly consistent with and friendly towards Agile practices. The 2 Pizza Team concept has been written about many times over the years (click here), and a close look shows that a 2 Pizza Team is basically a Scrum team without the Scrum. Teams at Amazon, especially 2 Pizza Teams, are stable and long-lived. Usually a development team reports to a single direct manager.

Knowledge

All it took to light the fire was someone who was willing to spend a little time educating interested parties about Scrum. Teams who learned about Scrum were able to make local decisions to implement it. Results were demonstrated that kindled interest in other teams.

Impetus

Over time, an email-based Scrum community formed. Scrum Master training was provided on an occasional basis by someone who simply wanted to do so. Basic Scrum education continued on an ad hoc and voluntary basis. Eventually enough teams had adopted Scrum that a need was seen and a position of Scrum Trainer/Coach was created. Having a full-time Trainer and Coach available made adoption easier and improved the quality of scrum implementations. By mid-2008 the community was able to support an Open Space Scrum Gathering within the company.

What was (and one assumes is still) missing was higher level engagement at the organization and enterprise levels. No executive support for Scrum ever emerged, and the transition was therefore limited primarily to the team level, with many organizational impediments still in place.

The success of Scrum at Amazon validates one easy, frictionless way to begin a Scrum transition.

  1. Establish stable teams
  2. Make Agile and Scrum information widely and easily available
  3. Give permission to adopt Scrum

The advantage of this approach is that it requires a minimum of enterprise-wide planning and it allows teams to select Scrum, rather than mandating it. All of the rest of an enterprise Scrum transition can be accomplished by simply responding to impediments as raised by the teams and providing management support for change. Based on experience, the impediments raised will include demand (pull) for coaching, scaling, training, organizational change, a Transition Team, PMO changes, and all of the other aspects of an enterprise transition that many organizations labor so mightily to plan and control. Leadership for this kind of transition can only be Servant Leadership from the C-level, which is exactly the right kind for an Agile initiative, isn’t it?

The only impediment to Scrum adoption at Amazon was lack of knowledge. Teams were in place, and permission was part of the culture. When knowledge was provided, teams adopted Scrum. The strength of this process was based on the fact that only teams that were interested in trying Scrum actually tried it. There was no mandate or plan or schedule for this uptake.  Nobody was forced to use Scrum. Teams made an independent, informed decision to try to solve some of their problems. Lean and Agile thinkers will recognize that this as a pull-based incremental approach and not a plan-driven, command and control, push-based approach.

What about the things that didn’t happen at Amazon? The transition stalled at the team level due to failure to engage either middle or upper management in a meaningful way.  Both of those groups are required to bring a transition to its full potential. Training for middle managers, in particular, is crucial, but will usually reach them only with executive sponsorship.  A Transition Team is crucial when organizational and enterprise-wide impediments begin to be unearthed. Support from a source of advanced knowledge and experience (trainer/coach) is key.

Was Scrum good or bad overall for Amazon? There is only spotty, anecdotal data to report. Certainly there are many stories of teams that used Scrum very successfully. The Amazon S3 project, which not only delivered on time after about 15 months of work, but nearly achieved the unusual result of having the entire development team take a week’s vacation leading up to the launch day. It was not the crunch-time, last minute, panic-drenched effort that is common with projects of this scope and complexity. There was the team that “hasn’t been able to deliver any software for 8 months” that, sure enough, delivered some software a month later at the end of their first sprint. Another team reported that their internal customers came to them some time after the team had adopted Scrum, asking that they implement a whole list of random features. “We know these aren’t your responsibility, but you’re the only team that is able to respond to our requests.” Finally, there was the platform team that had literally dozens of internal customers. When that team adopted Scrum, they organized their customers into a customer council of sorts and let them simply decide each month what the team would work on, for the good of all, in order of value to Amazon overall. But even if none of these anecdotes were available to tell, the mere fact that teams opted on their own to adopt Scrum implies that something about Scrum helped them be more successful and more satisfied professionally. If that were not true, then they would not have stayed with it at all, right?

Written by israelgat

July 20, 2009 at 12:20 am