The Agile Executive

Making Agile Work

Posts Tagged ‘Lean

“Big Agile”

leave a comment »

On January 30, 2012 12:00 pm EST, colleague and friend Hubert Smits and I will be doing a Cutter webinar entitled “Big Agile” is More than Just a Software Method.  We will follow on in February with a “Big Agile” issue of the Cutter IT Journal [CITJ] for which I am the guest editor. Coming April we are likely to discuss the topic some more in the Cutter Summit.

The heart of the webinar can be summarized in the following words:

Small is beautiful in software. While big software might not be beautiful, more often than not, it’s in the nature of what needs to be accomplished. This contrast between the beauty of small and the requirements of the big generates systemic tension in many software projects, organizations, and companies. Resolving this conflict is the focus of this webinar.

Discussing the webinar and the follow-on CITJ issue with various folks in the Agile, Lean and Kanban movement(s), I became painfully aware of the very many interpretation folks associate with the the term “Big Agile.” Hence, I would like to say a few words on what it means to me.  I am taking the very pragmatic view of a client on the subject. A VP of one department or another is chartered to implement an Agile roll-out at scale. The roll-out might not include all the teams. Nor might it include all functions within the company. The roll-out, however, affects a significant number of folks. Focusing the roll-out at the team level is not sufficient.

The typical VP that I run into under such circumstances is not an expert on software methods (and usually acknowledges it). He/she, however, is smart enough and experienced enough to understand that scale matters. He/she knows or feels intuitively that Big Agile is akin to Big Data. Data is data is data, but when it is Big Data you need to address various aspects that do not manifest themselves in dealing with ordinary amounts of data. Likewise for Agile IMHO.

We plan to make the webinar very interactive. Anne Mullaney will start with a few ‘warm up’ questions to enable us to ‘dig’ into the subject, thence turn it over to questions from the audience. We plan to take the discussion to wherever the participants might want to.

I hope you will join us whether you love, hate or indifferent to the term “Big Agile.” We are expecting a lively discussion…

Written by israelgat

January 22, 2012 at 10:51 pm

Balancing Agile – Guest Post by Alan Shalloway

with 3 comments

A fascinating difference exists between Agile and Business Service Management (BSM). Agile emphasizes continuous flow of value to the customer. In contrast, BSM focuses on the business – it aligns the deliverables of IT to the enterprise’s business goals. Subtle that the difference might be, the two methods evolved along quite different lines in spite of the common denominator – dealing with software.

In this guest post, Alan Shalloway – Founder and CEO of NetObjectives – discusses the implications of focusing on the business as distinct from focusing on the customer. His discussion is part of  a few thought-provoking threads he weaves around the Agile Manifesto. Alan perceives the Manifesto a product of the times. He thinks aloud whether today’s circumstances require a revised manifesto.

Alan is a man of passion. While I do not always agree with him, I have a lot of respect for his quest to find the deeper truths. Furthermore, I always learn from him. Whether you agree or disagree with the opinion Alan articulates in this post, “listening” to his thoughts is well worth your time.

Here is Alan:

The Agile Manifesto was a watershed event that has forever changed the landscape of software development.  So profound a positive impact of it has had, that few challenge whether it was actually correct.  Manifestos are often  a statement in reaction to something prevalent that needs changing.  This makes them very topical and temporal – and the exact intention needs to be restated when the landscape against which it was drawn has changed. The Agile Manifesto[1] states:

“Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.”

At the time of its edict, this was profound and well stated.  Too many software teams were:

  • Following a process dictated to them from outside their team
  • Managing according to an extensive set of artifacts that recorded where they supposedly were well before software existed
  • Were given a set of requirements to meet with little opportunity to discuss real needs (note, points 3 and 4 in the manifesto address this point in two ways – first recognizing that customer collaboration is necessary to discover their true needs and second that it is essential to take advantage of newly discovered information)

The manifesto represented a new paradigm from which to work – one in which the team would have better control over its destiny and where it was recognized that one had to make incremental, iterative movement towards one’s goals – both in discovering the true goal and in implementing it.

Unfortunately, the perspective from which the manifesto was created, or at least the methods which first followed the manifesto, have been extremely team centric.  Not a surprise, given the paradigm at the time gave development teams too little say in their own methods.  The impact of this has been, not surprisingly, success at the teams and difficult beyond the team. It is almost axiomatic now that companies will have successful team pilots only to bog down in their enterprise agile adoption efforts or even revert back to earlier methods.

I say “not surprisingly” because several things have been left out of the agile manifesto.  These are:

  • The role of management
  • The role of process
  • The role of planning
  • And indirectly, the role of guiding principles

It could also be said that the driver for agile development is misplaced.  I do not believe “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”  This makes software groups customer driven, not business driven.

There is a subtle, but important difference. Basically, conscious or not, the Agile Manifesto is driven with a team-centric view of satisfying customers – business and management play mostly secondary roles. Unfortunately, there is a significant difference between customer driven and business driven (see the figure entitled Alignment with Vision of Business to the right).  This is not apparent at the team level (the team is supposed to satisfy the customer).  But definitely at the business level. Not surprising, however, since the manifesto is a team perspective it states things in terms of customer value.

Ironically, it is the over-focus on the team, however, that is robbing teams of one of their greatest tools.  Clearly any method must have respecting people and providing those doing the work the ability to choose how they do their work. But this does not mean that process isn’t essential or that attending to certain laws of software development is optional.  Rather it means that process can’t be imposed on teams since to do so would both rob them of respect and almost certainly be the wrong way to do things – who knows more about how to get work done than the people doing the work themselves?

But “Individuals over process” as the first line has come to be called, makes it sound like people are it.  I do not think so – and I think this mindset has caused a lot of damage in many ways.

There is great evidence that the best approach is not merely get smart people onto a team and have them figure out how to solve their problems.  They must be properly equipped to do so.  Just being smart doesn’t mean you can solve the challenges facing you.  This should be readily apparent, but in many ways, the Agile community has mostly ignored it. Actually, there is not really an Agile community any more – there are factions that have significantly different beliefs.  For example, XP has long recognized the need for technical practices in Agile while the Scrum community is only just starting to get into what these are.

However, except for the Lean/Kanban community, few Agilists seem to espouse discovering and following the laws of product development flow (or even recognize their existence).  This, in my mind, has led to the low rate of success in scaling Scrum to the enterprise*[2]. Ironically, it is the over-focus on people that leads many in the Scrum community to assert this lack of success is a lack willingness to take the effort to improve. This is not surprising – if it’s up to the team to succeed, then when they don’t it must be something wrong with the team or their management when they fail.  I think not. I think it is the lack of understanding of the principles of software development flow.

These laws are not new.  Don Reinertsen, in his iconic book Managing the Design Factory lays out much of the rules of product development.  His more recent book The Principles of Product Development Flow: Second Generation Lean Product Development he lays out 175 of these principles.

To me, true respect for people means that one must equip them with what they need to get their job done.  Our thinking in the Agile community should change from “People over process” to one of “People times process.[3]This phrase emphasizes that if either are low, you get a low productivity.  Process does not ensure success. But a poor process requires heroes to succeed.  We’d like good, motivated, well-intentioned people to be able to succeed.

Our new agile perspective needs to include an understanding of what teams need to know to do their work. This opens up a role for managers to actively help teams get their job done and to coach them when they have challenges or lose their way.  While I will always be thankful for the Agile Manifesto, I am looking for a Business Agile Manifesto that will expand the focus from the team to the entire enterprise.

Footnotes:

[1] http://www.agilemanifesto.org

[2] Ken Schwaber, iconic leader of the Scrum community has said that “I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it.”

[3] I first heard this from Don Reinertsen.  Before that I used to say “People plus process.”

Harnessing Economies of Scale in Cloud Computing to Realize a Greener Computing Option

with 2 comments

Economies of Scale have been much discussed in The Agile Executive since the recent OpsCamp in Austin, TX. The significant savings on system administration costs  in very large data centers have been called out as a major advantage of Internet-scale Clouds. Unlike various short-lived advantages, the benefits to the Cloud operator, and to the Cloud user when the savings are passed on to him/her, are sustainable.

In this guest post, colleague and friend Annie Shum analyzes the various sources of waste in operations in traditional data centers. Like an Agilist with Lean inclinations who confronts an inefficient Waterfall process, Annie explains how economies of scale apply to the various kinds of waste that are prevalent in today’s small and medium data centers. Furthermore, she connects the dots that lead toward a Green IT option.

Here is Annie:

Harnessing Economies of Scale in Cloud Computing to Realize a Greener Computing Option

Scale Matters: “Over time, however, competitive advantage within categories shifts inexorably toward volume operations architecture.” – Geoffrey Moore, “Dealing with Darwin”

It is a truism that today’s datacenters are systemically inefficient. This is not intended as an indictment of all conventional datacenters. Nor does it imply that today’s datacenters cannot be made more efficient (incrementally) through right sizing and other initiatives, notably consolidation by deploying virtualization technologies and governance by enforcing energy conservation/recycling policies. There are a myriad of inefficiencies, however, that are prevalent in datacenters today.

Many industry observers lament the “staggering complexity” that permeates on-premises datacenters. Over time, most, if not all, enterprise IT datacenters have become amalgamations of disparate heterogeneous resources. Generally, they can be described as incohesive, perhaps even haphazard, accumulations. The datacenter components and configurations often reflect the intersections of organizational politics (LOB reporting structures leading to highly customized/organizational asset acquisitions and configurations), business needs of the moment (shifting corporate strategies and changing business imperatives to gain competitive edge or meet regulatory compliances) and technology limitations (commercial tools available in the marketplace). It should come as no surprise that human interactions and errors are considered a major contributor to the inefficiencies of datacenters: IBM reported that human errors account for seventy percent of the datacenter problems.

The challenge of maximizing energy efficiency begins fundamentally with the historical capital-intensive ownership model for computing assets to enable each organization to operate its own datacenter and to provide “24×7 availability” to its own users.  The enterprise IT staff has been required to support unpredictable future growth, accommodate situational demands and unscheduled but deadline-critical events, meet performance levels within SLAs and comply with regulatory and auditing requirements. Hence, datacenters generally are over-configured and over-provisioned. In addition to highly skewed under-utilization of distributed platform servers, ninety percent of corporate datacenters have excess cooling capacity. Worst of all, according to IBM, about seventy-two percent of cooling bypassed the computing equipment entirely. Further compounding these problems for a typical enterprise datacenter, is the lack of transparency and the inability to control energy consumption properly due to inadequate and often inaccurate instrumentation to quantify energy consumption and waste due to energy lost.

The economics of Cloud Computing can offer a compelling option for more efficient IT: by lowering power consumption for individual organizations and by improving the efficiency of a large number of discrete datacenters. Although the electricity consumption of Cloud Computing is projected to be one to two percent of today’s global electricity use, Cloud service providers can still cultivate sustainable Green I.T. effectively at lower costs by leveraging state-of-the-art super energy efficient massive datacenters, proximity to power generation thereby reducing transmission costs and, above all, harnessing enormous economies of scale. To better understand how Cloud Computing can offer greener computing in the Cloud and how will it help moderate power consumption by datacenters and rein in run-away costs, a good starting place is James Hamilton’s September 2008 study on Internet-Scale Service Efficiency” as summarized in the table below.

Resource Cost in

Medium DC

Cost in

Very Large DC

Ratio
Network $95 / Mbps / month $13 / Mbps / month 7.1x
Storage $2.20 / GB / month $0.40 / GB / month 5.7x
Administration ≈140 servers/admin >1000 servers/admin 7.1x

Table 1: Internet-Scale Service Efficiency [Source: James Hamilton]

This study concludes that hosted services by Cloud providers with super large datacenters (at least tens of thousands of servers) can achieve enormous economies of scale of five to seven times over smaller scale (thousands of servers) medium deployments.  The significant cost savings is driven primarily by scale. Other key factors include location (low cost real estate and electricity rate, abundant water supply and readily available fiber-optic connectivity), proximity to electricity and power generators, load diversity, and virtualization technologies.

Will this mark the beginning of the end for traditional on-premises datacenters? Can enterprise IT continue to justify new business cases for expanding today’s non-renewable energy powered datacenters? According to the McKinsey article, the costs to launch a large enterprise datacenter have risen sharply from $150M to over $500M over the past five years. The facility operating costs are also increasing at about twenty percent per year. How long will the status quo last for enterprise IT considering the recent trend of Cloud service providers? Major players such as Google, Microsoft as well as the U.S. government itself have invested in or are planning ultra energy-efficient mega-size datacenters (also known as “container hotels”) with massive commoditized containerization and proximity both to power source and less expensive power rates. Bottom line: will the tide turn if the economics (radical cost savings) due to enormous economies of scale become too significant to ignore?

Despite the potential for significant cost savings, it is premature to declare the demise of traditional IT or the end of enterprise datacenters. After all, the rationale for today’s enterprise IT extends well beyond simplistic bottom-line economics – at least for now. To most industry observers, enterprise datacenters are unlikely to disappear although the traditional roles of enterprise IT will be changing. A likely scenario may involve redistributing IT personnel from operating low-level system operational tasks to addressing higher-level functions involving governance, energy management, security and business processes. Such change not only would become more apparent but will likely be precipitated by the rise of hybrid Clouds and the growing interconnection linking SOA, BPM and social computing. Another likely scenario is the rise of the mega datacenters or “container hotels” for Cloud Utility Computing providers. Although the global economic outlook will undoubtedly play a key role in shaping the development plans/timelines of the mega datacenters, they are here to stay. Case in point: by 2012, Intel estimates that it will design and ship about a quarter of the server chips (it sells) to such mega-data centers.


Three Criteria for Qualifying as Agile

leave a comment »

Agile methods have been gaining popularity to the extent that one sees the term Agile used beyond the domain of software methods. Agile Infrastructure and Agile Business Service Management were used in this blog and elsewhere. Recently I have seen the term used in the domain of Business Process Management (BPM). For example, a presentations entitled Best Practices for Agile BPM will be delivered in the forthcoming Gartner Group Business Process Management Summit 2010.

I have no doubt the term Agile will be adopted in various fields. Using BPM as an example, I propose the following three criteria to differentiate between agile (small A) and Agile (capital A):

  1. Beyond software: A software team carrying out a BPM initiative might use Agile methods. This fact to itself does not suffice to make the initiative Agile BPM.
  2. Methodical specificity: Roles, forums/ceremonies and artifacts for the BPM initiative must be specified. Folks might be already applying Lean, TOC or other approaches to BPM, but a definitive Agile BPM method has not crystalized yet.
  3. Values: Adherence in spirit to the four principles of the Agile Manifesto. Replace the word “software” with “product” in the manifesto (just two occurences!) and you get a universal value statement that is not restricted to “just” software. It applies to BPM as well as to any other field in which products are produced and used.

You might be impressively agile in what you do but it does not necessarily make you Agile. The pace by which you do things must be anchored in a broader perspective that incorporates customers and employees. A forthcoming post entitled Indivisibility of the Principles of Operation will explore the connection between the Agile values (plural) you hold and the business value (singular) you generate.

Connecting the Dots: Operational Excellence, Strategic Freedom and the Pursuit of Passion

leave a comment »

My recent post The Headlong Pursuit of Growth, and Its Aftermath applied insights from Toyota Motor Corporation to Agile methods. Among various lessons to be learned, the post highlighted the relationship between mechanism and policy: 

Just like the Toyota Production System, your software method is a “vehicle” which is subject to policy decisions from above. It cannot, however, compensate for policy failures.

In other words, operational excellence in Agile methods is not a substitute for strategy/policy. It does not confer strategic freedom.

In another recent post – I Found My Voice; I did not Find My Tribe – the vicious cycle that leads to loss of passionate Agile talent was described as follows:

This “1.5” phenomenon is at the root of a vicious cycle that dilutes companies, particularly these days:

  1. A round of layoffs is implemented.
  2. Just about everyone takes notice and tries to exhibit the “proper behavior/values.”
  3. Folks in the “private tribe” don’t dare come out of the closet.
  4. The passionate person who found his/her voice in Agile is like a fish out of the water. Sooner or later he/she looks for a tribe elsewhere.
  5. The company becomes more diluted on folks who are willing to try new things and have the drive to make them happen.
  6. The products and the supporting processes continue to be mediocre.
  7. Goto step 1.

Reading the article Getting Toyota Out of Reverse, published in the December 18 issue of BusinessWeek, I found a fascinating linkage between the two posts:

“They say that young people are moving away from cars,” Toyoda said. “But surely it is us—the automakers—who have abandoned our passion for cars.”

One had better take notice when the president of Toyota speaks of the effects of loss of passion using phrases like “irrelevance or death” and “grasping for salvation”.

You need go no further than John Hagel‘s recent post Pursuing Passion for a resounding second opinion on the subject.

The Headlong Pursuit of Growth, and Its Aftermath

with 4 comments

The December 12-18, 2009 issues of The Economist features a couple of fascinating articles on Toyota Motor Corporation. According to The Economist, Toyota’s President  reached the following dire conclusion on the situation Toyota is facing:

Mr Toyoda’s alarm call last month appears partly to have been prompted by reading “How the Mighty Fall”, a book by Jim Collins, an American management writer, which identifies five stages of corporate decline. Mr Toyoda reckons that Toyota may already be at the fourth stage. Companies at this point, says Mr Collins, frequently still have their destinies in their own hands, but often flit from one supposed “silver bullet” strategy to another, thus accelerating towards the fate they are trying to avoid.

In the litany of things that went wrong, an interesting point is made about the Toyota Production System:

… the recalls continued and Toyota started slipping in consumer-quality surveys. A year later Consumer Reports, an influential magazine, dropped three Toyota models from its recommended list. The magazine added that it would “no longer recommend any new or redesigned Toyota-built models without reliability data on a specific design”. People within the company believe these quality problems were caused by the strain put on the fabled Toyota Production System by the headlong pursuit of growth.

Whatever Agile method you practice – Scrum, Lean, Kanban, Crystal, etc. – you need to be cognizant of three touch points with the Toyota experience reported above:

  • Just like the Toyota Production System, your software method is a “vehicle” which is subject to policy decisions from above. It cannot, however, compensate for policy failures.
  • If your company relentlessly pursues growth, the quality/technical debt liability it is likely to incur coud easily outweigh the benefits of growth. Consider the upside potential of growth vis-a-vis the downside of the resultant technical debt. When appropriate, monetize technical debt using the technique described in Technical Debt on Your Balance Sheet.
  • In addition to monetizing the technical debt, evaluate the various kinds of risks indicated in The View From The Executive Suite. A sense of how devastating those might be is given by Toyota’s own experience:

Just as Cadillac used to be synonymous with luxury and BMW with sportiness, Toyota was a byword for quality and reliability… The danger in all of this for Toyota is that its loyal (and mostly satisfied) customers in America have long believed that the firm was different from others and thus hold it to a higher standard. The moment that Toyota is seen as just another big carmaker, a vital part of the mystique that has surrounded the brand will have been rubbed away.

Please remember – unless you work for Toyota Motor Corporation, chances are your company would not be able to take the kind of risk Toyota can.

The Success of the Success Tour

leave a comment »

We started the 2009 Rally Agile Success Tour (AST) Series in March in Denver, CO; we just concluded it in London, UK. In between the AST “train” stopped at:

All in all we hosted about 1,000 participants in these cities. More than 40 panelists shared Agile experiences with their local colleagues. Some 200 meetings were held with various participants in conjunction with the events. Obviously, I cannot write here about the level of business generated by the success tour, but none of my Rally colleagues complained so far…

The breadth and depth of topics that were covered is mind-boggling. Here are a few of the most intriguing ones:

The success tour proved successful to a degree that actually perplexed me for quite some time. I had certainly expected a strong series of events from the outset and could point out to various things we were doing right along the way. Yet, the very simple ‘secret sauce’ that made the event series so remarkable eluded me until I collected my thoughts for writing this post:

The Agile Success Tour proved phenomenally successful because the Rally team is so much like the customers and prospects that participate in the events, license the Rally software and work hand-in-hand with Rally coaches.

A few words of explanation for what might seem on the surface like a somewhat banal statement. The various members of the Rally team – sales reps, coaches, technical account managers, marketing professionals and execs – resonated with participants in the events due to exceptionally high level of congruence in values, thinking and practices. If Ryan were the CTO of eBay he would probably have licensed Rally software; Jean would have re-architected the flow of eBay processes; Zach would have integrated the ALM tools eBay uses. As for Lauren, she would have single-handedly created a world-wide marketing events organization for eBay.

The power of being like your own customers is magnetic. Digital Equipment Corporation was immensely successful selling minicomputers to engineers like their own engineers in the 60’s and 70’s. Sun Microsystems rode the early Internet wave as their product designers were carbon copy of the folks who roamed the World Wide Web. Apple triumphed with the iPod because just about every Apple employee would have murdered for such a cool device. Nothing beats the intuitive understanding that comes with designing, marketing and selling the kind of product you will buy, play with and use yourself.

After the Santa Clara event, Forrester’s Tom Grant told me the following about Rally:

What a smart company – everyone gets it!

Though a slightly different perspective than mine, Tom had actually identified the outcome of the company-customer congruence I am highlighting in this post. Everyone at Rally gets it due to natural identification with his/her customers. Contexts and experiences of customers are exceptionally well understood and often replicated in Rally’s Boulder, CO headquarters and its various branch offices.

Fundamentally, the success of the success tour reflects the affinity between Rally and its clientele.

Agile Business Service Management

with 7 comments

Over the weekend we activated the BSM Review. It is a thought leadership website dedicated to next practices in Business Service Management (BSM) in a way that is appropriate for our era. To quote my colleague and friend Bill Keyworth:

This website is dedicated to the BSM dialogue by whoever wishes to participate.  There is no fee to join …no content that requires a subscription …and no censorship of reasonable ideas and questions.

My area of focus in this site is Agile Business Service Management. The term is defined as follows:

Agile Business Service Management (Agile BSM) is the fusion of modern software development methods with the prevailing preference to run IT from the perspective of the business customer. Instead of dividing the “world” to development on the one hand and operations on the other hand, Agile Business Service Management unifies the two to manage them as part of one continuum that improves the delivery and usage of the application to the targeted business end-user. By so doing, it crosses the metaphorical chasm between the R&D lab and the customer door (or laptop, or iPhone, or…)

My research agenda in the context of the BSM Review will be outlined in a forthcoming post. For now, suffice it to say it will primarily be driven by two themes:

  • Business alignment: At the heart of it, BSM is a discipline to better align business with IT; at its core, Agile is about “customer collaboration over contract negotiation.” The two are conceptually similar: they express the strong desire in both development and operations to carry out meaningful tasks that have business impact.
  • Continuous manufacturing: I view IT as a form of continuous manufacturing. If you accept this premise, the application of Agile concepts, principles and techniques to IT management makes perfect sense. Just as Agile has been influenced by Lean techniques from manufacturing, it has the potential now to influences (continuous) manufacturing in its IT incarnation.

If software development is your primary interest, you might find my forthcoming posts in BSM Review go a little beyond the traditional scope of software methods. If, however, you are interested in software delivery in entirety, you are likely to find good synergy between the topics I will address in BSM Review and those I will continue to bring up in The Agile Executive. Either way, I trust my posts and Cote’s will be of on-going interest to you.