The Agile Executive

Making Agile Work

Posts Tagged ‘Kanban

“Big Agile”

leave a comment »

On January 30, 2012 12:00 pm EST, colleague and friend Hubert Smits and I will be doing a Cutter webinar entitled “Big Agile” is More than Just a Software Method.  We will follow on in February with a “Big Agile” issue of the Cutter IT Journal [CITJ] for which I am the guest editor. Coming April we are likely to discuss the topic some more in the Cutter Summit.

The heart of the webinar can be summarized in the following words:

Small is beautiful in software. While big software might not be beautiful, more often than not, it’s in the nature of what needs to be accomplished. This contrast between the beauty of small and the requirements of the big generates systemic tension in many software projects, organizations, and companies. Resolving this conflict is the focus of this webinar.

Discussing the webinar and the follow-on CITJ issue with various folks in the Agile, Lean and Kanban movement(s), I became painfully aware of the very many interpretation folks associate with the the term “Big Agile.” Hence, I would like to say a few words on what it means to me.  I am taking the very pragmatic view of a client on the subject. A VP of one department or another is chartered to implement an Agile roll-out at scale. The roll-out might not include all the teams. Nor might it include all functions within the company. The roll-out, however, affects a significant number of folks. Focusing the roll-out at the team level is not sufficient.

The typical VP that I run into under such circumstances is not an expert on software methods (and usually acknowledges it). He/she, however, is smart enough and experienced enough to understand that scale matters. He/she knows or feels intuitively that Big Agile is akin to Big Data. Data is data is data, but when it is Big Data you need to address various aspects that do not manifest themselves in dealing with ordinary amounts of data. Likewise for Agile IMHO.

We plan to make the webinar very interactive. Anne Mullaney will start with a few ‘warm up’ questions to enable us to ‘dig’ into the subject, thence turn it over to questions from the audience. We plan to take the discussion to wherever the participants might want to.

I hope you will join us whether you love, hate or indifferent to the term “Big Agile.” We are expecting a lively discussion…

Advertisements

Written by israelgat

January 22, 2012 at 10:51 pm

Beyond Devops

leave a comment »

Based on feedback from participants in my Agile 2010 workshop “How We Do Things Around Here In Order To Succeed,” I am planning to offer the workshop as a one-day seminar. A tentative agenda for the seminar is as follows:

  • Introduction to Cultural Framework
  • Exercise #1: Determine Your Culture
  • Exercise #2: Strengths and Weaknesses of Your Culture
  • Change Behavior, Not Culture
  • When Cultures Meet
  • Exercise #3: Conflicts in Devops
  • The Agile Flywheel
  • Exercise #4: Using Technical Debt as a Boundary Object
  • Bringing Individuals and Organizations Together
  • Exercise #5: It is About Sharing the Process, Not Just Sharing the Information
  • Exercise #6: From success in devops to end-to-end success

Until I publish a full-fledged outline for the seminar, here is the central theme:

Beyond Devops

Inter-departmental flow in a corporate setting is often envisioned as the inner workings of a swiss watch. Wheels turn other wheels in a precise manner. Not only is effectiveness maintained, it is maintained in an efficient manner.

Problem is, many individuals and most departments hold distorted views of the departments they interact with. Reasonable distortions can be mitigated as long as the operational balance between departments is maintained. Once the operational balance is broken the “swiss watch” stops to function as the inter-departmental distortions block any attempt to restore the balance.

The most effective way to get dev and ops on a path of collaboration is for the two departments to jointly construct a boundary object. As dev and ops are joined in the hip through the code, and even more so through its quality, technical debt is well suited to serve as the core of a boundary object around which the two department share meaning while retaining operational autonomy.

Similar  boundary objects can be constructed between dev and other departments – customer support, professional services, marketing, sales and finance. When conceived and implemented in a manner that links numerous boundary objects together, Agile success in dev can be extended to both upstream and downstream functions.

Written by israelgat

August 11, 2010 at 5:11 am

A Recipe for Handling Cultural Conflicts in Devops and Beyond

with 4 comments

My Agile 2010 workshop “How We Do Things Around Here In Order To Succeed”  will weave together four trends that I am witnessing in my practice:

  • The ascendance of Agile portfolio management in a world characterized by loosely coupled processes
  • Devops dynamics are becoming more and more characteristic of end-to-end Agile/Kanban patterns
  • Viral spread of technical debt metrics in software governance
  • Increasing use of boundary objects in the enterprise context

The workshop is structured around three case studies/exercises that will take about two-thirds of the allotted time (the morning of August 9). The other third provides the theory and tools to be used in the three workshop exercises and (hopefully) in many future engagements participants in the workshop will carry out. Deep technical knowledge is not required – the workshop targets any Agile practitioner who has conceptual grasp of culture, software development, IT operations and portfolio management.

The #1 takeaway from the presentation is the details you need to know about creation and capture of lasting value through end-to-end Agile initiatives.

Here is the workshop agenda (still subject to some minor tweaking):

  • Introduction to Cultural Framework
  • Exercise #1: Strengths and Weaknesses of Your Culture
  • Change Behavior, Not Culture
  • When Organizations Clash
  • Exercise #2: Conflicts in Devops
  • The Agile Flywheel
  • Exercise #3: Using Technical Debt as a Boundary Object in Devops
  • Bringing Organizations Together Through Enlightened Governance Loops

I look forward to meeting you in the workshop and learning from your experiences and insights!

Israel

Toward An Event Driven Scrum Process

with 10 comments

Recent posts in this blog and elsewhere recommended using technical debt (TD) criteria as an integral part of the build process. One could fail the build under conditions such as:

  • TD(current build) > TD(previous build)
  • TD per line of code > $1
  • TD as % of cost  > 50%
  • Many others…

If you follow this recommendation while integrating continuously, chances are some fixing might need to be done multiple times a day. This being the case, the relationship between the daily stand-up meeting and the build fail event needs to be examined. If the team anyway re-groups whenever the build fails, what is the value add of the daily stand-up meeting?

Figure 1 below, taken from Agile Software Development with Scrum, examines the feedback loop of Scrum from a process control perspective. It shows the daily stand-up meeting providing the process control function.

Figure 1: Process Control View of Scrum

Figure 2 below shows the feedback loop being driven by the build process instead of by the daily stand-up meeting. From a process control standpoint it is essentially identical to Figure 1. Control through the regularly scheduled daily stand-up meeting has simply been replaced by event-driven control. The team re-groups whenever the build fails.

Figure 2: Build Driven Process Control

Some groups, e.g. the Scrum/Kanban teams under Erik Huddleston and Stephen Chin at Inovis have been practicing Event Driven Scrum for some time now. A lot more data is needed in order to validate the premise that Event Driven Scrum is indeed an improvement over vanilla Scrum. Until the necessary data is collected and examined, two intuitive observations are worth mentioning:

  1. Event Driven Scrum preserves the nature of Scrum as an empirical process.
  2. A control unit that is triggered by events (build fail) is in the very spirit of adaptive Agile methods.

Balancing Agile – Guest Post by Alan Shalloway

with 3 comments

A fascinating difference exists between Agile and Business Service Management (BSM). Agile emphasizes continuous flow of value to the customer. In contrast, BSM focuses on the business – it aligns the deliverables of IT to the enterprise’s business goals. Subtle that the difference might be, the two methods evolved along quite different lines in spite of the common denominator – dealing with software.

In this guest post, Alan Shalloway – Founder and CEO of NetObjectives – discusses the implications of focusing on the business as distinct from focusing on the customer. His discussion is part of  a few thought-provoking threads he weaves around the Agile Manifesto. Alan perceives the Manifesto a product of the times. He thinks aloud whether today’s circumstances require a revised manifesto.

Alan is a man of passion. While I do not always agree with him, I have a lot of respect for his quest to find the deeper truths. Furthermore, I always learn from him. Whether you agree or disagree with the opinion Alan articulates in this post, “listening” to his thoughts is well worth your time.

Here is Alan:

The Agile Manifesto was a watershed event that has forever changed the landscape of software development.  So profound a positive impact of it has had, that few challenge whether it was actually correct.  Manifestos are often  a statement in reaction to something prevalent that needs changing.  This makes them very topical and temporal – and the exact intention needs to be restated when the landscape against which it was drawn has changed. The Agile Manifesto[1] states:

“Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.”

At the time of its edict, this was profound and well stated.  Too many software teams were:

  • Following a process dictated to them from outside their team
  • Managing according to an extensive set of artifacts that recorded where they supposedly were well before software existed
  • Were given a set of requirements to meet with little opportunity to discuss real needs (note, points 3 and 4 in the manifesto address this point in two ways – first recognizing that customer collaboration is necessary to discover their true needs and second that it is essential to take advantage of newly discovered information)

The manifesto represented a new paradigm from which to work – one in which the team would have better control over its destiny and where it was recognized that one had to make incremental, iterative movement towards one’s goals – both in discovering the true goal and in implementing it.

Unfortunately, the perspective from which the manifesto was created, or at least the methods which first followed the manifesto, have been extremely team centric.  Not a surprise, given the paradigm at the time gave development teams too little say in their own methods.  The impact of this has been, not surprisingly, success at the teams and difficult beyond the team. It is almost axiomatic now that companies will have successful team pilots only to bog down in their enterprise agile adoption efforts or even revert back to earlier methods.

I say “not surprisingly” because several things have been left out of the agile manifesto.  These are:

  • The role of management
  • The role of process
  • The role of planning
  • And indirectly, the role of guiding principles

It could also be said that the driver for agile development is misplaced.  I do not believe “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”  This makes software groups customer driven, not business driven.

There is a subtle, but important difference. Basically, conscious or not, the Agile Manifesto is driven with a team-centric view of satisfying customers – business and management play mostly secondary roles. Unfortunately, there is a significant difference between customer driven and business driven (see the figure entitled Alignment with Vision of Business to the right).  This is not apparent at the team level (the team is supposed to satisfy the customer).  But definitely at the business level. Not surprising, however, since the manifesto is a team perspective it states things in terms of customer value.

Ironically, it is the over-focus on the team, however, that is robbing teams of one of their greatest tools.  Clearly any method must have respecting people and providing those doing the work the ability to choose how they do their work. But this does not mean that process isn’t essential or that attending to certain laws of software development is optional.  Rather it means that process can’t be imposed on teams since to do so would both rob them of respect and almost certainly be the wrong way to do things – who knows more about how to get work done than the people doing the work themselves?

But “Individuals over process” as the first line has come to be called, makes it sound like people are it.  I do not think so – and I think this mindset has caused a lot of damage in many ways.

There is great evidence that the best approach is not merely get smart people onto a team and have them figure out how to solve their problems.  They must be properly equipped to do so.  Just being smart doesn’t mean you can solve the challenges facing you.  This should be readily apparent, but in many ways, the Agile community has mostly ignored it. Actually, there is not really an Agile community any more – there are factions that have significantly different beliefs.  For example, XP has long recognized the need for technical practices in Agile while the Scrum community is only just starting to get into what these are.

However, except for the Lean/Kanban community, few Agilists seem to espouse discovering and following the laws of product development flow (or even recognize their existence).  This, in my mind, has led to the low rate of success in scaling Scrum to the enterprise*[2]. Ironically, it is the over-focus on people that leads many in the Scrum community to assert this lack of success is a lack willingness to take the effort to improve. This is not surprising – if it’s up to the team to succeed, then when they don’t it must be something wrong with the team or their management when they fail.  I think not. I think it is the lack of understanding of the principles of software development flow.

These laws are not new.  Don Reinertsen, in his iconic book Managing the Design Factory lays out much of the rules of product development.  His more recent book The Principles of Product Development Flow: Second Generation Lean Product Development he lays out 175 of these principles.

To me, true respect for people means that one must equip them with what they need to get their job done.  Our thinking in the Agile community should change from “People over process” to one of “People times process.[3]This phrase emphasizes that if either are low, you get a low productivity.  Process does not ensure success. But a poor process requires heroes to succeed.  We’d like good, motivated, well-intentioned people to be able to succeed.

Our new agile perspective needs to include an understanding of what teams need to know to do their work. This opens up a role for managers to actively help teams get their job done and to coach them when they have challenges or lose their way.  While I will always be thankful for the Agile Manifesto, I am looking for a Business Agile Manifesto that will expand the focus from the team to the entire enterprise.

Footnotes:

[1] http://www.agilemanifesto.org

[2] Ken Schwaber, iconic leader of the Scrum community has said that “I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it.”

[3] I first heard this from Don Reinertsen.  Before that I used to say “People plus process.”

Measuring Agile Success Rate the Right Way

with 5 comments

Much has been said recently about the success/failure rate of Agile projects. In particular, a debate arose around the success rate of Scrum vis-a-vis Kanban.  For example, in a post entitled Some Day Kanban will fail 75% of the Time, colleague Jurgen Appelo states as follows:

Unfortunately, some people arguing against Scrum include these ScrumBut teams in their evaluations of the “high failure rate” of Scrum. They love quoting that “at least 75 percent of Scrum implementations fail.” And I think “Yes of course, 75% fails when that includes the teams that don’t understand what they’re doing.”

I would like to add one other “dimension” to the discussion: boundary conditions.

Any Agile initiative – Crystal, Scrum, Kanban, etc. – typically starts from a certain state of affairs of the code that has already been developed using a Waterfall method or no method at all. Even brand new projects produce code that invariably interacts with other software components that are already deployed, warts and everything. Pristine environments with no technical debt for the Agile initiative to deal with are rare.

Like it or not, the Agile initiative is saddled from the outset with a certain amount of technical debt. Code has been duplicated, rules violated, complexity ran amuck, etc. A typical enterprise software team starts with hundreds of thousands $$ in technical debt, if not millions. This debt needs to be “paid back.” Probably not over night, but certainly over a period of time. As illustrated by the following figure from Jim Highsmith, things get ugly if the debt is not paid back over an extended period of time.

in-can-you-afford-the-software-you-are-developing

The evaluation of success or failure of the Agile initiative needs to take technical debt into account. A team of 50 with an accrued technical debt of $100,000 has a much easier job on its hands transitioning to Agile than a similar size team starting with $1M in technical debt on its hands.

Whatever criteria you use to determine whether an Agile initiative has been successful, I would suggest the following boundary condition needs to be satisfied:

Technical debt at the end of the project/initiative must be significantly lower than technical debt at the start of the project.

Use the techniques outlined in Using Credit Limits to Constrain Development on Margin to calculate technical debt before and after. In addition to qualifying your Agile success, quantifying technical debt will do a lot towards improving the quality of your software.

Definition: Agile Methodology

leave a comment »

Agile Methodology is actually a bit of a controversial termVarious authors consider Agile a method, as distinct from a methodology. Others, prefer methodology over method. For example, using the Merriam-Webster dictionaries, Alistair Cockburn makes the following distinction between methodology and method in Agile Software Development: The Cooperative Game :

  • Methodology: A series of related methods or techniques
  • Method: Systematic procedure

Alistair views Agile as a methodology in the sense defined above. For example, he discusses Crystal as a family of methodologies. The reader is referred to Alistair’s book for a an excellent analysis of the various aspects of methodologies. As a matter of fact, Alistair tracks down the confusion between method and methodology to certain inconsistencies between various versions of the Oxford English Dictionary.

On the other hand, best I can tell from various conversations with him, Jim Highsmith seems to prefer the term Agile Method. This preference is reflected in Agile Project Management: Creating Innovative Products. It is possible that Jim’s preference is due to writing his book from a project management perspective.

Rather than getting in-depth to the method versus methodology controversy, I would simply cite two definitions I find useful in capturing the essence of Agile methodology, or method if you prefer.

An interesting metaphor for Agile has been used by Jim Highsmith in a 2009 Cutter Advisory:

Visualize a house structure with a roof, a foundation, and three pillars… The roof is business goals — the rationale for implementing agile methods and scaling to larger agile projects. The foundation is agile values or principles — principles that need careful interpretation as to how to apply them to larger teams. And finally, the three pillars: organization, product backlog, and process/practice.

The simplicity of the metaphor makes it quite effective in communicating what Agile is in a concise way without losing the richness of the various elements in Agile.

Using Scrum as an example, colleague David Spann gives the following down-to-earth summary of the key structural components of Agile in a 2008 Cutter Executive Report:

Scrum, as a management methodology, is elegant in its design, requiring only three roles (i.e., product owner, ScrumMaster, and self-organized team), three ceremonies (sprint/iteration planning, daily Scrum/debrief, and sprint review meetings), and three artifacts (product and sprint backlogs and the burndown chart) — just-enough practical advice so agile teams do not overcomplicate the development lifecycle with too much ceremony and documentation.

Needless to say, the structural elements will change from one Agile methodology to another. However, examining an Agile methodology through the {roles, ceremonies, artifacts} “lens” is an excellent way to summarize an Agile methodology. Furthermore, it enables easy comparison between the ‘usual suspects’ of Agile – Crystal Methods, Dynamic Systems Development, Extreme Programming, Feature Driven Development, and Kanban. The reader is referred to The Business Value of Agile Software Methods: Maximizing ROI with Just-in-Time Processes and Documentation for detailed comparisons between the various methods/methodologies.