Posts Tagged ‘Agile Manifesto’
Balancing Agile – Guest Post by Alan Shalloway
A fascinating difference exists between Agile and Business Service Management (BSM). Agile emphasizes continuous flow of value to the customer. In contrast, BSM focuses on the business – it aligns the deliverables of IT to the enterprise’s business goals. Subtle that the difference might be, the two methods evolved along quite different lines in spite of the common denominator – dealing with software.
In this guest post, Alan Shalloway – Founder and CEO of NetObjectives – discusses the implications of focusing on the business as distinct from focusing on the customer. His discussion is part of a few thought-provoking threads he weaves around the Agile Manifesto. Alan perceives the Manifesto a product of the times. He thinks aloud whether today’s circumstances require a revised manifesto.
Alan is a man of passion. While I do not always agree with him, I have a lot of respect for his quest to find the deeper truths. Furthermore, I always learn from him. Whether you agree or disagree with the opinion Alan articulates in this post, “listening” to his thoughts is well worth your time.
Here is Alan:
The Agile Manifesto was a watershed event that has forever changed the landscape of software development. So profound a positive impact of it has had, that few challenge whether it was actually correct. Manifestos are often a statement in reaction to something prevalent that needs changing. This makes them very topical and temporal – and the exact intention needs to be restated when the landscape against which it was drawn has changed. The Agile Manifesto[1] states:
“Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a planThat is, while there is value in the items on
the right, we value the items on the left more.”At the time of its edict, this was profound and well stated. Too many software teams were:
- Following a process dictated to them from outside their team
- Managing according to an extensive set of artifacts that recorded where they supposedly were well before software existed
- Were given a set of requirements to meet with little opportunity to discuss real needs (note, points 3 and 4 in the manifesto address this point in two ways – first recognizing that customer collaboration is necessary to discover their true needs and second that it is essential to take advantage of newly discovered information)
The manifesto represented a new paradigm from which to work – one in which the team would have better control over its destiny and where it was recognized that one had to make incremental, iterative movement towards one’s goals – both in discovering the true goal and in implementing it.
Unfortunately, the perspective from which the manifesto was created, or at least the methods which first followed the manifesto, have been extremely team centric. Not a surprise, given the paradigm at the time gave development teams too little say in their own methods. The impact of this has been, not surprisingly, success at the teams and difficult beyond the team. It is almost axiomatic now that companies will have successful team pilots only to bog down in their enterprise agile adoption efforts or even revert back to earlier methods.
I say “not surprisingly” because several things have been left out of the agile manifesto. These are:
- The role of management
- The role of process
- The role of planning
- And indirectly, the role of guiding principles
It could also be said that the driver for agile development is misplaced. I do not believe “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” This makes software groups customer driven, not business driven.
There is a subtle, but important difference. Basically, conscious or not, the Agile Manifesto is driven with a team-centric view of satisfying customers – business and management play mostly secondary roles. Unfortunately, there is a significant difference between customer driven and business driven (see the figure entitled Alignment with Vision of Business to the right). This is not apparent at the team level (the team is supposed to satisfy the customer). But definitely at the business level. Not surprising, however, since the manifesto is a team perspective it states things in terms of customer value.
Ironically, it is the over-focus on the team, however, that is robbing teams of one of their greatest tools. Clearly any method must have respecting people and providing those doing the work the ability to choose how they do their work. But this does not mean that process isn’t essential or that attending to certain laws of software development is optional. Rather it means that process can’t be imposed on teams since to do so would both rob them of respect and almost certainly be the wrong way to do things – who knows more about how to get work done than the people doing the work themselves?
But “Individuals over process” as the first line has come to be called, makes it sound like people are it. I do not think so – and I think this mindset has caused a lot of damage in many ways.
There is great evidence that the best approach is not merely get smart people onto a team and have them figure out how to solve their problems. They must be properly equipped to do so. Just being smart doesn’t mean you can solve the challenges facing you. This should be readily apparent, but in many ways, the Agile community has mostly ignored it. Actually, there is not really an Agile community any more – there are factions that have significantly different beliefs. For example, XP has long recognized the need for technical practices in Agile while the Scrum community is only just starting to get into what these are.
However, except for the Lean/Kanban community, few Agilists seem to espouse discovering and following the laws of product development flow (or even recognize their existence). This, in my mind, has led to the low rate of success in scaling Scrum to the enterprise*[2]. Ironically, it is the over-focus on people that leads many in the Scrum community to assert this lack of success is a lack willingness to take the effort to improve. This is not surprising – if it’s up to the team to succeed, then when they don’t it must be something wrong with the team or their management when they fail. I think not. I think it is the lack of understanding of the principles of software development flow.
These laws are not new. Don Reinertsen, in his iconic book Managing the Design Factory lays out much of the rules of product development. His more recent book The Principles of Product Development Flow: Second Generation Lean Product Development he lays out 175 of these principles.
To me, true respect for people means that one must equip them with what they need to get their job done. Our thinking in the Agile community should change from “People over process” to one of “People times process.[3]” This phrase emphasizes that if either are low, you get a low productivity. Process does not ensure success. But a poor process requires heroes to succeed. We’d like good, motivated, well-intentioned people to be able to succeed.
Our new agile perspective needs to include an understanding of what teams need to know to do their work. This opens up a role for managers to actively help teams get their job done and to coach them when they have challenges or lose their way. While I will always be thankful for the Agile Manifesto, I am looking for a Business Agile Manifesto that will expand the focus from the team to the entire enterprise.
Footnotes:
[1] http://www.agilemanifesto.org
[2] Ken Schwaber, iconic leader of the Scrum community has said that “I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it.”
[3] I first heard this from Don Reinertsen. Before that I used to say “People plus process.”
Three Criteria for Qualifying as Agile
Agile methods have been gaining popularity to the extent that one sees the term Agile used beyond the domain of software methods. Agile Infrastructure and Agile Business Service Management were used in this blog and elsewhere. Recently I have seen the term used in the domain of Business Process Management (BPM). For example, a presentations entitled Best Practices for Agile BPM will be delivered in the forthcoming Gartner Group Business Process Management Summit 2010.
I have no doubt the term Agile will be adopted in various fields. Using BPM as an example, I propose the following three criteria to differentiate between agile (small A) and Agile (capital A):
- Beyond software: A software team carrying out a BPM initiative might use Agile methods. This fact to itself does not suffice to make the initiative Agile BPM.
- Methodical specificity: Roles, forums/ceremonies and artifacts for the BPM initiative must be specified. Folks might be already applying Lean, TOC or other approaches to BPM, but a definitive Agile BPM method has not crystalized yet.
- Values: Adherence in spirit to the four principles of the Agile Manifesto. Replace the word “software” with “product” in the manifesto (just two occurences!) and you get a universal value statement that is not restricted to “just” software. It applies to BPM as well as to any other field in which products are produced and used.
You might be impressively agile in what you do but it does not necessarily make you Agile. The pace by which you do things must be anchored in a broader perspective that incorporates customers and employees. A forthcoming post entitled Indivisibility of the Principles of Operation will explore the connection between the Agile values (plural) you hold and the business value (singular) you generate.
The Friction of Agile
Everything is very simple in War, but the simplest thing is difficult. These difficulties accumulate and produce a friction which no man can imagine exactly who has not seen War… This enormous friction, which is not concentrated, as in mechanics, at a few points, is therefore everywhere brought into contact with chance, and thus incidents take place upon which it was impossible to calculate…
IMHO, this excerpt from On War applies exceptionally well to Agile roll-outs these days:
- Simplicity: The four principles of the Agile Manifesto are intuitively compelling. You could (and probably should) use them as the core definition of what Agile is all about. Likewise, you do not need more than a single hand-drawn matrix to illustrate how WIP limits in Kanban work. In contrast to various other terms used in development and IT – e.g. SOA – the conceptual power of Agile methods is easy to grasp.
- Friction: Assume you were building a company from scratch without any pre-conceived notions of the organizational constructs you would put in place. Assume as well that you were developing your organization with end-to-end Agile effectiveness in mind. You would probably devise a holistically integrated organization. For example, you might opt for an organizational design in which each level of the organization will include all functions relevant to Agile – R&D, IT, Marketing, Support, Sales etc. In other words, ideally you will not go for the traditional organizational design: a vertical R&D stove pipe, a vertical Marketing stove pipe, a vertical Sales stove pipe, etc. As in reality you are unlikely to get the charter to start with a clean sheet of paper, the friction arises in each and every point in which your end-to-end organizational design for Agile deviates from the traditional organizational constructs your company uses.
- Not concentrated: As Clausewitz points out, the friction of war is not mechanical friction – you can’t address it by pouring in a ‘organizational lubricant’ in just a few places. Flooding the whole organization with the lubricant is likely to create a morass in which all agility will be lost.
I recommend four principles to minimize the organizational friction of Agile, as follows:
- Acknowledge you accrued organizational debt: It is conceptually quite similar to accruing technical debt – various organizational decisions and compromises taken along the way were rushed to the extent that they leave much to be desired. The organizational constructs and practices that sprang out of these decisions need to be refactored.
- Carry out the organizational refactoring work from the outside to the inside: A truly holistic Agile design will incorporate customers and partners. Start with the way you will integrate them, thence apply this very same way to the integration of the organizational entities within your company.
- Build on the strengths of your core corporate culture: As pointed out by Drucker:
… culture is singularly persistent… changing [organizational] behavior works only if it can be based on the existing ‘culture’… [Drucker, 1991]
- Use modern corporate for the fundamental organizing principles. See Relational Networks, Strategic Advantage: New Challenges for Collaborative Control by Hagel, Brown and Jelinek for a good recent article on the subject.
Since the end of the Cold War, a fair amount of debate has taken place around the applicability of the friction of war principles to armed conflicts in the information age. The conclusion is of interest to both military personnel, Agile practitioners and IT professionals:
… while technological advances might temporarily mitigate general friction, they could neither eliminate it nor substantially reduce its potential magnitude.
Definition: Agile Development
The difficulty to concisely define the term Agile Development stems from the very nature of the Agile Manifesto:
- The manifesto is a statement of values. By the very nature of values, people share them in a loose manner. Both definition and adherence (“But do they really practice Agile development?”) are qualitative and open to interpretation.
- The manifesto values are relative. The manifesto is quite explicit in stating “… while there is value in the items on the right, we value the items on the left more:”
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiations
Responding to change over following a plan
Agile development is often described in terms of the software method in use. For example, in his foreword to Agile Software Development with Scrum, Bob Martin summarizes Agile methods as “… people oriented software processes that work without getting in the way,” Martin Fowler emphasizes another aspect of Agile methods in his own foreword to the very same book:
… a new breed of software processes which are based on an empirical approach to controlling a project.
A more detailed definition is given by authors Rico, Sayani and Sone in their October 2009 book The Business Value of Agile Software Methods: Maximizing ROI with Just-in-time Processes and Documentation:
Agile methods are contemporary approaches for creating new software based on customer collaboration, teamwork, iterative development, and response to change. Combining communication and interpersonal trust with a flexible management and development framework, they contain just enough process to capture customer needs in the form of user stories and to rapidly create working software. However, the key to Agile methods are rich, high-context communications with customers along with cohesive teamwork.
On the other hand, such an authority (and signatory to the Manifesto) as Jim Highsmith does not seem to define the term Agile Development per se in the second edition of Agile Project Management: Creating Innovative Products. Instead, Jim defines Agility through two statements:
Agility is the ability to both create and respond to change in order to profit in a turbulent business environment.
Agility is the ability to balance flexibility and stability.
Likewise, in Agile Software Development, Alistair Cockburn focuses on discussing what is core to Agile, emphasizing the properties of Agility through the following citation:
Agility is dynamic, context specific, aggressively change-embracing and growth-oriented. It is not about improving efficiency, cutting costs or battening down the business hatches to ride out fearsome competitive ‘storms.’ It is about succeeding and about winning: about succeeding in emerging competitive arenas, and about winning profits, market share and customers in the very center of the competitive storms many companies now fear.
Rather than trying to reconcile all these worthy definitions, I would suggest five context-dependent approaches to the definition, as follows:
- For the reader who tries to understand what Agile is all about: It is the mindset that really matters. Read the Agile Manifesto and the corresponding History.
- For the reader who is anxious to put his/her hands around an Agile implementation: Pick a specific Agile method – any method – and study it with special emphasis on the roles, process and artifacts of the method. It could be Crystal, Extreme Programming (XP), Scrum, Dynamic Systems Development Method (DSDM), Kanban or any other method that shows promise as a good fit for your specific environment. Consult 10 Steps for Starting an Agile Start-up for a down-to-earth blueprint for implementation.
- For the reader who tries to assess whether a project team is really Agile: It is a maturity curve issue that manifests itself in quite a few disciplines. For example, see the various kinds of maturity models surveyed in the BSM Review blog. You will probably need to determine the maturity model that suits your environment and apply it to the method you are practicing.
- For the reader who needs to explore Agile in a business context: You need not worry about the technical aspects of Agile. See the category Benefits of Agile in this blog.
- For the reader interested in applying Agile beyond development: Extending Agile changes its definition. See the various posts on the subject by Eric Ries in Lessons Learned.
Please remember: when it comes to defining Agile Development, you have a problem of choosing, not of choice. It is the use to which you put the definition that determines the choosing.
The Changing Nature of Innovation: Part I — New Forms of Experimentation
Colleague Christian Sarkar drew my attention to two recent Harvard Business Review (HBR) articles that shed light on the way(s) innovation is being approached nowadays. To the best of my knowledge, none of the two articles has been written by an author who is associated with the Agile movement. Both, if you ask me, would have resonated big time with the authors of the Agile Manifesto.
The February 2009 HBR article How to Design Smart Business Experiments focuses on data-driven decisions as distinct from decisions taken based on “intuition”:
Every day, managers in your organization take steps to implement new ideas without having any real evidence to back them up. They fiddle with offerings, try out distribution approaches, and alter how work gets done, usually acting on little more than gut feel or seeming common sense—”I’ll bet this” or “I think that.” Even more disturbing, some wrap their decisions in the language of science, creating an illusion of evidence. Their so-called experiments aren’t worthy of the name, because they lack investigative rigor. It’s likely that the resulting guesses will be wrong and, worst of all, that very little will have been learned in the process.
It doesn’t have to be this way. Thanks to new, broadly available software and given some straightforward investments to build capabilities, managers can now base consequential decisions on scientifically valid experiments. Of course, the scientific method is not new, nor is its application in business. The R&D centers of firms ranging from biscuit bakers to drug makers have always relied on it, as have direct-mail marketers tracking response rates to different permutations of their pitches. To apply it outside such settings, however, has until recently been a major undertaking. Any foray into the randomized testing of management ideas—that is, the random assignment of subjects to test and control groups—meant employing or engaging a PhD in statistics or perhaps a “design of experiments” expert (sometimes seen in advanced TQM programs). Now, a quantitatively trained MBA can oversee the process, assisted by software that will help determine what kind of samples are necessary, which sites to use for testing and controls, and whether any changes resulting from experiments are statistically significant.
On the heels of this essay on how one could attain and utilize experimentally validated data, the October 2009 HBR article How GE is Disrupting Itself discusses what is already happening in the form of Reverse Innovation:
- The model that GE and other industrial manufacturers have followed for decades – developing high-end products at home and adapting them for other markets around the world – won’t suffice as growth slows in rich nations.
- To tap opportunities in emerging markets and pioneer value segments in wealthy countries, companies must learn reverse innovation: developing products in countries like China and India and then distributing them globally.
- While multinationals need both approaches, there are deep conflicts between the two. But those conflicts can be overcome.
- If GE doesn’t master reverse innovation, the emerging giants could destroy the company.
It does not really matter whether you are a “shoe string and prayer” start-up spending $500 on A/B testing through Web 2.0 technology or a Fortune 500 company investing $1B in the development and introduction of a new car in rural India in order to “pioneer value segments in wealthy countries.” Either way, your experimentation is affordable in the context of the end-result you have in mind.
Fast forward to Agile methods. The chunking of work to two-week segments makes experimentation affordable – you cancel an unsuccessful iteration as needed and move on to work on the next one. Furthermore, you can make the go/no-go decision with respect to an iteration based on statistically significant “real time” user response. This closed-loop operational nimbleness and affordability , in conjunction with a mindset that considers a “failure” of an iteration as a valuable lesson to learn from, facilitates experimentation. Innovation simply follows.
Software Moulding Methods
Christian Sarkar and I started an e-dialog on Agile Business Service Management in BSMReview. Both of us are keenly interested in exploring the broad application of Agile BSM in the context of Gartner’s Top Ten Technologies for 2010. To quote Christian:
Israel, where do agile practices fit into this? Just about everywhere as well?
The short answer to Christian’s good question is as follows:
I consider the principles articulated in the Manifesto For Agile Software Development http://agilemanifesto.org universal and timeless. They certainly apply just about everywhere. As a matter of fact, we are seeing the Manifesto principles applied more and more to the development of hardware and content.
The fascinating thing in what we are witnessing (see, for example: Scale in London – Part II, An Omen in Chicago, Depth in Seattle, and Richness and Vibrancy in Boston) is the evolution of the classical problem of managing multiple Software Development Life Cycles. Instead of dealing with one ‘material’ (software), we handle multiple ‘materials’ (software, hardware, content, business initiative, etc.) of dissimilar characteristics. The net effect is as follows:
The challenge then becomes the simultaneous and synchronized management of two or more ‘substances’ (e.g. software and content; software, content and business initiative; or, software, hardware, content and business initiative) of different characteristics under a unified process. It is conceptually fairly similar to the techniques used in engineering composite materials.
Ten years have passed since Evans and Wurster demonstrated the effects of separating the virtual from the physical. As software becomes pervasive, we are now starting to explore putting the virtual back together with the physical through a new generation of software moulding methods.
Agile Business Service Management
Over the weekend we activated the BSM Review. It is a thought leadership website dedicated to next practices in Business Service Management (BSM) in a way that is appropriate for our era. To quote my colleague and friend Bill Keyworth:
This website is dedicated to the BSM dialogue by whoever wishes to participate. There is no fee to join …no content that requires a subscription …and no censorship of reasonable ideas and questions.
My area of focus in this site is Agile Business Service Management. The term is defined as follows:
Agile Business Service Management (Agile BSM) is the fusion of modern software development methods with the prevailing preference to run IT from the perspective of the business customer. Instead of dividing the “world” to development on the one hand and operations on the other hand, Agile Business Service Management unifies the two to manage them as part of one continuum that improves the delivery and usage of the application to the targeted business end-user. By so doing, it crosses the metaphorical chasm between the R&D lab and the customer door (or laptop, or iPhone, or…)
My research agenda in the context of the BSM Review will be outlined in a forthcoming post. For now, suffice it to say it will primarily be driven by two themes:
- Business alignment: At the heart of it, BSM is a discipline to better align business with IT; at its core, Agile is about “customer collaboration over contract negotiation.” The two are conceptually similar: they express the strong desire in both development and operations to carry out meaningful tasks that have business impact.
- Continuous manufacturing: I view IT as a form of continuous manufacturing. If you accept this premise, the application of Agile concepts, principles and techniques to IT management makes perfect sense. Just as Agile has been influenced by Lean techniques from manufacturing, it has the potential now to influences (continuous) manufacturing in its IT incarnation.
If software development is your primary interest, you might find my forthcoming posts in BSM Review go a little beyond the traditional scope of software methods. If, however, you are interested in software delivery in entirety, you are likely to find good synergy between the topics I will address in BSM Review and those I will continue to bring up in The Agile Executive. Either way, I trust my posts and Cote’s will be of on-going interest to you.
Technical Debt on Your Balance Sheet
Colleague Jonathon Golden introduced me to a new plug in to Sonar. The plug in calculates the cost to fix the technical debt accrued in a product. For example, you might find an accrued technical debt amounting to $1M in a 500KLOC application. Obviously, you will need to spend $2 per each line of code to “pay back” your debt.
The expression of technical debt in monetary terms is intriguing. Unlike financial debt, there is no credit limit on technical debt. Hence, unless a team is proficient at refactoring on an ongoing basis, technical debt tends to grow over time as the underlying software decays. Beyond a certain level of debt, no good option is available. The code decayed to the point in which fixing anything in a hazardous proposition – every fix is likely to break something else. Under such circumstances, most/all of the development team gets sucked into maintaining the software instead of developing new features and functions.
Monetizing technical debt can have two far reaching implications, as follows:
- A credit limit on technical debt can be established. For example, when the technical debt reaches a certain level (say 25 cents per line of code), new functionality is put on hold. The team applies itself to aggressive refactoring to reduce the debt to an acceptable level.
- For companies who capitalize software, technical debt could become a line item on the balance sheet. It will simply be listed as a liability.
From a customer perspective, the monetized technical debt on the balance sheet of a software vendor is a proxy for the technical risk involved in licensing software from this vendor. Such monetization could be easily extended to report technical debt per product family. With such reporting in place, the technical risk associated with licensing a specific product can be assessed.
Software vendors might frown at the requirement to monetize technical debt. I would contend that such a reporting requirement is absolutely consistent with the spirit of the Agile Manifesto:
Customer collaboration over contract negotiations
In other words, if you are reluctant to list your monetized technical debt you can’t really claim you practice Agile.
The Agile Executive 