Archive for the ‘Business Service Management’ Category
The Devops Triangle
The Agile Triangles was introduced by Jim Highsmith as an antidote to the Iron Triangle. Instead of balancing development between cost, schedule and scope, the Agile Triangle strives to strike a balance between value, quality and constraints:
Figure 1 – The Agile Triangle (based on Figure 1-3 in Agile Project Management: Creating Innovative Products.)
Consider the Iron Triangle in the context of devops. Value, quality and constraints apply to IT operations as meaningfully as they apply to software development. IT can go beyond cost, schedule and scope to focus on value and quality just as the Agile software development team does. Between development and operations the specific tasks to be carried out change, but the principles embodies in the triangle remain invariant.
In addition to cost, schedule and scope, devops projects must cope with another constraint: compliance. For example, a bank that implements a ‘follow the sun’ strategy with respect to trading must finish reconciling transaction that took place in London before the start of trade in Wall Street. From the bank’s point of view, its IT department needs to be mindful of four constraints: compliance, cost, schedule and scope. This view is represented in Figure 2 below.
Figure 2 – The Devops Triangle
Balancing the four constraints – compliance, cost, schedule, and scope – is not a trivial task. However, just like the Agile Triangle, the Tradeoff Matrix used in Agile software development applies to IT. In its software development variant, the Tradeoff matrix is an effective tool to decide between conflicting constraints, as follows:
Table 1 – Tradeoff Matrix (based on Table 6-1 in Agile Project Management: Creating Innovative Products.)
For devops, the matrix is extended to include a compliance row and a Reluctantly Accept column as follows:
Table 2 – Tradeoff Matrix for Devops
The Devops Triangle and the corresponding Tradeoff Matrix demonstrate how governance a la Agile can be extended to devops projects as far as compliance goes. The proposed governance framework however is incomplete in the following sense: schedule in devops projects can be a much more granular and stringent constraint than schedule in “dev only” projects. The subject of schedule constraints in devops projects will be addressed in a forthcoming post.
Devops: It is Not About ITIL, It is About Proficiency
As you would expect, the Information Technology Infrastructure Library (ITIL) topic was brought up in the devops day held last Friday in a LinkedIn facility in Mountain View, CA. We, of course, had the expected spectrum of opinions about ITIL in the context of devops – from “ITIL will never work for a true continuous development shop” to “well, you can make ITIL work under such circumstances.” Needless to say, a noticeable level of passion accompanied these two statements…
IMHO the heart of the issue is not ITIL per se but system management proficiency. If your system management proficiency is high, you are likely to be able to effectively respond to 10, 20 or 50 deploys per day. Conversely, if your system management proficiency is low, ops is not likely to be able to cope with high velocity in dev. The critical piece is alignment of velocities between dev and ops, not the method used to manage IT systems and services. Whether you use ITIL, COBIT or your own home-grown set of best practices is irrelevant. Achieving alignment of velocities between dev and ops is a matter of proficiency in system management.
How to Initiate a DevOps Project
Source: 17th/21st Lancers c. 1922-1929 “THE FIGHTING SPIRIT!”
Agile consultants on a development project often start by helping the team construct a backlog. The task is sufficiently concrete to get all stakeholders (product management, project management, development, test, any others) on a collaborative track through the creation of a key artifact. The backlog establishes a base line for the tasks to be carried out in the project.
For a DevOps project, start by establishing the technical debt of the software to be released to operations. By so doing you build the foundations for collaboration between development and operations through shared data. In the DevOps context, the technical debt data form the basis for the creation and grooming of a unified backlog which includes various user stories from operations.
Apply the same approach when you are fortunate to be able to include folks from operations in the Agile team from the very beginning. You start with zero technical debt, but you track it on an ongoing basis and include the corresponding “fix-it” stories in the backlog as you accrue the debt. Running technical debt analytics on the source code every two weeks is a good practice to follow.
As the head of development, you might not be comfortable sharing technical debt data. This being the case, you are not ready for DevOps.
The Agile Flywheel
Readers of The Agile Executive have been exposed to the “All In!” strategy used by Erik Huddleston to transform the software engineering process at Inovis and make it uniquely streamlined. In this post we follow up on the original discussion of the subject to explore the effect of Agile on IT Operations. As the title implies, Agile at Inovis served as a flywheel which created the momentum required to transform IT Operations and blend the best of Agile with the best of ITIL.
This guest post was written by Ray Riescher – a Six Sigma Black Belt, Agile evangelist and a business process change agent. Ray is currently responsible for business process management and IT governance at Inovis, a leading provider of business-to-business (B2B) e-commerce services, in Alpharetta, GA
Here is Ray:
When we converted to an Agile Scrum software methodology some 24 months ago, I never imagined the lessons I’d learn and the organizational change that would be driven by the adoption of Scrum.
I’ve lived by the philosophy that managing a business is managing its processes and that all of those processes, especially the operational processes, are interconnected. However, I don’t think I was fully prepared for effect Agile Scrum would have on our company operations.
We dove head first into Agile Scrum and adapted to it very quickly. However, it wasn’t until we landed a very large and demanding customer that Scrum was really put to the test. New enhancements, new features, and new configurations were all needed ASAP. Scrum delivered with rapid development and deployment in the form of releases that were moving into production with amazing velocity. Our release cadence hit warp drive and at one point we experienced several months where multiple teams’ production releases were deploying at the end of every two week sprint.
We’ve subscribed to the ITIL service support processes for Release, Change, Incident, Problem and Configuration Management. ITIL has served us well, giving us a common language and a clear understanding of process boundaries.
As the Scrum release cadence kicked in, the downstream ITIL processes had to keep up, adapt, and support the dynamics of rapid production changes. What happened was enlightening and maybe a bit ground breaking.
The Release Management process had to reassess its reliance on artifacts for gate keeping. The levels of sign offs had to be streamlined, the heavyweight deployment documentation had to be lightened, yet the process still had to control the production release to ensure deployment success. The rapidity of the release cycles meant that maintenance window downtime would be experienced too frequently by customers, so “rolling bounce” deployment strategies were devised and implemented.
Change requests could no longer wait for a weekly Change Management review board to approve and schedule the changes. Change management risk models had to be relied on for accurate detection of risky changes.
Early on in this dynamic environment, we weren’t quite as good as we needed to be and our Incident Management process was put to the test. Faster releases meant more opportunity for problems with service degradation and outages. This reality manifested itself more frequently than we’d ever experienced. Monitoring, detecting and repairing became paramount for environment stability and customer satisfaction.
What we found out was that we became very agile at this break/fix game. We developed a small team approach to managing incidents and leveraged the ITIL Problem Management process to rapidly perform root cause analysis. Once the true root cause was determined, a fix would be defined and deployed. Sometimes the fix was software related and went through the Scrum process, sometimes the fix was hardware related and went through the Configuration Management process, other times it was more operational and the fix took the form of training or corrections to procedural documentation.
The point is we’ve become agile across the entire IT spectrum. Whether it’s development via Scrum, the velocity with which we now operate our ITIL processes, or the integrated break/fix operational support processes, we are performing all of these with an agile mindset and discipline. We have small teams, working on priorities, and completing what needs to be completed now.
Scrum set the flywheel in motion and caused the rest of the IT process life cycle to respond. ITIL’s processes still form the solid core of service support and we’ve improved the processes’ capability to handle intense work velocity. The organization adapted by developing unprecedented speed in the ability to deliver production fixes and to solve root cause problems with agility.
What I think we are witnessing is a manifestation of Agile Business Service Management; a holistic agile methodology running across the IT process spectrum that’s delivering eye popping change and tremendous results.
OpsCamp Through an Internet-scale Lens
Like Agile Roots in Salt Lake City in June 2009, OpsCamp in Austin last week demonstrated how powerful grass roots conferences can be. We might not have had big names on the roster, but we sure had a productive dialog on the tricky issues lurking in the cusp between software development and IT operations in Cloud environments.
The conference has been amply covered by Michael Cote, John Willis, Mark Hinkle, and Damon Edwards (to name a few). This post restricts itself to commenting on one fundamental aspect of the cloud which IMHO does not get the attention it deserves. It might be implied in various discourses on the subject, but I believe it needs to be called out as a fundamental assumption for just about anything and everything one might consider doing with respect to the cloud. I am referring to economies of scale.
As pointed out in a forthcoming book on Cloud Computing by colleague and friend Annie Shum, the cloud phenomenon is fundamentally driven by substantial economies of scale in very large data centers. The operational costs of running such data centers are close to an order of magnitude lower than these prevailing in small and mid-sized data centers. User benefits are primarily derived from these compelling economies of scale.
I will be asking Annie to write a detailed guest post on the subject for readers of The Agile Executive. Until her post is published here, I would recommend we primarily consider the Cloud as a phenomenon that only becomes meaningful at scale. In particular, Private Clouds are not likely to yield Internet-scale efficiencies. Folks who regard their company’s conventional data center as a private cloud might be missing up on the ‘secret sauce’ of cloud computing.
The various agile system administration schemes discussed at the Austin OpsCamp are essential to attaining the requisite economies of scale in cloud services. Watch out for follow-on OpsCamps in other cities for developments to come in this all important space.
Agile Infrastructure
Ten years ago I probably would not have seen any connection between global warming and server design. Today, power considerations prevail in the packaging of servers, particularly those slated for use in large and very large data centers. The dots have been connected to characterize servers in terms of their eco foot print.
In his Agile Austin presentation a couple of days ago, Cote delivered a strong case for connecting the dots of Agile software development with those of Cloud Computing. Software development and IT operations become largely inseparable in cloud environments. In many of these environments, customer feedback is given “real time” and needs to be responded to in an ultra fast manner. Companies that develop fast closed-loop feedback and response systems are likely to have a major competitive advantage. They can make development and investment decisions based on actual user analytics, feature analytics and aggregate analytics instead of speculating what might prove valuable.
While the connection between Agile and Cloud might not be broadly recognized yet, the subject IMHO is of paramount importance. In recognition of this importance, Michael Cote, John Allspaw, Andrew Shafer and I plan to dig into it in a podcast next week. Stay tuned…
Between Agile and ITIL – Part II
The July 2009 post Between Agile and ITIL introduced the application of Agile principles to system management with the following words:
You do not need to be an expert in Value Stream Mapping to appreciate the power of speeding up deployment to match the pace of Agile development. By aligning development with deployment, you streamline “production” with “consumption.” The rationale for so doing is aptly captured in the first bullet of the Declaration of Interdependence: “We increase return on investment by making continuous flow of value our focus.”
Yesterday’s press release about the acquisition of Phurnace by BMC validates the projection given in the afore-listed post. Colleague and friend Michael Cote puts his finger on the heart of the acquisition in his post in People Over Process:
The interesting part is also that this is automation – I’m assuming – at the application layer, where as most automation talk in past and present is at the infrastructure layer. Of course, the thought leaders in this area – folks like Reductive Labs (Puppet),OpsCode (Chef), and in a more general sense cloud management outfits – are doing a helpful job of blurring the distinction between traditional IT layers like application and infrastructure with their dev/ops angled automation. Check out this white paper done by Reductive Labs and dto solutions on the topic for a nice toe-dip. And, I’d expect to see more application layer automation from VMWare/SpringSource. Older automation portfolios like BMC’s Blade Logic line need to keep a close eye on these developments, hopefully, taking in the proven parts of that work.
One can, of course, automate IT tasks without embracing Agile. The fundamental question to be answered is whether one considers ITIL as an “empirical” process control model or as a “defined” process control model (or possibly a hybrid).
Extending a True Epiphany
In Agile Software Development with Scrum, Ken Schwaber describes a true epiphany he experienced as a result of his 1995 meetings with DuPont’s process control experts:
They [DuPont’s process control experts] inspected the system development processes that I brought them. I have rarely provided a group with so much laughter. They were amazed and appalled that my industry, system development, was trying to do its work using a completely inappropriate process control model. They said system development had so much complexity and unpredictability that it had to be managed by a process control model they referred to as “empirical.” They said it was nothing new, and all complex processes that were not completely understood required the empirical model…
… I realized why everyone in my industry had such problems building systems. I realized why the industry was in such trouble and had such poor reputation. We were wasting our time trying to control our work by thinking we had an assembly line when the only proper control was frequent and first-hand inspection, followed by immediate adjustments…
Based on this insight, I have since formulated with others the Scrum process for developing complex products, particularly software systems.
Fast forward to November 2009. During a lovely dinner in Boulder with Dean Leffingwell, we got into the subject of connecting Agile with ITIL. This conversation really registered with me. I actually recalled how years ago Ray Paquet characterized IT as a “continuous manufacturing” process. If you accept Ray’s premise, the chain {DuPont –> Scrum –> IT} is quite intriguing.
Re-reading Software Evolution recently, I was struck by the observation Tom Mens makes in the Introduction:
… due to the fact that the activity of software evolution is a continuous feedback process, the chosen process model itself is likely to be subject to evolution.
I can’t help wondering whether Tom’s observation applies to IT. If so, what are the implications with respect to IT operations and system management?!
Opinions please!
Predicting the Year Ahead
Cutter Consortium has published predictions for 2010 by about a dozen of its experts. My own prediction, which examines the crash of 1929, the burst of the “dot-com bubble” in 2000 and the financial collapse in 2008, is actually quite bullish:
I expect 2010 to be the first year of a prolonged golden age. Serious as the various problems we all are wrestling with after the 2008-2009 macro-economic crisis are, they should be viewed as systemic to the way a new generation of revolutionary infrastructure gets assimilated in economy and society.
In addition to the techno-economic view expressed in the Cutter prediction, here are my Agile themes for 2010:
- Agile moves “downstream” into Release Management.
- Agile breaks out of Development into IT (and beyond) in the form of Agile Infrastructure and Agile Business Service Management.
- SOA and Agile start to be linked in enterprise architecture and software/hardware/SaaS organizations.
- Kanban starts an early adoption cycle similar to Scrum in 2006.
Acknowledgements: I am thankful to my colleagues Walter Bodwell, Sebastian Hassinger, Erik Huddleston, Michael Cote and Annie Shum who influenced my thinking during 2009 and contributed either directly and indirectly to the themes listed above.
Prior to Sprint Zero: A Note on Jakob Nielsen’s “Agile User Experience Projects”
Dr. Jakob Nielsen published the results of a follow-on study to his 2008 report Agile Development Methods and Usability. The bottom line from the 2009 study (entitled Agile User Experience Projects) is as follows:
The two main recommendations for ensuring good usability in Agile projects remain the same as in our original research:
Separate design and development, and have the user interface team progress one step ahead of the implementation team. That way, when it comes time to build something, it’s already been designed and tested. (And yes, you can do both in a week or two by using paper prototypes and discount user testing.) Maintain a coherent vision of the user interface architecture. Create the initial vision during a “sprint zero” period — before any implementation has started — and maintain it through annual (or semi-annual) design vision sprints. You can’t just design individual features; they have to fit together into a coherent whole — a whole that must be designed as well. Bottom-up user interface design equals a confused total user experience (the Linux syndrome).
I would like to highlight one implicit sub-aspect of Dr. Nielsen’s good counsel to maintain a coherent vision of the user interface architecture:
- Ensure coherence with the underlying application paradigm
To illustrate the point, think of a Business Service Management Application. You might monitor any number of servers, routers, databases and applications in order to ensure that a service satisfies the corresponding Service Level Agreement. However, the user interface architecture should have service as its fundamental concept. The architecture should certainly enable zooming in on any component of the service. But, the status of any such component (or sub-component) is merely means to an end: reflecting the status of the service and initiating as appropriate action(s) to fix it. Forming a service piecemeal from a number of constituent elements like those mentioned above – servers, routers, databases, applications, etc. – is no substitute to “service orientation” of the user interface.
The reason for my strong emphasis on the service as the most fundamental user interface concept is nicely captured in the article “How to Spell BSM” by BSM Review‘s Tom Bishop:
Most businesses today are so dependent on IT that, if an IT organization does not understand how the business depends on its services, or does not manage those services with that business perspective in mind, they are dooming the business to slow, steady death….
Dr. Nielsen’s recommendation to conceive the initial user interface architecture prior to beginning any implementation work is very consistent with this imperative need in BSM to manage the services from a business perspective. I would actually go one step further and contend that whenever the underlying paradigm changes in a manner as dramatic as the servers –> services in the BSM example above, demonstration of the core concept(s) of the user interface might need to precede the “sprint zero” period. In the context of the overall planning and budgeting process which governs the Agile process, such demonstration could actually be a pre-requisite to launching “sprint zero.”
If you consider this “prior to sprint zero” approach a bit heavy-handed, I would offer a simple test to assess its reasonableness. Play with a number of IT Service Management (ITSM) products that you picked in random. Once you did so, compare the numbers of those that clearly have services at their core, to the number of those that integrated services into their user interface as an afterthought.
The Agile Executive 
