Toward An Event Driven Scrum Process
Recent posts in this blog and elsewhere recommended using technical debt (TD) criteria as an integral part of the build process. One could fail the build under conditions such as:
- TD(current build) > TD(previous build)
- TD per line of code > $1
- TD as % of cost > 50%
- Many others…
If you follow this recommendation while integrating continuously, chances are some fixing might need to be done multiple times a day. This being the case, the relationship between the daily stand-up meeting and the build fail event needs to be examined. If the team anyway re-groups whenever the build fails, what is the value add of the daily stand-up meeting?
Figure 1 below, taken from Agile Software Development with Scrum, examines the feedback loop of Scrum from a process control perspective. It shows the daily stand-up meeting providing the process control function.
Figure 1: Process Control View of Scrum
Figure 2 below shows the feedback loop being driven by the build process instead of by the daily stand-up meeting. From a process control standpoint it is essentially identical to Figure 1. Control through the regularly scheduled daily stand-up meeting has simply been replaced by event-driven control. The team re-groups whenever the build fails.
Figure 2: Build Driven Process Control
Some groups, e.g. the Scrum/Kanban teams under Erik Huddleston and Stephen Chin at Inovis have been practicing Event Driven Scrum for some time now. A lot more data is needed in order to validate the premise that Event Driven Scrum is indeed an improvement over vanilla Scrum. Until the necessary data is collected and examined, two intuitive observations are worth mentioning:
- Event Driven Scrum preserves the nature of Scrum as an empirical process.
- A control unit that is triggered by events (build fail) is in the very spirit of adaptive Agile methods.
The Agile Executive 
So if the build doesn’t break for 4 days then you don’t meet as a team for 4 days? A working build doesn’t necessarily equate to “everything is going well”. Skipping daily Scrums is not a good thing since obviously you are reducing feedback loops and collaboration. You can get this same basic effect by including a “stop-the-line” build fail mentality into a Scrum process.
Dave Updike
May 12, 2010 at 7:17 am
One needs to be mindful of the operational characteristics and adjust parameters accordingly. A build fail once a day is quite likely for a team that builds multiple times a day. The technical debt criteria are probably too relaxed if a build fail has not been experienced for a dozen consecutive builds. This being the case, one needs to tighten the technical debt criteria. For example, use a criterion such as TD per line of code <$0.8 instead of <$1.
Israel
israelgat
May 12, 2010 at 10:51 am
Questions:
Is there a reason to change the timing of a time-boxed process control? Do build fails really need to “stop the line” when there’s continuous integration and testing? Aren’t fails encouraged and if so, which are best handled at the lowest scrum level? Scrum of scrums? Level
The concern is meeting-creep. Seems to me that if a daily meeting doesn’t do it, the team may have a capacity issue.
R. Mullen
May 12, 2010 at 4:29 pm
The thing that drives my thinking about technical debt criteria as integral part of the build process is the on-going growth of debt many of my clients experience. I am becoming more and more convinced that technical debt is the #1 issue for many/most development shops.
If you accept this premise, the focus of the “control unit” needs to shift toward reducing/eliminating technical debt. One could, of course, do so during the daily stand-up meeting. However, doing it through Event Driven Scrum makes it intrinsic.
While responsible failure is encouraged in enlightened shops, the question is what to do about it. A developer in an Event Driven Scrum meeting might say something like the following:
1. Since the previous meeting I completed module XYZ and checked it in.
2. The build, however, failed due to increase in the overall amount of technical debt. I plan to spend the rest of the afternoon bringing it back in line.
3. My hunch is that the culprit is lack of cohesion of methods. I need help wrestling this “alligator” to the ground. There are some subtle issues of cohesion that I am not certain about. Who can help me?!
Finally, I believe that the core issue is not team capacity but “feature mania.” Event Driven Scrum enables the team to put a damper on the feature mania at the earliest possible time, before technical debt accumulates. It reinforces intrinsic quality in the face of relentless pressure to add features and functions as if there was no tomorrow.
Best,
Israel
israelgat
May 12, 2010 at 8:48 pm
Regina, to your good question “… are not failures encouraged?” I just recalled that Don Reinertsen said in his keynote presentation at LSSC10 that the maximum amount of information is generated when you fail 50%!
Best,
Israel
israelgat
May 13, 2010 at 8:47 pm
Hi Israel,
Your post triggers some thinking. Thank you!
I have a hard time with the strong relationship you seem to pout between the handling of the build failed event and the daily scrum.
What happens for example, if the build does not fail?
The questions I am currently asking myself are:
– What are the events that are significant enough to trigger an immediate process control event?
– Is build failed the right one? Other events?
Any thoughts?
I personnally think that the daily scrum still needs to be the main process control mechanism.
In the ‘stop the line’ spirit, here is a suggestion of questions the team should ask itself in the build fail standup :
What needs to be fixed?
Is there a way to fix it so that the problem will never occur again?
Any topic to park for restrospective?
Again, any thoughts?
Cheers,
~françois
François
May 14, 2010 at 6:04 am
Thanks for the kind words, Francois!
I would say there is no “right” or “wrong” trigger for Event Driven Scrum. If your circumstances are such that quality is a major issue, failing the build based on technical debt criteria makes a lot of sense. If, for example, you are primarily concerned about flow, WIP limit violations constitute a good trigger (this trigger was suggested to me by Stephen Chin.)
In other words, Event Driven Scrum has two components to it: a mechanism and a policy. The mechanism is the readiness (and arrangements) to get together as a team to work an issue when a significant event occurs. The policy is determining the parameters that will trigger an event. One could actually use multiple kinds of events. For example, both technical debt limits and WIP limits. It is a very effective way of exercising enlightened governance.
The point you are making about fixing it so that the problem will never occur again is on the cusp between repetitive processes v. knowledge work. As a developer, I might be weak on a topic such as complexity. Once I messed up the build big time, the team might self organize to have me do XP together with a programmer who is very strong on Java metrics. This teaming does not guarantee I will not send complexity through the roof again. However, it increases the prospects that over time I will develop my skills to program with lower complexity.
Best,
Israel
israelgat
May 14, 2010 at 12:34 pm
An equally important event to drive the process is WIP limit violations. This can either be at the team level (for teams using Kanban rather than Scrum) or the portfolio level.
At the portfolio level, you can imagine triggers like:
• In Process stage limit exceeded – No more features can be started. Prioritize and drop features to improve cycle time on top requirements.
• Deploy stage limit exceeded – Too many features piled up to go in production. Remove impediments to production deployment.
Even if you still have regular planning and checkpoint meetings, driving them off event triggers like this changes the focus from status reporting into action and problem resolution.
Stephen Chin
May 18, 2010 at 4:17 pm
Israel – I feel very exposed asking such a basic question, but seeing that this is an important issue and needing to minimize my ignorance I must ask it –
What do YOU define as Technical Debt?
Harry
June 3, 2010 at 9:20 am
Glad you asked the question, Harry. While the metaphor is intuitively compelling, various folks interpret it in somewhat different ways.
Technical debt is usually perceived as a measure of expediency. You borrow a little (time) with the intent of paying it back as soon as possible. To quote Ward Cunnigham:
“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it.”
My own pragmatic definition, rooted in the ‘time is money’ quip, is as follows: The monetized value of all design/coding/testing/documenting quality deficits.
For example, we talk about a project saddled with $1M in technical debt. What it means is that about 10 men years (assuming the cost of a man-year is $100K) of efforts will be needed to eliminate all quality deficits. Such elimination of deficits is ‘paying back’ debt in entirety. Often times the debt is reduced but not completely eliminated. For example, the debt could be reduced from $1M to (say) $300K
Best,
Israel
israelgat
June 3, 2010 at 5:51 pm