Archive for August 2009
Mapping the Agile thinking, theory and practices to the realities of the target company is a tricky part of making Agile happen in a sustainable manner. HubOne’s Nick Beaugeard, known to readers of this blog from his post Enterprise Product: $50,000 and 8 months – You Must be Kidding, shares with us his recipe for so doing in a start-up environment. He manages to weave the pragmatic details with the core principles behind software development in general and Agile methods in particular. For example, consider the following insight provided by Nick:
In fact, when developing an API, the unit tests are your clients!
Readers of this blog might want to compare and contrast the thoughts Nick expresses in this post with those of:
- Ryan Martens on prescriptive versus adaptive rollout of Agile (click here).
- Eric Ries on iterating on the problem definition and developing the customer base in parallel with iterating on the solution (Click here).
Here is Nick:
After my last post, where I discussed the concept of implementing the tools and process before you get people on board, I though it appropriate to provide some prescriptive guidance on how we achieved the process. This post is primarily aimed at start-ups where you have total control over your infrastructure, computers, network and internet connection. If there is enough interest, I’ll produce another post of how to perform the same, but in a corporate environment.
I believe, and my experience has shown, that preparing your work environment, tools and process before the team starts coding helps eliminate costly and lengthy discussions about tools and process. In fact, in my experience, most developers are pleasantly surprised to find the environment ready and working and slip into the processes extremely quickly.
So, please find below my ten steps. Following this process really helped us get up and running quickly. Whilst we used Microsoft Development tools, this equally applies to their open source equivalents, so feel free to substitute specific tools, just not the requirements and process.
- Authentication, Network and VPN. Setting up the core of your environment is critical. As you are more than likely working on secret software at the outset, you need good, auditable mechanisms for authentication and logon. In addition, we don’t want our developers to have to do anything except start their PC or laptop and login, and we really want them to be able to work remotely. If you don’t feel qualified or able to complete the steps below, any good local IT Pro should be able to set this all up for you. To perform this, we implemented the following:
- Network Connection – we are in Australia so our networks are not fantastic. We subscribed to a good ADSL 2+ plan (24Mb) with a 80Gb limit. We implemented a modern ADSLwireless modem/router and configured it correctly. This gave us acceptable internet connectivity.
- Domain – We implemented a domain controller running on Windows Server 2008. This gave us corporate authentication, auditing and identification. The domain controller was hosted on our private network (see 1.c)
- Routing and VPN –our internal development network needed to be protected from internet connectivity so we implemented a Windows Server 2008 machine with two network cards (called multihomed). One card was connected to the router and one connected to an Ethernet switch. We used a private IP subnetfor our development Local Area Network (LAN) and enabled Microsoft Routing and Remote Access. This gave us the ability to authenticate domain users to VPN into the private network for remote working. We then configured our router to allow access to the server for VPN Access.
- DNS –one of the issues you face with ADSL is that your Internet IP Address changes often. The solution for this was to use a solution from DYNDNS (www.dyndns.com) which allowed us to register a host name, coupled with a client application which ensured our host name for VPN always pointed at the correct IP Address.
- DHCP –it’s a real pain, especially when using VPN when your client machines are configured for static IP addresses. We used Microsoft Dynamic Host Configuration Protocol (DHCP) to ensure every machine had a unique IP Address and that our networking became just “plug-and-play”
- Version Control and issue tracking – in my experience, there are four critical systems needed for any software development team. These are:
- Version Control – Also known as source repositories, these systems allow control over check-in, check-out and versioning of source files. I cannot recall the number of times we backed out an individual source file to a previous “working version”. Without this in place, I can guarantee you’re going to struggle maintaining a good code base!
- Bill of Materials– the way I use these is to highlight each of the key deliverables in a project. When building Enterprise Profile Management, our BOM had 345 separate items, everything from the corporate website, to graphic design, to core components of the API. The Bill of materials is a way to track the overall progress of the project. Each item in the Bill of Materials should have at a minimum a title, description, owner, due date, % complete and %tested. We also use the bill of materials to determine our release criteria (more on this later).
- Build System – the biggest mistake a development manager can make is to not build the software regularly. We build our software system every time there is a new check-in (called continuous integration). Even with a very small team, the ripple effect of changes could go un-noticed for ages without building regularly. As an aside, we also have a Nabaztag (www.nabaztag.com) bunny which tells us whenever there is a build, whose check-in caused it and whether it succeeded or failed. While this is really annoying, it focuses the developers on good check-ins. We might also make the person who broke the builds buy us a round of beer, but that’s a secret!
- Issue Management – More important than email, IM, or indeed any other form of communication in the development team is issue management. I believe that in a project of any size, you’ll be hard pressed to ever finish if you don’t have good issue management. Issues should contain a title, description, history, assignee, status and indicate which version/iteration of which product the issue applies to. I don’t actually think a spreadsheet will cut it here. If there’s one investment you make, make sure it’s issue management.
We used Microsoft Visual Studio Team System for all of the above. Being an ex-microsoft product team member, I am familiar with the way the product works, but there are lots of plug-ins available for scrum, agile (MSF) and CMMI. To do this on a startup budget, we were able to join the Microsoft BizSpark program (http://www.bizspark.com) which gave us instant access to Microsoft’s developer tools. I’d highly recommend taking a look at that program!
- Backup – Now is the time to do a backup and recovery operation. You have no real data in the system and how you installed it all is probably fresh in your mind. Trust me, every first recovery operation fails. You need to imagine your office has been hit by lightning and you have no servers, and just a backup. If you can successfully recover your environment in under 24 hours, you’re in a good place. Document how to do it and test it regularly.
When we were developing Enterprise Profile Management, our server with all issue management, reporting, builds and version control failed (the processor fried). We were taking backups, but the restore failed. It took me 22 hours to perform a forensic recovery of our production platform. Luckily the developers could work offline, but we still introduced a ton of integration bugs. Don’t skimp on backup.
- Email, IM and Web Conferencing – You’re going to be working remotely at some time, whether you think you will or not. We quickly implemented the following tools:
- Coding Standards – Getting at least some coding standards in place before you write any code is really important. Good Version Control Systems should be able to validate code against your standards on check-in. My key coding standards are:
- No String Literals – All strings must be externalised in resource files for later localization (a process called globalization)
- Commenting – I insist that every class, method, property, event and interface be commented. There’s method in my madness here; obviously code commenting is important but with proper commenting (especially in C#) and tools like sandcastle, you can automatically create documentation like ours at http://api.hubone.com.
- No Short Cuts – This one was a little contentious – Our code will in the future (hopefully) be worked on by people who had no idea of the concept. Writing out If…then constructs and property accessors in long-hand makes the code much more readable for novices, junior programmers, support teams and architects alike.
- Unit Test Everything –I insist that every method, property, event etc, etc has an associated unit test. These we execute as a part of the build. My goal is to have over 80% of any code written covered by unit tests. In fact, when developing an API, the unit tests are your client! Although this increases the programming effort up front, it actually reduces the total time taken to ship the product. I havelost count of how many times our massive unit test library has saved us from nasty regressions that we could never have found from UI testing.
- Write some code –Don’t worry, you don’t need to be the world’s best developer to write some code. All you need to do is think carefully about the different items in the Bill of Materials and start prototyping the methods, properties and events that will make it all work. Ensure you adhere to your coding standards. The goal of this exercise is to effectively build a skeleton of your application before the developers get on board. This can make them hyper productive when they start. They’ll probably end up getting rid of every line of your code, but it will get them in the rhythm of how you want the platform to look. As the architect of our platform, I used this period to prototype and test all of my assumptions of how things would work. By the time the developers got on board, I had a passable working prototype of an API with unit tests and coding standards, although there is almost none of my code in the final product, this enabled us to get running really quickly.
- Don’t write a test plan – Write the user documentation instead, and use this to test your application. That way you’ll know your application does everything the user guide says it does and you won’t double up on a test plan and the user guide (which should end up being pretty similar anyway). I wrote our user guide because I knew what I wanted the product to do and it’s also a fantastic way (sometimes they only way) you get to communicate with your customers.
- Give ownership, responsibility and praise – You will have no choice. As you get developers, testers, writers and others on board, you must give them ownership of components. I find that if I give someone total ownership, they always deliver. People are proud when they do great work and if they feel and really have ownership over something, it will be their best work. Not only that, but they’ll pull out every stop to impress you, the team and your customers. If you hire correctly, you’ve got great professionals in the team. Respect them as professionals and the results will be awesome.
- The best wire framing tools are not wire framing tools – You’re a start-up, right? Then you’ve discussed your ideas with potential customers and you deeply understand their needs. You could sit down with any number of wire framing tools and attempt to design components. How about you take a different approach? Write functional code which does what you (and therefore your customers want) – Developers find it far easier to code from a working model and they can re-use portions of your code. All of this streamlines the process and you can take your models to customers (with caveats on your poor coding skills and lack of stylistic ability).
- Constantly review and drive – We spend some time every day in front of a whiteboard discussing options, vision, checklists and almost everything else, but in a start-up, it’s not really a democracy. I run my start-ups like a benevolent dictatorship – I hold the final decision, but let everyone have input. I’m the one talking to customers, and sometimes I bring customers in to explain a requirement, but at the end of the day, how it works is up to me (I have the most to lose if it doesn’t work!)
Notice here, we never had a specification, requirements specification or design, we build the code a piece at a time and made it work. We’ve gone back and documented some core functionality, flowcharts and features, but at the outset, it’s more important to make it work. I’ll share a caveat, though, I’m also the software architect so I can hold the requirements for scalability, security and reliability, and guide the team to accomplish those goals.
Hiring the team can happen sometime between points 7 and 8, I don’t recommend you do it before!
Abstract: The presentation applies Agile thinking to critical aspects of strategy and execution at a time of uncertainty and disruption. The essential point is simple and logical: Agile values and principles are indivisible. To succeed, they must be applied not just to R&D, but also to customer and company, simultaneously. This requires reconfiguration of customer relationships, employee policy, software development, and the relationship that binds the three. The resulting paradigm shift could lower the cost of software and produce prosperity similar to the one induced by ultra-cheap oil in the 50’s.
Perspective: In addition to being a ‘think-piece,’ the presentation offers pragmatic recommendations for the Agile champion in three critical areas:
- It explains how the Agile champion can cross three chasms that tend to form in the course of large scale Agile rollouts.
- It explores how to apply Agile priciples to software deployment and operations.
- It shows how earned value management can utilize ‘real time’ customer feedback in companies that embrace end-to-end Agility.
In this episode, we’re joined again by Andrew Shafer to talk about Agile Infrastructure (or “Agile Operations” as some folks call it).
- The in problems in IT that cause us to start wanting Agile Infrastructure. The high-level problem is enabling change (that works) more often: configuration drift, intentional complexity, walls of confusion everywhere, hero-driven incentives. Israel also mentions the theory that you have to change up your incentive structures often so that people don’t get locked into incentive-driven thinking vs. “doing the right thing,” so to speak.
- Leading us into the practices, Israel asks Andrew about including the operations folks in the Agile team, just as you do developers, QA, documentation, and so on. This gets into a discussion on “fractal teams.” We then get into other practices and technologies that help with Agile Infrastructure:
- Version control – getting beyond .bak files. You need some kind of version control system. What do you put in there? All your configuration files, to start with. Perhaps your scripts next. Puppet and other tools can help do more. The tools, really, can be the same as used in development: git, subversion, CVS, and so on. In fact, Andrew says you should really use whatever development is using for consistency.
- Always ship trunk
- “Dark launches” – staging the release of features to test back-end tasks before exposing it to the user, and then finally giving the user access to the new system. This lets you test out the impact of the “background” tasks in the production system of new features without exposing it to users.
- An over-arching theme here is to reduce the fixed cost of deployment, trying to get it to zero as much as possible.
- Some other practices: test-driven infr, deploy early/deploy often, tagging everything with who/what/when, time synchronizing, and a few more.
Rally’s Alan Atlas shares with us his insights on picking a pilot project for Agile. His post nicely complements the account Sue McKinney and Pollyanna Pixton gave about their approach to bootstrapping Agile at IBM (click here). It also touches on some of the points made in our post The First Decision to Make. Whether you agree or disagree with Alan, his thoughts are always intriguing. You will find additional insights by Alan in The Scrum Mechanic.
Alan has been professionally involved in high tech for nearly thirty years. His career spans top technology companies such as Bell Labs and Amazon as well as various intriguing start-ups. He brought to market numerous products, including OSF Motif and Amazon’s S3. His passion for Scrum has recently led him to make a career switch into full-time Agile Coaching and Training with Rally Software.
Here is Alan:
Picture this: You’re an Agile Coach and you arrive for the first day of your new, monster engagement at a large enterprise that has hired you to help them become Agile. You’re very excited as you walk into your first training session with a select group of employees. As you start the training, you are greeted with questions from your happy, excited audience. “Can we get this over with early?” “I don’t want to be here. My manager said I had to come.” “What is this all about, anyway?” “I have a friend who got fired for advocating Scrum. I don’t want anything to do with it.” “Why am I here?” Is this any way to start an Agile transformation?
A necessary step in Agile adoptions is picking where to start. Consultants often help management or transition teams with the selection of so-called pilot projects. Project teams are then notified that they are the lucky winners in the Scrum Lottery. The result can be a (not entirely incorrect) feeling amongst employees that they are being forced to become Agile, which can lead to the scene I just described.
This command-and-control (C&C) approach can easily erode trust, destroy motivation and handicap an adoption program because it ignores the critical contribution that team buy-in and ownership can make toward successful implementation of the new development process. At best, it leaves a mild bad taste in the mouths of the employees and gives them a good reason to believe that this Scrum thing is just another management flavor of the week. At worst, it removes the single most important success factor in Agile adoption: the active and enthusiastic participation of the team.
Is there a reasonable way to avoid the pitfalls of the C&C approach and still meet the legitimate needs of management? Can we start a transformation with excited, enthusiastic employees instead of sullen ones? Can management make a decision to ‘go Agile’ and still establish a collaborative relationship with employees? Can we take advantage of the inherent appeal of Agile methods to increase our chances of success? I think the answer to all of these is, “Yes!”
Agile’s people-based approach tells us to view the world from a team-centric point of view and not a project-centric point of view. Applying this to our Agile transition itself, we realize that we don’t need to assign Scrum to projects. Instead, we can let teams choose to adopt Scrum (or not to adopt, to be fair). It’s the team that will make the Agile process work and lead to success, not the project itself. This approach will maximize the likelihood of success by finding the teams that want to make it work.
The way to ensure the success of early Agile transformation efforts and simultaneously to align management and employees without coercion is to provide teams with the permission and knowledge to make their own decisions, and then for management to support those decisions. Teams that choose to ‘go Agile’ will make it work. Teams that are told to ‘go Agile’ might or might not make it work.
Implementing this isn’t hard. Start by making introductory Agile training available to the target organization (a few two-hour classes spread throughout a week is a good start for all but the largest organizations). Announce that teams are welcome to try Scrum, and tell them how to request further team training and coaching (don’t neglect to make sure their immediate management is on board). There might be reasons to give priority to certain teams and possibly to delay others, but in general the teams that want to do Agile should be allowed to do it. The company is now supporting and leveraging teams that want to transition to Agile and allowing those that don’t want to change to continue as they are. This in itself is an important message to all that the Agile philosophy is being taken seriously by management.
The result of using this approach is that there is no bad taste among employees, the most enthusiastic teams self-select to participate in the new experiment, nobody is forced, and management demonstrates its willingness to support employees in the new endeavor.
No managers were harmed in the filming of this scenario. 🙂 Seriously, we put the focus on teams without negating any of the needs of the enterprise. The ability of teams to get further training can still be managed. The identities of the teams can be known so that there can be followup assessments, monitoring, and support. In the cases where it is necessary, teams that have interdependencies or other complexities can be staged appropriately. The biggest and most important difference between this scenario and the more traditional C&C scenario is that here the teams that are excited and interested about Agile become the first in the water.
Using this method for starting your transition doesn’t change anything else about your organizational Agile initiative. You still have all of the cultural, technical, engineering, management, organizational, and human issues to deal with. It just gives you a way to pick the starting point in a more positive and Agile way.
InformIT has just posted my interview with Jim Highsmith. While the interview naturally focuses on the the new edition of Agile Project Management, Jim makes quite a few observations on deep truths. For example, in response to my asking him to do a quick “retrospective” of the period since he signed the Manifesto, Jim gives both perspective and retrospective. Here is an excerpt from his answer:
If the Agile movement is to continue, we have to better understand what the core Agile principles really are, and not just our personal interpretation, and then find ways to incorporate thoughts and ideas that may seem in conflict with our own ideas. Just because some Agile camps may have a more widespread audience, that doesn’t make them the source for all things Agile. The essence of change is tolerance for new ideas that conflict with our own.
Enjoy reading the full interview!
With the successful release of Enterprise Profile Management 1.0, I asked Nick Beaugeard – Founder and Managing Director of HubOne – to capture and share with us his Agile development experience. Some of the threads he describes might have been mentioned in other posts in this blog. Others, like “Invest in your method and tools before you hire people, not after you hire people!”, appear for the first time. Whether mentioned here before or not, the ‘secret sauce’ is Nick’s ability to pull so many threads together. The power of so doing is nicely expressed in Nick’s guest post below.
Nick’s been involved in Tech for 27 years (if you include the games he wrote for the Apple II aged 9) and has been involved with companies like Microsoft, CSC, JP Morgan and many others. He’s also ran a number of start-ups, from web 2.0 to services to solutions companies and has been successful with a number of those. Nick has a passion for all things Agile, but prefers to design the methodology to suit the purpose, rather than rely on a one size fits all method. Notwithstanding this, Nick has some notoriety within Dimension Data for passionately changing solution development from failing waterfall to a test based development approach, suiting solutions and services.
Nick lives on Sydney’s northern beaches with his wife, four kids, two mice and a framed copy of the fabled W.W. Royce “waterfall” speech to the IEEE.
Here is Nick:
I’ve ran start-ups before. I’ve even managed the development of products before, but this is the first time I’ve set up a start-up with the sole goal of developing a product and bringing it to market. Thinking back on the last 8 months, I’m not sure I’d have taken the challenge on in retrospect, but we did, and we succeeded and so in this post, I’d like to share with you how a micro-startup with a budget of $50,000 can design, develop, release and certify an Enterprise quality product without going both broke or mad….
During my career, which spans everything from support, through solutions and has a great deal of focus on Systems Management, I’ve been constantly frustrated by something which should be really simple, how to find people with specific skills or responsibility. Seeing that the problem still existed, I set out to build a solution which would attempt to solve such a problem.
The first step on the road was to meet with potential customers. At this stage, I had barely a germ of an idea. It could be written in 20pt type on a single piece of paper. Nevertheless, I met with over 20 different potential customers and attempted to pitch the “solution” to them. The feedback I got was extensive, but the main and core piece of information was that the solution had value and merit and if we could deliver then these customers might just buy the software. I even went as far as to ask them how much they’d pay for such a solution and got a fairly consistent figure of about $20 per year per user.
I realised at once that the problem I was trying to solve was not a database problem. In fact it was a directory problem so I set about investigating whether I could produce a solution using an LDAP directory.
Right at the beginning, though, I made an investment in tools. I implemented great source control, issue management, task management and a build platform, just like I was in a huge team.
I also made sure I adhered to issue and task management and made sure that I was forced to complete the criteria for each check-in. I know that sounds stupid, and it must have seemed so… The CEO of a start-up assigning himself issues only to resolve them on check-in. However I believed it to be easier to implement the processes I wanted before I had a team on board, demonstrate me using them and it would be WAY easier to get the team to use them!
To test if my architectural hypotheses was correct, I wrote a very, very basic API. This small piece of code was able to perform some CRUD (Create, Retrieve, Update and Delete) operations on my own custom directory objects.
Let’s read the above again… See I had no spec, some interesting comments from customers and just went ahead and wrote a prototype API. Next I wrote some Unit tests which exercised the API and went back to my customers to see if I was on the right track.
That bit was tough. It takes a fair amount of imagination to see a simple set of unit tests and then extrapolate that to a completed product. However my friendly customer s were very patient as I explained that a response of “Passed” actually meant what I just explained had happened.
Once again that feedback helped and I felt confident enough to prototype most of the API and almost completely complete the schema. Once again I created a bunch of unit tests, but this time I focussed on the core scenarios as discussed with our customers.
Still at this stage, we had spent almost nothing, had a good, working and testable prototype API and a schema which worked.
It was then time to go and get a couple of developers. I knew a couple of my close colleagues had just resigned from another company so I asked them if they’d come and work for me, at a vastly reduced rate, for some stock in my new start-up.
Just the stage for a spec, huh? We’ve now got more than one developer so obviously need to write down then entire goal of the solution…
I thought not. My API was just a prototype and needed to be secured, consolidated, better documented, globalized and tested for performance and the unit tests demonstrated exactly how things should work. My new, fresh team set to work on tidying up and completing the API, working on issues and tasks and performing check ins. They also got told whenever they broke a build.
We were then seriously thinking about building a web services layer. This had several advantages, not the least being that we could support clients on multiple platforms. However, I was still a little uncomfortable with my API hypotheses and really wanted to build a client that exercised the entire API before we invested in SOAP and web services.
Therefore we set the challenge and therefore the scope of version 1. Version 1.0 would deliver a server (LDAP with an Installer) and a client (Windows XP and above) which talked to the LDAP server using our API.
That made the next challenge was the UI. This is a problem for me as I’m not one for having any design skills at all. However I could have sat down and tediously wire framed every screen and interaction I and my customers wanted.
However I knew a better wireframing tool – the IDE we used to write code instead. I sat down and started to craft a UI, using a lot of trial and error. In 7 days, I had a UI which exercised the main aspects of the API and the code base…
And that’s how our development stayed, I’d prototype (wireframe with code) a feature and the devs would then go and finish it.
We found bugs in the API, sure, but never needed to write new API methods to make the client work.
In the final weeks, we just worked on testing and fixing bugs… The test plan you ask? – Well that was the user guide! A full test pass meant running through each page in the user guide and doing everything it said.
A recent code review showed there was none of my prototype code left in the platform, but it didn’t matter. We shipped the product and spent less than the $50,000.
In conclusion our method was:
· Speak to your customers. Not lots, all the time!
· Invest in your method and tools before you hire people, not after you hire people!
· Make the software function, before you spend time making a UI function.
· Continuously test, fix and test again and don’t be afraid to throw prototypes away.
Last Friday we released version 1.0, go download it, you be the judge of how we did!
Various posts in this blog (click, for example, here, here, and here) brought up noteworthy threads from the Q2 Rally Agile Success Tours events in Santa Clara, Atlanta and Washington, DC. The Rally team and I are gearing up now for the following Q3 Success Tour events:
- September 17, 2009 in Boston, MA
- October 1, 2009 in Seattle, WA
- October 15 in Chicago, IL
- October 29, 2009 in London, UK
I have started teleconferencing with the panelists in these four cities. It is clear even at this stage of the preparations we will have an eclectic mix of panelists and various intriguing topics that might not have been fully addressed during Q1 and Q2. For example, organizing your reality and Agile can’t be localized are two of the topics that will be discussed in the Boston event.
As I did during Q1 and Q2, I will report here highlights from the events. Stay tuned…