It’s well known in the IT industry that companies just do not take an Agile / Scrum approach to the infrastructure organization. After all, those methodologies were written for software development, right!
Well, yes, but if you’ve read any general book on the subject, you’ll also know that people make the argument that you can “scrum anything.” I’ve heard of people scrumming their wedding, their home improvement projects, or simply daily tasks.
At my company, we did start scrumming in the Infrastructure organization. With that said, it’s not 1:1 identical as software development. Over the last 10 months we had to bend and mold the framework to fit our needs. Here is a list of ways in which Agile / Scrum in Infra can be different. I call it the Infrastructure Agile & Scrum lessons learned.
- The product owner role is different in either one of the following ways:
- The PO is either the manager of the team OR
- The PO is a team/tech lead OR
- The PO is the business sponsor providing the requirements AND GENERALLY
- The PO writes fewer stories and the team writes more stories
This is because the diversity of systems in infra is much larger than software. In software, there is ONE application to maintain. Thus, the Product Owner owns that one system. In infrastructure, you may have a PO for network, server, and storage each being a different person. So, if your cross-functional team is composed of each of these three areas, you could have 3 PO’s.
Solution: Find someone above them (Director) who preferably has tie-ins with all three areas. However, this may not be totally possible. If not, you will have to go up another level until you can find a common denominator. i.e. Sr Director; SVP etc.
- Engineering vs. Operations
The ultimate goal is to have a team that does nothing but projects (engineering) and another team that does nothing but production and operations support. When team members have to do both, their context switching is increased and their overall output decreased.
Solution: It is very unlikely that anyone would ever be 100% project oriented. The goal should be a minimum of 80% project and only support as necessary. You can do this by either increasing the size of the team or by out-sourcing initiatives that give additional leverage to the project teams.
- Like it or not, we are “WaterScrum”
The nature of hardware is waterfall. There is a clear path to completing most hardware projects: order hardware, install, setup and configure, and implement into production. Because of this:
- More up front planning and requirements are generally needed. If a piece of hardware is not ordered up front for example, you could add several weeks to the overall timeline.
- You cannot do the firewall connectivity before the servers are built and you cannot build the servers until the hardware is installed.
- However, it is often the case that once the physical hardware portion is done, many projects require unique Features that are more Agile in nature i.e. a migration consisting of 100 URLs. These can be ordered and put into sprints which equals a release plan.
Solution: While the order of operations may be “waterfall”, there is nothing preventing us from loading the work into sprints. Each task can still be broken down as stories and planned in an iterative fashion which still allows for release and road map planning.
- MVP’s are useful but not in the SAFe sense
In software development, it is possible to complete a project without finishing 100% of the planned features. If the Product Owner is happy, they can stop 60% of the way through and save 40% of the budget.
In infrastructure, nearly all MVP’s are required to be complete. Thus:
- MVP’s are less about scope and more about the order of completion
Solution: Utilize MVP’s as an ordering mechanism: MVP 1 is hardware installation, MVP2 is configuration, MVP3 is production implementation etc. This also allows for cross-train dependencies to be clearly communicated.
- We release throughout the sprint, not at one time at the end
There is no single release time in infrastructure. The servers are built on production hosts, firewall rules are pushed into production and apps are installed and configured. This means that:
- There is not necessarily a “done” increment at the end of the sprint and thus
- There is generally no need for a sprint review
Solution: There is no need to have a sprint review because the PO/Sponsor has already realized the benefits of the system. If a test environment is built and finished midway through a sprint, the app teams start testing. If bugs are found they let us know. We don’t necessarily wait until the end of the sprint to show them the test system.
- Scrum Teams, while they can be multi-skill set, are still fairly component-based
If a team is composed of a server person, a network person, a storage person, and an app person, each of those members can only do that one specialty. In the software world, you could have one person develop, test, QA, and implement into prod. It is not so in infrastructure because:
- Most Engineers specialize in one area. You very rarely see someone who knows networking, server, storage, OS, app, and database at a very high level.
Solution: While you don’t have some of the cross-skill set factor, you still gain a lot simply by having each member on the same team and in the same room. You decrease the communication barrier as well as the amount of hand-offs required. Another solution is to have your scrum teams sit in the same area. This is generally not done because server still sits with server, network still sits with network etc. This may only be possible once the division of Engineering and Operations takes place.
- Scrum teams may NOT be long lived
Scrum calls for long lived teams – that is, teams that once they form stay together for a long duration of time – possibly a year or more. This is typically so that things like velocity stay consistent and don’t change based on newly added/removed team members.
Unfortunately, in Infrastructure, because teams are often built around projects, as old projects are closed and new projects are started, the team may change depending on the requirements.
- Swarming and cross-training are mutually inclusive (paired programming)
Infrastructure teams generally do not swarm on projects. That is, they do not focus on one project at a time but rather many. This is due in part to the issue above (specialty skill set). If there are three people on a team, and all three have a different skill set, then naturally they’ll all be working on a different project.
Solution: Support the idea of paired programming. That is, put two people with like skill sets together on one project. While one person is doing most of the work the others are learning a new skill set. Now you have two people to do the same work and you only worked one project (swarming).