Agile / Scrum and Infrastructure

It’s well known in the IT industry that companies just do not take an Agile / Scrum approach to the infrastructure organization. After all, those methodologies were written for software development, right!

Well, yes, but if you’ve read any general book on the subject, you’ll also know that people make the argument that you can “scrum anything.” I’ve heard of people scrumming their wedding, their home improvement projects, or simply daily tasks.

At my company, we did start scrumming in the Infrastructure organization. With that said, it’s not 1:1 identical as software development. Over the last 10 months we had to bend and mold the framework to fit our needs. Here is a list of ways in which Agile / Scrum in Infra can be different. I call it the Infrastructure Agile & Scrum lessons learned.

  • The product owner role is different in either one of the following ways:
    1. The PO is either the manager of the team OR
    2. The PO is a team/tech lead OR
    3. The PO is the business sponsor providing the requirements AND GENERALLY
    4. The PO writes fewer stories and the team writes more stories

This is because the diversity of systems in infra is much larger than software. In software, there is ONE application to maintain. Thus, the Product Owner owns that one system. In infrastructure, you may have a PO for network, server, and storage each being a different person. So, if your cross-functional team is composed of each of these three areas, you could have 3 PO’s.

Solution: Find someone above them (Director) who preferably has tie-ins with all three areas. However, this may not be totally possible. If not, you will have to go up another level until you can find a common denominator. i.e. Sr Director; SVP etc.

  • Engineering vs. Operations

The ultimate goal is to have a team that does nothing but projects (engineering) and another team that does nothing but production and operations support. When team members have to do both, their context switching is increased and their overall output decreased.

Solution: It is very unlikely that anyone would ever be 100% project oriented. The goal should be a minimum of 80% project and only support as necessary. You can do this by either increasing the size of the team or by out-sourcing initiatives that give additional leverage to the project teams.

  • Like it or not, we are “WaterScrum”

The nature of hardware is waterfall. There is a clear path to completing most hardware projects: order hardware, install, setup and configure, and implement into production. Because of this:

  1. More up front planning and requirements are generally needed. If a piece of hardware is not ordered up front for example, you could add several weeks to the overall timeline.
  2. You cannot do the firewall connectivity before the servers are built and you cannot build the servers until the hardware is installed.
  3. However, it is often the case that once the physical hardware portion is done, many projects require unique Features that are more Agile in nature i.e. a migration consisting of 100 URLs. These can be ordered and put into sprints which equals a release plan.

Solution: While the order of operations may be “waterfall”, there is nothing preventing us from loading the work into sprints. Each task can still be broken down as stories and planned in an iterative fashion which still allows for release and road map planning.

 

  • MVP’s are useful but not in the SAFe sense

In software development, it is possible to complete a project without finishing 100% of the planned features. If the Product Owner is happy, they can stop 60% of the way through and save 40% of the budget.

In infrastructure, nearly all MVP’s are required to be complete. Thus:

  1. MVP’s are less about scope and more about the order of completion

Solution: Utilize MVP’s as an ordering mechanism: MVP 1 is hardware installation, MVP2 is configuration, MVP3 is production implementation etc. This also allows for cross-train dependencies to be clearly communicated.

  • We release throughout the sprint, not at one time at the end

There is no single release time in infrastructure. The servers are built on production hosts, firewall rules are pushed into production and apps are installed and configured. This means that:

  1. There is not necessarily a “done” increment at the end of the sprint and thus
  2. There is generally no need for a sprint review

Solution: There is no need to have a sprint review because the PO/Sponsor has already realized the benefits of the system. If a test environment is built and finished midway through a sprint, the app teams start testing. If bugs are found they let us know. We don’t necessarily wait until the end of the sprint to show them the test system.

  • Scrum Teams, while they can be multi-skill set, are still fairly component-based

If a team is composed of a server person, a network person, a storage person, and an app person, each of those members can only do that one specialty. In the software world, you could have one person develop, test, QA, and implement into prod. It is not so in infrastructure because:

  1. Most Engineers specialize in one area. You very rarely see someone who knows networking, server, storage, OS, app, and database at a very high level.

Solution: While you don’t have some of the cross-skill set factor, you still gain a lot simply by having each member on the same team and in the same room. You decrease the communication barrier as well as the amount of hand-offs required. Another solution is to have your scrum teams sit in the same area. This is generally not done because server still sits with server, network still sits with network etc. This may only be possible once the division of Engineering and Operations takes place.

  • Swarming and cross-training are mutually inclusive (paired programming)

Infrastructure teams generally do not swarm on projects. That is, they do not focus on one project at a time but rather many. This is due in part to the issue above (specialty skill set). If there are three people on a team, and all three have a different skill set, then naturally they’ll all be working on a different project.

Solution:  Support the idea of paired programming. That is, put two people with like skill sets together on one project. While one person is doing most of the work the others are learning a new skill set. Now you have two people to do the same work and you only worked one project (swarming).

Posted in project management, Agile, Scrum | Tagged , | Leave a comment

Just do it…even if it’s not right

One of my business pet peeves is not doing something when something needs to be done. There is a well known quote attributed to Einstein that reads: “The definition of insanity is doing the same thing over and over and expecting a different result.”

I’ve seen too many times where changes need to be made or resources need to be swapped but nothing happens. This lack of direction can be crippling to a business. The second part of this post’s title is “even if it’s not right.” I truly believe that doing something is better than doing nothing even if it turns out to be the wrong thing.

Doing something shows initiative and creativity. Doing nothing shows, well, no initiative. Sitting on your hands is paramount to keeping the car in neutral when it needs to be in drive to get off the train tracks. Obviously there are times when you can’t do anything due to regulations or business controls.

Doing something leads to new information and new strategies. Even if what you did fails you are able to learn from mistakes and take a new direction, or pivot, your newly discovered ideas.

Strategy: Always be in learning mode . Make mistakes. It’s the only way to get better. If a team member needs to be replaced – replace them. If head count is needed – add head count. Don’t wait until it’s too late either as there can be a point of no return for something to be effective.

Posted in Uncategorized | Leave a comment

The better elevator

I’ve never seen an improved elevator. I’ve seen improved cars, trains, and planes. Never elevators though. You would think that there would be more to elevators today than simply up, down, and selecting buttons.

What does this have to do with anything? Well, I’m a stickler for efficiency and optimization. During our in-house Agile training class, we set aside time to play a game called “What’s your MVP”. This game allows people to get into teams and come up with either a real or made up product and come up with it’s list of MVPs.

Since I’ve played the game several times, mine is always elevator improvement. I mean, 10% of my day is riding up and down elevators. They are extremely slow and out dated. They may have new parts if they were recently built, but that’s it. So, if I owned an elevator company, these would be my improvements:

1.) First, there is no arrangement. People crowd in first come first serve, or if there are gentlemen, they will let the Ladies on first. Inevitably, though, people that end up in the back get off first pushing through others and people in the front get off last.

Solution: Create some type of “grid” or system where those that are getting off on certain floors stand in a certain place. For example, in a 9 floor building, floors 1-3 get stand in the front 1/3rd of the elevator; floors 4-6 stand in the middle; and floors 7-9 stand in the back 1/3rd.

2.) While there is a weight limit to elevators it’s just a maximum. You could literally let 15 people on and still meet the limit; yet the elevator would be extremely overcrowded. Still, even if the capacity was full (not the weight limit) the elevator will still stop on floors assuming more people could get on thus wasting time.

Solution: Have a built in sensor that can tell when there are too many people on the elevator. Typically, any more than about 6 people is just too many. This would prevent elevators from stopping on floors when no other people would fit. It would still be a weight limit, so something like 6 (people) x avg. Weight ($170 lbs?).

3.) At the end of the day, and since I’m on the top floor, we play a game called “guess how many floors we stop on today before we can go home.” The point is, you can’t tell which floors an elevator is going to stop on. It’s really annoying!

Solution: Add an extra light panel allowing for different color lights on the buttons. If the buttons are normally orange, make them red when the elevator is going to make another stop.

4.) Elevators never seem to be there when you need them. There are three times of the day that there is excess use: Morning, lunch, and closing. It seems in the morning you have to wait 5 minutes for an elevator to be available.

Solution: Come up with a better algorithm for these specific times. For example, between 7 AM and 9 AM have all of the elevators automatically go down to the first floor vs. staying on the floor that was let out.You’d do the same for lunch but have the elevators evenly dispersed among floors, and at close you’d have a majority of them at the higher levels.

Posted in Uncategorized | Leave a comment

The Shortest Distance

“All complex systems that work evolved from simpler systems that worked.” – John Gall; Gall’s Law

We’re always looking for ways to get better and faster at what we do. I was thinking the other day about how in Geometry the shortest distance between two points is a straight line.  Look at this image I created:

Straight Line

“A” and “B” represent directions. Perhaps these include research, design, and implementation. In the middle there is a decision point. Either way, whichever direction you go, you’re not taking the most optimal path. Both “A” and “B” have right angles, or in this example, extra work.

Now look at this image:

Straight Line

A new path, “C”, has been established. This is the most optimal path to take. It has a shorter distance and is a simpler solution.

Strategy: Keep it simple! The best way to think of this is that by introducing “turns” we increase complexity. These are simple drawings and most projects would have many more factors to consider. Imagine the drawing with 10 or 15 angles. Keeping things simple, in order, and straight should be a goal. Obviously, scope changes and things happen, but overall the “keep it simple” model is what I always try to use.

Note: The shortest distance between two points being a straight line is only true on flat surfaces. Introducing depth changes this fact.

Posted in philosophy, project management | Tagged , , | Leave a comment

Priority Consideration Table

It can be difficult when trying to determine which project should get priority over another. If left up to business leaders, they would all get priority. I’ve actually seen instances where different business units could not come to an agreement on the top priority because no one would give an inch.

This table, if used, can actually prevent this scenario from even happening:

Untitled

This can be edited to better suite your organization. The key take away is that each project should be put up against something similar to this.

Strategy: Assign points to each category: 5 points for High, 3 points for Medium, and 1 point for Low. For each check box selected, add up the total points. The project with the highest point count gets the #1 priority tag. A project with two high priority categories trumps a project with three Mediums for example.

Posted in Management | Tagged , | Leave a comment

Nerf Wars!

Untitled

I was recently taking  a training class across town. During the class the instructor said, “Oh, at about three o’clock the room next to us will have a nerf gun war. So, sorry if they get loud.”

Most of us in the room looked at each like “what?”. Well, right at three o’clock, not only did we hear the nerf gun war but we saw nerf bullets flying by as well.

I have to be honest, this put a smile on my face. What a cool and fun way to let off some steam! I could tell this place was all about fun because they also had a ping pong table in the middle of the floor with plenty of balls and paddles as well as snacks and soft drinks freely available.

Boosting morale is something that companies should have high on their to do list. Happy employees should be a top goal – and is something that gets pushed aside due to “business as usual.” It’s not always expensive and difficult to achieve this. It may just take a few toy guns and a few minutes each day.

Posted in Morale | Tagged , | Leave a comment

Prioritization between projects

The most efficient way to prioritize projects and complete the work is one item at a time. See the below diagram for further explanation:

Prioritization

This diagram was borrowed from the book “Scrum” by Jeff Sutherland (which I highly recommend). The explanation is as follows:

1.) The top line shows that traditional project management follows the line of thinking that everything is important so work on every project at once. By working multiple things at once, you lose a % of time to context switching i.e. waste. For example, working on three projects at once means you lose 40% purely to waste and only leaves 20% per project. (Scrum pg. 91).

2.) The bottom line shows that Agile says to focus only on one project at a time. You do not incur waste due to context switching and you finish all three projects sooner than focusing on all three at once.

Strategy: Focus on as few items as possible. This goes along with WIP limits. As well, it may be necessary to enforce hard project limits per person i.e. no more than two projects at any given time.

Posted in Management | Tagged , , | Leave a comment