The agile development methodology is nothing new these days. Since its inception in the spring of 2000, it has proliferated through the industry like a wildfire through the dry brush. Every one of us has heard of it and most of us are practicing it. Several good books and articles have been written (Scrum: The art of doing twice the work in half the time by Jeff Sutherland, for example), new specialized jobs created (Agile coach, for example), YouTube videos recorded, and talks given. Despite all of that, there are visible inconsistencies in Scrum implementation, even within the same organization. It takes focus, attention to detail and periodic refreshers to run Scrum right. Let’s face it – Scrum doesn’t come naturally to humans. Result-oriented teams tend to shift their focus from the process to the product over time, which causes them to drift away from the Scrum principles. In the end, the team and the product take the hit. I was inspired to write this post as a refresher for myself and I hope a few fellow engineers will find it useful too.
The roots
Scrum originates from Toyota Production System that achieved higher quality and lower manufacturing time per car than GM, Lexus, and BMW. In fact, they did that with the workers who had been fired from the GM plant – they rehired them back and only changed the management and the process.
Scrum very closely resembles the Scientific method – a closed loop that starts with an observation, that leads to a question, followed by a hypothesis, that is tested in an experiment, which is then analyzed and conclusions are drawn. Scrum has the “OODA” loop that contains 4 steps Observe ⟶ Orient ⟶ Decide and Act. Looks similar, doesn’t it?
If you think about it – it isn’t a rocket surgery. There are plenty of real world examples that use the scientific method:
- The car navigation system runs the loop a few times a second. The unit “observes” the GPS coordinates of the car, “orients” itself relative to the destination, calculates the route and “decides” whether it needs to stay on the current path or take a detour, and “acts” announcing a route change or staying silent.
- A child playing with lego blocks uses the OODA loop to build a masterpiece. She “observes” the structure she produced after every few blocks, “orients” herself relative to the structure she imagines, “decides” on the next few blocks to take, and “acts” by taking the blocks and stacking them.
Business justification
Typically, the vast majority of customers use only 20% of product features. The rest of the features are almost never touched. Economists refer to this phenomenon as the law of diminishing returns. Yet, companies hold the product release until the last developer finishes coding some esoteric feature. Obviously, it costs money and time to the manufacturers. It also causes customer dissatisfaction who have to wait longer to get their hands on the product. Did you know you could change the image print order on your Canon DSLR? Yeah, 99.9% of other customers didn’t either.
When it comes to large projects that take years to build the situation is even direr. By the time the entire requirements set is fulfilled and the contractor delivers the project, the extracted value can be much lower than anticipated because original requirements are no longer relevant due to the passage of time. Imagine the magnitude of waste if someone decided to build a complex document processing software using Adobe Flash in 2018.
Scrum delivers small bits of value into the customer’s hands at regular short intervals. The important part is delivery into the customer’s hands for the OODA cycle to take place. When customers get their hands on a barely functioning product, they can immediately provide feedback on the feature set. The team can improve certain features and completely cut the others to focus engineering resources on important capabilities. The final product may look very different than initially anticipated at the start of the project and that’s by design. The research and development cycle can also be cut short the moment most useful features are done. No need to keep building the remainder of the product if no one will use that.
Philosophy
The “why” and the “how” of Scrum are usually lost in translation in organizations that practice it. Organizations follow the process mechanics but lose sight of the reasons “why” certain things are done a certain way. If there’s one thing you take away from this entire post it should be this paragraph. Scrum is built upon ideas of empowerment, continuous learning, and waste elimination.
Empowerment
Every participant of the scrum process is empowered to perform their roles to the fullest, regardless of their titles in the organization.
- The team is empowered to stop the production process and swarm a problem (called “impediment“) that gets in the way. This is analogous to any worker on Toyota’s plant halting the entire production line if something didn’t go as planned. Management would recognize and reward such behavior.
- The scrum master is empowered to protect the team from interference and work injection. She can terminate the sprint if the product owner or management attempts to influence the scope after the sprint had started.
- The product owner is empowered to determine the priority of user stories that go into the backlog exclusively based on her market insights and expertise.
Within the scrum team, all participants are equal, regardless of their positions and titles in the organization. When they join the scrum team, they check their ego and their badge at the door. This means that any team member can pick up any work from the backlog. On the ground, such an approach means that junior engineers in the team can work on complex tasks, while senior engineers pick up trivial/grunt work.
If you are paying close attention, you may have noticed that management has no role to play in the scrum team. Their job is to provide tools and resources to empower the scrum team to perform effectively.
Continuous learning
The scrum team is evaluating their success not by whether they hit specific production dates or not, but rather by velocity trend. Velocity is a metric that represents how many story-points the scrum team has burned per sprint. A story point, in turn, is a measure of effort a scrum member must put into the task to complete it, not the time it takes to complete the task. Successful scrum teams demonstrate a continuous increase in velocity sprint after sprint. This means that teams increase mastery of their craft sprint after sprint and are capable of delivering more complex user stories.
Waste elimination
The idea is to remove obstacles as soon as the team identifies them. For example, if one of the team members needs to configure their development environment, it would be wise for the whoe team to swarm and configure it quickly for that person. Or perhaps there is a build break in the master branch – the team should swarm and fix it before continuing development work.
It may not be practical to address some impediments on the spot. If they don’t block the progress it may be appropriate to keep track of the list of those impediments and schedule them for the subsequent sprint. However, it is important to drain the impediment backlog consistently. It is not enough to identify inefficiencies and leave them there for posterity.
Roles
We have touched on the 3 roles in the scrum team already and most people understand general delineation between them. For completeness, it is worth taking a deeper look at each one of them.
Development team
This is a team of people, with special skills in the field where the development or manufacturing process takes place. This is the group of people that will be responsible for the final product. It is important for scrum teams to be self-sufficient – meaning, they should include representatives from most fields that will be necessary to deliver the product. In the software engineering field, this team could include developers, testers, UX designers, researchers, and security assurance, for example.
There has been plenty of research on the topic of ideal team size, which we won’t be covering in depth. Here are a few prominent influences in this area:
People are most effective working in groups of 5-9 people. In larger teams, people tend to cluster around interest areas and mesh communication disappears. Also, with the larger team, it takes more time to go through rhythms (planning, stand-ups, reviews, etc.)
Responsibilities of the development team:
- Evaluate user stories for meeting the definition of ready and reject those that do not meet
- Cost user stores in terms of effort and assign story-points
- Pick user stories from the sprint backlog and deliver functionality that meets the definition of done
- Perform SPIKEs to answer specific questions
Product owner (PO)
The product owner is a subject-matter expert in the field where development takes place. This individual doesn’t need to have technical expertise in the development process. She must be able to describe functionality clearly such that the development team can reason about the work at hand.
The product owner does not have direct authority over the scrum team or the scrum master. She does not manage people or the process – her purpose is exclusively the product. She is connected to customers, understands their business needs, and can showcase new functionality after every sprint.
Responsibilities of the product owner:
- Populate and maintain a clear product backlog of user stories for many sprints ahead
- Determine the priority of each user story in the product and sprint backlog
- Provide a clear definition of ready (DoR) and definition of done (DoD) for each user story
- Guide the development team during the sprint as questions come up
The product owner is a full-time job. One can’t be an effective product owner if they spend less than 1-2 days a week on the product backlog refinement. This is a proactive role, looking ahead of the current development effort, onto what’s next. The product owner determines when the product reaches the “good enough” stage fulfilling 80% of the customer value.
Scrum master (SM)
The scrum master is a team captain. She facilitates the process and ceremonies and is responsible for smooth execution. An individual in the scrum master role does not (should not) have direct authority over the scrum team, like a direct manager. Instead, she should be coaching the team and influencing without authority. If you think of the team as a car engine, then the scrum master would be the mechanic.
Responsibilities of the scrum master:
- Organize and run the rhythms for the team
- Quickly remove impediments
- Coach team members to do their best work, encourage and inspire
- Protect the team from external influence and interruptions during the sprint
- Communicate execution metrics to organizational leadership
Ceremonies
Each of the ceremonies in scrum requires a dedicated paragraph to cover, which we won’t be doing in this post. We will simply list all scrum ceremonies (rhythms) to cover the ground.
| Ceremony | Purpose | Frequency |
|---|---|---|
| Team agreement | When a new team comes together it helps to get everyone to agree to a set of rules for everyone to follow | 1 hour, once per scrum team |
| Backlog refinement | Backlog must be full of user stories, sorted by customer value for a few sprints ahead | 1-2 hours, weekly |
| Stand-up | Get the whole team together to discuss work in progress and impediments. No design discussions. | 15 minutes, daily |
| Sprint review | This is a product-focused review of the progress made in the sprint | 1 hour, end of sprint |
| Retrospective | This is a team-focused review of the execution – what went well and what needs to be improved | 1 hour, end of sprint |
| Sprint planning | Populate the next sprint with the user stories that are costed (pointed) and well understood | 1 hour, before next sprint |
| Sprint demo | This is an opportunity to demonstrate new features to the stakeholders and customers | 30 minutes, end of sprint |
Daily stand-up
This is the most frequent of the ceremonies and the one that sucks the time like a blackhole. It is absolutely feasible to fit the stand-up into 15 minutes every day, if team pays attention and discusses only relevant items. For stand-up, each team member should cover only 3 things:
- What did you do yesterday?
- What will you do today?
- Are there any impediments to the progress?
It is up to the scrum master to keep the team on track and shut down design discussions during stand-up. It is very natural for people to jump into details, but they should resist that urge. Stand-up is a status meeting. The team can determine the need for a follow-up design discussion during stand-up.