Scrum Theory

The Scrum Guide – 2020

Scrum is founded on empiricism and lean thinking. Empiricism asserts that knowledge comes from experience and making decisions based on what is observed. Lean thinking reduces waste and focuses on the essentials.

Scrum employs an iterative, incremental approach to optimize predictability and to control risk. Scrum engages groups of people who collectively have all the skills and expertise to do the work and share or acquire such skills as needed.

Scrum combines four formal events for inspection and adaptation within a containing event, the Sprint. These events work because they implement the empirical Scrum pillars of transparency, inspection, and adaptation.

© 2020 Ken Schwaber and Jeff Sutherland

[This is not the end of the chapter, however, the commentary is added directly to the main sections.]

Commentary

Lean thinking is a fresh reference in the 2020 Scrum Guide, the authors did not mention it before. It is pretty appropriate to draw a parallel between Scrum and Lean. They have the same origins in the Japanese automotive industry, and both were applied in the Toyota Production System well before Western managers discovered them. The five Lean principles are also applicable to Scrum, furthermore, Scrum values are intrinsic to Lean thinking, too.

We should recall that Scrum – like other Agile frameworks and practices – is not a result of science or engineering. The evolution of Agile is all about experimenting. If something works, it would so without an established scientific theory. If something does not work, the weight of science would not help it either. Scrum theory is based on practical success.

The main rule of empiricism is that we should rely on what we can sense and prove as true. In contrast, while rationalism can help us to create a hypothesis and deduct conclusions from it, empiricism requires proof to accept the conclusions. Actually, this is how science works. For instance, Einstein deduced from the theory of relativity that gravitational waves should exist. In his time, it was merely a hypothesis coming from a model. A century later, scientists proved his idea with physical measurements.

Turning back our focus to Scrum theory, we can apply the same chain of thoughts to project management. While creating a project plan, we can only assume how long a body of work would take place, what its quality would be, and whether we managed to gather all requirements and dependencies. Our entire plan is hypothetical. Real-life events will prove whether it was complete or realistic. Therefore it matters what we can do when life proves our plan was somewhat flawed, which is always the case with complex projects. Let’s compare how the traditional approach and the Agile address this problem!

Traditional or Waterfall Approach

The traditional project management approach organizes the work as a sequence of one-off activities:

Based on its visual representation, it is also called the Waterfall approach. In the Waterfall approach, every activity is executed once. The consecutive activities are planned with the assumption that the dependencies will be delivered in time, and in flawless condition. Any findings with an impact on the already delivered pieces of work mean a serious problem. That is to say, the delivery process needs to jump back to a previous stage for a fix and flow through the waterfall again before it can continue to progress. As a consequence, any change request has a detrimental impact on the project.

It is good to know that ‘waterfall’ was never a definitive method or a scientific approach. It is just a parallel. Traditional project management proved its merits in large-scale construction works and it is based on common sense. What is more, the plan – execute – verify process is valid and foundational in agile software development, too, though in a fine-tuned way (described later). However, the traditional approach is literally not agile in software development. It is very rigid and gives little room for corrections. That is why in reality pure Waterfall never existed, many tactics and techniques emerged to mitigate its shortcomings. Nevertheless, traditional software development is definitely more rigid and fail-prone than its agile counterparts, therefore when we discuss ‘Waterfall’ in relation to Scrum theory it is not just a strawman argument.

Agile approach

One of the major advantages of Agile is the regular assessment of the product and the environment throughout the lifecycle of the product. Every iteration ends in a situation that is clear and gives a good opportunity to revise all our plans in the short as well as in the long run. The entire process is transparent, inspection and adaptation take place continuously.

In Agile, only the next iteration has to be planned in detail. The further an iteration is, the less detailed the plans are. This way the plans for the consecutive iterations can take into account the lessons learned in previous Sprints and can react to the changes in the environment. Thus, can rely on what is already known.

Benefits of the Agile approach

• Reduces waste: detailed plans for the far future are usually created in vain since by the time they are implemented, will be obsolete. The details of plans should reflect the uncertainty involved. (See the cone of uncertainty).
•  Reduces risks: a working solution is at the disposal of the organization from Sprint 1. Resources are not locked in: changes are encouraged at any time without incurring waste.
•  Improves the product: since the inspection is feasible and changes are welcome, there are more opportunities for finding good ideas and implementing them.
•  Time to Market is shorter, meaning the organization can start realizing benefits earlier.

The iterative, incremental approach requires a different attitude from the organizations and the professionals. That is why agility cannot be achieved at the project management level. It needs self-managing teams with committed and empowered team members and many techniques and tactics that support the short delivery cycles.

The Three Pillars

Previously the Scrum Theory chapter stated that the pillars of Scrum’s ‘Empirical Process Control’ are transparency, inspection and adaptation, as mentioned e.g. in the 2017 Scrum Guide. In fact, Scrum was never based on ’empirical process control theory’ because there is no such thing. Empirical process control is used in stochastic engineering processes and has nothing to do with complex teamwork. On the other hand, admittedly it can serve as an analogy to some aspects of project management and product development. Fortunately, this expression is not in the Scrum Guide anymore. Nevertheless, Scrum is still empirical and the three pillars are just as important as before, we are going to visit them in the next three chapters.

Transparency

The emergent process and work must be visible to those performing the work as well as those receiving the work. With Scrum, important decisions are based on the perceived state of its three formal artifacts. Artifacts that have low transparency can lead to decisions that diminish value and increase risk.

Transparency enables inspection. Inspection without transparency is misleading and wasteful.

Commentary

As the proverb states, knowledge is power. That is to say, transparency gives knowledge to the participants of an endeavor. For instance, access to the artifacts and reports, invitations to the events, and the opportunity to inspect the increment at the end of every sprint largely contribute to the common understanding of the goals and the status of a product.

In contrast, let’s evoke how typical projects look like in a traditional environment:

• A project manager handles a proprietary desktop Gantt-charting tool which is the single source of truth regarding who works (believed to work) on which task, when it is expected to be delivered, and what are the key milestones of delivery. In addition, only one person can update the project plan, and few have access to view it.
•  A large RACI matrix is maintained, and a detailed stakeholder communication plan is implemented to ensure people have access to information. The opposite may also occur: nobody exactly knows who should be informed about decisions and changes. As a consequence, sometimes important stakeholders learn about changes too late.
•  There are different versions of the truth for the management and for the development and business teams. Managers or even peers single out individuals for the blame of late delivery.

Scrum – if implemented correctly – largely eliminates these types of problems.

Before the 2020 changes, the Guide had an Artifact Transparency chapter that details artifact-specific transparency requirements. Even though it is not part of the Guide anymore, it is still worth reading.

Inspection

The Scrum artifacts and the progress toward agreed goals must be inspected frequently and diligently to detect potentially undesirable variances or problems. To help with inspection, Scrum provides cadence in the form of its five events.

Inspection enables adaptation. Inspection without adaptation is considered pointless. Scrum events are designed to provoke change.

Commentary

In this section, the Guide specifically mentions Scrum artifacts and the progress toward agreed goals as the subjects of inspection. There is a daily inspection during the Daily Scrum and an inspection takes place during Sprint Review when the stakeholders and the Scrum Team scrutinize the product and the increment. The Sprint Retrospective is an inspection of the processes and practices applied. The progress toward a project goal itself is also subject to inspection.

It is important to notice that this is just a sub-section of the Guide and there are further chapters detailing inspection.

Adaptation

If any aspects of a process deviate outside acceptable limits or if the resulting product is unacceptable, the process being applied or the materials being produced must be adjusted. The adjustment must be made as soon as possible to minimize further deviation.

Adaptation becomes more difficult when the people involved are not empowered or self-managing. A Scrum Team is expected to adapt the moment it learns anything new through inspection.

Commentary

Anyone can be an inspector within or outside of the scrum team. It depends on the subject of inspection and when the inspection takes place. The wording of the Guide reflects the origins of Scrum, so the text is very generic and applicable to engineering, production, software development, etc. In such a broad context adaptation is difficult to define. We may say that adaptation is the action to get the process/product back on track. The benefit of frequent inspection is that deviations from the desired parameters are discovered early. In consequence, the odds are good to quickly fix the problem.

Since the majority of Scrum practitioners work in software development, the commentary provides examples best suitable for them in the Events for Inspection and Adaptation section.

The following sentence deserves detailed analysis:

Scrum combines four formal events for inspection and adaptation within a containing event, the Sprint.

In order to avoid confusion: the four Scrum events have more specific goals beyond inspection and adaptation, but all of them should serve these high-level goals. How do these events foster inspection and adaptation? Let’s go through the list!

Sprint Planning

Inspection: In theory, inspection should focus on the Product Backlog and Sprint Backlog items and the feedback from the previous Sprint. Team members can receive clarification of the items from the Product Owner and they together can discuss the findings of the previous Review. The Retrospective should identify process improvement opportunities serving as input for Sprint Planning. In practice, teams may deal with less fortunate issues, such as leftovers and ad hoc tasks.

Adaptation: The Planning itself is adaptation. The Product Owner assesses the current state of the product and the items on the Product Backlog, and the team considers its capacity and past performance to make a forecast for the coming Sprint. They take into account the feedback from the Review and the findings of the Retrospective. The definition of “Done” guides the Development Team in assessing the effort needed to turn Product Backlog Items into Increments and to plan the Sprint accordingly.

Daily Scrum

Inspection: as the name of the event suggests, this activity serves to inspect the status of the Sprint day-by-day. This way, the team can follow up on the progress through different metrics and charts. For instance, the most popular example is the sprint burndown chart which represents the remaining tasks of the Sprint in the proportion of the planned tasks.

Adaptation: The Daily Scrum is a good occasion to synchronize the activities of the team members, communicate findings, ask for a remedy for impediments, notify the Product Owner if delivering the planned items became improbable, and level the burden across team members. The Daily Scrum is an occasion to update our plans in order to meet the Sprint Goal.

Sprint Review

The Review (often mentioned as a demo, however, the Review serves broader goals) is a great way to inspect the increment delivered. This is a good occasion for the stakeholders to take a look at the product, consider the status of the project, etc.

The goal of the Sprint Review is broader than just inspecting the Increment. It should also address how to progress with product development. Furthermore, even the way the team organizes the Review is subject to inspection and adaptation.

Sprint Retrospective

This event is mainly for inspection. The team inspects its own practices and discovers ways to improve its work. They may solve some issues right away at this event, thus adaptation may also take place.

It is important to point out that the above examples are just examples. Inspection and adaptation can happen in any form, at any time.

External Content: Lean Thinking, Empiricism and Scrum Theory

Let’s watch the below embedded YouTube videos to get diverse views on the topic.