Watch Your Scrum!

Scrum is the most popular framework in software project management. However, introducing and operating Scrum isn’t always smooth. This article inspects a frequently overlooked problem area in Scrum: the relationship between team and management. Fraunhofer IESE as a leading research institute in software engineering has deep experience in software project management, including in the application of agile methods.

Why Scrum?

In software development, Scrum is an incremental framework for managing software projects in interdisciplinary teams. The increments of the Scrum development process are so-called sprints, which typically last one to four weeks. At the end of each sprint, the self-organizing team delivers a working, tested, integrated, and deployable software product that can be handed over to customers and stakeholders for feedback. This method counters the drawbacks of Waterfall-like approaches with their cumbersome definition and planning phases, which often fail to produce a suitable project organization or satisfactory result, utimately delivering what was planned instead of what is needed. In contrast, Scrum, as an agile incremental software development method, promises to unleash the development team’s full creative potential through self-organization, reveal misguided concepts early on, and give stakeholders and customers a timely but non-invasive influence on the development.

So, what’s the problem?

Success can become its own curse. The industry’s enthusiasm for agile software development in general – who doesn’t want to be „agile“? – and Scrum in particular has created profitable business opportunities for consultants and agile coaches. Many of these coaches are IT professionals who provide valuable advice, while others rather resemble Michael Küsters‘ „self-proclaimed ‚experts‘ … who claim to understand and be doing Scrum“ („The Scream Guide“). The combination of inflated management expectations caused by clever Scrum marketing, the self-interest of consultants to please their sponsors, and agile coaches who know the Scrum rulebook but don’t understand software development can result in Ron Jeffries‘ „Dark Scrum„.

What makes the Scrum framework especially appealing to managers in software development is Scrum’s systematic and periodic nature, with regular „rituals“, increments, and reviews. Transparency tools like Scrum boards and sprint burndown charts come as a bonus. However, these Scrum features are intended for the development team’s self-organization and feedback loop, and can create the false impression that Scrum is a perfect control instrument for management. Below, we will present how to spot dangerous Scrum abuse.

We need to be aware that Scrum is a popular software development framework, but not the only one. And we should be aware that part of its success relies on its appeal to management sponsors – but often for the wrong reasons. This leads us to the first myth about Scrum: „Scrum is the best way to develop all software.“

Ever heard of the small elephant?

Agile coaches love the story of the small elephant. In this tale, there’s the waterfall way to develop a full-grown elephant in response to a given specification. Of course, after a long development time, the elephant will inevitably end up being a Frankensteinian monster of a software.

In contrast, Scrum’s incremental approach produces a viable, functioning, and walking elephant in each sprint. The elephant starts as a baby with few features and develops towards ever more functionality over time. The many product increments allow customers and stakeholders to give timely feedback in case the development doesn’t move into the right direction.

In Scrum marketing terms, this allegory is a careful and effective choice. We all know that elephants are not assembled from parts, we know that they grow organically from small to big. The question many forget to ask: „Is every software an elephant?“ What if your software is a plane? Here the metaphor breaks. If you build a small plane in sprint 1, there’s no way that you can scale the tiny turbine or fuselage up to the next larger iteration without starting all over again. A plane refuses to grow organically; it needs to be assembled from complex modules. Don’t let yourself be fooled: Not all software is an elephant.

Are you serious about Scrum?

Before Scrum is imposed top-down on a software development organization, management should be aware that the advantages of Scrum come at a cost:

• Self-organized teams need more time for internal communication.
• The teams have to organize their work packages around fixed sprint cycles. This causes software increments not to be determined by functional criteria but by an arbitrary working interval. Forcing teams to deliver in fixed intervals creates overhead.
• Scrum works best if customers get to play with the sprint increment and give feedback. This requires an infrastructure, organization, and stakeholders willing to invest time and participate. If this participation isn’t secured, the overhead of sprint deliverables is largely for nothing.
• Management likes long-term plans. True Scrum requires not only the team to be agile, but also the roadmap – and in extension the management and the contracts.
• Scrum demands protection of sprint goals. Last-minute demands and short-term priority changes by management don’t work with Scrum.

Feature-heavy Software as a Service (SaaS) software with continuous integration and deployment is mostly well suited for Scrum development. In this environment, feature collections can often be neatly packaged in sprint increments and be directly delivered to the customer via CI/CD. However, not every type of software is so nicely suited to the Scrum framework, for a variety of reasons. Just a few examples:

• Limiting technical environment: Embedded software can’t be developed independent of the hardware. Hardware changes are difficult and expensive, so control software has to stay largely within the specifications.
• Fixed budget: Customer-specific software projects should be the prime example for Scrum development. However, customers typically don’t accept open-ended efforts in bigger projects. Mostly, this leads to detailed up-front specifications with little room for deviations.
• Initial investment: Framework-heavy software products often require long initial development efforts until any meaningful feedback can be collected. This type of software is more of a plane than an elephant.

Even in these cases it is still possible to work in a „Scrum-ish“ way, but efficiency and effectiveness will decrease compared to a Scrum-friendly environment. Top IT consultants will be able to advise whether to employ Scrum, Kanban, Scrumban, or other methods in a certain environment, but this requires know-how in software development – something people won’t acquire with just a Scrum certificate.

How it feels when Scrum is done right

In classic software project management, developers await specific directions on what to do and when to do it. In Scrum, developers are incentivized to take on more responsibility and express their points of view, all aimed at a higher degree of self-organization. In this approach, the team decides which load of tasks to take on for the next sprint and how to implement it. In doing so, the team commits itself to the sprint goal announced by the Product Owner and the team is motivated to achieve it. Although this autonomy might be unsettling for management and the Product Owner at first, the team’s velocity can be measured over time and provide predictability for planning purposes. When Scrum values are respected, the management supports and motivates the team, which makes developers realize that they have direct influence on their work. By seeing the fruits of their decision-making, collaboration, and dedication, each completed task feels more rewarding.

At first contact with Scrum, developers may find the meetings tiresome or not productive. The short dailies seem to provide only little information, but they strengthen collaboration by raising awareness of the project’s progress and by providing an opportunity to seek advice or address important topics relevant for the whole team. Plannings and refinements are a numbing experience until developers discover that they can be creative and directly influence and improve their tasks. Retrospectives, when done in a really safe environment to encourage the team to be open and truthful about what was good or bad in the last sprint, are essential for optimizing cooperation, workflow, and social interaction.

When the whole team understands Scrum and participates in it, constant communication becomes a habit, continuous improvement becomes natural, and, as a consequence, the collaborative work becomes smoother.

Warning signs for Scrum abuse

Scrum can’t prevent bad leadership style like micromanagement or authoritarian, toxic rule. Through its systematic nature, Scrum can amplify such abuse and turn it into a regular series of pain for the developers.

But even apart from such toxicity and with the best of intentions, Scrum implementation might be inadequate due to deficiencies in the management-team relationship. The result is an illusion of Scrum that fails to deliver the intended benefits and creates frustration among all those involved.

On an organizational level, one main problem can be the management’s failure to realize that Scrum requires them to change their behavior and expectations, too. The warning signs are:

• Uniformity of sprints, rituals, Scrum methods, etc. is imposed on all development teams in parallel to suit management – irrespective of the teams‘ individual needs.
• Management expects the same detail of long-term planning and forecast as with Waterfall practices – not realizing that Scrum strives for the best, but possibly yet unknown solution.
• Management expects that it can continue changing the development teams‘ current priorities or imposing new, urgent tasks at any given time.

Another main source of problems is a lack of trust in the development teams‘ work ethics and judgment. These are some of the warning signs:

• Dev team gets scolded by management for not meeting the sprint goals. There’s no better way to discourage a motivated team that intended to tackle a challenging mission.
• Dev team gets scolded by management for not planning enough work in the sprint. Either the teams are self-organized or not. In the latter case, Scrum will not work properly.
• Management is heavily invested in the team’s metrics, e.g., sprint burndown charts or velocity. These metrics are for the team’s self-optimization, not for identifying low performers or comparing different teams.

Other warning signs are on the psychological side:

• Developers remain passive and wait for instructions instead of being proactive and presenting their ideas.
• Developers perceive Scrum as a straightjacket instead of a liberating experience.

How to break the cycle

The presence of the above warning signs is a reason to scrutinize the management-team relationship. If the team concludes in honest assessment that the management is the problem, the next part is difficult: telling those who hold power over you that they need to change.

How to deal with this issue varies according to the category of warning signs:

• Organizational-level warning signs: This indicates an information deficit about Scrum itself in management. In a mild case, management might listen to a group of developers or the Scrum master. A heavy case requires some sort of mediator.
• Lack of trust: The remedy is to have the Scrum Master and the Product Owner increase the transparency of team efforts and impediments. Management needs to understand why the speed isn’t what they had hoped for.
• Psychological warning signs: The reasons for discontentment in the team need to be clearly identified and named. Again, only in a mild case can the help of a mediator be skipped.

The mediator should be an experienced external consultant. Why external? Because management tends to rather listen to external criticism rather to internal criticism. In any case, the mediator should have IT know-how in order to judge the overall setting and determine whether Scrum is the correct method in the first place. This means that the mediator should be able to understand the difference between an elephant and a plane.

However, management needs to be convinced to put money into external assistance. The level of dissatisfaction – possibly on both sides – could be the argument to seek help for Scrum workflow optimization. Of course, the team needs to draw the mediator’s focus to the problematic management behavior.

Success is not guaranteed, but not resolving Scrum abuse is unsustainable in the long run. A positive workplace is worth the fight.