Why does Agile actually work?

I was first introduced to agile principles through Scrum. While intuitively they made sense, they also created more questions for me than provided answers, with one in particular: Why does this work?

At the time I was working with a group of embedded software developers in the automotive industry. My feeling was that, while a good start, Scrum was not the best possible solution for this environment. Thus I began my journey through the agile and lean world and found a lot of other stuff (e.g. XP, Pair Programming, Kanban,..). While all of these things made a reasonable amount of sense and answered my question of how it works, they still fell short of answering why.

Motivation Theory

The first thing that sent me off into the right direction was a hint from a colleague to check out Dan Pink and his talk/book called Drive. Based on research in motivation theory, it states that there are three main factors that define if people enjoy what they are doing. Those aspects are Autonomy, Mastery and Purpose.

  • Autonomy – the desire to live our own lives: Give employees the freedom they deserve. Instead of micromanaging, control should be exchanged for trust.  People should manage themselves and make their own decisions. Additionally, employees should work on projects they are interested in so they become their “baby” instead of tasks they are told to do.
  • Mastery – the urge to become better at something: Ensure extensive interaction between employees so they can learn from each other, understand the big picture and thus have the chance to grow every day. Additionally, the people should have the chance to experiment a lot and test their assumptions in order to improve their knowledge even further and create innovative and unique things.
  • Purpose – feeling like you are part of something bigger: Intrinsic motivation is achieved if employee beliefs are aligned with organizational values. Providing a big picture view helps people to understand why they are doing what they are doing. In addition, putting them in touch with the customer ensures that employees get feedback and recognition for their daily efforts.

Flow Theory

The next thing I stumbled upon, which provides yet another way of looking at these issues, is Mihaly Csikszentmihalyi’s flow theory.  According to this theory, people achieve a state of optimal experience and do their best work (and enjoy doing it) when they are completely focused and engaged on the task at hand. People can reach this state when they encounter a challenge that 1) requires a specific skill set that they possess, and 2) is neither too easy or to hard. In this sweet spot, the person is eager to solve the issue, sees progress in doing so, continuously learns and improves and is exhausted but happy when finished. Having accomplished such a task supports personal growth and overall well being.



The third clue I found was in a blog post which looks at this topic from the perspective of Neuroscience. The main aspects here are two regions in our brain: the prefrontal cortex (PFC) and the limbic system. The PFC is responsible for decision-making, higher-level thought, goal-oriented behavior, mental flexibility, and emotion control. On the other hand the limbic systems goals are to minimize danger and maximize reward by recognizing stored patterns, triggering emotions and informing the decision-making process. This is commonly know as “gut feeling”.


The interesting takeaway here is this: we make our best possible decisions and in turn do our best work when the PFC reaches its peak performance. This happens when we are in between a state of low brain activity (bored, distracted,..) and a state of high activity (stress). Due to evolution, our limbic systems will take over if either of these extremes are encountered in an effort to save energy. When motivated and empowered, however, the PFC will take over and people begin to think clearly and strategically, get into a good flow, and work towards reaching goals.


While much more research is needed to fully answer the question posed today, the psychological aspects described here provide a good starting point. A solid scientific foundation is important for building upon and continuously improving these principles.

Leave a Reply

Your email address will not be published. Required fields are marked *