Before we dive into Agile Project Management, it is helpful to look at how complex software projects were managed before it existed. We will also study the waterfall model, which was popular in the 1970's, and how Agile methodologies were inspired by shortcomings in the waterfall model.
The Waterfall Model
The waterfall model was originally defined by Winston W. Royce in the1970s. It is a software development approach that proceeds sequentially through a series of phases:
The waterfall model attempts to complete each phase before moving to the next but has built-in feedback loops which allow you to return to a previous phase if you discover problems. Progress flows from one stage to the next, just like a waterfall cascading down a hill! The process even looks like a waterfall. Compare the diagram above to the vector diagram of a waterfall below:
The Phases of Waterfall Development
Now that you know how the phases are sequenced, let's take a closer look at what should be done in each one:
Requirements Analysis - This is when all requirements for the system are gathered. This includes talking to the stakeholders and end users.
Design - Software architects take the information gathered during the requirements phase and design a structure which supports the required needs. During this phase, some user interface design may also be carried out.
Implementation - Software developers take all requirements and design documents and code the solution.
Testing - The internal testing team and the client both test the system. The client should validate that the system performs and behaves as expected. In practice, this step is often skipped. In that case, the client tests the system in the maintenance phase.
Maintenance - While using the solution, the client identifies any defects so that the system can be adapted to function as needed.
Pros and Cons of the Waterfall Model
Let's examine the advantages and disadvantages of the waterfall model:
Many design errors are identified and captured early in the project. This means that the relative cost of fixing them is much smaller at this stage of the project than it would be if you had written lots of code and then had to go back to make changes.
Measuring progress is easy as there are well-defined milestones as you move from one phase to another according to an estimated timeframe.
Accurate cost estimates can be provided - even for large projects. Toward the end of the design phase, you will know the plan and can, therefore, provide an accurate cost estimate for the entire project.
Change is difficult with the waterfall model. Let's say that you took two months gathering requirements, spent one month on design, and have worked for two weeks on implementation. Then you realize that the requirements are wrong. In that case, you have to go back to the beginning and re-do the Requirements Analysis phase. This may require considerable administrative overhead and not politically possible. The team members may resist the change even if they know the requirements are not fit-for-purpose.
Projects often run late with the waterfall model and the estimated delivery date is usually not met. This is because, by nature, the planning of large projects carries many unknowns. If you try to plan resources and costs for a nine-month project, then those costs and resource estimates will only be accurate if no requirements or circumstances change over the duration of the project. That is often not realistic.
The major downside of the waterfall model application to software development and building products is that it assumes that you know all the requirements up-front before you write a line of code. There are occasions when you will know all the requirements ahead of time, but for most web development projects, you will not. You may think that a certain feature is vitally important to your target clients, or that users will understand the interface you designed. However, until the design is tested and you receive feedback, you cannot know these things for sure.
Although managers of the 1970's and 1980's loved the waterfall model because of its logical flow, in the 1990's a new form of software development called Iterative Development came to prominence.
With iterative development, some requirements could lead to some design which would lead to some implementation, etc. While a team would be implementing a particular subset of the project, requirements could still continue on another subset, and so on.
If you want to compare iterative development to the waterfall model, let's apply each approach to building a doghouse. In the waterfall method, the team would gather all requirements, make a design, build the house completely, test it, and then show it to the client. Delivering everything (the finished doghouse) to the client at the end of the project characterizes the waterfall model.
Rather than one delivery, the iterative approach would have several cycles. At the end of each cycle, something of value would be delivered to the client. For example, imagine that your doghouse project was broken up into three iterations:
Iteration 1: Create a frame outline of the dog house in wood.
Iteration 2: Add the roof and walls to the wooden frame.
Iteration 3: Paint the house and add decorations, including the dog's name above the door.
The value of iterative product development becomes even more obvious if you look more closely at each iteration. Let's start the first iteration:
Imagine you've finished the wooden frame and you test it, only to find out that your dog doesn't fit in comfortably (or perhaps, in the meantime, you've adopted a slightly bigger dog 🐕). Once you put the frame in your small backyard, you will see how much space it takes up and decide that you want a smaller doghouse (but one that the dog still fits in). With iteration development, your team provides a deliverable to the client who then reacts to what has been delivered.
Now imagine that you are happy with the frame and you move on to the second iteration phase. You instruct the team to build the walls and roof. Now that the team knows that the size of the frame won't change, they can start with more confidence. Their focus is now on the walls and roof that they will deliver at the end of the phase.
During this process, let's say that the team discovers a problem with the roof. The interior of the doghouse is wet after a night of rain. In that case, the roof is leaky and must be replaced. Or, the team uncovers a different issue: adding the roof increases the width of the doghouse and the doghouse no longer fits into the intended location in the backyard. With the iterative method, the issue can be fixed right away.
The third iteration phase can begin because you know that the shape of the doghouse is good (Iteration 1 feedback) and the walls and roof are acceptable (Iteration 2 feedback). Now you are ready to varnish and paint the wood, treat the roof, add the dog's name, and even a toy bone above the door!
The beauty of the iterative approach is that you learn from each iteration and take that learning to the next. This ability to absorb learning means that you can adapt to misunderstandings, change of context, or new knowledge in a productive way.
The Birth of Agile Software Development
The popularity of iterative software/product development in the 1990's, in turn, motivated a new set of methodologies such as Extreme Programming (XP), SCRUM, and Kanban. These new methodologies were iterative in nature but added additional elements. They became recognized as effective approaches to building software and products that customers loved.
in 2002, some visionaries in the field of software development condensed what they had learned into a set of principles called the Agile Manifesto, which crystallized the arrival of Agile development practices into the world. In the next few chapters, we will look at the Agile Manifesto, discuss the principles of Agile development, take a quick look at Kanban, and then spend most of the rest of this course looking in-depth at SCRUM, which is one of the more popular agile methodologies.
In the 1970's, the waterfall model of software development was popular. It prescribed building in a set of phases (Requirements Analysis, Design, Implementation, Testing & Maintenance) and only moving to the next phase when the current phase was complete.
During the 1990's, iterative product development was born out of a frustration with the waterfall model and in particular, its need to define all requirements up-front as well as the inability to manage changing requirements. Iterative product development breaks a project up into iterations and delivers something of value to the client at the end of each phase.
In the 1990's and 2000,s, Agile methodologies (all iterative such as SCRUM, Kanban, and XP) were built on this principle and are very popular today.