DevOps is often associated with the acronym C.A.L.M.S., which represents the methodology's five pillars as follows:
Culture
Automation
Lean
Measurement
Sharing
In this chapter, we'll learn what each of these words means and which practices they involve.
Culture
DevOps is designed primarily to resolve communication problems and issues relating to cross-team responsibilities. This makes DevOps similar to Agile methodologies, where various teams such as operations, testers, designers, developers, project and product managers, and anyone else whose skills are required, are involved in the process to deliver a quality product.
This then results in a product-oriented team, with all the skills required to deliver the best application and experience possible, shifting away from distant teams grouped into silos based on skills. These teams work towards a common goal and share the plan to get them there. One way to create this team spirit could be to include the Ops team in daily stand-up meetings, where they can share any issues to be included in future sprints. Equally, the Development team could ask operations to help them resolve any problems they're experiencing.
Shared practice between developers and operations staff makes deployment and problem resolution so much more efficient. This process is known as cross-pollination. A good way to tell if DevOps culture has been adopted is to ask questions such as:
Are problems resolved by the entire product team?
Do all team members share a common vision?
Are sprint retrospectives focused on resolving problems rather than assigning blame for an error?
If you can answer yes to these three questions, it's highly likely that your team has adopted the DevOps culture! This is what we call the culture of shared responsibility.
I often say that team-building activities with the Dev and Ops teams is a useful DevOps practice, because it reinforces team spirit and the feeling of working toward a common goal. It’s also a great way of getting to know people better. That way, if a problem arises, it's easier to go and see the person and work together to resolve that problem rather than raising a ticket in some tool or other and waiting for someone to deal with it.
Automation
Let’s have a look at the advantages of automation.
Deploy more frequently
With automation, applications can be deployed more often. This requires the deployment process to be reliable. Traditional organizations tend to carry out deployments a few times a year, while companies like Google, Amazon and Facebook—which are often held up as examples of automation—can deploy into production up to 17 times a minute.
There are a number of techniques to ensure the deployment process doesn't introduce bugs or break any existing functionality. One example is Blue/Green deployment, in which two servers are maintained: one "blue" and one "green". At any given time, only one of these servers is handling public requests, acting as a production environment (e.g. the blue server), while the second one is used as a staging environment (e.g. the green server) to which changes are deployed. After successful testing of the latest deployment on a green server, the public requests are re-routed to it, making it a production server. Using this deployment method allows for quick roll-back to a previous state if anything goes wrong.
Configure Your Deployments Automatically
Configurations can be managed automatically by tools. In what we’d call traditional organizations, developers often write an implementation plan in a document that explains to production teams how to deploy and configure the software.
Because of the wall of confusion and the lack of interaction between the Dev and Ops teams, the configuration parameters are often sent via email or written in the implementation plan. This makes it very easy for errors to occur when the application is deployed, as you might miss a step in the process or forget to run a script. You could end up with a system that doesn’t work properly or even at all. And because a human is involved, it’s often difficult to document the configuration parameters in a way that ensures the application is correctly configured.
All of these problems lead to the situation we mentioned earlier, with organizations only deploying a few times a year because they're worried about breaking the system.
Production changes are often carried out outside of normal working hours or at the weekend, to avoid affecting users. However automation means that changes can be deployed during normal working hours, to reduce stress for Ops teams.
Create Environments On Demand
Environments can be created on the fly. By using Infrastructure-as-Code tools, you can create on-demand test and deployment environments for your applications, reducing time spent waiting for non-production environments to be made available.
For example, instead of waiting several days or even weeks for a production team to provide an environment for you, you can create one almost immediately (well, in a matter of minutes). You’ll then be able to create as many environments as you need to facilitate testing and also to support the various Agile ceremonies.
These environments are codified and placed under version control, which makes it easy to quickly recreate an identical environment. To save money, it’s recommended to completely uninstall the environment once it's no longer required.
In the large corporation where I worked, by using Infrastructure-as-Code, I was able to create identical copies of various types of environments while saving the company money. There used to be an environment that was permanently available throughout the month for testing, and it was therefore invoiced for the full month. While this environment was being used, it was unavailable for any other types of testing, which is a problem when different developers want to test different functionalities at the same time.
By using Infrastructure-as-Code, I could not only create three identical environments, delivering them to developers for their testing purposes, but also save money by immediately uninstalling any environments that were no longer needed.
Test more frequently
Tests can be run more often. Even when we use techniques such as Test-Driven Development or Behavior-Driven Development to optimize our unit testing, where the speed of execution means that we can obtain results in just a few minutes, sometimes we need to focus on certain types of testing, such as performance testing, load testing, or end-to-end testing.
These types of testing require an environment that is configured to simulate production environment constraints and that includes the full chain of inter-connected applications. As tests and infrastructure versions are aligned with the versions of code, any change to one of these three elements (test, infrastructure, code) will automatically rerun the end-to-end test process. In such a setup, it's easy to run the full regression test in just a few minutes, or hours if the end-to-end chain covers a high percentage of the functionality.
Lean
Avoid Waste With Lean
There are seven types of waste:
overproduction
waiting
transportation
over-processing
inventory
motion
defects
In terms of DevOps, Lean is concerned with delivering added value to the end-client (the general public in the case of an app used by the outside world), while minimizing elaborate, costly processes that don't add value. In a way, Lean when applied to DevOps is similar to Agile in the sense that we’re interested in continuous improvement and acceptance of errors. The Japanese word Kaizen summarizes this way of thinking. It literally means "change" (kai) and "better" (zen). But make sure you don’t cut corners on application quality by skipping any of the continuous integration steps to save on environments!
Understand the Value Chain Using the VSM
Looking back to the adoption of DevOps at Contoso, the VSM revealed that creating and providing environments was the longest step in the process, with waiting times of up to several weeks. While waiting for the environments, development was paused and deployment delayed. By using Infrastructure-as-Code, this waiting time was reduced from several weeks to a few hours, or even a few minutes. Hopefully, this gives you a good idea of why it's useful to create a VSM in your organization, as well as to use Lean processes in your day-to-day work.
Someone with a DevOps mindset will be able to find opportunities for continuous improvement almost everywhere they look. For example, setting up retrospectives or postmortems is a good way to improve processes and the way team members work together, by identifying what didn't go so well during development or deployment and ensuring it doesn’t happen again.
Errors are inevitable! To quote Werner Vogels, CTO of Amazon, “Everything fails all the time”. It could be a server that reboots at inconvenient times, a network that has lost connectivity or just an implementation that didn’t run smoothly due to human error. In DevOps, it's important not to blame individuals for deployment issues, but instead to encourage the team to talk openly when an error occurs so that it can be pinpointed, its root cause identified and a process put in place to ensure it doesn't happen again. Some call this process “building antifragility”, a term coined by Nassim N. Taleb.
Measurement
Fortunately, there are many tools available on the market to measure the performance of an application or transformational process. For example, customer conversion rate is a KPI often used in online marketing to understand how well a marketing campaign has performed.
Let’s look into Amazon as an example. The website collects and stores a huge variety of data, which is used to measure if newly deployed changes in the software brought improvements compared to previous versions. For example, did the change bring about an improved conversion rate? It's important to understand that this KPI also has a direct impact on business. Just 100 milliseconds of delay when displaying the Amazon homepage will see the company lose 1% of sales.
A number of different KPIs can be used:
How long did it take for a new feature to go from development to implementation?
How often does a recurring bug appear?
How many people use the product in real time?
How many users have we gained or lost in one week?
These indicators are also useful when prioritizing backlogs with business stakeholders.
For example, if a Product Owner can use factual evidence to demonstrate that a business unit has spent $100,000 on a particular feature that is only used once a month, wouldn't it have been better to spend this money on a feature that's used more often, saving processing time within the application?
We’ll see in the next chapter that these measurements are very important when defining KPIs such as SLAs (Service Level Agreements).
Sharing
If developers start to understand how to optimize operations and get to know the role of operations engineers, they’ll be more inclined to help those who actually run the applications. This doesn’t mean that every developer needs to be a brilliant system operator. But it does mean that operations engineers and developers must work alongside each other throughout the application’s full lifecycle.
Teams that adopt the DevOps approach often have a pivoting role. Developers resolve problems for end-users but also get involved in resolving production issues. They react to urgent issues, create patches where a problem requires a quick response, and fix bugs raised by users. They get to know how their application is used by end-users on a day-to-day basis. By making themselves available to the Ops teams, developers build a trusting relationship with them, fostering a spirit of mutual respect.
Let's Recap!
So now you know more about DevOps and its five pillars. Let’s summarize them again:
Culture: DevOps is about an organization’s culture between Dev and Ops teams, rather than a set of tools or role-specific practices. Dev and Ops teams must work together with a common vision towards shared objectives.
Automation: everything that can be automated should be automated! This is achieved using planned, frequent deployments, with test environments being created on demand and automated testing.
Lean: within the DevOps methodology, focus is on value and reducing resource waste. The Value Stream Map can help with this, as it's used to understand your value chain.
Measuring performance: you need to be able to measure everything you do so that you can automate and optimize. Setting up relevant KPIs to guide your deployments will enable you to do this.
Sharing: in DevOps, Dev and Ops teams spend time together and share responsibilities. This is at the core of the DevOps approach.
In the next chapter we’ll look at good and bad practices for implementing DevOps in an organization.
