You’re probably wondering where to start when it comes to driving the SOC’s continuous improvement. That’s completely normal!
As is often the case, we need to look at the big picture to manage this process. That’s exactly what we’re doing in the final chapter of the course.
Learn From Incident Management With SIRP
At the end of each incident, you need to carry out a post-incident review (PIR). As mentioned in the “Close the Security Incident” chapter, this PIR should identify:
what worked well.
what didn’t work well.
what was missing.
what was done incorrectly.
mistakes to avoid.
etc.
To properly implement continuous improvement, each of these areas for improvement need to be transformed into a clear action plan with:
a creation date.
a clear, practical action.
the criteria for assessing whether the action has been completed.
related incidents.
a person in charge.
a target completion date.
| Example 1 | Example 2 |
Clear, practical action | “Deploy EDR in a legacy network zone” | “Implement middleware maintenance monitoring” |
Criteria | “EDR deployed on a list of 20 machines” | “Maintenance schedule added to SOAR observables” |
Related incidents | An incident on the security perimeter where the EDR was not available to conduct the investigation | The list of false positives related to maintenance |
A manager | Contact the EDR deployment project manager | An analyst |
Measure the Performance of Your Processes Over Time
Use SIRP to monitor the progress of these plans! The SIRP is already the place where you communicate within the SOC and with other teams to organize resolution actions. These plans to improve the SOC must be monitored and communicated in the same way as resolution actions.
To track the SOC’s improvement, you’ll need to select the metrics you want to monitor. For example:
The rate of false positives: the number of alerts identified as false positives after classification or investigation.
The rate of false positives investigated: the number of investigations resulting in false positives that were not identified during the classification step.
The rate of true positives: the percentage of alerts reflecting an actual incident.
The Mean Time to Detect (MTTD): the time between the start of a confirmed security incident and the creation of the associated alert in the SOAR.
The Mean Time to Respond (MTTR): the time between the start of a confirmed security incident and the closure of the incident.
Monitor Risk Coverage
Measure the Risk Analysis Coverage
There are also useful metrics that exist for taking context into consideration.
Remember that your job is to reduce the risks facing the organization. These risks were defined in the risk analysis outlined in the first part of the course. Have all these risks been detected?
Measure the Coverage of the MITRE ATT&CK Matrix
What actions have been detected beyond the assets covered by the SOC?
You’ll want to refer to the MITRE ATT&CK matrix. In the first part of this course, we explained how detection rules are used to determine whether an attacker is using a technique from this matrix. So, ask yourself which rules are detected by the SOC and which are not. Is this coverage consistent with the priorities defined in the risk analysis?
Use Audits to Measure the SOC’s Relevance
Once again, to get a more relevant overview, you can call on security teams to test and audit your SOC. Specialized teams can conduct penetration tests to simulate real-world attacks.
We’re talking about Red Team missions where the aim is to test whether a specific goal is achievable by any means possible. Examples include access to a manager’s email or a customer password database. Generally, the aim is to test the capabilities of the SOC by seeing if the auditors can achieve their objectives without being detected. This can include phishing, the use of infected USB drives, and other techniques.
Conversely, the goal for the Purple Team is to support the SOC. Here, too, auditors have to simulate a real-world attack by any means possible. The mission is organized in successive stages, so that the auditors playing the attackers (Red Team) communicate with the SOC (Blue Team) at the end of each stage to understand what the attackers have done, what they should have detected, and how.
Leverage Your Metrics
What can be done to improve the various metrics measured?
You can improve the MTTD and your coverage of the MITRE ATT&CK matrix by improving the detection rules.
Increase the security perimeter of the SOC by continuing to add new assets under the supervision of the SOC.
Reduce your MTTR by improving your procedures and automating as many actions as possible, as mentioned at the beginning of this section.
Reduce the rate of false positives by implementing post-incident feedback and investigating detection errors.
You can also improve the information available to you during the classification step.
Choose Your Battles and Define Your Objectives
All of these projects can feel overwhelming because there are so many things to do. So, how do you prioritize them?
Set goals for yourself! These goals should be realistic. You won’t be able to work on all these metrics at the same time or achieve a perfect SOC in just a month!
Just don’t lose sight of your assignment. It’s not about improving your numbers; it’s about protecting your organization’s business. What’s the priority for this assignment?
To help you define your goals, ask yourself the following questions:
According to your risk analysis, what is the most significant risk that is not covered or not adequately covered?
Which phase needs to be improved based on the most recent incidents?
If you are a complex organization, are your response processes effective enough?
If your SOC is relatively new, how can you improve detection and coverage?
In short, monitor the evolution of metrics over time to understand what’s most effective!
Communicate Constantly
To ensure that your improvements are sustainable, you need to communicate clearly, collaborate with other teams, and share your victories!
Communicate Changes
Inform the organization about the actions you are taking. This will allow you to support the required changes. Use past incidents as an example to communicate what you do:
Risks that were avoided
How the measures you’re implementing reduce these risks
What the SOC is doing to ensure that other teams keep working as usual
Emphasize the risks. That’s the purpose of the SOC—and your mission.
Transform Business Teams Into Allies
Other teams often view security as something that slows them down or even prevents them from doing their job.
The good news is that this viewpoint is easily corrected by constant communication about the work you do. Through effective communication, you can anticipate common problems that certain business team practices may cause for the SOC and, conversely, problems that improved security may cause to business teams.
You can overcome opposition between business and security. After all, the more that business teams know about the SOC, the more likely they are to help you investigate and detect incidents through their knowledge of the security perimeter.
To round out this chapter, watch this interview with Guillaume and Raphaël to learn everything you need to know about continuous improvement.
Share With the Community
When it comes to detection, there’s no real competition since everyone wants to protect themselves against a common enemy. Helping each other is essential!
Don’t forget that other SOCs are working on the same issues as you. Share your challenges and processes with this community.
Look online for rules and tools. You can even share your own tips on a blog or GitHub for feedback!
Monitor your threat intelligence with regard to vulnerabilities, as well as tools and new techniques.
Follow detection professionals on social media, like Twitter or LinkedIn.
The Adventure Continues After This Course!
Now that you’ve completed this course, you know all the essentials about security incidents, detection, and response!
You now know how to assess the risks the organization is facing in order to design an appropriate detection system. You have what it takes to organize and implement this detection.
You know how to respond to alerts and investigate in order to understand the incident. You also know how to respond to emergency situations, limit the spread of the incident, deal with the damage caused, and address the root cause to achieve a long-term resolution.
And finally, you know all about the role of the SOC and its interactions with the organization and the various detection agencies. You can contribute to—and even organize—its growth!
Let’s Recap!
To improve the SOC’s capabilities, take a critical look at past incidents by analyzing the data in SIRP and setting goals.
Improve your mean time to detect (MTTD) and mean time to respond (MTTR).
Specialized teams can conduct penetration tests to simulate real-world attacks. These penetration tests can take the form of Red Team or Purple Team missions, the difference being that the Red Team evaluates your detection at the end of the mission and the Purple Team continuously evaluates it.
Use SIRP to draw lessons from incident management. Identify the mistakes to avoid, the processes that can be prepared in advance, and the tools that were missing.
Constantly communicate with business teams to anticipate current problems. The more you communicate, the more they’ll be able to help you detect incidents at an early stage.
You’re now ready to join a SOC like the one at Méditronique. The ball is in your court!
