How to take your Incident Response plan to the next level
‘Practice does everything’ – Periander (Often misquoted as ‘Practice makes perfect’)
You’ve gone through the work of creating an incident response (IR) plan, created some runbooks to deal with likely, known threats, and you’re feeling a lot better about the ability of your organization’s ability to detect a cyber incident in your network. But how do you know it works? How can you be sure the teams and individuals named in the plan know their roles and responsibilities, who to report to, or even where the IR documents are stored?
Think about when you learned to drive. First you had to study the rules of the road, and take a test to prove you understood them. Even with this knowledge, getting behind the wheel of a car on your own and driving would have been foolhardy. Actually, driving the car takes practice, usually with an instructor or parent to guide you during the process, and until you receive your license.
So, similar to how driving without the proper training can have drastic consequences, trying out your IR plan for the first time during an active incident can increase time to containment and remediation. Lack of knowledge regarding chain of command, roles and responsibilities, service levels, compliance and legal obligations, and communication plans can cause chaos during the response activities, and result in greater damage to systems or data, or larger financial or reputational repercussions.
How best to put rubber to the road then, and test your IR plan? The most common way is to hold a tabletop exercise (TTX). This activity involves planning a session where a scenario is presented to the IR team as a paper exercise around a meeting room table; although the “paper” is generally a PowerPoint slide deck. Does anyone use actual paper anymore? The make-up of your IR team will vary based on your organization, but generally will include representatives from the IR or Security Operations team, one or more members of each operational technology teams (Server, Network, Desktop, Database, Applications), and possibly some representatives from HR, Legal, Corporate Communications and Corporate Security.
A facilitator should be present to moderate the session, introduce the scenario, and add some complications or details as the session progresses. The facilitator will also lead the discussion and guide the team as they work through the possible incident from identification to remediation according to the IR plan. The session is closed with a “Lessons Learned” activity where participants are asked to discuss how well the steps of the IR plan performed during the run-through, and where the plan needs to be improved or modified to ensure a more successful outcome on the next trial or an actual incident. In addition, the attendees are asked to evaluate, without blame or judgement, how the team performed, and suggest where additional training or practice are required.
A paper TTX can be planned for a variety of audiences as well; from the C-level focusing on higher-level policies, decision-making and compliance topics, to business units targeting business continuity processes in the face of an outage, and of course, technical teams to practice specific runbooks or stages of IR. Exercises can target specific compliance topics including PCI or HITRUST-related incidents, or familiarize an organization with common threats such as ransomware or DDOS attacks.
Once your organization is comfortable with the IR plan by completing one or more TTX, the way to up the ante a little is to move away from just paper, and introduce a level of simulated attack to your exercise. This can be as simple as deploying a simulated phishing email or social engineering calls targeting a specific group of users or randomly at the entire organization, or involve a full-scale attack on your network by a trusted “red” team while your “blue” team defends.
A red team is made up of trusted internal or third-party attackers who engage in a live attack on your network; usually based on pre-agreed targets or methodology. A blue team is made up of internal and possibly trusted third-party network defenders, incident responders, forensic and malware analysts who actively defend your network against intruders, both real and simulated.
This type of exercise not only tests your IR plan, but also the ability of your organization to actually detect and respond to real security events, determine if chains of those events constitute ongoing malicious activity, and work to contain and eradicate the active threats.
Lastly, the simulated attack can be elevated in complexity again by conducting a blind exercise. In this type, the attack is still performed by a trusted red team, but knowledge of the attack is limited to very few personnel in your organization. This allows the reaction or non-reaction to the red team’s actions to be a true test of your IR team’s ability to detect and respond. While this can be very effective in identifying opportunities for improvement in your organization’s IR plan and general security posture, it can also be demoralizing if the attack is very successful. Therefore, it is highly recommended that this type of exercise only be attempted against a more sophisticated blue team that has gone through a number of paper TTX, smaller “known” simulated attack exercises, and perhaps even a few penetration tests. This allows the gaps identified in previous Lessons Learned sessions to have been remediated and the team to have learned over time.
Like any other policy or procedure in your document library, an IR plan is a living document and requires constant upgrades and modifications based on the changes in your environment over time, and the growth and maturity level of your security program. Practicing and training your plan on a regular basis using paper and simulated exercises is the best way to ensure it remains effective.
The old saying goes, “Practice makes perfect”. While it is doubtful that responding to a cyber incident in your organization will ever go perfectly, don’t let the first time you test your IR plan to be the day you need it to succeed the most.
This blog post was originally written for the Cisco Security Blog
