By Paul Lee, Yuri Kramarz and Martin Lee.

Adversaries are always growing their capabilities and changing their tactics, leading to a greater number of incidents and data breaches. This is supported by organizations such as ITRC who reports that the number of data breaches in 2021 is already greater than that of 2020. This is why defenders must become proactive, not reactive. Many forms of traditional protection are reactive, like host-based antivirus, firewalls and secure web gateways. An overlooked aspect of cybersecurity is the proactive planning and policy that goes into defense.

Having a policy that defines how an organization can respond to cybersecurity incidents, and a plan on how to deal with those incidents can play a major role in resolving them with minimal cost and downtime.

This is especially important in a high-stress scenario like responding to an active ransomware infection, supply chain compromise, or critical data breach. You don’t want to find your organization in the midst of a critical incident without a clear plan of what to do and who to call. Luckily an incident response plan (IRP) is just that.

An IRP is a document that serves as a framework of an organization’s incident response capabilities, and how to respond to that incident in a methodical manner. The end goal is to return the organization to normal business operations through a set of repeatable actions that minimize operational costs and errors.

The IRP should cover all areas of incident response (IR) within the organization. Organizations should consider the following items when creating an IRP:

• Defining what an incident is, including incident severity and categorization that can give a quick and reliable classification of what is happening on the ground. This core part of the framework will shape various other elements of the incident response process.
• Describe the IR process, and how IR is to be performed at high and low levels.
• Establishing the roles and responsibilities of incident response team members who will support the entire process from start to end.
• Determining escalation and notification requirements paths. Some of these will come from regulations and frameworks which business follows. This is especially important for organizations handling regulated data.
• Listing information that should be collected from affected systems (network logs, host-based artifacts, file samples, etc.).
• Creating a mapping of appropriate contacts and staff who will help during an outbreak and can be mobilized to action quickly.
• If needed, create a working relationship with vendors who can assist at critical junctions of the incident response and make sure they are part of the IRP.

All these areas combine to form a one-stop document for all things IR for the organization. That isn’t to say the IRP is the only document covering IR, but it will be the core document helping organizations in a difficult time. Additional documents such as IR Playbooks can and should be used to supplement IRP with detailed information on specific handling procedures.  

Why do you need a plan?

Many might still ask the question of whether or not an IRP is necessary. Technically, all aspects of IR can be performed without a plan or policy in place. There is nothing stopping a security team from investigating an incident on their own, containing infected hosts, and remediating without an IRP. This is known as an ad-hoc approach to incident response and can work for smaller organizations.

But the biggest risk of performing IR in an ad-hoc manner is that the lack of a formal process — it’s entirely reliant upon the individual. A lack of a formal process means that everyone is doing whatever actions they believe are best in the given scenario and not necessary actions that are justified. This usually means a lack of documentation, weak communication, and repeated actions. Two incident responders may end up working on the same incident, investigate the same set of artifacts, and enact the same containment actions, wasting a lot of valuable time during critical moments. This is especially important as incidents can happen at any point during the day or night so reducing mishaps by documenting all the steps might come really handy at 3 a.m. on a Sunday.

Who is part of an IRP?

Incidents do not happen in isolation and various teams need to support the process to be successful at recovery and containment. Starting from the IT team that usually works to restore the systems and monitor the recovery from backups this goes all the way to the legal departments assessing regulatory effects of a potential breach. The typical roles of various units can be broken down as follows:

Plans must be tested

Having an IRP also brings in a testing and training capability to your IR process. New or junior analysts can refer to the IRP to understand what they should be doing during an incident. An IRP that clearly outlines how IR should be performed can be used by newer IR team members to get up to speed sooner. The instructions provided in the IRP can be referenced during incidents to reduce the chances of a mistake.

An established plan should also serve as a basis for training exercises such as a tabletop, purple team, red team, or penetration test. By comparing the actions of the analyst during the exercise to what is supposed to be done according to the IRP, feedback can be provided. There might be some kinks in the process that are difficult to notice without live testing. For example, an exercise may reveal some failures in the communication process. An exercise can also provide familiarity with the process in a way that document review cannot. The goal here is to get as close to an incident without actually being an incident. For example, your organization could partake in one of CTIR’s tabletop exercises to discover potential gaps or improvements in your IR plan. Finally, lessons learned, or after-action reviews performed during post-incident activity of an actual incident response must be incorporated back into the IRP, so the IRP contains the most up-to-date organizational IR capabilities.

How to start an IRP?

CTIR can guide our customers in addressing the many considerations that may go into making an IRP. For example, a business continuity-oriented response plan that can be used during a business-halting incident can be created, or a boilerplate plan for a newer IR practice that can grow with the team. This does not happen in isolation, however, and some steps need to be taken before a working plan is put together:

• An asset map should be performed to identify the data and systems that need to be protected by the IRP. Trying to recover all components of the business at the same time simply won’t be feasible, so priority should be given based on the criticality of the asset, process or data that is stored on a given asset.
• Determine what kind of data is in your environment, with an emphasis on regulated data. There are strict notification and protection requirements for PII and PHI, and harsh penalties for failure to meet those requirements.
• If an external vendor is chosen to support the efforts during an emergency, make sure that their engagement process is known and that your organization knows how to contact them. Make their telephone numbers/emails or other contact details part of your plan. Also, identify who is authorized to active those third parties.
• Interview your employees and verify that they understand the roles and responsibilities that are expected from them during an emergency response.
• A workforce continuity plan should be created and included in IRP. Incidents run 24/7, so staff rotation, chain of command and other processes need to support continuous operations. Your employees will need to take a break and sleep so knowing who can carry on recovery efforts is important.
• Ensure that critical systems can operate with backups, so if they go down, a quick restore procedure can bring them up again. Test backup procedures on a regular basis and as part of BCP and make sure that the IR plan includes appropriate references and processes related to the recovery of operations.
• Establish a working process for performing backup and recovery procedures so there are no surprises when IRP is triggered.
• Establish a working process for collecting forensic evidence. Many IR vendors need that evidence to perform an investigation.  
  

So, you have a plan now?

Is your current IRP regularly reviewed or updated? Has the IR process in your organization been tested with something like a Tabletop Exercise, Red Team Exercise, or Penetration Test? Is there a review or lessons-learned process following major incidents? If you’ve answered no to any of those questions, then you should know that CTIR also includes an IRP review and update service.

Having an updated and reviewed process is incredibly important, as the IRP should be a living document that grows with the organizations, and responds to the constantly changing threat landscape. Having an IRP gathering dust does not work in today’s dynamic IT environments so regular testing, and updates are needed to ensure that the IRP continues to provide value.

• We have created IRPs for countless organizations in the past that span virtually every industry, including a new organization that did not have an existing IR plan or policy and wanted to begin the process of formalizing IR within their organization. CTIR worked with the organization to create a plan that addresses the biggest considerations of IR while remaining open to changes and relevant to internal processes and technology.
• A service provider in an industry that deals with regulated data that had an existing IRP but felt it did not provide enough information and in-depth guidance. CTIR began the IRP engagement by reviewing their existing IR-related documentation, to include the IRP and built it up to be more expansive.  
  

Conclusion

The goal of any cybersecurity department is to prevent harm from being incurred by the organization through detecting, responding to, and recovering from incidents. The best kind of response is one that prevents or detects an attack long before it impacts a sensitive system due to the efforts of security professionals, but that isn’t always the case. The cybersecurity department needs to be successful all the time, but an adversary only needs to be right once for an incident to occur.

This is why we must be open to the possibility that not every attack will be blocked. No defense system will ever be 100 percent effective; attackers may seek weak points to bypass defenses, or an attacker may be a malicious insider with legitimate access to a system. We must be humble enough to recognize that no security defenses are infallible.

Planning for an incident is a vital part of any cybersecurity posture. Incident planning doesn’t have to be onerous, but any plan must be workable and can be implemented on a moment’s notice.

Cisco Talos Incident Response services have much experience in responding to incidents, helping organizations recover from breaches, and working through incident response plans. One of the services available to clients is to use our expertise to help prepare plans, or to provide a tabletop exercise of a realistic scenario that can be used to test drive a plan in order to identify any weaknesses before a real incident arises.

Talos Incident Response analysts aren’t only available in case of emergency but can be used to strengthen responses in advance of an incident. As with so much in life, when it comes to the secret of success, preparation is key.