OVH Fire and Disaster Recovery – Lessons Learnt and What You Can Do to Avoid Data Loss

OVH Fire and Disaster Recovery - Lessons Learnt and What You Can Do to Avoid Data Loss

The hosting community was recently rocked due to a major fire incident at one of hosting provider OVH’s Strasbourg-based data centers. In the aftermath, OVH itself as well as its customers are scrambling to salvage what they can and ensure history does not repeat itself. Seemingly helpless, OVH has urged its clients to “activate their disaster recovery plans.”

But, what does a fire have to do with your business’ cybersecurity? As is clear from cybersecurity efforts today, data is one of a business’ most important assets. Ransomware, and similar threats, have shown how valuable data is to companies, and it’s loss can lead to a significant blow to operations and bottom lines.

From a holistic perspective, all threats to the security and integrity of your data is considered a cybersecurity threat. A fire that destroys your servers can be every bit as damaging, if not even more thorough, than a data breach, successful ransomware deployment, etc. The resiliency of your business in the face of threats to its data comes down to more than just your malware and hacking countermeasures.

LIFARS CYBER RESILIENCY PROGRAM provides the manpower and expertise to immediately respond and remediate to cyber incidents and breaches, in addition to providing a full array of services to increase your company’s cyber resiliency.

In the aftermath of this still-developing situation, two things have become abundantly clear:

  1. Unexpected, disastrous events can still occur at any time with little or no warning
  2. Many were completely unprepared to deal with this type of incident, from individuals to multinational corporations to government agencies
  3. When it comes to the security of your valuable data, it’s best to take your fate into your own hands

What is the OVH Fire Incident?

On Wednesday 10 March 2021 at 00:47, a fire broke out in one of OVH’s four Strasbourg-based data centers. Firefighters responded immediately to cordon off the scene and limit the spread of the fire which continued to rage throughout the early morning hours, destroying the SBG2 data center. However, it was finally contained by 05:30 AM.

The complete loss of a large number of servers resulted in a massive loss of data as well as complete service interruptions for many of OVH’s customers. As the largest hosting provider in Europe, and third-largest in the world, the impact has been tremendous, particularly for individuals and businesses in France.

Among OVH’s clients are not only individuals and small businesses, but A-list companies, such as game developer Rust, cryptocurrency exchange Derabit, and news outlet eeNewsEurope. OVH also hosted all other types of important institutions, from banks to government agencies.

Some without a disaster recovery plan and who happened to have warehoused all their data in the SBG2 data center, experienced a 100% loss.

What is a Fire Disaster Recovery Plan? How to Create One?

In short, a fire disaster recovery plan shares the same objectives as any other incident recovery plan:

  • Mitigate or minimize the negative impact of the event (loss of data, for example)
  • Restore business operations as soon as possible
  • Establish failsafes, roles, and processes to activate during and after an incident to facilitate the recovery process

With that in mind, a fire disaster recovery (DR) plan is similar to any other recovery plan aimed at minimizing your loss of data. While you might need to tailor it to your unique context, it roughly follows these steps:

  1. Inventory hardware & software: Clear visibility of all your assets is crucial in order to accurately estimate the damages and swiftly recommission resources to resume operations.
  2. Establish metrics for downtime and loss: Define acceptable Recovery Point Objective (RPO) and Recovery Time Objective (RTO) to ensure you can implement a cost-effective approach to DR.
  3. Assign roles and responsibilities: Make sure that every stakeholder knows what their role and responsibilities are during the disaster recovery phase. Having clearly defined roles as well as an established chain of command is crucial to ensure an efficient process.
  4. Establish communication channels: Establish internal channels among team members, external channels with third-party vendors or OEMs, and top-down channels for disseminating status updates and critical information. This will avoid cross chatter and miscommunication.
  5. Implement redundant backups: All copies of your live and backup data should never be stored on the same medium and within the same physical assets. You can make your adata backups more resilient by using systems such as the 3-2-1 principle and storing backups across different media (for example, on-premise and cloud) as well as ensuring your backups live in separate physical locations.
  6. Ensure SLA’s include disaster recovery: Make sure that your hosting or data center operator’s SLA’s includes measures to protect against and recover from incidents involving fire.
  7. Test and revise your plan: Your plan needs to be regularly revised according to changes in your assets, technologies, and warehoused data. If an incident should occur, processes must be in place to analyze your recovery performance and establish ways to improve in the future.