On March 10, 2021 a large fire took an OVH datacentre in Strasbourg offline, and destroyed the entirety of SBG2 and four halls of SBG1. A lot of people lost data that morning. One story caught my eye. A sound mastering engineer who ran his website on a VPS in SBG2 lost everything. He had backups, but they were made by OVH, and were unusable.
It is my understanding, that the backups were not lost, but were corrupted. Technical details like these are lost when the press runs the story.
In this post, I'd like to talk about backups, and why just one backup is useless, and why just having a backup is not everything you need to worry about.
At all times, store 3 copies of your data.
On 2 different types of media.
Of which 1 is off-site.
You have a server running your WordPress blog. That WordPress installation is your first copy of your data. There's currently no data-loss protection.
You make an incremental backup every hour to a separate directory, but still on the same server. This is your second copy of your data. Now you're protected against attacks on your WordPress instance leading to data-loss, as well as accidents caused by you. There's still no protection against losing the server.
You clone your local backups to S3. That's your third copy. It's also off-site. It's even on a different type of media. Sort of. It's no longer just on a single disk. It's kept safe on a platform that has never lost a file, and probably never will.
There's something, so damn important, that many still just seem to forget. Check your backup system! Do a sample restore. Simulate a disaster to see if you're actually prepared.
Ensure you know if a backup fails. You don't want to find out, that your latest backup is 4 weeks old, when you've lost everything.
If what you're backing up is not business critical, you may instead store your local (second) copy in the cloud, and the third copy in Amazon Glacier.
So, what's the lesson? Did that guy lose his website because a datacentre burned, and his backup provider messed up? No. He only had two copies, and never checked that the backup even worked. Slobby. Good for some things, but for backups.