Backups

There are two types of data: Backed up data, and data you're prepared to lose.

The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.

Threats to Your Data

Keep these threats in mind when planning a backup strategy:

• Accidental deletion or modification
• Bitrot and file corruption
• Burglary and theft
• Fires
• Harddrive failure
• Malware, viruses, ransomware, etc.
• Natural disasters
  • Storms and floods
  • Tornadoes
• Power irregularities
  • Dips and brownouts
  • Lightning strikes
  • Surges and spikes

A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

What to Backup

First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:

• Photos
• Documents
• Music
• Movies
• TV
• Books/Audiobooks
• Applications
• Operating systems (see Install Kits below)
• Chinese cartoons
• Games
• Porn
• School files
• Work files
• Password databases
• Wallpapers
• Offline code repositories
• Android roms

You may have your files in an unsorted mess, such as a catch-all "Downloads" directory. No problem! Make another subdirectory under big data:

• Unsorted

Some files can't be moved to your big data directory because they either live in a specific place, or don't exist until you manually create them:

• Application and Operating System configurations
• Browser bookmarks
• Offline email
• Video game save files
• Things in "My Documents"

To have a copy of these things in your big data directory, you may have to manually "export" or "backup" from the applications they're used with. Some of this you will be able to automate with a script. Don't forget to keep the script in your big data directory too!

You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!

Install Kits

You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT network access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.

To do this, you will need copies of a few things:

• Your Operating System(s) isos
  • Tools to put the iso onto a thumbdrive such as Rufus
• Drivers for all of your hardware
  • Wired and Wireless NICs
  • GPUs
  • USB3
  • Soundcard
  • Bluetooth
  • Printers
• Firmware for your motherboard/peripherals/devices
• Applications you use
  • Configuration files for them, including window managers and ricing configs
  • Plugins (Web Browsers, etc)
    • Configuration files for the plugins (adblocklists etc)
• Windows Specific stuff
  • Activation tools such as DAZ Loader.
  • dotNet
  • DirectX
  • msxml
  • vcredist 2005, 2008, 2010, 2012, 2013
  • xnafx
  • Games for Windows Live
  • Java
  • Flash

Make sure all of the installers you have are full installers and not web setups. These can be tricky to find, like the Offline Flash Player Installer.

Alongside your main backup, have a USB thumbdrive with your operating system installer on it. It's handy to have all of your drivers, applications and configuration on this thumbdrive too.

Once a month or so, check on these files to see if new versions are available. Perhaps when you run your regular backup?

Android

If you have a custom bootloader applied such as ClockworkMod, you can likely create a full backup of your device including data and configuration to an external storage card.

Once complete, copy the backup from your storage card to your "big data" directory.

How to Backup

With all your data in a central location, you're ready to back it up.

Depending on your needs, investing in a file server, NAS or SAN may be worthwhile. A simpler option is to get yourself an external hard drive of sufficient capacity for you data. Encrypt it if you want.

Once it's mounted:

• $ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/
  • -v being verbose
  • --delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/
  • -a being "archive mode, which includes:
    • -r recursive mode
    • -p preserve permissions
    • -t preserve times
    • -g preserve groups
    • -o preserve owner
    • -l copy symlinks as symlinks

rsync will update any modified files and create any new ones.

There are many other software options which work in a similar way to rsync. See Unison (all platforms) or SyncToy (Windows).

Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.

Cloud Storage

Cloud storage is a bad idea because if you:

• don't have network access
• have a slow connection
• have a data limited connection which costs extra for excessive data
• lose your password
• get phished/hacked
• get banned from the service

then you don't have a backup and have lost your data.

Other considerations:

• Unencryped backups can be read the cloud provider
• Cloud provider could go offline/shutdown without warning
• Cloud provider may be in a different legal jurisdiction to you

When to Backup

This depends on how often your data changes. Once a week is a good. Once a month at least.

You may want to run an ad hoc backup when something important changes such as encryption keys or password databases.

Testing the Backup

Most of your backup (except drivers) can be tested in an isolated virtual machine to see if you've missed anything or if some of your install files require internet access.

If you're fairly confident in your backup, wipe your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.