We are still actively working on the spam issue.

Difference between revisions of "Backups"

From InstallGentoo Wiki
Jump to: navigation, search
m (What to Backup: alphabetical order)
m (change games link from linux games to games in general)
 
(10 intermediate revisions by 4 users not shown)
Line 1: Line 1:
There are two types of data: Backed up data, and data you're prepared to lose.
+
There are two types of data: Backed up data, and data you're prepared to lose. The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.
  
The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.
+
== Threats to your data ==
 
 
== Threats to Your Data ==
 
 
Keep these threats in mind when planning a backup strategy:
 
Keep these threats in mind when planning a backup strategy:
* Accidental deletion or modification
 
 
* Bitrot and file corruption
 
* Bitrot and file corruption
 
* Burglary and theft
 
* Burglary and theft
 
* Fires
 
* Fires
* Harddrive failure
+
* Hard drive failure
* Malware, viruses, ransomware, etc.
+
* Human error
 +
** Accidental deletion or modification
 +
** Misplacement of data storage
 +
* Intentional sabotage (by someone you may know personally)
 +
* [[Malware]], [[Malware#Viruses | viruses]], [[Malware#Viruses | ransomware]], etc.
 
* Natural disasters
 
* Natural disasters
** Storms and floods
+
** Floods
** Tornadoes
+
** Tornadoes and storms
 
* Power irregularities
 
* Power irregularities
 
** Dips and brownouts
 
** Dips and brownouts
Line 21: Line 22:
 
A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.
 
A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.
  
== What to Backup ==
+
== What to backup ==
  
 
First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:
 
First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:
 
* Applications with addons, extensions and plugins
 
* Applications with addons, extensions and plugins
* Books/Audiobooks
+
* [[Books |Books/Audiobooks]]
 
* Documents
 
* Documents
 
** Email
 
** Email
Line 31: Line 32:
 
** Work files
 
** Work files
 
* Drivers
 
* Drivers
* Games
+
* [[Games]]
* Music
+
* [[Music]]
* Movies
+
* [[Movies]]
 
* Offline code repositories
 
* Offline code repositories
* Operating systems (see Install Kits below)
+
* [[:Category:Operating systems | Operating systems]] (see [[Backups#Install kits | install kits]], below)
* Password databases
+
* [[Security#Password_manager | Password databases]]
 
* Photos
 
* Photos
* Porn
+
* [[Movies#Pornographic | Porn]]
* TV Shows
+
* [[Movies | TV shows]]
 +
* [[VirtualBox |Virtual machines]]
 
* Wallpapers
 
* Wallpapers
  
Line 51: Line 53:
 
**NoScript whitelist
 
**NoScript whitelist
 
**RequestPolicy whitelist
 
**RequestPolicy whitelist
**Saved passwords and logins
+
**Saved [[Passwords | passwords]] and logins
 
* Calendars
 
* Calendars
 
* Chat logs
 
* Chat logs
 
* Contacts
 
* Contacts
* Database/FTP/RDP/SSH/TeamViewer server bookmarks
+
* [[Databases | Database]]/FTP/RDP/SSH/TeamViewer server bookmarks
 
* Offline email
 
* Offline email
 
* Things in "My Documents" and "AppData"
 
* Things in "My Documents" and "AppData"
* Video game save files
+
* [[Games | Video game]] save files
 
* VOIP/Teamspeak server bookmarks and identity keys
 
* VOIP/Teamspeak server bookmarks and identity keys
  
Line 65: Line 67:
 
You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!
 
You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!
  
=== Install Kits ===
+
=== Install kits ===
You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT network access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.
+
You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT [[:Category:Networking | network]] access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.
  
 
To do this, you will need copies of a few things:
 
To do this, you will need copies of a few things:
* Your Operating System(s) isos
+
* Your [[:Category:Operating systems | Operating System(s)]] isos
 
** Tools to put the iso onto a thumbdrive such as [http://rufus.akeo.ie/ Rufus]
 
** Tools to put the iso onto a thumbdrive such as [http://rufus.akeo.ie/ Rufus]
 
* Drivers for all of your hardware
 
* Drivers for all of your hardware
 
** Wired and Wireless NICs
 
** Wired and Wireless NICs
** GPUs
+
** [[GPU]]s
 
** USB3
 
** USB3
 
** Soundcard
 
** Soundcard
Line 79: Line 81:
 
** Printers
 
** Printers
 
* Firmware for your motherboard/peripherals/devices
 
* Firmware for your motherboard/peripherals/devices
* Applications you use
+
* [[Recommended software | Applications]] you use
** Configuration files for them, including window managers and ricing configs
+
** Configuration files for them, including window managers and [[Ricing | ricing]] configs
** Plugins (Web Browsers, etc)
+
** Plugins ([[Web Browsers]], etc)
 
*** Configuration files for the plugins (adblocklists etc)
 
*** Configuration files for the plugins (adblocklists etc)
* Windows Specific stuff
+
* [[Windows]] Specific stuff
 
** Activation tools such as DAZ Loader.
 
** Activation tools such as DAZ Loader.
 
** dotNet
 
** dotNet
 
** DirectX
 
** DirectX
 
** msxml
 
** msxml
** vcredist 2005, 2008, 2010, 2012, 2013
+
** vcredist 2005, 2008, 2010, 2012, 2013, 2015
 
** xnafx
 
** xnafx
** Games for Windows Live
+
** [[Video Games |Games]] for Windows Live
 
** Java
 
** Java
 
** Flash
 
** Flash
Line 105: Line 107:
 
Once complete, copy the backup from your storage card to your "big data" directory.
 
Once complete, copy the backup from your storage card to your "big data" directory.
  
== How to Backup ==
+
== How to backup ==
 
With all your data in a central location, you're ready to back it up.
 
With all your data in a central location, you're ready to back it up.
  
Line 111: Line 113:
  
 
Once it's mounted:
 
Once it's mounted:
* $ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/
+
  $ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/
** -v being verbose
+
* -v being verbose
** --delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/
+
* --delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/
** -a being "archive mode, which includes:
+
* -a being "archive mode, which includes:
*** -r recursive mode
+
** -r recursive mode
*** -p preserve permissions
+
** -p preserve permissions
*** -t preserve times
+
** -t preserve times
*** -g preserve groups
+
** -g preserve groups
*** -o preserve owner
+
** -o preserve owner
*** -l copy symlinks as symlinks
+
** -l copy symlinks as symlinks
  
 
rsync will update any modified files and create any new ones.
 
rsync will update any modified files and create any new ones.
Line 128: Line 130:
 
Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.
 
Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.
  
=== Cloud Storage ===
+
=== Cloud storage ===
 
Cloud storage is a bad idea because if you:
 
Cloud storage is a bad idea because if you:
 
* don't have network access
 
* don't have network access
Line 143: Line 145:
 
* Cloud provider may be in a different legal jurisdiction to you
 
* Cloud provider may be in a different legal jurisdiction to you
  
== When to Backup ==
+
== When to backup ==
 
This depends on how often your data changes. Once a week is a good. Once a month at least.
 
This depends on how often your data changes. Once a week is a good. Once a month at least.
  
You may want to run an ad hoc backup when something important changes such as encryption keys or password databases.
+
You may want to run an ad-hoc backup when something important changes such as encryption keys or password databases.
 
 
== Testing the Backup ==
 
  
Most of your backup (except drivers) can be tested in an isolated virtual machine to see if you've missed anything or if some of your install files require internet access.
+
== Testing the backup ==
 +
Most of your backup (except drivers) can be tested in an isolated [[VirtualBox | Virtual Machine]] to see if you've missed anything or if some of your install files require internet access.
  
If you're fairly confident in your backup, wipe your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.
+
If you're fairly confident in your backup, [[Data Destruction | wipe]] your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.
  
 
[[Category:HowTo]]
 
[[Category:HowTo]]
 +
[[Category:Tutorials]]

Latest revision as of 11:21, 27 March 2016

There are two types of data: Backed up data, and data you're prepared to lose. The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.

Threats to your data

Keep these threats in mind when planning a backup strategy:

  • Bitrot and file corruption
  • Burglary and theft
  • Fires
  • Hard drive failure
  • Human error
    • Accidental deletion or modification
    • Misplacement of data storage
  • Intentional sabotage (by someone you may know personally)
  • Malware, viruses, ransomware, etc.
  • Natural disasters
    • Floods
    • Tornadoes and storms
  • Power irregularities
    • Dips and brownouts
    • Lightning strikes
    • Surges and spikes

A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

What to backup

First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:

You may have your files in an unsorted mess, such as a catch-all "Downloads" directory. No problem! Make another subdirectory under big data:

  • Unsorted

Some files can't be moved to your big data directory because they either live in a specific place, or don't exist until you manually create them:

  • Application and operating system configurations
  • Browser bookmarks
    • Adblock custom filters
    • NoScript whitelist
    • RequestPolicy whitelist
    • Saved passwords and logins
  • Calendars
  • Chat logs
  • Contacts
  • Database/FTP/RDP/SSH/TeamViewer server bookmarks
  • Offline email
  • Things in "My Documents" and "AppData"
  • Video game save files
  • VOIP/Teamspeak server bookmarks and identity keys

To have a copy of these things in your big data directory, you may have to manually "export" or "backup" from the applications they're used with. Alternatively you may need to backup a copy of the corresponding AppData or .config folder. Some of this you will be able to automate with a script. Don't forget to keep the script in your big data directory too!

You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!

Install kits

You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT network access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.

To do this, you will need copies of a few things:

  • Your Operating System(s) isos
    • Tools to put the iso onto a thumbdrive such as Rufus
  • Drivers for all of your hardware
    • Wired and Wireless NICs
    • GPUs
    • USB3
    • Soundcard
    • Bluetooth
    • Printers
  • Firmware for your motherboard/peripherals/devices
  • Applications you use
    • Configuration files for them, including window managers and ricing configs
    • Plugins (Web Browsers, etc)
      • Configuration files for the plugins (adblocklists etc)
  • Windows Specific stuff
    • Activation tools such as DAZ Loader.
    • dotNet
    • DirectX
    • msxml
    • vcredist 2005, 2008, 2010, 2012, 2013, 2015
    • xnafx
    • Games for Windows Live
    • Java
    • Flash

Make sure all of the installers you have are full installers and not web setups. These can be tricky to find, like the Offline Flash Player Installer.

Alongside your main backup, have a USB thumbdrive with your operating system installer on it. It's handy to have all of your drivers, applications and configuration on this thumbdrive too.

Once a month or so, check on these files to see if new versions are available. Perhaps when you run your regular backup?

Android

If you have a custom bootloader applied such as ClockworkMod, you can likely create a full backup of your device including data and configuration to an external storage card.

Once complete, copy the backup from your storage card to your "big data" directory.

How to backup

With all your data in a central location, you're ready to back it up.

Depending on your needs, investing in a file server, NAS or SAN may be worthwhile. A simple option is to get yourself an external hard drive of sufficient capacity for you data. Encrypt it if you want.

Once it's mounted:

 $ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/
  • -v being verbose
  • --delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/
  • -a being "archive mode, which includes:
    • -r recursive mode
    • -p preserve permissions
    • -t preserve times
    • -g preserve groups
    • -o preserve owner
    • -l copy symlinks as symlinks

rsync will update any modified files and create any new ones.

There are many other software options which work in a similar way to rsync. See Unison (all platforms) or SyncToy (Windows).

Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.

Cloud storage

Cloud storage is a bad idea because if you:

  • don't have network access
  • have a slow connection
  • have a data limited connection which costs extra for excessive data
  • lose your password
  • get phished/hacked
  • get banned from the service

then you don't have a backup and have lost your data.

Other considerations:

  • Unencryped backups can be read by the cloud provider
  • Cloud provider could go offline/shutdown without warning
  • Cloud provider may be in a different legal jurisdiction to you

When to backup

This depends on how often your data changes. Once a week is a good. Once a month at least.

You may want to run an ad-hoc backup when something important changes such as encryption keys or password databases.

Testing the backup

Most of your backup (except drivers) can be tested in an isolated Virtual Machine to see if you've missed anything or if some of your install files require internet access.

If you're fairly confident in your backup, wipe your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.