Backups: Difference between revisions
m add to Software category |
|||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
We take [[LDAP]] and [[Kerberos]] backups twice daily. Backups on auth1 occur at 4am and on auth2 they occur at 4pm. On each machine, backups are stored in /backup/{ldap,krb5} (in the case of auth1, this is /backup exported from the [[NetApp]]). |
|||
We take nightly [[Kerberos]] and [[LDAP]] backups. The backups are stored on [[Machine_List#artificial-flavours|artificial-flavours]] in /var/backups/krb5 and /var/backups/slapd respectively. These are executed as cronjobs in /etc/cron.daily. |
|||
Miscellaneous backups (including backups of home directories for expired accounts) are stored in /users/backup (accessible on any system that NFS-mounts [[User-data|/users]]). |
Miscellaneous backups (including backups of home directories for expired accounts) are stored in /users/backup (accessible on any system that NFS-mounts [[User-data|/users]]). |
||
== |
== /users Backup == |
||
=== Initial Sync === |
|||
As of January 2010, backups of [[MySQL]] databases are run weekly and stored in /users/backup/mysql, a git repository accessible only to members of [[Systems_Committee|syscom]] (for obvious reasons). In there is a .sql file containing a backup of each database present in the MySQL database. |
|||
/users primary is held on ranch, whilst it's backup is held on tequila via incremental zfs snapshots. |
|||
On Tequila I made a dedicated destination dataset called backup/csc0_backups (backups is a big pool); |
|||
zfs create backup/csc0_backups |
|||
Then on ranch I ran a snapshot, and sent it over; |
|||
zfs snapshot -r csc0@base |
|||
zfs send -R csc0@base | ssh root@tequila zfs recv -F backup/csc0_backups |
|||
In the future we need to setup incremental sends on ranch |
|||
zfs snapshot -r csc0@snap_$(date +%Y%m%d) |
|||
zfs send -R -i csc0@base csc0@snap_$(date +%Y%m%d) | ssh root@tequila zfs recv -F backup/csc0_backups |
|||
=== Recurring Backups === |
|||
For recurring backups, we use Sanoid, and Sycnoid |
|||
This is the Sanoid config on Ranch:<syntaxhighlight lang="text"> |
|||
[csc0/users] |
|||
use_template = production |
|||
[csc0/backup] |
|||
use_template = production |
|||
[csc0/music] |
|||
use_template = music |
|||
[template_production] |
|||
frequently = 0 |
|||
hourly = 36 |
|||
daily = 30 |
|||
monthly = 3 |
|||
yearly = 0 |
|||
autosnap = yes |
|||
autoprune = yes |
|||
[template_music] |
|||
frequently = 0 |
|||
hourly = 0 |
|||
daily = 0 |
|||
monthly = 12 |
|||
yearly = 3 |
|||
autosnap = yes |
|||
autoprune = yes |
|||
</syntaxhighlight>Syncoid is run via a crontab on the root account with the following command<syntaxhighlight> |
|||
15 * * * * (date; /usr/sbin/syncoid --quiet --recursive root@ranch:csc0 backup/csc0_backups) >> /var/log/syncoid.log 2>&1 |
|||
</syntaxhighlight> |
|||
== MySQL == |
|||
We don't currently backup MySQL. See the historical version of this page for previous backup strategy. |
|||
This backup is run automatically from /etc/cron.weekly/mysql-backup on [[Machine_List#caffeine|caffeine]]. A git repository was chosen because the SQL dumps are by default text files which are easily handled by git (notably, git handles deltas for this format extremely well). As a result, it is possible to retrieve an old backup of a database or view differences from week to week by using the regular git tools. git stores compressed deltas for all previous revisions, so this has the effect of storing many old copies of databases at minimal storage cost. |
|||
[[Category:Software]] |
[[Category:Software]] |
||
Latest revision as of 14:32, 1 September 2026
We take LDAP and Kerberos backups twice daily. Backups on auth1 occur at 4am and on auth2 they occur at 4pm. On each machine, backups are stored in /backup/{ldap,krb5} (in the case of auth1, this is /backup exported from the NetApp).
Miscellaneous backups (including backups of home directories for expired accounts) are stored in /users/backup (accessible on any system that NFS-mounts /users).
/users Backup
Initial Sync
/users primary is held on ranch, whilst it's backup is held on tequila via incremental zfs snapshots.
On Tequila I made a dedicated destination dataset called backup/csc0_backups (backups is a big pool);
zfs create backup/csc0_backups
Then on ranch I ran a snapshot, and sent it over;
zfs snapshot -r csc0@base
zfs send -R csc0@base | ssh root@tequila zfs recv -F backup/csc0_backups
In the future we need to setup incremental sends on ranch
zfs snapshot -r csc0@snap_$(date +%Y%m%d)
zfs send -R -i csc0@base csc0@snap_$(date +%Y%m%d) | ssh root@tequila zfs recv -F backup/csc0_backups
Recurring Backups
For recurring backups, we use Sanoid, and Sycnoid
This is the Sanoid config on Ranch:
[csc0/users]
use_template = production
[csc0/backup]
use_template = production
[csc0/music]
use_template = music
[template_production]
frequently = 0
hourly = 36
daily = 30
monthly = 3
yearly = 0
autosnap = yes
autoprune = yes
[template_music]
frequently = 0
hourly = 0
daily = 0
monthly = 12
yearly = 3
autosnap = yes
autoprune = yes
Syncoid is run via a crontab on the root account with the following command
15 * * * * (date; /usr/sbin/syncoid --quiet --recursive root@ranch:csc0 backup/csc0_backups) >> /var/log/syncoid.log 2>&1MySQL
We don't currently backup MySQL. See the historical version of this page for previous backup strategy.