Backups

We take LDAP and Kerberos backups twice daily. Backups on auth1 occur at 4am and on auth2 they occur at 4pm. On each machine, backups are stored in /backup/{ldap,krb5} (in the case of auth1, this is /backup exported from the NetApp).

Miscellaneous backups (including backups of home directories for expired accounts) are stored in /users/backup (accessible on any system that NFS-mounts /users).

MySQL

As of January 2010, backups of MySQL databases are run weekly and stored in /users/backup/mysql, a git repository accessible only to members of syscom (for obvious reasons). In there is a .sql file containing a backup of each database present in the MySQL database.

This backup is run automatically from /etc/cron.weekly/mysql-backup on caffeine. A git repository was chosen because the SQL dumps are by default text files which are easily handled by git (notably, git handles deltas for this format extremely well). As a result, it is possible to retrieve an old backup of a database or view differences from week to week by using the regular git tools. git stores compressed deltas for all previous revisions, so this has the effect of storing many old copies of databases at minimal storage cost.