|
Size: 1770
Comment:
|
← Revision 30 as of 2010-09-22 07:18:44 ⇥
Size: 2107
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from EMEN2/Backups | |
| Line 3: | Line 4: |
| EMEN2 supports three types of backups: Log Archive, Cold Backup, and Hot Backup. | An EMEN2 installation consists of several components: |
| Line 5: | Line 6: |
| == Log Archive == | * EMEN2 database environment ([[EMEN2/config.yml|DB_HOME]]) * File storage area ([[EMEN2/config.yml|BINARYPATH]]) * Image browser cache area ([[EMEN2/config.yml|TILEPATH]]) * Web server log, log file archive and database backup ([[EMEN2/config.yml|LOGPATH]], [[EMEN2/config.yml|ARCHIVEPATH]], [[EMEN2/config.yml|BACKUPPATH]]) |
| Line 7: | Line 11: |
| EMEN2 uses Berkeley DB as the underlying database technology. To provide guarantees about transaction atomicity and durability, Berkeley DB writes changes to a log file on stable storage before a transaction is marked as committed. Database files are not updated until this step is complete. In the event of a crash or hardware failure, the database files can be checked against the log files to correct any errors or missing data. | <<TableOfContents>> |
| Line 9: | Line 13: |
| The log files are stored in $DB_HOME/log as log.XX, where XX is a sequential integer starting from 1. With default settings, the files are 8 MB each. As one log file is finished, the next log file in the sequence is created and used as the active log file. | = EMEN2 Database Environment = There are several ways to backup your EMEN2 database environment. |
| Line 11: | Line 17: |
| Log files that are not being used by any open transaction may be archived after a "Checkpoint" is made. This frees up disk space in the DB_HOME environment, and lets the administrator move the logs to long term archival storage. | In the simplest case, you can perform a normal copy of the database environment. This is a "cold backup", and is the easiest and most reliable mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive (see "long answer"). If your uptime requirements are not stringent, performing cold backups once a night or once a week using normal shell tools (see simple answer below) may be all you need. |
| Line 13: | Line 19: |
| The EMEN2 backup utility will archive finished logs when run with the "--archive" flag. Finished log files will be moved to the location specified by the "ARCHIVE" configuration setting. | If you want to perform very frequent backups, or do not want to stop the database environment, you can perform a "hot backup," which can be performed even while the database is active. |
| Line 15: | Line 21: |
| {{{ backup.py --archive }}} |
|
| Line 19: | Line 22: |
| Example: | = File Storage Area, Logs, etc. = |
| Line 21: | Line 24: |
| {{{ # python ./cmdlineutils/backup.py --archive ... snip: startup ... Opening Database Environment: /home/emen2/db/ Log Archive: Checkpoint Log Archive: /home/emen2/db/log/log.0000000303 -> /home/emen2/db_archive/log.0000000303 Log Archive: /home/emen2/db/log/log.0000000304 -> /home/emen2/db_archive/log.0000000304 Log Archive: /home/emen2/db/log/log.0000000305 -> /home/emen2/db_archive/log.0000000305 }}} == Cold Backup == |
The other directories are just normal files on disk and can handled with standard shell tools. |
| Line 35: | Line 28: |
| == Hot Backup == | = Backup: Simple Answer = Shutdown the web server (see [[EMEN2/emen2control.py]]) and perform a normal cold backup of everything by rsync'ing to a remote backup server. This will be sufficient for most installations. Example (default config, everything in /home/emen2): {{{ [emen2@ncmidb ~/emen2]# python ./emen2control.py --shutdown [emen2@ncmidb ~/emen2]# python ./emen2control.py --recover [emen2@ncmidb ~]# cd /home/emen2/ [emen2@ncmidb ~]# rsync -vr db db_backup log_archive applog emen2files emen2tiles emen2 emen2@remotebackup:~/emen2backup/ .... }}} = Hot backup, incremental backup, log files, etc. = For a more detailed discussion, see [[EMEN2/BackupDiscussion|Backup Discussion]] |
EMEN2 Maintenance and Backups
An EMEN2 installation consists of several components:
EMEN2 database environment (DB_HOME)
File storage area (BINARYPATH)
Image browser cache area (TILEPATH)
Web server log, log file archive and database backup (LOGPATH, ARCHIVEPATH, BACKUPPATH)
Contents
EMEN2 Database Environment
There are several ways to backup your EMEN2 database environment.
In the simplest case, you can perform a normal copy of the database environment. This is a "cold backup", and is the easiest and most reliable mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive (see "long answer"). If your uptime requirements are not stringent, performing cold backups once a night or once a week using normal shell tools (see simple answer below) may be all you need.
If you want to perform very frequent backups, or do not want to stop the database environment, you can perform a "hot backup," which can be performed even while the database is active.
File Storage Area, Logs, etc.
The other directories are just normal files on disk and can handled with standard shell tools.
Backup: Simple Answer
Shutdown the web server (see EMEN2/emen2control.py) and perform a normal cold backup of everything by rsync'ing to a remote backup server. This will be sufficient for most installations.
Example (default config, everything in /home/emen2):
[emen2@ncmidb ~/emen2]# python ./emen2control.py --shutdown
[emen2@ncmidb ~/emen2]# python ./emen2control.py --recover
[emen2@ncmidb ~]# cd /home/emen2/
[emen2@ncmidb ~]# rsync -vr db db_backup log_archive applog emen2files emen2tiles emen2 emen2@remotebackup:~/emen2backup/
....
Hot backup, incremental backup, log files, etc.
For a more detailed discussion, see Backup Discussion