|
Size: 5441
Comment:
|
← Revision 30 as of 2010-09-22 07:18:44 ⇥
Size: 2107
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from EMEN2/Backups | |
| Line 9: | Line 10: |
<<TableOfContents>> = EMEN2 Database Environment = |
|
| Line 10: | Line 15: |
| The EMEN2 database environment requires special backup procedures because it is an active database environment. The other directories are just normal files on disk, however, and can handled with standard backup tools (e.g. rsync). | There are several ways to backup your EMEN2 database environment. |
| Line 12: | Line 17: |
| There are several ways to backup your EMEN2 database environment. At the simplest level, you can just copy all the files directly to another location. This is a type of "cold backup," and is the simplest and most reliable backup mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive. Alternatively, you can perform a "hot backup," which can be performed even while the database is active. A hot backup copies database log files to an existing cold backup to bring it up-to-date with the current state. I recommend running cold backups once a week, with hot backups daily. | In the simplest case, you can perform a normal copy of the database environment. This is a "cold backup", and is the easiest and most reliable mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive (see "long answer"). If your uptime requirements are not stringent, performing cold backups once a night or once a week using normal shell tools (see simple answer below) may be all you need. If you want to perform very frequent backups, or do not want to stop the database environment, you can perform a "hot backup," which can be performed even while the database is active. |
| Line 15: | Line 22: |
| == backup.py == | = File Storage Area, Logs, etc. = |
| Line 17: | Line 24: |
| EMEN2's database core includes some methods to help manage database environment backups. These methods (db.archivelogs, db.coldbackup, db.hotbackup) can be invoked easily from the cmdlineutils/backup.py script. | The other directories are just normal files on disk and can handled with standard shell tools. = Backup: Simple Answer = Shutdown the web server (see [[EMEN2/emen2control.py]]) and perform a normal cold backup of everything by rsync'ing to a remote backup server. This will be sufficient for most installations. Example (default config, everything in /home/emen2): |
| Line 20: | Line 35: |
| Usage: backup.py [options] Options: --help Print help message -h HOME, --home=HOME DB_HOME -c CONFIGFILE, --configfile=CONFIGFILE --archive archive log files --cold cold backup --hot hot backup |
[emen2@ncmidb ~/emen2]# python ./emen2control.py --shutdown [emen2@ncmidb ~/emen2]# python ./emen2control.py --recover [emen2@ncmidb ~]# cd /home/emen2/ [emen2@ncmidb ~]# rsync -vr db db_backup log_archive applog emen2files emen2tiles emen2 emen2@remotebackup:~/emen2backup/ .... |
| Line 31: | Line 42: |
| = Hot backup, incremental backup, log files, etc. = | |
| Line 32: | Line 44: |
| == Cold Backup == To create a cold backup, shut down any open database processes (see [[EMEN2/emen2control.py|emen2control.py]] to stop the web server), and use the EMEN2 backup utility with the "--cold" argument. {{{ backup.py --cold }}} This will run a database checkpoint, and create a cold backup in the path specified by [[EMEN2/config.yml|BACKUPPATH]]. The database files, highest numbered log file, and configuration files will be copied. To prevent overwriting an existing cold backup, the script will not run if the target directory exists. You should move or remove the existing cold backup first. Example: {{{ # python ./cmdlineutils/backup.py --cold ... snip: startup ... Opening Database Environment: /home/emen2/db/ Cold Backup: Checkpoint Cold Backup: Copying data: /home/emen2/db/data -> /home/emen2/db_backup/data Cold Backup: Copying config: /home/emen2/db/config.yml -> /home/emen2/db_backup/config.yml Cold Backup: Copying config: /home/emen2/db/DB_CONFIG -> /home/emen2/db_backup/DB_CONFIG Cold Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311 }}} Once you have created a cold backup, it can be updated by running a hot backup. == Hot Backup == {{{ backup.py --hot }}} {{{ # python ./cmdlineutils/backup.py --cold ... snip: startup ... Opening Database Environment: /home/emen2/db/ Hot Backup: Log archive Log Archive: Checkpoint Log Archive: /home/emen2/db/log/log.0000000303 -> /home/emen2/log_archive/log.0000000303 Log Archive: /home/emen2/db/log/log.0000000304 -> /home/emen2/log_archive/log.0000000304 Log Archive: /home/emen2/db/log/log.0000000305 -> /home/emen2/log_archive/log.0000000305 Log Archive: /home/emen2/db/log/log.0000000306 -> /home/emen2/log_archive/log.0000000306 Log Archive: /home/emen2/db/log/log.0000000307 -> /home/emen2/log_archive/log.0000000307 Log Archive: /home/emen2/db/log/log.0000000308 -> /home/emen2/log_archive/log.0000000308 Log Archive: /home/emen2/db/log/log.0000000309 -> /home/emen2/log_archive/log.0000000309 Log Archive: /home/emen2/db/log/log.0000000310 -> /home/emen2/log_archive/log.0000000310 Hot Backup: Copying log: /home/emen2/db/log/log.0000000303 -> /home/emen2/db_backup/log/log.0000000303 Hot Backup: Copying log: /home/emen2/db/log/log.0000000304 -> /home/emen2/db_backup/log/log.0000000304 Hot Backup: Copying log: /home/emen2/db/log/log.0000000305 -> /home/emen2/db_backup/log/log.0000000305 Hot Backup: Copying log: /home/emen2/db/log/log.0000000306 -> /home/emen2/db_backup/log/log.0000000306 Hot Backup: Copying log: /home/emen2/db/log/log.0000000307 -> /home/emen2/db_backup/log/log.0000000307 Hot Backup: Copying log: /home/emen2/db/log/log.0000000308 -> /home/emen2/db_backup/log/log.0000000308 Hot Backup: Copying log: /home/emen2/db/log/log.0000000309 -> /home/emen2/db_backup/log/log.0000000309 Hot Backup: Copying log: /home/emen2/db/log/log.0000000310 -> /home/emen2/db_backup/log/log.0000000310 Hot Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311 Log Archive: Checkpoint Log Archive: Removing /home/emen2/db/log/log.0000000303 Log Archive: Removing /home/emen2/db/log/log.0000000304 Log Archive: Removing /home/emen2/db/log/log.0000000305 Log Archive: Removing /home/emen2/db/log/log.0000000306 Log Archive: Removing /home/emen2/db/log/log.0000000307 Log Archive: Removing /home/emen2/db/log/log.0000000308 Log Archive: Removing /home/emen2/db/log/log.0000000309 Log Archive: Removing /home/emen2/db/log/log.0000000310 }}} |
For a more detailed discussion, see [[EMEN2/BackupDiscussion|Backup Discussion]] |
EMEN2 Maintenance and Backups
An EMEN2 installation consists of several components:
EMEN2 database environment (DB_HOME)
File storage area (BINARYPATH)
Image browser cache area (TILEPATH)
Web server log, log file archive and database backup (LOGPATH, ARCHIVEPATH, BACKUPPATH)
Contents
EMEN2 Database Environment
There are several ways to backup your EMEN2 database environment.
In the simplest case, you can perform a normal copy of the database environment. This is a "cold backup", and is the easiest and most reliable mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive (see "long answer"). If your uptime requirements are not stringent, performing cold backups once a night or once a week using normal shell tools (see simple answer below) may be all you need.
If you want to perform very frequent backups, or do not want to stop the database environment, you can perform a "hot backup," which can be performed even while the database is active.
File Storage Area, Logs, etc.
The other directories are just normal files on disk and can handled with standard shell tools.
Backup: Simple Answer
Shutdown the web server (see EMEN2/emen2control.py) and perform a normal cold backup of everything by rsync'ing to a remote backup server. This will be sufficient for most installations.
Example (default config, everything in /home/emen2):
[emen2@ncmidb ~/emen2]# python ./emen2control.py --shutdown
[emen2@ncmidb ~/emen2]# python ./emen2control.py --recover
[emen2@ncmidb ~]# cd /home/emen2/
[emen2@ncmidb ~]# rsync -vr db db_backup log_archive applog emen2files emen2tiles emen2 emen2@remotebackup:~/emen2backup/
....
Hot backup, incremental backup, log files, etc.
For a more detailed discussion, see Backup Discussion