Differences between revisions 2 and 18 (spanning 16 versions)

EMEN2 Maintenance and Backups

An EMEN2 installation consists of several components:

EMEN2 database environment (DB_HOME)
File storage area (BINARYPATH)
Image browser cache area (TILEPATH)
Web server log, log file archive and database backup (LOGPATH, ARCHIVEPATH, BACKUPPATH)

The EMEN2 database environment requires special backup procedures because it is an active database environment. The other directories are just normal files on disk, however, and can handled with standard backup tools (e.g. rsync).

There are several ways to backup your EMEN2 database environment. At the simplest level, you can just copy all the files directly to another location. This is a type of "cold backup," and is the simplest and most reliable backup mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive. Alternatively, you can perform a "hot backup," which can be performed even while the database is active. A hot backup copies database log files to an existing cold backup to bring it up-to-date with the current state. I recommend running cold backups once a week, with hot backups daily.

backup.py

EMEN2's database core includes some methods to help manage database environment backups. These methods (db.archivelogs, db.coldbackup, db.hotbackup) can be invoked easily from the cmdlineutils/backup.py script.

Usage: backup.py [options]

Options:
  --help                Print help message
  -h HOME, --home=HOME  DB_HOME
  -c CONFIGFILE, --configfile=CONFIGFILE
  --archive             archive log files
  --cold                cold backup
  --hot                 hot backup

Cold Backup

To create a cold backup, shut down any open database processes (see emen2control.py to stop the web server), and use the EMEN2 backup utility with the "--cold" argument.

backup.py --cold

This will run a database checkpoint, and create a cold backup in the path specified by BACKUPPATH. The database files, highest numbered log file, and configuration files will be copied.

To prevent overwriting an existing cold backup, the script will not run if the target directory exists. You should move or remove the existing cold backup first.

Example:

# python ./cmdlineutils/backup.py --cold
                ... snip: startup ...
Opening Database Environment: /home/emen2/db/
Cold Backup: Checkpoint
Cold Backup: Copying data: /home/emen2/db/data -> /home/emen2/db_backup/data
Cold Backup: Copying config: /home/emen2/db/config.yml -> /home/emen2/db_backup/config.yml
Cold Backup: Copying config: /home/emen2/db/DB_CONFIG -> /home/emen2/db_backup/DB_CONFIG
Cold Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311

Once you have created a cold backup, it can be updated by running a hot backup.

Hot Backup

backup.py --hot

# python ./cmdlineutils/backup.py --cold
                ... snip: startup ...
Opening Database Environment: /home/emen2/db/
Hot Backup: Log archive
Log Archive: Checkpoint
Log Archive: /home/emen2/db/log/log.0000000303 -> /home/emen2/log_archive/log.0000000303
Log Archive: /home/emen2/db/log/log.0000000304 -> /home/emen2/log_archive/log.0000000304
Log Archive: /home/emen2/db/log/log.0000000305 -> /home/emen2/log_archive/log.0000000305
Log Archive: /home/emen2/db/log/log.0000000306 -> /home/emen2/log_archive/log.0000000306
Log Archive: /home/emen2/db/log/log.0000000307 -> /home/emen2/log_archive/log.0000000307
Log Archive: /home/emen2/db/log/log.0000000308 -> /home/emen2/log_archive/log.0000000308
Log Archive: /home/emen2/db/log/log.0000000309 -> /home/emen2/log_archive/log.0000000309
Log Archive: /home/emen2/db/log/log.0000000310 -> /home/emen2/log_archive/log.0000000310
Hot Backup: Copying log: /home/emen2/db/log/log.0000000303 -> /home/emen2/db_backup/log/log.0000000303
Hot Backup: Copying log: /home/emen2/db/log/log.0000000304 -> /home/emen2/db_backup/log/log.0000000304
Hot Backup: Copying log: /home/emen2/db/log/log.0000000305 -> /home/emen2/db_backup/log/log.0000000305
Hot Backup: Copying log: /home/emen2/db/log/log.0000000306 -> /home/emen2/db_backup/log/log.0000000306
Hot Backup: Copying log: /home/emen2/db/log/log.0000000307 -> /home/emen2/db_backup/log/log.0000000307
Hot Backup: Copying log: /home/emen2/db/log/log.0000000308 -> /home/emen2/db_backup/log/log.0000000308
Hot Backup: Copying log: /home/emen2/db/log/log.0000000309 -> /home/emen2/db_backup/log/log.0000000309
Hot Backup: Copying log: /home/emen2/db/log/log.0000000310 -> /home/emen2/db_backup/log/log.0000000310
Hot Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311
Log Archive: Checkpoint
Log Archive: Removing /home/emen2/db/log/log.0000000303
Log Archive: Removing /home/emen2/db/log/log.0000000304
Log Archive: Removing /home/emen2/db/log/log.0000000305
Log Archive: Removing /home/emen2/db/log/log.0000000306
Log Archive: Removing /home/emen2/db/log/log.0000000307
Log Archive: Removing /home/emen2/db/log/log.0000000308
Log Archive: Removing /home/emen2/db/log/log.0000000309
Log Archive: Removing /home/emen2/db/log/log.0000000310

-  ⇤ ← Revision 2 as of 2010-03-29 23:10:15 → 
  Size: 1176
  Editor: root
  Comment:
+   ← Revision 18 as of 2010-03-30 20:10:03 → ⇥
  Size: 5441
  Editor: root
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-EMEN2 supports three types of backups: Log Archive, Cold Backup, and Hot Backup.
+An EMEN2 installation consists of several components:
 Line 5:
-== Log Archive ==
+ * EMEN2 database environment ([[EMEN2/config.yml|DB_HOME]])
 * File storage area ([[EMEN2/config.yml|BINARYPATH]])
 * Image browser cache area ([[EMEN2/config.yml|TILEPATH]])
 * Web server log, log file archive and database backup ([[EMEN2/config.yml|LOGPATH]], [[EMEN2/config.yml|ARCHIVEPATH]], [[EMEN2/config.yml|BACKUPPATH]])
 
The EMEN2 database environment requires special backup procedures because it is an active database environment. The other directories are just normal files on disk, however, and can handled with standard backup tools (e.g. rsync).
-Line 7:
+Line 12:
-Berkeley DB provides guarantees about atomicity and durability by first writing changes to a series of log files. A transactions is not marked as committed until it has been flushed to stable storage. This allows recovery from a crash without loss of data.
+There are several ways to backup your EMEN2 database environment. At the simplest level, you can just copy all the files directly to another location. This is a type of "cold backup," and is the simplest and most reliable backup mechanism, but requires that you stop all database writes for the duration of the process to ensure the integrity of the archive. Alternatively, you can perform a "hot backup," which can be performed even while the database is active. A hot backup copies database log files to an existing cold backup to bring it up-to-date with the current state. I recommend running cold backups  once a week, with hot backups daily.
-Line 9:
+Line 14:
-In the event of a crash or hardware failure, the log files will be examined. All committed transactions will be written out to the BTree files (in $DB_HOME/data) and any uncommitted transactions will be aborted.
-Line 11:
+Line 15:
-The log files are stored in $DB_HOME/log as log.XX, where XX is a sequential integer starting from 1. With default settings, the files are 8 MB each. As one log file is finished, the next log file in the sequence is created and used as the active log file.
+== backup.py ==
-Line 13:
+Line 17:
-Log files that are not being used by any open transaction may be archived after a "Checkpoint" is made. This frees up disk space in the DB_HOME environment, and lets the administrator move the logs to long term archival storage.
+EMEN2's database core includes some methods to help manage database environment backups. These methods (db.archivelogs, db.coldbackup, db.hotbackup) can be invoked easily from the cmdlineutils/backup.py script.
-Line 15:
+Line 19:
-.. more to come ..
+{{{
Usage: backup.py [options]

Options:
  --help                Print help message
  -h HOME, --home=HOME  DB_HOME
  -c CONFIGFILE, --configfile=CONFIGFILE
  --archive             archive log files
  --cold                cold backup
  --hot                 hot backup
}}}
-Line 20:
+Line 34:
+To create a cold backup, shut down any open database processes (see [[EMEN2/emen2control.py|emen2control.py]] to stop the web server), and use the EMEN2 backup utility with the "--cold" argument.
-Line 21:
+Line 36:
+{{{
backup.py --cold
}}} 

This will run a database checkpoint, and create a cold backup in the path specified by [[EMEN2/config.yml|BACKUPPATH]]. The database files, highest numbered log file, and configuration files will be copied.

To prevent overwriting an existing cold backup, the script will not run if the target directory exists. You should move or remove the existing cold backup first.

Example:

{{{
# python ./cmdlineutils/backup.py --cold
  ... snip: startup ...
Opening Database Environment: /home/emen2/db/
Cold Backup: Checkpoint
Cold Backup: Copying data: /home/emen2/db/data -> /home/emen2/db_backup/data
Cold Backup: Copying config: /home/emen2/db/config.yml -> /home/emen2/db_backup/config.yml
Cold Backup: Copying config: /home/emen2/db/DB_CONFIG -> /home/emen2/db_backup/DB_CONFIG
Cold Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311
}}}

Once you have created a cold backup, it can be updated by running a hot backup.
-Line 23:
+Line 60:
+{{{
backup.py --hot
}}} 

{{{
# python ./cmdlineutils/backup.py --cold
  ... snip: startup ...
Opening Database Environment: /home/emen2/db/
Hot Backup: Log archive
Log Archive: Checkpoint
Log Archive: /home/emen2/db/log/log.0000000303 -> /home/emen2/log_archive/log.0000000303
Log Archive: /home/emen2/db/log/log.0000000304 -> /home/emen2/log_archive/log.0000000304
Log Archive: /home/emen2/db/log/log.0000000305 -> /home/emen2/log_archive/log.0000000305
Log Archive: /home/emen2/db/log/log.0000000306 -> /home/emen2/log_archive/log.0000000306
Log Archive: /home/emen2/db/log/log.0000000307 -> /home/emen2/log_archive/log.0000000307
Log Archive: /home/emen2/db/log/log.0000000308 -> /home/emen2/log_archive/log.0000000308
Log Archive: /home/emen2/db/log/log.0000000309 -> /home/emen2/log_archive/log.0000000309
Log Archive: /home/emen2/db/log/log.0000000310 -> /home/emen2/log_archive/log.0000000310
Hot Backup: Copying log: /home/emen2/db/log/log.0000000303 -> /home/emen2/db_backup/log/log.0000000303
Hot Backup: Copying log: /home/emen2/db/log/log.0000000304 -> /home/emen2/db_backup/log/log.0000000304
Hot Backup: Copying log: /home/emen2/db/log/log.0000000305 -> /home/emen2/db_backup/log/log.0000000305
Hot Backup: Copying log: /home/emen2/db/log/log.0000000306 -> /home/emen2/db_backup/log/log.0000000306
Hot Backup: Copying log: /home/emen2/db/log/log.0000000307 -> /home/emen2/db_backup/log/log.0000000307
Hot Backup: Copying log: /home/emen2/db/log/log.0000000308 -> /home/emen2/db_backup/log/log.0000000308
Hot Backup: Copying log: /home/emen2/db/log/log.0000000309 -> /home/emen2/db_backup/log/log.0000000309
Hot Backup: Copying log: /home/emen2/db/log/log.0000000310 -> /home/emen2/db_backup/log/log.0000000310
Hot Backup: Copying log: /home/emen2/db/log/log.0000000311 -> /home/emen2/db_backup/log/log.0000000311
Log Archive: Checkpoint
Log Archive: Removing /home/emen2/db/log/log.0000000303
Log Archive: Removing /home/emen2/db/log/log.0000000304
Log Archive: Removing /home/emen2/db/log/log.0000000305
Log Archive: Removing /home/emen2/db/log/log.0000000306
Log Archive: Removing /home/emen2/db/log/log.0000000307
Log Archive: Removing /home/emen2/db/log/log.0000000308
Log Archive: Removing /home/emen2/db/log/log.0000000309
Log Archive: Removing /home/emen2/db/log/log.0000000310

}}}