ID: 1227

Backup Zoom Servers

It is always important when installing Zoom to have a backup plan for key files in the unfortunate event that a system failure occurs. Fortunately, backing up the Zoom system can be configuring any enterprise backup system. Zoom’s RevDB database allows the database to be backed up even while Zoom is running. Every time Zoom Server starts, it performs a checkpoint of database to ensure the database is correct. All the database files as well as the check-pointed state of the repository is saved under the db/ folder. You can setup a periodic job that does an incremental backup of this folder.

What files to Backup

For an effective backup strategy, the following items need to be backed up:

conf folder

The conf folder contains all of the information for how the Zoom Server is configured and your metadata specifications. It is important to keep a backup of this folder so that in the event of having to reinstall the software, all of your settings are preserved. It can be found in the following locations:

OS Path
Linux [ZoomDir]/conf
Windows Server C:\Program Files (x86)\Evolphin\DAM\conf

db folder

The db folder contains the RevDB database and is one of the most crucial items to backup on a regular basis as this contains the contents of the entire Zoom repository.

The location of the db folder can vary from installation to installation.

 

The following locations are the default locations of the db folder:

OS Path
Linux [ZoomDir]/db
Windows Server C:\Program Files (x86)\Evolphin\DAM\db

If you are not quite sure where the db folder is located in your installation, you can check by doing the following:

  1. In your web browser, log into the Web Administration Console (http://[YourServerURL]:8443)
  2. In the sidebar, click on Server Control Panel.
  3. Click Database Settings.
  4. Under the field DB root dir you will find the path to your db folder.

logs folder

audit logs under the <zoom-server--install-dir>/logs/ folder are recommended. It is not necessary to backup rest of the log files.

zm.conf

This file applies only to Zoom installations running 4.1-b6026 and older. This file is deprecated in future version and all the functions are now accessible within the preview-server.xml file in the conf folder.

For older Zoom installations, the zm.conf file is the configuration file for the Preview Server. This allows the server to be distributed on different servers than the Zoom Server. By default, this file is not created but should be backed up in the even this file is created.

OS Path
Linux [ZoomDir]/webcontext/servers/PreviewServer
Windows Server C:\Program Files (x86)\Evolphin\DAM\webcontext\servers\PreviewServer

Pre-backup procedure

Zoom server checkpoints the database periodically from its transactional redo logs based on a schedule specified via the Zoom web console. To avoid copying both redo logs and check-pointed data twice, it is highly recommended but not necessary that a full backup be started post check-pointing.

When the check-pointing is over, Zoom will spit out a message in the server log similar to the log message below, that can be monitored by a script to kick off the full database backup. The time to complete a checkpoint depends upon the amount of data in the repository that was changed since the last check-point and the sustained disk IO throughput. Usually the initial import takes the longest to check-point as multi terabytes of data might have been imported. After that <10% of repository data might change every month because of Zoom’s de-duplication features.

Below is an example of check-point start and end logging performed by Zoom server:

=========== BEGIN Zoom LOG ==============
Start Time: Fri Jun 15 13:52:11 PDT 2012

Product Version: 3.6-b5004
===========  LOG HEADER =================

INFO: Fri, 15 Jun 2012 13:52:11.054 PDT  [TransactionRedoLogManager.setup main] REDO_CHECKPOINT_START
INFO: Fri, 15 Jun 2012 13:52:11.743 PDT  [TransactionRedoLogManager.setup main] REDO_CHECKPOINT_END 

Checkpointing Frequency

A checkpoint creates a known good point from which the Zoom RevDB database can start applying changes contained in the redo log during recovery after an unexpected shutdown or crash.  Without a checkpoint the recovery will take forever as the server would have to look at every single redo log.

Checkpoint also help distribute the disk IO load across multiple files within the Zoom transnational file store. Over a longer period of time the system will perform better w.r.t disk IO with checkpointed redo log.

Eventually RevDB will implement background checkpoints currently these are done at server restart only to ensure no writers can come in whil ethe redo logs are being garbage collected/deleted during checkpoint. This downtime is a non-issue with Zoom NonStop as there will be a node that can server the readers & writers during downtime of another node.

Check-pointing allows the Zoom server to apply the transactions written to the Zoom RevDB’s redo logs to the the actual database on disk. Transactions in Zoom span both file data and metadata changes. Check-pointing ensures that changes in transactions to one or more asset is atomically written out to the database on disk.

Check-pointing occurs at either Zoom DAM server restart or periodically based on a schedule specified in the Zoom Web Console -> Server Control Panel -> Automatic Server restart Settings. Out of the box, periodic restart or checkpoint is disabled. Using the Web Administration console’s GUI the check-pointing via restart can be enabled, start date, start time and a frequency (in days) can also be specified in the web based form.

Frequency of restart and thus check-pointing is based on how much data in the transaction redo log can accumulate between restart time period. For example if in a 7 day period, 250 GB of changes collect in the transaction log and the storage transfer rate is 50 Gbps, it could take around 40 seconds to finish the check-point. During the check-pointing the server goes into a standby state and does not serve any client requests. If the system administrator requires a smaller check-point interval they may decide to change the frequency for instance from 7 days to 3 days based on the expected time for storage transfer IO rate.

With the Zoom NonStop product, system administrators can have a Zoom server perform a check-point while letting a second Zoom Server attached to the shared disk storage continue to serve client requests with 0 down-time during check-pointing.

Incremental backups

Incremental backups of the Zoom folders can be performed at any time. Usually the best strategy is to perform incremental backups between two check-points. For instance if the check-pointing is done on a weekly basis, say every Sunday morning, during the week from Mon-Sat, IT administrator can setup incremental backups with a full backup on Sunday night.

Zoom de-duplication and backups

Zoom clients send just deltas or changes whenever a file is edited on the end-user’s desktop. Beyond the first version, other versions are always stored as compressed deltas in the Zoom database folder. The backup strategy is not impacted by how Zoom de-duplication mechanism stores the data on its back-end.

From the administrator perspective, the Zoom folders are to be treated as a black-box that need to be periodically backed-up. The administrator need not know the internal data structure used to store the database records and file data.

In case of a disaster that wipes out the server storage or the storage disk are corrupted, the administrator needs to stop the Zoom Dam server, mount a new Zoom db/ folder on a clean storage device, copy the content of the last valid backup to this folder and restart the Zoom service. Zoom will then automatically restore the data from the backup. It would check-point a new database state after the restore and re-start operation complete.

1. Shutdown the Zoom servers (DAM & Preview both)
2. Move the DAM server’s current db/ folder if restoring just the database to a safe location
3. Restore the contents of the db/ folder from the backup
4. If end-user’s working copy contain files that are not going to be present in the restored backup, delink their working copy
5. Delete preview cache to ensure it doesn’t contain files that are no longer there in the restored database
6. Restart both servers
7. Verify via Asset Browser the files in the restored database

If the administrator failed to backup the entire db/ folder and files were only backed up partially, the Zoom database server may not be able to restart upon recovery from the backup. Zoom NonStop HADR (High Availability & Disaster Recovery) product can safe guard against such failures. Please contact sales for further information on Zoom NonStop product.

 

To speed up previews it is a good idea to backup the preview cache though it is not required. If the preview cache is deleted, Zoom can always recreate the missing previews at the cost of additional compute time. Please check the location of the preview cache as mentioned here.

If full text indexing service is turned on, please ensure the indices are backed up to recover from a disk failure that wipes out the existing text index files.