Setting up Archive/Restore with VideoFX (Zoom 5.x – 7.2)

This post is about adding archive/restore functionality to an existing VideoFX setup. If you have to set up VideoFX first, then check here for details.

Zoom offers a built-in internal archive system. It also allows extending the internal module by using external archive modules like SGL, S3, and DIVA. With VideoFX, we can only use external archive modules. You can skip these configuration steps if you do not need archive with VideoFX.

If you have run the script installer on all servers as described here, then you are ready to set up the external archive module for VideoFX. We will first set up basic archiving in Zoom. Then, we will set up the external archive modules to build on the basic functionality. These sections are covered:

Follow each section to go through the configuration steps in detail.

Check Archive license

Archiving is an optional feature that is available with an appropriate license. You can check your license information in Zoom’s Web Management Console.

To check the license information, log on to your Web Management Console, for example, http://MyZoomServer:8443/

Open the License Management node under the System node in the Admin Menu sidebar. Click License Information to view your license.

If you hold a license that allows archive operations then the Archive Module would be enabled here.

From Zoom 7.3.1 onwards, for customers that hold a Hierarchical Archive license, it would also be enabled here as Hierarchical Archive. For Zoom 7.3 only, Hierarchical Archive is listed as Multi-tiered Archive here.

If you would like to purchase the Archive Module or a Hierarchical Archive license then contact Evolphin Support.

Enable basic archive in your setup

When the Archive Module is included as a part of the Zoom license, Zoom can be set up to archive assets to a single designated archive location using its internal Basic Archive Module. This folder location must exist on the Zoom MAM server in order for the configuration to work. Enable and set up the Basic Archive by following these steps:

  1. Open the Web Management Console for your Zoom Server.
    ex. http://localhost:8443 or http://<zoomserver>:8443
  2. Log in using your admin credentials.
  3. In the Admin Menu sidebar, click Server Control Panel under the System node.
  4. Click Archive Management on the Server Control Panel page.

  5. Click Enable Archive to enable it.
  6. Specify a local path on Zoom MAM Server as the Archive Location.
    If using an external archive module, like SGL or S3, the archived files are moved to the external archive systems from this path. Check Getting started with Hierarchical Archive for more details.
    Ex. E:\zoom\archive\ or /mnt/Archive on the Zoom MAM Server.
  7. If you are not setting up External Archive then pre and post scripts for archive and restore are optional. You can still set up scripts here to execute before and after the archive/restore operations.
  8. If you are setting up External Archive, then you need these script files:
    1. Specify the path for Pre-script for Archive from the location where the script installer placed the script files (as described here). For example, if you had specified /home/evolphin as the root path for the script installer, then the path will be /home/evolphin/zoom-deploy/ArchivePreHook/
    2. Similarly, specify the path for Pre-script for Restore from the location of the restore pre-script in the script files on Zoom MAM Server. For example, as per the above example, this path will be /home/evolphin/zoom-deploy/RestorePreHook/
  9. Update Limit on Arguments on Command-line to be 0. This is needed to remove the limit on the number of asset IDs passed in a Zoom command while using archiving.
  10. Click Save.
  11. You will be prompted to restart the server. Click Yes.
  12. Refresh your web browser.

Basic Archive settings are now saved.

Archiving to multiple target locations

This step is not needed for external assets. It is only applicable to direct ingest assets.

As a media manager, you may decide that assets from some projects in Zoom may be less relevant and could be archived to low-cost tape storage, while assets from other projects may have to be kept more accessible on a fast SAN. You could achieve this out of the box starting with Zoom 7.1 and above.

You can set up these configuration details from the Web Management Console. Log in as an admin to the Web Management Console. Open Server Control Panel under the System node in the Admin Menu sidebar. Click Archive Management

Path-specific archive targets

Enter the Zoom paths and their corresponding file system locations to which assets must be archived in the Path-specific Archive Targets box.

For example, the entry may look like this on a Linux server:

/defproj=/data/Archives/misc/ ; /Library = /mount/commons

And like this on a Windows server:  

/defproj=\\NY-NAS\backups\defproj; /Library=L:\Users Data\Users1\akila\Others

Click Save. Choose Yes to restart the Zoom Server to save the configuration changes.

At this point server.xml will show a new entry in <archivespec>, <targets>

<targets>/defproj=\\NY-NAS\backups\defproj; /Library=L:\Users Data\Users1\akila\Others</targets>

The paths may be a mapped drive letter path or a UNC path, or a local or network file system path. As long as the Zoom server process has the correct permissions into the target locations, the archive operations will work just fine. 

Any assets belonging to paths given here will get archived according to the locations configured here; the assets belonging to paths not mentioned in this field will still get archived into the default archive location. 

Though it is usual to put the paths corresponding to Zoom projects here, it is not mandatory. You could specify any folder path in the configuration and partition your archive locations even within the project-level boundaries. 

This configuration is only allowed when there are no assets that are currently archived. Therefore it is recommended that you configure this during the initial Archive setup.
Configuration of default archive location is still mandatory, and this path must also physically exist and be accessible from the Zoom Server. 

Check archive on ingest setup

Check archiveonIngest flag in the [Archive] section of the config.ini file that was used to save the configuration for the script installer as described here.

If you find that this flag was set to 1, the archive/restore is set to be triggered automatically at the time of high-res ingest and mid-res creation.

If this flag was 0, archive/restore is not automatic. The user can trigger archive/restore for specific assets manually.

Check metadata schema

While setting up your Zoom MAM Server for VideoFx, we had set up some metadata fields to be used as the folder schema to save the various high-res, low-res, mid-res, or direct-ingest assets. We can also use the fields set up in a similar way to set up the folder schema for Archived assets. Check here for how to set up a metadata schema on your Zoom MAM Server.

Add dynamic forms

We need to add dynamic forms for Archive and Restore flow.

  1. Navigate to the dynamic-forms directory in the conf directory inside the default Zoom installation folder (Windows: [ZoomInstallDir]\conf\dynamic-forms\ and Linux: $home$/zoom/conf/dynamic-forms/)
  2. Open createjobfromtemplate.xml for editing.
  3. Add a new form each for archive and restore as follows:
    <form heading=”Archive” name=”Archive” tooltip=”This is tooltip” formFor=”*Archive*”>
    <!– Must specify name and id to each tag. Name field is used to store at backend and Id field is required by Flex form framework –>
    <checkbox id=”ArchiveHires” name=”archiveHires” enable=”true” label=”Archive Native File” checked=”true”/>
    <checkbox id=”ArchiveProres” name=”archiveMidres” enable=”true” label=”Archive Prores File” checked=”true”/>
    <form heading=”Restore” name=”Restore” tooltip=”This is tooltip” formFor=”*Restore*”>
    <!– Must specify name and id to each tag. Name field is used to store at backend and Id field is required by Flex form framework –>
    <checkbox id=”RestoreHires” name=”restoreHires” enable=”true” label=”Restore Native File” checked=”true”/>
    <checkbox id=”RestoreProres” name=”restoreMidres” enable=”true” label=”Restore Prores File” checked=”true”/>
  4. Save and close the file.

Update workflow executables

The executable files for Archive workflow are placed by the script installer when running on the Zoom MAM Server. These have to be set as default automatic workflow executables in the Web Management Console. 

  1. Open Web Management Console.
    ex. http://localhost:8443 or http://<zoomserver>:8443
  2. Log in using your admin credentials.
  3. In the Admin Menu sidebar, click on Server Control Panel under the System node.
  4. Click on Workflow Settings on the Server Control Panel page.
  5. Update the Automatic Work Executables Directory to point to the workflow executables folder placed by the script installer (as described here). For example, if you had specified /home/evolphin as the root path for the script installer, then the workflow executables path will be /home/evolphin/zoom-deploy/AutoTasks.

  6. Click Save.
  7. You will be prompted to restart the server. Click Yes.
  8. Refresh your web browser.

Add workflow templates

You need to create workflows in Web Management Console and assign archive and restore scripts to these. To learn more about workflows in Zoom, click here.

  1. Open Web Management Console.
    ex. http://localhost:8443 or http://<zoomserver>:8443
  2. Log in using your admin credentials.
  3. In the Admin Menu sidebar, click on Manage Workflow under the Team Workflow node.

  4. Click on the + sign at the lower bottom corner of the page to add a new workflow.

  5. Drag and drop the Auto task symbol into the design space and attach it to the Start and End tasks using arrowheads that appear while hovering over any task.

  6. Select the auto task and click on the drop-down for Project in the Owner box below the design space. Select your desired project.
  7. Next, click on the Executable drop-down and choose Archive/ from the list.

  8. Carefully click on the Save as Template button on the top bar (not Save button) and save this workflow template as something like ArchiveS3 and click Save.

  9. Click on the + sign again to add another workflow.
  10. Again, drag and drop the Auto task and attach it with the Start and End tasks.
  11. Select the Auto task and select the desired project from the Project drop-down.
  12. Choose the executable as Restore/ from the Executable drop-down.
  13. Carefully click Save as Template and save the workflow template as something like RestoreS3.

  14. The workflow templates for Archive and Restore flow are now set.

You are now set to use the Archive module. Move on to setting up your Zoom Clients to work with VideoFX as described here.