Zoom 7.6.x series has been specifically built towards maximising the ease of working remotely. The setup usually involves the Zoom servers running on the cloud. The high-res original assets are stored centrally on S3 and shared to users’ desktops on demand via Zoom. This article walks you through the setup required, the end-user experience and ways of troubleshooting when there are hiccups.
Prerequisites
- Zoom 7.6.2+ server is installed and all the services are up and running.
- Zoom 7.6.2+ version is installed on client machines and the necessary basic configurations like setting up of the default Zoom server etc is complete.
- VideoLX is configured on the server and corresponding client-side setup is complete.
- This would include setting up of the third party mount points – the locations where the original high-res assets will be placed on ingest, and from where users could link them into their work files like Premiere Pro projects etc. This is also the location therefore, from where assets will get archived to S3, or restored from S3.
Hub Controls on the Zoom Server
All configurations with respect to the archive hubs as well as a lot of tracking, happen via the Web Management Console.
Hub Configurations
As of Zoom 7.6.2, there are two types of hubs that can be set up in a deployment.
- One is the server-side hub which will service archive / restore requests on direct-ingest assets and low-res proxy data; they also play a role in hi-res ingest from the WebClient application.
- The second type of hub is the client-hub, that is bound to a single Zoom client, and services all archive / cleanup / restore requests operating between that client machine and the tier-2 S3 storage.
The basic parameters for both the types are the same. Go to System -> Hub Settings -> Hub Configuration Panel. Take a look at the screenshot below:
Look at this article for more details about each of the configuration parameters. That article talks about a different topology – that of on-premises deployment, and central server-hub. But the parameters of each hub are practically the same in 7.6.2 release as well.
The key thing to keep in mind vis-a-vis the server-hub vs client-hub is:
- Server-hub configuration is specific to one hub, and is tied to a specific VideoLx location.
- Client-hub configuration is a template or a preset. A client-hub connects to the Zoom server when it starts up, fetches this preset, and then initializes itself based on the parameters defined in the preset. Thus, a client-hub preset configuration is applicable to multiple hubs.
By default, Zoom 7.6.2 comes packaged with 2 default configurations – one server-hub configuration that maps to the default VLX location “Global” and a client-hub preset called the C3_HUB, which all the client-hubs are wired to use by default.
Adding more Hubs, or Presets
Sometimes it is useful to create separate presets for different sets of client-hubs depending on the resources available at the client end namely the processor speeds, RAM, internet bandwidth speeds etc. For example, you could define a client-hub preset called “High-speed Hub” and set the S3 Part Size parameter to 100 MB, and make the download thread count as 24, if the client machine has sufficient resources. Similarly you could define a preset called “Low-power Hub” whose download thread count could be just 4 for client machines that are very basic in terms of hardware, or are required to devote a bulk of their resources for other processes.
Approving / Blocking Specific Users
The client-hub directly transfers data to and from cloud storages like S3, and as there may be a cost aspect to this, the users eligible to have this privilege will have to be manually approved by the Zoom administrators. There are two ways to enforce access control.
- Go to Hub Settings -> Hub User Panel. Mark specific users as Hub users. You can also remove users from this privilege at a later point when it is no longer required.
- The second way is described in the section below. This involves disabling a hub instance instead of a specific Zoom user.
Viewing All Hubs
Go to Hub Settings -> Hub Accounts Page to see the list of all hubs in the Zoom deployment. You will find details like whether it is a server hub or a client-hub, the user associated with each hub, the preset associated if it is a client-hub, the location associated if it is a server-hub and several other details.
This page also provides a means of activating or deactivating a hub instance in the setup. You can do that using the drop-down labeled Change hub account state at the top. See screenshot above.
Setting Default Client-Hub Preset
It is also possible to assign a specific hub preset as the default for all client-hubs connecting to the Zoom server after that. This lets the administrators push a completely new set of hub configuration parameters across all client-hubs which will come in handy say, if the entire deployment facility undergoes a hardware upgrade. There are two ways to do this:
- From the Hub Settings -> Hub Account Panel. See screenshot below
- From the Hub Settings -> Hub Configuration Panel. See screenshot below
Tracking Hub Jobs
The Hub Jobs Panel gives a comprehensive list of all the archive module jobs that are executed in the Zoom ecosystem. The page opens to a list of all hubs in the setup, with their corresponding summary statistics of jobs in various statuses like Pending / In-Progress / Completed and Failed.
On selecting a hub, and then double-clicking on any of the columns, you can see the details of all jobs in that particular status that are handled by that particular hub. See screenshot below – it shows details of the asset ID, job type (cleanup / purge etc), the time when progress status was last updated, and a host of other details.
It is also possible to clear jobs in pending or in-progress status, if absolutely necessary. However, this is a last-resort step and is best left to Evolphin support team to execute if at all required. Deleting jobs without adequate care and analysis may result in the jobs getting permanently stalled, causing problems for the team. So please refrain from doing this.
Remote Debugging Panel
This is an advanced tool for debugging issues encounted on the hubs. This is exclusively for the Evolphin support team to use when required.
Mapping Hubs to Actions
In a deployment, we could add multiple server-hubs and assign specific tasks or actions for each of those, in order to distribute the job load across many nodes. To do this, use the Action to Location page under the Hub Settings.
In Zoom 7.6.2, the url-upload is not supported.
Setting Up Client Hub
To get the client-hub up and running in a Zoom client machine, all it takes is a flag in the zoom.properties.
ENABLE_C3=true
In addition, to be able to see the client-hub status, and to start / stop the process from the Z-menu, add the following flag in the zoom.properties.
SHOW_HUB_MENU_ITEM_ON_Z=true
Both these properties may also be auto-synced from the Zoom server to all the relevant clients; therefore practically no manual configuration is required for the client-hub to operate with the default parameters.
Switching to a new preset
As discussed in the previous section, it is possible to pick up a new preset for your client-hub based on your workstation’s capabilities. In this case, all you need to do is specify the name of the new preset in the zoom.properties as shown below, and restart the Client Proxy service. Restarting the ClientProxy implicitly also restarts the client-hub itself. When the client-hub comes up again, it will have wired itself up with the configuration parameters corresponding to the new preset configured.
HUB_PRESET=Terra Firma Hub
Archive / Cleanup, Restore – In Action
Refer to this article and this article to get a sense of how Zoom ingest works, and how that ties into the client-hub operations with its interactions with the cloud (S3).
Troubleshooting Archive Ops
This article covers the commonly faced issues in the archive module, and how best to resolve them.
- My colleague ingested a hi-res media, and I want to now use it for editing in my local machine. I triggered
Restoreto fetch the asset from S3, but it is not working. Why?- Check the status of the asset from the metadata properties. If the status says
Pending Archive/Submitted Archive/Ongoing Archive, then the asset has not yet reached the S3. Wait for some time, and then retry when the status becomesCopied-To Archive. - To know how long it might take for the copy job to be done, your colleague can check the hub dashboard; the dashboard shows the real-time progress in the data transfer.
- Check the status of the asset from the metadata properties. If the status says
- I ingested a hi-res asset. But its status is stuck in Pending Archive. What is wrong?
- Check if your client-hub is running. If it is not, then you can start it from Z-menu->Hub (Stopped) -> Start.
- If you do not see this option in your Z-menu, then add the following line in the zoom.properties and save. After this, be sure to restart the ClientProxy from the Z-menu.
SHOW_HUB_MENU_ITEM_ON_Z=true
- If the asset is still stuck in the same Pending Archive statatus, then check if you have this flag in the zoom.properties set to true. If it is false, set it to true, and then restart the ClientProxy service.
ENABLE_C3=true
- If after all the above steps, you find that the asset is not copied to S3, check with your Admin as to whether client-hub operations are allowed for your user account.
- I ingested a hi-res asset, and I can see that the metadata status inside Zoom is stuck at
Ongoing Archive. Based on past ingest operations, I feel this operation should have completed by now. How can I check what is going on?- Go to your hub dashboard (Z-menu -> Hub (Running) -> Dashboard). Look for the asset ID in the table, and see what status it shows. If the hub says that the File Copy Completed, then the data transfer part has been completed successfully and the asset is on S3. It is possible that the status update to indicate successful copy (
Copied-To Archive) has failed. This could be either because the Zoom server itself is down, or due to some network disruptions between your client machine and the server. If that is the case, then it will resolve itself whenever the server is accessible again.
- Go to your hub dashboard (Z-menu -> Hub (Running) -> Dashboard). Look for the asset ID in the table, and see what status it shows. If the hub says that the File Copy Completed, then the data transfer part has been completed successfully and the asset is on S3. It is possible that the status update to indicate successful copy (
- I see that one of the assets I ingested, is marked with status
Manualin the hub dashboard. What does this mean? What should I do?- This generally means that the job could not be executed, because the file was not accessible.
- Check if your TPM path (
THIRD_PARTY_MOUNT_POINT) is correctly configured in the zoom.properties. If this is incorrect, none of the jobs would go through successfully. - If the TPM path configured in the zoom.properties is correct, and you have seen ingest operations work successfully in the past, then the problem may be because the TPM path is no longer accessible.
- If it is an externally mounted drive, check that it is mounted correctly; unmount and remount if required.
- Sometimes, for external drives, the drive letters may change on unmount / remount, or system restart. Check if the drive letter for the TPM mount is the same as the value configured in the zoom.properties. Update the value if required, and restart the ClientProxy service from Z-menu.
- Check if your TPM path (
- This generally means that the job could not be executed, because the file was not accessible.
- I am trying to Restore an asset. It is failing, and the hub dashboard says that the file is locked. What to do now?
- Restart the hub (Z-menu -> Hub (Running) -> Stop / Start).
- I am the Zoom Administrator in my company. I have a user who is not able to copy hi-res assets to S3 when she ingests into Zoom. She has the appropriate flags enabled in her zoom.properties. What is going wrong?
- Check if her user account has been marked as a hub user. Go to Web Management Console -> System -> Hub Settings -> Hub User Panel. Select the user who is facing the problem. If the Hub User column says
false, then you need to go to Hub User Operations menu at the top, and click on Mark as Hub User.
- Check if her user account has been marked as a hub user. Go to Web Management Console -> System -> Hub Settings -> Hub User Panel. Select the user who is facing the problem. If the Hub User column says
- I am the Zoom Administrator in my company. I have a user who has recently upgraded to a new workstation. I want her to ingest and copy to S3 from the new workstation, but disallow any hub activity from her old workstation. How can I enforce this?
- Go to Web Management Console -> System -> Hub Settings -> Hub Account Panel. Identify the hub instance corresponding to her old workstation. The name of the hub will be a combination of the user name, and her old machine name itself. Select this row, and then click on Change Hub State, and Mark as Inactive.
- I am the Zoom Administrator in my company. I want to disallow a user from running data transfers to S3. I still want to keep the user for other Zoom operations. How can I enforce that?
- Go to Web Management Console -> System -> Hub Settings -> Hub User Panel. Select the user you want to disallow client-hub operations for. Click on the Hub User Operations menu at the top, and click on Remove from Hub User.
- I am the Zoom Administrator in my company. We have a disgruntled employee and we want to make sure that she doesn’t have access to any hub features from her machine. What can I do?
- Block the user from hub operations from the Hub Settings -> Hub User Panel page.
- Block the hub instance from the Hub Settings -> Hub Account Panel page.
- I am trying to ingest a hi-res asset and it keeps failing with an error message that there is already another file of the same name on the server. I have already deleted the old file of the same name, but still I am getting this error. What should I do?
- It is not enough to simply delete a file to vacate the place for a new asset of the same name. In the case of hi-res / image-sequence files, the asset may have been copied to the tier-2 storage, S3. It may also be present in the TPM path of the users who are working with it. The best way in this case is to purge the asset. On purging an asset, all the data associated with it – whether it is hi-res original data, or low-res proxy data – all are removed from wherever one may find them. Once the data is successfully removed from the S3 and the Zoom database, you can retry the operation.
- I am trying to ingest a hi-res asset and it keeps failing with an error message that there is already another file of the same name on the server. I have already PURGED the old file of the same name, but still I am getting this error. What should I do?
- Even after purging, if you still see the error, chances are that the purge operation is still in the process of removing data from S3 etc. This activity may be going on in the background, and depending on network speeds, and other job load, it may take a little while before the purge completes. Please retry after sometime.
- Meanwhile, if you want to know whether the purge is complete or not, you could open the Web Management Console -> Hub Settings -> Hub Jobs Panel, select your hub, and check under the Pending Jobs, if the purge job for the particular asset ID is listed. If it is fully processed, the entry will move from Pending to Completed Jobs. Another place to check from would be the hub dashboard, where it would list the purge job along with its current status.
- Even after purging, if you still see the error, chances are that the purge operation is still in the process of removing data from S3 etc. This activity may be going on in the background, and depending on network speeds, and other job load, it may take a little while before the purge completes. Please retry after sometime.
- I have a hi-res file in my TPM path. But I am not sure it has been downloaded from S3 correctly. I triggered
Restoreagain but the file is still messed up. How can I resolve this?- If a hi-res file is already present in the TPM location, subsequent
Restorerequests simply skip the operation to prevent incurring wasteful costs by unnecessary downloads from S3. If you think the file is corrupted, and want a fresh copy, delete the file from the TPM path, and then triggerRestore.
- If a hi-res file is already present in the TPM location, subsequent
- I am the Zoom Admin in my company. I see that the server-hub in our deployment is failing to start. I have tried restarting the service multiple times now. What to do?
- Check the following settings. Make changes where required, and restart server hub to see if the problem gets resolved.
- Go to Web Management Console -> System -> Hub Settings -> Hub Configuration Panel. Select the server hub in question, and look up the location parameter. Ensure that it maps to a legitimate VideoLx location defined.
- Open the hub.xml file present in the conf/ folder of the server hub deployment. Ensure that you have entered a valid Zoom username and password in the appropriate sections.
- Go to Web Management Console -> System -> Hub Settings -> Hub User Panel. Ensure that the server hub user is marked as a hub user.
- Go to Web Management Console -> System -> Hub Settings -> Hub Account Panel. Ensure that the server hub is marked as active.
- Check the following settings. Make changes where required, and restart server hub to see if the problem gets resolved.
- I am trying to ingest hi-res assets from WebClient. But the ingest is failing with an error that no hub is configured.
- You (or your Zoom Admin), must configure a server hub to handle the ingests from the WebClient app.
- Go to Web Management Console -> System -> Hub Settings -> Action to Location. Ensure that the
s3-uploadaction is pointing to a correct location here. - Go to Web Management Console -> System -> Hub Settings -> Hub Configuration Panel. Ensure that there is a valid server-hub for the location defined in the above step.
- Go to Web Management Console -> System -> Hub Settings -> Action to Location. Ensure that the
- You (or your Zoom Admin), must configure a server hub to handle the ingests from the WebClient app.
- We have a central server-hub running in our deployment. We ingest hi-res assets and they get copied to the S3 successfully. But the files are immediately removed from the shared P-SAN and so we are not able to use them in our editing work. Why is that?
- Check the hub.xml in the server-hub conf/ folder. Look for the tag
alwaysDeleteFromPsan. If this is set to true, then the files get removed soon after copy to S3. Reset it to false, and restart the server-hub for this to take effect. That should resolve the issue. - An aside here: this flag should be set to true if your server-hub is running on the cloud, and disk storage is scarce there. Otherwise, there is a risk of running out of disk space on cloud.
- Check the hub.xml in the server-hub conf/ folder. Look for the tag