ID: 5024

Print Friendly, PDF & Email

Curator Configuration

From Zoom 6.0 onwards, search speeds have enhanced with the use of a dedicated index and search engine that ships with the Zoom Server installation package. This indexing service is called the Curator.

Curator was available with earlier Zoom versions too, however, only in a limited form – it was used only for file-content searches. Starting from Zoom 6.0, non-file content, like metadata, revision data etc. can also be indexed and looked up quickly. Search operations, even with complex query parameters, are executed very fast, giving the end users a very responsive search experience.

For versions prior to Zoom 6.0, refer to the article: File Content Search.

 

If you are upgrading from an older version of Zoom to 6.0 or above, then the existing Curator data indices must be removed before proceeding. Follow these steps:

  1. Stop Zoom Curator Service.
  2. Open server.xml (Windows: [ZoomInstallDir]\DAM\conf\server.xml Linux: [ZoomInstallDir]/conf/server.xml)
  3. Find tag <solrDataDir> inside <ftsSpec>
  4. Delete the value specified in the <solrDataDir> tag. This value is the directory where the Curator Server stores its data. By deleting this value, all indexes will be removed. The next time when you start the Curator Service, indexing will start from the beginning.

 

The Curator service is not enabled by default in Zoom 6.0

On Linux:

  • Copy the solr-server_init.d.sh from the Zoom_Installation_Directory/bin/ directory to the /etc/init.d/ directory.
  • Rename the newly copied file to solr-server
  • Edit the solr-server file in a text editor, if required, to ensure the values of the following 2 variables match the Zoom setup:
    • ZOOM_HOME=Zoom_Installation_Directory; by default ZOOM_HOME=/home/evolphin/zoom
    • ZOOM_USER=Service_Running_User; by default ZOOM_USER=evolphin
  • Open the Terminal and issue the following commands as the root user:
    • chkconfig --add solr-server
    • chkconfig --list

    Ensure that the solr-server service is listed in the output of the second command.

  • Now the Curator service can be controlled, like other Zoom Services, with the following commands on the Linux Terminal:
    • service solr-server status (Check the status of Curator service)
    • service solr-server start (Start the Curator service)
    • service solr-server stop (Stop the Curator service)
    • service solr-server restart (Restart Curator service)

On Windows:

  • From the Windows Command Prompt, cd to the directory Zoom_Installation_Directory/bin/
  • The Curator service can be installed as a Windows service using the following command:
    • solrserver install
  • After the service installation, Zoom Solr Service can be controlled (start, stop, restart etc.), like other Zoom services, from the Windows Services Manager or Task Manager.
  • Alternatively, the Curator service can be controlled using the following commands, from the Command Prompt:
    • solrserver start (Start the Curator service)
    • solrserver stop (Stop the Curator service)
    • solrserver restart (Restart the Curator service)
  • To uninstall Curator as a Windows services, use the following command in the Command Prompt:
    • solrserver remove
If you find an access denied error on Windows, try running the Command Prompt as an Administrator.

 

From Zoom 7.0 onwards, the Curator service has been renamed from Solr to Curator.

On Linux:

  • Copy the curator-server_init.d.sh from the Zoom_Installation_Directory/bin/ directory to the /etc/init.d/ directory.
  • Rename the newly copied file to curator-server
  • Edit the curator-server file in a text editor, if required, to ensure the values of the following 2 variables match the Zoom setup:
    • ZOOM_HOME=Zoom_Installation_Directory; by default ZOOM_HOME=/home/evolphin/zoom
    • ZOOM_USER=Service_Running_User; by default ZOOM_USER=evolphin
  • Open the Terminal and issue the following commands as the root user:
    • chkconfig --add curator-server
    • chkconfig --list

    Ensure that the curator-server service is listed in the output of the second command.

  • Now the Curator service can be controlled, like other Zoom Services, with the following commands on the Linux Terminal:
    • service curator-server status (Check the status of Curator service)
    • service curator-server start (Start the Curator service)
    • service curator-server stop (Stop the Curator service)
    • service curator-server restart (Restart Curator service)

On Windows:

  • From the Windows Command Prompt, cd to the directory Zoom_Installation_Directory/bin/
  • Install the Curator as a Windows service using the following command:
    • curatorserver install
  • After the service installation, Zoom Curator Service can be controlled (start, stop, restart etc.), like other Zoom services, from the Windows Services Manager or Task Manager.
  • Alternatively, the Curator service can be controlled using the following commands, from the Command Prompt:
    • curatorserver start (Start the Curator service)
    • curatorserver stop (Stop the Curator service)
    • curatorserver restart (Restart the Curator service)
To uninstall Curator as a Windows services, use the following at the Command Prompt: curatorserver remove
If you find an access denied error on Windows, try running the Command Prompt as an Administrator.

 

These steps are for Curator configuration on a non-HADR setup. For Curator configuration on a Zoom HADR setup, refer to the article on Setup of Curator with HADR.

These additional initial configuration steps are only needed if the Curator service is being set up on a separate machine. If the Curator is installed on the same machine as the Zoom Server, then skip to the next section.

  1. On the Zoom Server, stop services for Zoom (all Zoom services installed on the machine). The order to stop services is Zoom Curator, Zoom Preview, Zoom and then Zoom HADR services.
  2. Open server.xml for editing (Windows: [ZoomInstallDir]\conf\ and Linux: [ZoomInstallDir]/conf/).
  3. Locate the tag <ftsSpec>.
  4. Inside the <ftsSpec> tag, set host tag value to Curator Server’s host name or IP address.
    e.g. <host>zm04.company.com<host>
  5. Set solrDataDir tag value to the Curator Server’s database directory; see below for a detailed description.

    Curator Server’s database directory is recommended to be kept on a file-system which provides fast disk access, like local SSD drives, fibre-channel connected drives, etc.

  6. Save the file.
  7. Start services for Zoom (the order to start services is Zoom HADRZoomZoom Preview, and then Zoom Curator services).
  8. On the Curator Serverstop services for Zoom (all Zoom services installed on the machine). The order to stop services is Zoom Curator, Zoom Preview, Zoom and then Zoom HADR services.
  9. Copy server.xml file edited above from the Zoom Server to the Curator Server‘s conf folder (Windows: [ZoomInstallDir]\conf\ and Linux: [ZoomInstallDir]/conf/). Alternatively, edit the server.xml file on the Curator Server in the same manner as above.
  10. Start services for Zoom on this Curator Server (the order to start services is Zoom HADRZoomZoom Preview, and then Zoom Curator services).
  11. Configure the remaining parameters from the the Zoom Web Management Console on the Curator Server. See the next section for steps.
 
  1. Open Zoom Web Management Console in a web browser.
    Sample URL for the Admin Console: http://localhost:8443/
  2. Open the Server Control Panel from the left navigation panel.
  3. Open Curator-based Search Management from the server control panel options.
  4. Change the parameters as described below.

    The configurable parameters are:

    • File Content Index: Check to enable indexing of the contents of assets.
    • File Content Search: Check to enable full text search on the contents of assets.
    • Non-File Content Index: Check to enable indexing of properties of assets (like file name, metadata etc.).
    • Non-File Content Search: Check to enable full text search on non-content data of assets (like file name, metadata etc.).
      If Curator-based search is being enabled after a large number of assets are already present in the Zoom repository, then the initial indexing of the existing assets may take a long duration. It is recommended that only the Indexing flag(s) are enabled initially without enabling the corresponding Search flag(s). This will prevent any disruptions to searches done by users while the indexing is going on. Once the initial indexing is complete, then the Search flag(s) can also be enabled. See this post about how to check Curator’s global indexing status.
    • Audit log Index: Check to enable indexing of audit log files (this is only available from Zoom 7.0 onwards).
    • Port: This is the port on which the Curator service runs. The default value is 8983.
    • Log Level: This is the logging level used for Curator logs. The default value is INFO.
    • Commit Frequency (ms): Enter the frequency (in milliseconds) with which the the Curator Server will check for new indexes and then commit unsaved data to disk.
    • Solr Data Dir: This is the path to the Solr database where the indexing information of the Zoom repository files is stored. The default value is [ZoomInstallDir]/db/solr-db.
      The Solr Data Dir cannot be changed if the directory is no longer empty. That is, it cannot be changed after the first file gets indexed successfully.
    • Solr Base Context: This is the URL append for accessing Curator Server.
    • Failed Documents Notification: This is the frequency for sending reports to super-admins about indexing failures. It can be set to daily, hourly, weekly, or, disable. The default value is daily.

The other parameters are either non-configurable, or, do not usually require any change from the defaults. Additional details about these are in the section Advanced Curator configuration.

Whenever new data is submitted in the Curator-based Search Management form of the Server Control Panel, the submitted details are first sent from Zoom to Curator for verification, and are saved only when these are acceptable.

The communication between Zoom and Curator to establish the correctness of input parameters could be slow on certain setups.

 

In Zoom 7.0, when any Curator parameters are changed and saved in the Web Admin, an email is sent out to the email address(es) set-up to receive Zoom notifications. This is a warning message sent as a reminder to check that the new configuration settings work. You should ignore this message. Here is the sample email message:

 

 

The file-types, for which content indexing and searching is available, are all listed in the IndexableFileType.properties file, present in the conf folder (Windows: [ZoomInstallDir]\conf Linux: [ZoomInstallDir]/conf).

Administrators can add any file-type extension in this file to enable content indexing for that file-type. Removing an entry from this properties file will cause that file-type to be excluded from indexing from then on.

Indexing of the JSON, CSV and XML file formats is performed as plain text files.

From Zoom 6.0 onwards, file content searching is not restricted to any specific set of languages by default.

 

On searching for a particular word, all files containing the input word in full or as part of another word will be returned as matches.

Search for “data” will fetch files containing “data”, “metadata”, “undatable” etc.

Phrase Search or Exact Match Search

To search for an exact phrase or word, enclose it with quotes – ” “.

List of Unsupported Special Characters for “Search Everywhere” in Asset Browser

# < >

List of Special Characters for “Advanced Search” in Asset Browser

The Asset Browser’s Advanced Search supports regular-expression search (with shell wild-card *). So when non-content search is enabled, if the following special characters need to be searched literally, devoid of their regular-expression meaning, they must be prefixed with a \.

& < > \ { } # +

e.g. text\# \<tag\>

If you encounter out of memory exceptions, then increase the Java heap size in conf/solr-server.conf inside the Zoom_Installation_Directory.
wrapper.java.additional.9=-Xmx2g
Here 2g represents 2GB RAM; increase this value accordingly.

If you encounter PERFORMANCE WARNING error often in bin/logs/solr.log, you can increase the number of maxWarmingSearchers. Do this only when confident of the available system resources (RAM, CPU, etc…).
Update the count <maxWarmingSearchers>2</maxWarmingSearchers> in conf/solr-core/ID/conf/solrconfig.xml file inside the Zoom_Installation_Directory.