ID: 5024

Curator Configuration

In Zoom 6.0 and above, search speeds have been enhanced with the use of a dedicated indexing and searching engine that ships with the Zoom server installation package. This indexing service is known as Curator.

Curator was available in earlier Zoom releases too, however, only in a limited form – it was used only for file-content searches. From Zoom 6.0 onwards, non-file content, like metadata, revision data etc. can also be indexed and looked up. Search operations, even with complex query parameters, are executed very quickly, giving the end users a very responsive search experience.

  • If you are switching from an older version of Zoom to Zoom 6.0, or above, then the existing Curator data indices must be removed before using Curator.
  • Refer to the article on cleaning older indexes.

 

For versions prior to Zoom 6.0, refer to the article: File Content Search.

The Curator service is not enabled by default in Zoom 6.0

Linux machine:

  • Copy the solr-server_init.d.sh from the Zoom_Installation_Directory/bin/ directory to the /etc/init.d/ directory.
  • Rename the newly copied file to solr-server
  • Edit the solr-server file in a text editor, if required, to ensure the values of the following 2 variables match the Zoom setup:
    • ZOOM_HOME=Zoom_Installation_Directory; by default ZOOM_HOME=/home/evolphin/zoom
    • ZOOM_USER=Service_Running_User; by default ZOOM_USER=evolphin
  • Open the Terminal and issue the following commands as the root user:
    • chkconfig --add solr-server
    • chkconfig --list

    Ensure that the solr-server service is listed in the output of the second command.

  • Now the Curator service can be controlled, like other Zoom Services, with the following commands on the Linux Terminal:
    • service solr-server status (Check the status of Curator service)
    • service solr-server start (Start the Curator service)
    • service solr-server stop (Stop the Curator service)
    • service solr-server restart (Restart Curator service)

Windows machine:

  • From the Windows Command Prompt, cd to the directory Zoom_Installation_Directory/bin/
  • The Curator service can be installed as a Windows service using the following command:
    • solrserver install
  • After the service installation, Zoom Solr Service can be controlled (start, stop, restart etc.), like other Zoom services, from the Windows Services Manager or Task Manager.
  • Alternatively, the Curator service can be controlled using the following commands, from the Command Prompt:
    • solrserver start (Start the Curator service)
    • solrserver stop (Stop the Curator service)
    • solrserver restart (Restart the Curator service)
  • To uninstall Curator as a Windows services, use the following command in the Command Prompt:
    • solrserver remove
If there is an Access Denied error on Windows, try running the Command Prompt as Administrator.

 

For Curator configuration on a Zoom HADR setup, refer to the article on Setup of Curator with HADR.

These additional initial configuration steps are only needed when the Curator service is being set up on a separate machine. If the Curator service is to be run on the same machine installation as the Zoom service, then skip to the next section.

  1. Manually edit the Zoom_Installation_Directory/conf/server.xml on the Zoom server in the following manner.
    • Open the server.xml in a text editor.
    • In the editor, move to the ftsSpec section.
    • Set the host tag value to the Curator service machine’s host name or IP.
      e.g. <host>zm04.company.com<host>
    • Set the solrDataDir tag value to the Curator service database directory; see below for a detailed description.

      The data directory is recommended to be located on a file-system which provides fast disk access, like local SSD drives, fibre-channel connected drives, etc.

    • Save the file.
  2. Copy the server.xml file edited above, to the Curator server machine, overwriting Zoom_Installation_Directory/conf/server.xml. Alternatively, edit the server.xml file on the Curator server in the same manner as above.
  3. Start/Restart the Curator Service
  4. Start/Restart Zoom Service
  5. Configure the remaining Curator service parameters from the the web-based admin console. See the section below.

If Curator-based search is being enabled after a large number of assets are already present in the Zoom repository, then the initial indexing of the existing assets may take a long duration. It is recommended that the Indexing flag(s) is enabled initially, without enabling the corresponding Search flag(s). This will prevent any disruptions to searches done by users while indexing is going on. Once the initial indexing is complete, then the Search flag(s) can be enabled. See this section on how to check Curator’s global indexing status.

 
  1. Open the Web-based Admin Console in a web browser.
    Sample URL for the Admin Console: http://localhost:8443/
  2. Open the Server Control Panel from the left navigation panel.
  3. Open the Curator-based Search Management form in the central control panel.
  4. Change the config, as required, and then save the form. The descriptions of the various parameters are given below.

Curator-based Search Management Form

The configurable parameters signify the following:

  • File Content Index: Specify whether Content Indexing has to be enabled or not.
  • File Content Search: Specify whether Content-based Searching has to be enabled or not.
  • Non-File Content Index: Specify whether Non-Content Indexing has to be enabled or not.
  • Non-File Content Search: Specify whether Non-Content based Searching has to be enabled or not.
  • Host: Curator service machine host name or IP; as will be accessed from the Zoom service (Default: localhost)
  • Port: The port on which the Curator service runs (Default: 8983)
  • Log Level: The logging level to be used for Curator logs (Default: INFO)
  • Failed Documents Notification: Frequency for sending email notifications to the super-admins about documents which failed to get indexed. It can be set to daily, hourly, weekly, or, disable. (Default: daily).
  • Solr Data Dir: Path to the Solr database directory on the Curator service machine; where the indexing information of the Zoom repository files is stored. (Default: “Zoom_Installation_Directory/db/solr-db” )

    The Solr Data Dir cannot be changed if the directory is no longer empty. That is, it cannot be changed after the first file got indexed successfully.

The other parameters are either non-configurable, or, do not usually require any change from the defaults. Additional details about them can be found in the advanced Curator config specification.

Whenever a new change is submitted in the Curator-based Search Management form of the Server Control Panel, the submitted details are first sent from Zoom to Curator for verification, and are saved only when they are acceptable.

The communication between Zoom and Curator to establish the correctness of input parameters could be slow on certain set-ups.

 

The file-types, for which content indexing and searching is available, are all listed in the IndexableFileType.properties file, present in the Zoom_Installation_Directory/conf folder.

Administrators can add any file-type extension in this file to enable content indexing for that file-type. Removing an entry from this properties file will cause that file-type to be excluded from indexing from then on.

Indexing of the JSON, CSV and XML file formats is performed as plain text files.

With Zoom 6.0 onwards, by default, file content searching is not restricted to any specific set of languages.

 

On searching for a particular word, all files containing the input word in full or as part of another word will be returned as matches.

Search for “data” will fetch files containing “data”, “metadata”, “undatable” etc.

Phrase Search or Exact Match Search

To search for an exact phrase or word, enclose it with quotes – ” “.

List of Unsupported Special Characters for “Search Everywhere” in Asset Browser

# < >

List of Special Characters for “Advanced Search” in Asset Browser

The Asset Browser’s Advanced Search supports regular-expression search (with shell wild-card *). So when non-content search is enabled, if the following special characters need to be searched literally, devoid of their regular-expression meaning, they must be prefixed with a \.

& < > \ { } # +

e.g. text\# \<tag\>

If you encounter out of memory exceptions, then increase the Java heap size in conf/solr-server.conf inside the Zoom_Installation_Directory.
wrapper.java.additional.9=-Xmx2g
Here 2g represents 2GB RAM; increase this value accordingly.

If you encounter PERFORMANCE WARNING error often in bin/logs/solr.log, you can increase the number of maxWarmingSearchers. Do this only when confident of the available system resources (RAM, CPU, etc…).
Update the count <maxWarmingSearchers>2</maxWarmingSearchers> in conf/solr-core/ID/conf/solrconfig.xml file inside the Zoom_Installation_Directory.