Configure Amazon AI in Zoom

After setting up the Hub Server and configuring Amazon settings on it, we also need to configure detailed Amazon AI parameters in Zoom using the Web Management Console. We need to set Amazon as the AI Provider, provide configurations for AI engines, and then map AI rules to AI engines. Follow these steps:

AI Providers

You need to add one or more AI providers to work with Zoom. Follow these steps to add a provider:

  1. Log in to the Web Management Console and click AI Providers under the AI Configuration node in the Admin Menu sidebar. You need to add an AI Provider here.

  2. Inside the AI Providers panel, click Add and choose Amazon from the list of AI Providers.

  3. Select the newly added row for Amazon. Its details will be loaded in the right side Information panel.

  4. Add/update these values in the Information Panel:
    1. aws_secret_access_key – specify your AWS secret access key. It is generated from amazon/aws console.
    2. aws_access_key_id – specify your AWS access key id. It is also generated from amazon/aws console.
    3. region – specify the region where this AI Provider will run the AI jobs. The default value is us-west-2.
    4. minConfidence – specify the minimum confidence needed for any match to be returned to Zoom as the result for an AI category. You can specify a value between 0.00 to 1.00. The default value is 0.55, which shows 55% probability.
    5. maxLabels – enter the maximum number of results that you want to receive for an AI category. The default value is 100, which is also the maximum number of results that Amazon can send per AI category. You can specify a value of 100 or less.
    6. enable – this is true by default to enable this provider. Setting this value to false will disable this provider in Zoom, so it will not be used further. If this value is missing in the configuration files, it will still be treated as true.
      The values specified here are all case-sensitive.
      For other providers, different sets of values may be needed.
  5. After updating the provider values, move to configure this provider on the AI Configurations page in the next section.

AI Configurations

You need to add one configuration per AI Engine provided by your AI Provider. You would get the Engine ID and type from them for each AI engine. Follow these steps to add the necessary configurations:

  1. In the Web Management Console, click Configurations under the AI Configuration node in the Admin Menu sidebar. You need to add AI configurations here for your AI Provider.

  2. Inside the Configurations panel, click Add to add a new configuration.

  3. Provide a name for the new configuration. It is best to use names relating to the task performed by the engine and the names should be unique.

  4. Select the newly added row for the new configuration. Its details will be loaded in the right side Properties panel.

  5. Choose an AI provider from the AI Providers dropdown box above the Properties panel. Choose the provider, Amazon, that we had added in the previous section.
  6. Click Add in the Properties panel for each property that you want to add for the new AI Configuration.

  7. You need to fill Label and Value for each property at the minimum. The Description is optional.

    For example, for the AI Type property, you should add the Label as type and Value as its AI type. So, Label will be type and Value may be transcription.
  8. Each configuration that you add should have exactly one AI Engine and one AI Type property in it. Some configurations will also need additional properties depending on the AI Engine that is being configured. Each group of AI Engine and AI Type together should be exclusively configured in one configuration only, and these cannot be repeated. Add these properties for the new configuration:
    Label Value
    type Add the AI type that should be associated with this configuration. This is the AI Engine’s AI Type that is provided by Veritone.
    enable Optionally, set this property to false to disable this AI Engine. The Engine will be disabled and no further jobs will be created using this AI Engine.
    Similarly, setting ‘enable’ as false for an AI Provider will disable that provider and all the engines associated with it.
    minConfidence Specify the confidence level in the range from 0.00 to 1.00. Only data that has a confidence level equal or higher will be added to the Zoom database. The rest will be discarded. The value of minconfidence here overrides the same value from the AI Provider configuration above. The value from AI Provider configuration will be used if it is not specified here.
    maxLabels The default value is 100. Amazon can give a maximum of 100 values per AI category. You can set lower than 100 if fewer values are required in the result. The value of maxLabels here overrides the same value from the AI Provider configuration above. The value from AI Provider configuration will be used if it is not specified here.
    type should be specified for each AI Engine.

    enable, minConfidence, and maxLabels are optional.

    All property values specified are case-sensitive.
  9. After updating the configurations for engines inside your AI providers, move to map the engines with configurations on the AI-Mappings page in the next section.

AI-Mapping

Finally, you need to map AI engines with Zoom, to control what assets will be analyzed by each engine. Follow these steps to map asset rules to an engine:

  1. In the Web Management Console, click Mapping under the AI Configuration node in the Admin Menu sidebar.

  2. Click Add to add a new mapping rule.

  3. Specify these values:
    1. Rule Name: specify a name for the mapping -rule.
    2. File Format: specify a comma-separated list of file extensions associated with this mapping. For e.g.: MP4, MXF. Only files matching the given extension(s) will be sent for analysis to the AI Engine. If not specified, then all file types are considered valid for this mapping.
    3. Engine: choose an AI engine for this mapping. Each mapping needs an engine. The associated engine will be used to extract the AI data as per the rules specified in this mapping.
    4. Metadata Rule: specify metadata key-value pairs to be matched for selecting a file. Multiple metadata property value pairs can be specified in this field, separated by commas only (no spaces). Only those assets that have these values for the specified properties will be sent for AI analysis.

      Values can be specified like these examples:

    1. IPTC_City=Milan (Only assets with city value Milan will be selected for this engine).
    2. IPTC_COUNTRY=Italy,AND,IPTC_Season=2019 (Assets with the country Italy and Season value 2019 will be selected for this engine)
      Multiple metadata values should be separated by commas. AND/OR operators are also supported.
      Value comparison for metadata properties is case insensitive, which means Zoom Server will match Japan with japan.
      When nothing is specified for this field, all assets are selected.
    1. Paths: specify the path to match assets from. Only files present in this path are selected for AI analysis.
    2. Run Once Per Asset: if this is enabled then an asset is analyzed with AI only once throughout its lifetime, even after it is moved, renamed, or a new revision is checked in.
      Rule name and Engine cannot be empty. Any other field left blank will match everything. For example, leaving Paths blank will match all paths on the Zoom Server.
  4. Click Save to save the mapping rule.

Amazon’s AI configuration in Zoom is now complete.

AI Types supported in Zoom

Zoom supports a fixed number of AI types. AI type is the way data is matched by the AI engine. These AI types are provided by your AI provider. These are later configured in Zoom as shown in the AI Configurations section above. The following AI types are supported in Zoom:

  1. face – denotes and categorizes AI data related to human faces. This data has various information related to faces, such as age, emotions, race, etc. 
  2. object – identifies and names objects and items in the real world, like glasses, bicycles, crayons, ties, suits, etc.
  3. text – denotes extracted text (like OCR) from images or videos.
  4. celeb – denotes well-known celebrities and is attached to their name.
  5. all – this is used to fetch all types of AI data from Amazon. It is generally not recommended.

The AI type transcription is not supported by Amazon.

You should check with your AI provider about the AI types provided with their AI engines.