SAYT Configuration File

The SAYT module sifts through all your product data this data will automatically be made available to the SAYT web service.

If no SAYT configuration file is included, client key will be taken from the data configuration file and the default collection will be used by default, as well as field title for keywords.

The accepted file encoding is UTF-8 without BOM.

  • SAYT content will be generated 15 minutes after the upload as completed.
  • All fields are case sensitive.

Available parameters

Below is a table of the configuration parameters for SAYT.

Parameter Required Description
clientKey Yes The client key authorizes your upload and should be entered as found in Command Center.
searchField Yes Limit of 2. Which fields within your record should be used to build the SAYT index. At least one field must be set as a searchField. See more below for restrictions.
navigationField Yes Limit of 5. Which fields within your record should be used to build the Navigation suggestions within your SAYT. At least one field must be set as a navigationField. See more below for restrictions.
searchTermMinOccurrences No Optional: The number of weighted times a term has to appear before it will appear in the SAYT autocomplete service. This is used to prune down the final search term list to prevent overloading the service. Default is 4.
collection No The collection from which SAYT data will be generated. If this is not specified, the default collection will be used.
languageField No If your collection contains multiple languages, this parameter should contain the name of the field that defines the language of a given record. Please refer to language definitions for available languages. If you define a languageField, you must define the language at SAYT query time.
filterField No Limit of 1 If you want to filter SAYT data to a subset of your overall collection, this parameter should contain the field you wish to use as the filter. If you define a filterField, you must define the filter value at SAYT query time.
popularNav No Popular navigations are optionally returned for each keyword to let the user pick a keyword WITHIN a category. E.g. “shoe” can be popular in “Nike”, “Adidas” Brand refinements. This parameter lets you set how many navigation suggestions are returned within each navigation, for each of the keywords. Each keyword will have the popular navigations returned based on all fields defined in navigationField parameter. Therefore, if you have defined 3 items for navigationField and 2 items for popularNav, each keyword will get 2 “popular navigation” suggestions per navigation. Defaults to 1. Note: the navigation must already be configured as a navigationField, and a dynamic navigation in Command Center.
popularNavThreshold No Popular Navigation Threshold determines how many times a keyword must be found within a category before the navigation is suggested as a popular nav. Defaults to 60. Note: the navigation must already be configured in Command Center for it to be returned in SAYT.
stopWords No A list of words that should not be allowed in the first or last word. For example, “Jimmy the Knife” would make it into the type ahead. But, “The Knife” or “knife the” would not.
excludePattern No You can use regular expressions to exclude values from being included in both the navigation search and search field. Some useful regular expressions are provided in the sample config.
searchTermWeightField No Defines a weight field for the search terms generated from that specific record. The input for this value must only be numeric data, i.e. 2 or 2.6. If this value is not specified for a record, it will be given a default weight of 1.
disabled No true or false, false by default. Use this to stop any SAYT data from being generated.
field No A list of fields (field per line) of the fields you wish to white list in the SAYT product results.

SAYT Keyword Generation

Keywords can be generated from your catalog, and shown off partially typed text to help your customers get to the right search term faster.

The keywords are generated following certain rules that you control within the upload configuration, and based off your product catalog data.

Keyword indexing

The upload configuration lets you define 2 fields within your data that will be indexed via searchField.

The first 20 words, or the first 20 Chinese-Japanese-Korean characters of the value will be indexed.

Indexing on multiple phrases

Please note, these fields are broken into every possible three word n-gram and this will generate a lot of data.

If a product has the terms shaggy black dog within searchField this will generate these keywords:

  • “shaggy”
  • “black”
  • “dog”
  • “shaggy black”
  • “black dog”
  • “shaggy black dog”

It will not wrap around and break it down to “dog shaggy” or “dog shaggy black”.

Frequency

By default, a term has to appear 4 times to be added to the SAYT keyword index.

This can be changed by changing searchTermMinOccurrences in the SAYT Upload Configuration.

Keywords and searchField

Fields specified by searchField are used for auto complete keyword suggestions.

  • You are limited to a maximum of 2 search fields.

  • Only the first 20 CJK characters or non-CJK words of the value will be indexed.

  • The weighting of a field (to make it’s content more valuable for auto complete) can be specified by adding a number, separated by a comma, after the field name. If not set, the weight defaults to 1. e.g.

    searchField: title, 6 searchField: brand, 3

  • The weight number works as a multiplier. That is, if you have title and brand, and you say title,6 and brand,3 then words found in title will be counted twice as many times as the same phrase in brand.

  • Please note, these fields are broken into every possible three word n-gram and this will generate a lot of data. Autocomplete could become unresponsive if not used with care. Decide which fields are most important to your user experience.

Stopwords

You can define additional stopwords and apply them via excludePattern. This will affect the keyword index as follows.

Let’s say you add “the” as a stopword, and we have searchField has Shaggy the dog.

This will generate these keywords:

  • “Shaggy”
  • “dog”
  • “Shaggy the dog”

“The dog” or “Shaggy the” would not be added to the SAYT index.

You would add stopwords by these lines:

stopWords:it|that|than|of|the
# starts with a stop word.
excludePattern:^(STOP_WORDS) .*
# ends with a stop word.
excludePattern:.* (STOP_WORDS)$

Navigation Suggestions

navigationFields allow you to suggest to the customer which Navigations are available as they type in their search. Common navigation suggestions are off Brand or Category.

  • You are limited to a maximum of 5 navigation fields.
  • If the value of the field has more than 500 characters, it will be skipped.
  • If the field type is defined as PartNumber, returned results will be similar to a PartNumber search

Whitelisting Product Results

By default, only the title field will be returned for SAYT product results. You can specify any fields you wish to white list here. This is done as the SAYT product results don’t require your client key, and therefore you want to ensure no sensitive data is exposed in the products results. More than one field can be defined.

Here is an example list of exposed fields:

field: title
field: product.price
field: product.image
field: product.url

Example Configuration File

Below is a typical configuration for generating the SAYT data during upload.

txt
# Mandatory
# Your client key as found in the command center.
clientKey: --clientKey--

# Mandatory
# In SAYT there are three sections visible to the user.  Autocomplete, navigation
# search and product recommendations.  Include navigationFields to define
# which navigation options should be used for navigation search.
# navigationField:categoryName
# navigationField:type
navigationField:brand

# Mandatory
# Fields specified by searchField are used for auto complete. In addition,
# a weight can be given to terms found in each specified field. This can
# be done by adding a number, separated by a comma, after the field name.
# If weight is not set, the weight defaults to 1. Please note, these fields are
# broken into every possible three word n-gram and this will generate a
# lot of data. Autocomplete could become unresponsive if not used with care.
# Decide which fields are most important to your user experience.
searchField:title
#searchField:title,4
#searchField:title,4.5

# Optional
# This parameter represents the sub-collection and must match the sub-collection
# defined in one of your Upload Module configurations. If no sub-collection was defined
# in the configuration for the upload, this parameter does not need to be overwritten.
# collection: products

# Optional
# A flag to disable generation of SAYT data
# disabled: true

# Optional
# A list of words that should not get into the type ahead's first or last word.
# For example, "Jimmy the Knife" would make it into the type ahead.
# But, "The Knife" or "knife the" would not.
stopWords:although|have|also|yes|no|you|and|for|to|the|that|with|which|is|in|allows|a|in|any|on|your|be|without|has|an|these|then|we|them|or|their|it|that|than|of|from|at|as|can|by|not|does|if|what|when|no|all

# Optional
# You can use regular expressions to exclude values from being included
# in both the navigation search and search field.
# https://www.regular-expressions.info//
# starts with a symbol.
excludePattern:^[^a-z^0-9]+ .*$
# ends in a symbol.
excludePattern:^.* [^a-z^0-9]+$
# is a total of 1 character.
excludePattern:^.{1}$
# starts with a number. (disabled)
#excludePattern:^[0-9]+ .*$
# ends with a number. (disabled)
#excludePattern:^.* [0-9]+$
# is only a number.
excludePattern:^[0-9]+$
# ends with a space and single or double character.
excludePattern:^.* .{1,2}
# ends with a space and a single or double character.
excludePattern:^.* .{1,2}
# starts with a single or double character.
excludePattern:^.{1,2} .*$
# has a single middle ngram
excludePattern:. .{1} .
# starts with a stop word.
excludePattern:^(STOP_WORDS) .*
# ends with a stop word.
excludePattern:.* (STOP_WORDS)$
# is entirely a stop word
excludePattern:^(STOP_WORDS)$
# contains a colon in the middle.
excludePattern:.*:.*
# has a comma or full stop then a space.
excludePattern:.*[\.,] .*
# ends with a comma or a full stop
excludePattern:.*[\.,]$

# Optional
# A language field can be specified to switch between character-based (for languages such as Thai,
# Chinese, Korean or Japanese) and the default (word-based) processing. Records with supported
# character-based languages will generate terms from all defined searchFields by splitting on
# characters rather than by words. Please refer the documentation for a list of supported languages.
# Please note, The "lang_" prefix is optional. For example, the field value can either be "lang_en"
# or "en". If the language provided in the field is not recognized, the default word-based processing
# is used.
#languageField: language

# Optional
# A filter field can be specified to filter suggestions to a subset of your overall collection (e.g. products for a specific
# site.)
# filterField: variants.stores.storeID


# Optional
# By default, all the fields specified here are returned by the SAYT product search endpoint. As the endpoint is not secure
# (does not require your client key to query) it is imperative that this "field" parameter is used to
# tell the service which data should be returned in the allMeta section.  More than one field can be defined.
#field: title
#field: price
#field: image