Field Definitions

This page will cover the various parameters used in Field Definitions:

1. Field Types
2. Wildcard Definitions
3. Search Type
4. Refinable & Sortable Configurations
5. Deprecated Fields

Field Type

Field Type: String

The type will default to String if not provided

String, Integer, Float and definitions can only be applied to values, or arrays of values, within your data.

Field Type: Integer, Float

Use Integer or Float to define a field as numeric.

Integer defined fields will round the contained value if it is not an integer.

Fields should be defined as numeric in order to be used in Numeric Biasing or to be sorted numerically.

Field Type: ChildRecord

ChildRecord definition can only be applied to arrays of values (specifically, arrays of objects within a single record) in your data. This is a special definition, that is used for supporting a variety of in-record variant structures.

Use the ChildRecord type to denote a field as a child record of the parent record. This will allow search refinements to match fully against child records.

Basic ChildRecord

For example, let’s say we have the following record:

Use the ChildRecord type to denote a field as a child record of the parent record. This enables you to have multiple versions of one product (such as size or color variants, or pricing for different stores) within a single record.

At the most basic, ChildRecords manage refinements against the variants in an accurate manner. For example, let’s say we have the following record with two variants, of size and color:

{
 "title": "shirt",
 "variants": [
   {
     "color": "red",
     "size": "XL"
   },
   {
     "color": "blue",
     "size": "M"
   }
 ]
}

By default, refining on variants.color:“red” and variants.size:“M” will bring back the record even though the product doesn’t exist as “M” and “red”, only as “M” and “blue”, because the match will extend across all attributes.

Setting variants to type: ChildRecord will force refinements to match fully within the children on the record.

Configuration file:

fieldDefinition: {name: "variants", type: "ChildRecord"}

When we set that, “M” will not come back as an available option for variants.size.

ChildRecord: Filtering

fieldDefinition: {name: "pricingRate", type: "ChildRecord", childAction: "Filter"}

Once a field is denoted as a ChildRecord, we can tell the engine to only return the relevant child records.

The business use case is a situation where you could have multiple stores with different inventory counts, or pricing models in a B2B relationship, and you’d like to segregate some parts of the record from polluting the recall or relevancy.

For example, given the following record:

 {
   "title": "shirt",
   "pricingRate": [
     {
       "group": "corporate",
       "price": "5.99"
     },
     {
       "group": "individual",
       "price": "8.99"
     }
   ]
 }

We want to ensure that Corporate users only see their pricing, and Individuals only see theirs - and that there is no chance of an individual seeing the corporate rate.

By default, refining on pricingRate.group:“individual” will bring back the full record, which we don’t want as it would return the corporate pricing within the record.

The childAction of Filter will restrict the returned child records to just the matching ones, so that we only get the correct pricing back.

Using the same example above, when refining on pricingRate.group:“individual”, the record returned would be

{
  "title": "shirt",
  "pricingRate": [
    {
      "group": "individual",
      "price": "8.99"
    }
  ]
}

instead of the full record.

This means dynamic navigation and front-end rendering will not show the other pricing rates.

ChildRecord: Ordering

fieldDefinition: {name: "variants", type: "ChildRecord", childAction: "Order"}

We can use the ChildRecord model to automatically re-order subvariants within the record by relevancy. The kinds of business requirements this could meet is when we have multiple visual variants within one record - different sizes, or commonly, different colors.

Once a field is denoted as a ChildRecord with an action of order, we can tell the engine to place the most relevant child records at the top.

The relevancy will be calculated using the entire sub-record, using the fields for refinement, or, if this was a search, using searchable fields defined within the sub-record. This relevancy will take into account the biasing profile that is being applied.

For example, given the following record:

 {
   "title": "shirt",
   "variants": [
     {
       "color": "red",
       "size": "XL"
     },
     {
       "color": "blue",
       "size": "M"
     }
   ]
 }

By default, searching for “blue” will bring back the full record (child records in the original upload order). Setting the child action on variants to Order will place the most relevant child records at the top.

Using the same example above, when refining on variants.color:“blue”, the record returned would be

 {
   "title": "shirt",
   "variants": [
     {
       "color": "blue",
       "size": "M"
     },
     {
       "color": "red",
       "size": "XL"
     }
   ]
 }

When searching for “blue” the record returned would also be:

  {
    "title": "shirt",
    "variants": [
      {
        "color": "blue",
        "size": "M"
      },
      {
        "color": "red",
        "size": "XL"
      }
    ]
  }

instead of the original upload order:

 {
   "title": "shirt",
   "variants": [
     {
       "color": "red",
       "size": "XL"
     },
     {
       "color": "blue",
       "size": "M"
     }
   ]
 }

Field Type: PartNumber

Fields defined as PartNumber will allow partial alpha-numeric search where special characters are ignored for match.

For example, for the following field definition and records:

fieldDefinition: {name: "title", type: "PartNumber", search: "All"}

 {"id": "1", "title": "abc123"}
 {"id": "2", "title": "abc 12"}
 {"id": "3", "title": "abc-123"}

Query	Search Result Ids
“abc”, “c12”	[1,2,3]
“123”, “abc123”, “abc 123”, “abc-123”	[1,3]

Field Type: LengthAgnosticPartialMatch

Fields defined as LengthAgnosticPartialMatch will allow partial alpha-numeric search where the length of the string is ignored for match.

For example, for the following field definition and records:

fieldDefinition: {name: "title", type: "LengthAgnosticPartialMatch", search: "All"}

  {"id": "1", "title": "Symphony in C"}
  {"id": "2", "title": "Symphony in D minor"}
  {"id": "3", "title": "Symphony from the New World"}

When searching for “Symphony”, all three records above would be considered equally relevant based on the search term regardless of the title length.

Field Type: Keywords

Fields defined as Keywords are useful for a list of short words that tag/categorize your records. For example, for the following records,

{"id": "1", "title": "Blue Jeans", "tags": ["Pants", "<company_brand1>"]}
{"id": "2", "title": "Leather Jacket", "tags": ["Jacket", "Leather", "<company_brand2>"]}
{"id": "3", "title": "Leather Belt", "tags": ["Leather", "<company_brand2>"]}

a good candidate for using Keywords would be the field tags, which would be defined as follows:

fieldDefinition: {name: "tags", type: "Keywords", search: "All"}

Field Type: displayName

You can set a displayName for a given field that will be used to bring back a user friendly name for refined fields that are not setup as a Dynamic Navigation within Command Center.

If you apply a refinement on a field within your data that’s not defined in Dynamic Navigation, that field will be listed in selectedRefinements portion of the response. If a field is not defined in Dynamic Navigation, then the displayName set at upload time will be added into that portion of the response. If it is defined in Command Center Dynamic Navigation, then the value in Command Center will override the one at upload.

This allows you to have graceful fallback if you do not wish to have to define display names for all the fields that are used for refinements, and you do not wish to use those fields as Dynamic Navigation.

Field Type: additionalId

You can mark multiple fields as being an additionalId, so that the engine can correctly map the record via those IDs to other cloud services, like Recommendations.

This is useful if you have multiple fields within your record that mark the product or the SKU id for the record - in particulary, if the record id field does not match the productId that is sent with Recommendations.

To flag a field as an Additional ID, set it in the field definition as follows:

fieldDefinition: {name: "variants.productSku", additionalId: true}

Unless it is flagged, a field is not an Additional ID as a default, so it is never necessary to include “additionalId: false” in your field definitions.

You may define as many Additional ID fields in a collection as you require.

A field of type ChildRecord cannot be flagged as an Additional ID field; a field nested within the ChildRecord, however, can be.

The core strength of Additional IDs is how they work with the Recommendations engine. If you are using clickboost and/or conversion boost with your cloud data, you are able to track product views and orders by a field other than the Product ID if you flag it as an Additional ID. The engine will always attempt to match your data to Recommendations metrics by Product ID first, so it is okay to have tracking on the Product ID and Additional ID(s) of the same record. However, to prevent intersections, it is important to ensure that Additional ID values are unique among not only themselves, but of all Product ID values as well. If it is possible for a field is to have same value as a Product ID, reconsider it from acting as an Additional ID.

Wildcard Definitions

You can define fields in bulk. For example, if you want to make all fields that contain "number" to be an Integer, you can do it with one line:

fieldDefinition: {name: "*.number.*", type: "Integer"}

The field name supports two types of wildcard characters: ? and *. The ? wildcard will match one character. For example "????_price" will match "sale_price", but not "sales_price". The * wildcard will match any number of characters. For example, a name of "*_price" will match both "sale_price" and "sales_price", but not "sale_prices". It is important to note that for field definitions containing wildcard characters, the defined type applies to all matching fields.

Wildcarded field definitions should be ordered from most specific to most generic. In the examples below, with the defined order, field.specific.text has the type "Integer" while all other fields that match the pattern field.*.text have the type "String".

Note that "field.wrong.text" will not be defined as Integer - it is overwritten by the generic configuration above it.

    fieldDefinition: {name: "field_name", search: "All", type: "String"}
    fieldDefinition: {name: "field.specific.text", type: "Integer"}
    fieldDefinition: {name: "field.*.text", type: "String"}
    fieldDefinition: {name: "field.wrong.text", type: "Integer"}
    fieldDefinition: {name: "field.*", search: "All"}

When defining a specific field, explicitly define all of it’s settings. Once a field is defined, the definitions below will not apply to it.

In other words: fieldDefinitions do not cascade - the definition highest in the file wins. In the example above, only field_name will be searchable.

When defining a field as a String, Float, or Integer, it must contain a value (not be an object). In JSON, for example, you can define this type of record structure (pretty printed below, but needs to be in one line for upload):

{
    "id":"782",
    "title":"Navy dress",
    "description":"Pair this dress with your favorite jacket to work all day in comfort and style.",
    "items": [
      { "color": "light-blue",
        "price":"99",
        "brand":"lion",
        "fabric":"cotton",
        "size":["6","8","10","12"]
      },
      { "color": "light-blue",
        "price":"68",
        "brand":"lion",
        "fabric":"cotton",
        "size":["2"]
      } ],
    "categoryId": "3"
  }

As:

fieldDefinition: {name: "title", search: "All"}
fieldDefinition: {name: "description", search: "RelevancyOnly"}
fieldDefinition: {name: "items.price", type: "Float"}
fieldDefinition: {name: "items.size", type: "Integer"}
fieldDefinition: {name: "items", type: "ChildRecord"}
fieldDefinition: {name: "items.*", search: "All"}

This will set items.brand and items.fabric as searchable. This will not set items.price and items.size as searchable. This definition is valid for items.size, because it contains an array of Integers.

Search Type

Use "All", "None", "RelevancyOnly" or "RecallOnly" to specify how a field should influence relevancy and recall.

"All" will influence both recall and relevancy. "RelevancyOnly" won’t influence recall, but will still influence relevancy. This is useful for a field such as description where you may not want a search to return a record if a search value is found only within description, but if the record is returned from other fields, this field will boost it’s relevancy. "RecallOnly" does influence recall, but will not influence relevancy. However, this field can still be affected by biasing and sorting. "None" will not influence either. This is the same as not making a field searchable.

At least one field must exist with search set to "All".

Legacy Support

Using true and false for search is deprecated. Use the three new options instead. However, using true will set search to "All", and using false will set search to "None".

Refinable and Sortable Configurations

A field can be set as searchable, sortable, refinable, or any combination of the 3.

All fields will be returned in the response (if they included in the fields portion of the query), regardless of their sortable or refinable status.

Flags

Here are the flags that must be set to true if you wish to have the functionality listed available.

If one field has multiple requirements, a single fieldDefinition combining all the flags must be created.

Functionality	Searchable	Refinable	Sortable	Type
Content is searchable	Yes	No	No	Any
Biasing by exact value	No	Yes	No	Any
Biasing without an exact value	Yes	Yes	No	Any
Biasing, numeric	No	Yes	No	Float or Integer
Navigation or refinement, Value	No	Yes	No	Any
Range or Dynamic Range navigation, or Range refinements	No	Yes	No	Float or Integer
Sorting, A-z	No	No	Yes	Any
Sorting, numeric	No	No	Yes	Float or Integer

Biasing by “searched” for value is used when you wish to influence relevancy by specifying which fields should be used to boost, or bury, records when the searched content hit against them.

See more under Biasing and Relevancy within the Command Center section.

Why use these flags?

Searchable fields are used to generate search results - you will want to mark fields that you want to be indexed as searchable. Explicitly defining sortable and refinable fields will optimize the indexing performance - if you anticipate loading large amounts of data, or loading data very frequently, this will speed up the process of loading your data.

Please note, if you are whitelisting which fields are sortable, you need to set these flags as well as other information about each field - e.g. if it is Numeric.

What are the defaults

By default no fields are searchable, and all fields are sortable and refinable.

If refinable or sortable are not set per field, they will take on the value of defaultRefinable and defaultSortable respectively (which is by default, true).

We recommend that you keep the default configuration and retain all fields as sortable and refinable, unless indexing performance is of paramount performance.

How to use Sortable

If you want to be able to sort by a value, the field should be sortable.

To set a field as sortable, set sortable: true:

fieldDefinition: {name: "field_name", sortable: true}

How to use Refinable

If you want to filter, or bias by a field, it should be refinable. To set a field as refinable, set refinable: true:

fieldDefinition: {name: "field_name", refinable: true}

Internationalization

If your collection uses languages other than English, you can define what language is a given field in, by using the languages attribute as part the fieldDefinition. One or more languages can be defined on a field. If no languages are defined, lang_en is used by default. See the Language Reference for a list of available languages. You should define any languages that are not English as it will improve recall and relevancy for searches in that language.

Sample definition of a field in 2 languages:

fieldDefinition: {name: "field_name", languages: ["lang_en", "lang_fr"]}

Sample definition of a field in one language:

fieldDefinition: {name: "field_name", languages: "lang_zh-CN"}

Secured Collections

A field can be set as an access control field. If set, the collection requires a securedPayload to be sent on every request. The securedPayload will be an encrypted query that the user obtains once logging into your site, which will restrict the user to only view a subset of the data. If the securedPayload is not included, the request will fail.