Technical Specifications and Usage Limits

June 2019

This document provides information about system limits in the data, business rule and query portions of the search engine. Many of these limits are configurable - please reach out to your Customer Advocate if you believe you need to go beyond the listed value.

These limits might have recommended values for acceptable performance or limits that are intentionally built into the system and cannot be changed. For most of these you shouldn’t need to go beyond the listed limit. You can use this information to help you plan your deployment and to configure your system for optimal performance. The information is organized into sections based on feature.

Upload and Data Model

Data item Limit Notes
Number of collections 20 Can be increased by request.
Characters used in collection name 450 Collection names must be alphanumeric.
Individual record size, XML 200 KB
Individual record size, JSON 32 MB
Record count, collection 10,000,000 Can be increased by request.
Record count, end-point 50,000,000 Can be increased by request.
Unique fields per record, JSON 1,000
Searchable fields per collection, JSON 1,024 When have wildcard searches on.
Maximum nesting depth, JSON 8 levels More detail available under Data format
Field names Characters in [a-zA-Z0-9-_ ]
Field name length Total of 80 characters or less
Reserved field name categories
Mandatory field, all records title
Mandatory field, all records id
Number of invalid records allowed for upload 100 Only applies when failOnFirstError in upload config is false
Navigations, bulk upload 20,000 This can only be done with Bulk Upload and not via Command Center. The engine will only evaluate up to 250 within a single query.
Number of simultaneous uploads to 1 collection 3 Any uploads sent over this limit will be rejected with a 429 - Too Many Requests message. If you are streaming multiple partial updates the best practice is to have a retry period roughly the length of your average upload (which will depend on the size of your uploads).

Query and Response

Data item Limit Notes
Search query 60 characters If exceeded, the engine will truncate the search query and return a warning. If the query term cannot be truncated, a 413 error will be returned.
Search query 10 terms
Search query 300 bring to top ids If exceeded, the list of ids will be truncated.
Refinements within the query 300 refinements This is the number of individual refinements that can be passed within a single query to the search engine.
Response, number of records per page 120
Response, pagination limit 10,000 Pagination is supported up to the 10,000 inclusive record. See more below.
Response, number of records evaluated 100,000 The engine will return up to 100,000 records with each query. See full example below.
Navigations, evaluated for response 250 This can be managed via includedNavigations or defining navigations within Command Center.
Navigations, returned in response 50 A navigation example: Brand, Color, Size.
Navigations, refinements returned in initial response 20 A navigation refinement example: red, green within Color. This limit is for the number that is returned with the initial response. See more below.
Navigations, refinements returned with Refinements API 10,000
Navigations, bulk upload 20,000 This can only be done with Bulk Upload and not via Command Center. The engine will only evaluate up to 250 within a single query.
Sort By ID 1,000 A warning will be generated if this limit is exceeded. All IDs beyond this limit will be ignored.

Response pagination limit

The engine will return records up to 10,000th one. In other words, pagination is supported up to the 10,000 record.

This is based on the expected functional behavior that no shoppers will paginate above 10,000 products.

Query record response limit

Let’s say your data set has 200,000 dresses: half (100,000) are green, and half are red.

On an unfiltered search for dress, we will return the first 100,000 records that are available. This will be a blend of red and green dresses from the total available set of 200,000.

A query that has hit the 100,000 limit will have a metadata.recordLimitReached: true flag in the response.

Your biasing profile, or sort, will then be applied to the 100,000 returned records.

When you apply a filter, let’s say into green only, the engine will return all 100,000 green dresses - not just the ones that were shown in the first, unfiltered query.

This means that even though any given result is limited to 100,000 records, your customer can filter in and be certain that we’re returning all available products.

This is done to protect the fast response time, and out of known customer behavior that very few scenarios will have a shopper go through 100,000 results - likely a broad query like that will then be refined further with filters.

Navigation value response limit

The engine will return up to 20 values per navigation as part of the initial response. These 20 refinements are built off the 100,000 records that are returned (see Query record response limit, above).

If more than 20 refinements exist, the response will include a moreRefinements: true flag for that specific availableNavigation.

If more navigations are available, they can be fetched via secondary call to Refinements API.

The refinements API will return all possible navigations, not subject to the 100,000 record limitation.

Command Center

Data item Limit Notes
Synonyms 200
Synonyms, tokens per 100
Rules 200 Can be increased by request.
Redirects 200
Spell corrections 200
Navigations, visible in Command Center 250 Bulk uploaded navigations will appear in Command Center but cannot be edited through the user interface.
Content Zone Contents 200KB
Content Zone in Templates 100
Rich Content Zone in Templates 50
Query Zone in Templates 10
Expert Query Zone in Templates 10
Push to top, ids 300

SAYT

Data item Limit Notes
Indexing, fields for keywords 2 This limit is in place for performance optimization - the speed of processing and the quality of the results is better when keywords are generated off smaller meaningful fields.
Indexing, fields for navigations 5
Indexing, characters per navigation 500 characters E.g., if you are using a category name as a suggested Navigation Field in SAYT, this will be indexed up to 500 characters of any given category name.
Indexing, Maximum suggestions generated for searchField 1,000,000
Indexing, Maximum suggestions generated for navigationField 2,500,000