Technical Specifications and Usage Limits
June 2019
This document provides information about system limits in the data, business rule and query portions of the search engine. Many of these limits are configurable - please reach out to your Customer Advocate if you believe you need to go beyond the listed value.
These limits might have recommended values for acceptable performance or limits that are intentionally built into the system and cannot be changed. For most of these you shouldn’t need to go beyond the listed limit. You can use this information to help you plan your deployment and to configure your system for optimal performance. The information is organized into sections based on feature.
Upload and Data Model
| Data item | Limit | Notes |
|---|---|---|
| Number of collections | 20 | Can be increased by request. |
| Characters used in collection name | 450 | Collection names must be alphanumeric. |
| Individual record size, XML | 200 KB | |
| Individual record size, JSON | 32 MB | |
| Record count, collection | 10,000,000 | Can be increased by request. |
| Record count, end-point | 50,000,000 | Can be increased by request. |
| Unique fields per record, JSON | 1,000 | |
| Searchable fields per collection, JSON | 1,024 | When have wildcard searches on. |
| Maximum nesting depth, JSON | 8 levels | More detail available under Data format |
| Field names | Characters in [a-zA-Z0-9-_ ] |
|
| Field name length | Total of 80 characters or less | |
| Reserved field name | categories |
|
| Mandatory field, all records | title |
|
| Mandatory field, all records | id |
|
| Number of invalid records allowed for upload | 100 | Only applies when failOnFirstError in upload config is false |
| Navigations, bulk upload | 20,000 | This can only be done with Bulk Upload and not via Command Center. The engine will only evaluate up to 250 within a single query. |
| Number of simultaneous uploads to 1 collection | 3 | Any uploads sent over this limit will be rejected with a 429 - Too Many Requests message. If you are streaming multiple partial updates the best practice is to have a retry period roughly the length of your average upload (which will depend on the size of your uploads). |
Query and Response
| Data item | Limit | Notes |
|---|---|---|
| Search query | 60 characters | If exceeded, the engine will truncate the search query and return a warning. If the query term cannot be truncated, a 413 error will be returned. |
| Search query | 10 terms | |
| Search query | 300 bring to top ids | If exceeded, the list of ids will be truncated. |
| Refinements within the query | 300 refinements | This is the number of individual refinements that can be passed within a single query to the search engine. |
| Response, number of records per page | 120 | |
| Response, pagination limit | 10,000 | Pagination is supported up to the 10,000 inclusive record. See more below. |
| Response, number of records evaluated | 100,000 | The engine will return up to 100,000 records with each query. See full example below. |
| Navigations, evaluated for response | 250 | This can be managed via includedNavigations or defining navigations within Command Center. |
| Navigations, returned in response | 50 | A navigation example: Brand, Color, Size. |
| Navigations, refinements returned in initial response | 20 | A navigation refinement example: red, green within Color. This limit is for the number that is returned with the initial response. See more below. |
| Navigations, refinements returned with Refinements API | 10,000 | |
| Navigations, bulk upload | 20,000 | This can only be done with Bulk Upload and not via Command Center. The engine will only evaluate up to 250 within a single query. |
| Sort By ID | 1,000 | A warning will be generated if this limit is exceeded. All IDs beyond this limit will be ignored. |
Response pagination limit
The engine will return records up to 10,000th one. In other words, pagination is supported up to the 10,000 record.
This is based on the expected functional behavior that no shoppers will paginate above 10,000 products.
Query record response limit
Let’s say your data set has 200,000 dresses: half (100,000) are green, and half are red.
On an unfiltered search for dress, we will return the first 100,000 records that are available. This will be a blend of red and green dresses from the total available set of 200,000.
A query that has hit the 100,000 limit will have a metadata.recordLimitReached: true flag in the response.
Your biasing profile, or sort, will then be applied to the 100,000 returned records.
When you apply a filter, let’s say into green only, the engine will return all 100,000 green dresses - not just the ones that were shown in the first, unfiltered query.
This means that even though any given result is limited to 100,000 records, your customer can filter in and be certain that we’re returning all available products.
This is done to protect the fast response time, and out of known customer behavior that very few scenarios will have a shopper go through 100,000 results - likely a broad query like that will then be refined further with filters.
Navigation value response limit
The engine will return up to 20 values per navigation as part of the initial response. These 20 refinements are built off the 100,000 records that are returned (see Query record response limit, above).
If more than 20 refinements exist, the response will include a moreRefinements: true flag for that specific availableNavigation.
If more navigations are available, they can be fetched via secondary call to Refinements API.
The refinements API will return all possible navigations, not subject to the 100,000 record limitation.
Command Center
| Data item | Limit | Notes |
|---|---|---|
| Synonyms | 200 | |
| Synonyms, tokens per | 100 | |
| Rules | 200 | Can be increased by request. |
| Redirects | 200 | |
| Spell corrections | 200 | |
| Navigations, visible in Command Center | 250 | Bulk uploaded navigations will appear in Command Center but cannot be edited through the user interface. |
| Content Zone Contents | 200KB | |
| Content Zone in Templates | 100 | |
| Rich Content Zone in Templates | 50 | |
| Query Zone in Templates | 10 | |
| Expert Query Zone in Templates | 10 | |
| Push to top, ids | 300 |
SAYT
| Data item | Limit | Notes |
|---|---|---|
| Indexing, fields for keywords | 2 | This limit is in place for performance optimization - the speed of processing and the quality of the results is better when keywords are generated off smaller meaningful fields. |
| Indexing, fields for navigations | 5 | |
| Indexing, characters per navigation | 500 characters | E.g., if you are using a category name as a suggested Navigation Field in SAYT, this will be indexed up to 500 characters of any given category name. |
Indexing, Maximum suggestions generated for searchField |
1,000,000 | |
Indexing, Maximum suggestions generated for navigationField |
2,500,000 |