ProductABC Clusters
Info/Context
Type: End-user documentation
Audience: CompanyName users and employees
Purpose: A product overview with information about the server clusters that CompanyName sets up on behalf of each ProductABC user.
Note: Some words and phrases are enclosed in [brackets] to represent hyperlinks to other docs not included in this sample library.
Since [ProductABC] is designed to provide pre-bid IVT predictions in real time, it's important to minimize latency as much as possible. One step we take to minimize latency is providing each ProductABC user with their own unique set of ProductABC servers, collectively known as a cluster.
Each cluster contains multiple servers (or nodes) to handle your requests. Whenever you send data to ProductABC, you'll send that data to your own dedicated cluster; likewise, every time you receive an IVT prediction from ProductABC, that prediction will come from your cluster. No other ProductABC users will send requests to your dedicated cluster, and your requests will never reach other users' clusters.
Cluster deployment
To support our users in all regions, CompanyName deploys ProductABC clusters in various data centers across the globe. As part of the ProductABC onboarding process, we'll set up a new server cluster as close to you possible; however, to ensure optimal response times, each of your own data centers must be within 200 miles of the data centers where your dedicated ProductABC servers reside.
Service providers
CompanyName uses Amazon Web Services (AWS), Google Cloud Platform (GCP), and Equinix Metal data centers for our standard cluster deployments.
It is not possible to deploy ProductABC on your own servers.
Cluster size
The size of your ProductABC cluster depends on the amount of requests you anticipate making to ProductABC—in other words, if you need a high volume of IVT predictions, you'll need more nodes per cluster.
Each node in a ProductABC cluster can process a certain number of queries per second (QPS). This number varies based on which ProductABC integration option you're using:
-
If your integration is built on the ProductABC SDK, which automatically handles connection pooling and load balancing via round-robin DNS, each node can process a maximum of 20,000 QPS.
-
If your integration is built on the ProductABC API, each node will have a lower QPS limit compared to an integration build on the ProductABC SDK. The actual limit depends on how well you handle load balancing and connection pooling, although even an optimized API integration generally performs below 20,000 QPS per node.
Example calculation
We use this QPS threshold to calculate the number of nodes you'll need in your ProductABC cluster. For example, if your integration is built on the ProductABC SDK, and you anticipate a maximum threshold of 90,000 QPS, CompanyName would provision six nodes for your cluster:
90,000 / 20,000 = 5 [rounded up]
+ 1 [redundancy]
--------------------
6 [total]
When we calculate the number of nodes required to support your projected QPS, we always add an extra node for redundancy. As such, all clusters must have at least two nodes, even if your projected QPS is less than the limit of a single node.
Limiting connections
Even with load balancing in place, a sudden increase in connections can overwhelm a node and prevent ProductABC from responding to lookup requests—regardless of your integration type. Because of this, we recommend opening a small number of long-lived connections and sending as many requests through that connection as possible.
Traffic spikes
If you anticipate a QPS spike of 20% or more in the near future, please contact CompanyName as soon as possible to discuss cluster resizing. We may need to add more nodes to your cluster to prevent increased latency or other performance issues.