Operations · 2026-08-28 · HippoAPI Documentation Team

Handle rate limits and concurrency

Design bounded retries, queues, and concurrency controls for limits that can vary by model and account.

Expect limits to vary

Available request rate and concurrency can depend on the account, model, provider capacity, and current platform controls. Do not hard-code one global number unless it is part of a written agreement.

Use bounded exponential backoff

Retry only transient failures, add jitter, cap the delay and total attempts, and respect server guidance when present. Never retry authentication, invalid-request, or insufficient-balance errors as if they were capacity errors.

Control concurrency in your application

Place a queue or semaphore before model calls, define timeouts, and apply backpressure to callers. This is more reliable than allowing traffic spikes to create synchronized retries.