Define the boundary
Only test endpoints you own or are authorized to test. Decide whether the test measures the gateway, a model route, an asynchronous generation task, or the full user journey. Mixing them produces a number that is difficult to act on.
Warm up, then measure
Run a short warm-up to establish connections and caches, then record a fixed measurement window. Report request count, concurrency, payload size, model ID, region, timeouts, and retry policy. For asynchronous image or video tasks, measure submission latency and completion latency separately.
Report the full outcome
Never delete failed requests from the denominator. If retries are enabled, publish both first-attempt and final success rates so the retry policy does not hide instability.
Start with a bounded tool
The AI API Load Tester emits latency, throughput, status, and retry evidence for OpenAI-compatible endpoints. Begin with a small authorized test and increase load gradually.
Use supported models through one APIMART account.
Confirm current model availability, pricing, and limits before routing production traffic.
Create an APIMART account