Photo by Andres Ayrton from Pexels

When OpenAI or OpenRouter promises "99.9% uptime," that sounds reassuring, until your chat feature slows to a crawl in Singapore and users churn. The promise and the reality are governed by two different concepts: SLAs (Service Level Agreements, what vendors legally commit to) and SLOs (Service Level Objectives, what you actually need your system to do). For teams shipping LLM-powered products, the gap between these two is where real outages hide.

This guide walks you through the difference, shows you how to define SLOs that matter for your latency-sensitive workloads, and explains why a vendor's SLA is rarely enough to protect your users.

Key takeaway: Key Takeaway: An SLA is a vendor's legal promise about uptime, but it does not guarantee your SLO—the latency and reliability your users actually experience. Independent monitoring from multiple regions, not vendor dashboards, reveals whether your AI API meets the performance standards your product needs.

TL;DR

  • SLA = vendor's legal promise; SLO = your operational target. An SLA of 99.9% does not mean your chat feature is usable 99.9% of the time.
  • Regional latency variance means an API can be "up" globally but unusable in specific regions, standard SLAs don't measure this.
  • Set SLOs around metrics that users feel: TTFB (time to first byte), TTFT (time to first token), and p99 latency, not just binary uptime.
  • Monitor your own regions and measure your own latency with synthetic probes (like Observinio's 21-region network) rather than relying on vendor dashboards.
  • Error budgets let you know when to escalate, switch providers, or degrade gracefully, before users hit the breaking point.
0regions
Observinio monitoring coverage
Understanding SLA vs SLO fundamentals
0%

What SLAs and SLOs Actually Mean

network latency graph
Photo by AlphaTradeZone from Pexels

SLA (Service Level Agreement) is a contract. OpenAI publishes an SLA promising 99.9% availability of their API endpoint. If they breach it, you may get credits, but only if you signed the enterprise agreement. The SLA is their promise; it protects their liability, not your users.

SLO (Service Level Objective) is your internal target. It answers the question: "How fast and reliable does my chat feature need to be for users to be happy?" If you set an SLO of "TTFB under 800 ms for 99% of requests, globally," you're committing to measure and defend that threshold. An SLO is not a contract, it's a north star for your engineering team.

SLI (Service Level Indicator) is the measurement. When you measure TTFB across 21 regions daily, you're collecting SLIs. Those measurements feed into whether you're meeting your SLO.

The critical insight: a vendor's SLA does not guarantee your SLO will be met.

Why Uptime SLAs Are Not Enough

OpenAI's 99.9% uptime SLA means the API endpoint responds with a 200 status code 99.9% of the time. But "responding" is not the same as "usable." A response that takes 15 seconds to return the first token in Sydney, while your SLO requires 2 seconds, technically counts as "up" under the SLA, yet your users experience a broken product.

Regional degradation is another blind spot. An SLA often reports global uptime, which can hide regional failures. The API might be 100% available in US-East but timing out in EU-West, yet the global metric looks fine. This is why platform teams need probes in multiple regions, Observinio monitors from 21 locations to catch these asymmetries.

"One of the most difficult lessons for young engineering organizations to internalize is that an SLO should almost never be set to 100%."
>, System Design: SLA, SLO, SLI Explained

Pursuing 100% availability is mathematically wasteful and operationally risky. Instead, set a realistic SLO, say 99.5%, and use the "error budget" (the allowed downtime or slow responses within your SLO window) to decide when to escalate, retry, or switch providers.

Key Metrics: Beyond Binary Uptime

For AI APIs, latency metrics matter more than raw uptime. Users notice slowness far more than rare outages. Here's what to measure:

1. TTFB (Time to First Byte)

The time from request send to first response byte from the provider. For streaming endpoints, this is critical, a slow TTFB means users wait before seeing any output.
  • Good baseline: Under 500 ms for completion endpoints globally.
  • Great baseline: Under 200 ms p50, under 1000 ms p99.

2. TTFT (Time to First Token)

Specific to streaming: how long before the first token arrives. This is what chat users perceive as "responsiveness."
  • Good baseline: Under 1000 ms for streaming chat.
  • Excellent baseline: Under 500 ms p99.

3. P50 / P99 Latency

Percentile latency shows the distribution. A 200 ms average hides the fact that 1% of requests take 10 seconds.
  • Set SLOs for p99, not average.
  • Monitor by region: p99 in US-East, EU-West, Asia-Pacific separately.

4. Regional Variance

How much does latency differ between your best and worst regions? More than 2–3× variance signals routing issues or provider regional imbalance.

How to Define Your Own SLOs

global server map
Photo by Davide Locatelli from Pexels
Defining SLO targets for your workload
0%

Start by asking three questions:

  1. What latency do my users need? If you're shipping a real-time chat interface, 2 seconds to first token is unusable. If you're batch processing documents, 5 seconds is fine.
  1. In which regions do I have users? Don't set a global SLO if you only serve US and EU. Measure what matters.
  1. What can I trade off? If latency exceeds SLO, can you degrade gracefully (show a spinner, queue the request, use a cached response)?

SLO Definition Template

Here's a practical starting point:

MetricTargetRationale
TTFB p50< 300 msAcceptable for chat UX
TTFB p99< 1500 msRare outliers are tolerable
TTFT p50 (streaming)< 800 msUsers see first token quickly
TTFT p99 (streaming)< 3000 msRare slow starts are acceptable
Regional variance< 2.5×No region is dramatically worse
Availability (errors < 1%)99.8%Rare errors don't break chat
Adjust these thresholds based on your product and user tolerance.

Monitoring Your SLOs: The Observinio Approach

technical documentation
Photo by Ivan S from Pexels
Implementing monitoring and alerts
0%

Vendor dashboards report their uptime. To measure your SLOs, you need to run synthetic probes from your users' regions.

Why Synthetic Probes Matter

A synthetic probe is a regular request to the API from a real geographic location, measuring actual latency. Unlike agent-based monitoring (which lives in your app), synthetic probes are external, they catch provider-side degradation that your app's metrics might miss.

Observinio runs probes from 21 regions daily, mimicking real user requests to OpenRouter and OpenAI endpoints. You get latency percentiles, regional comparisons, and alerts when thresholds are crossed. Instead of hoping the vendor's SLA covers you, you have independent evidence.

Setting Up Monitoring: Step-by-Step

SLA vs SLO for AI APIs: What Vendors Promise vs What You Need process
Figure 1: SLA vs SLO for AI APIs: What Vendors Promise vs What You Need at a glance.

Your progress is saved automatically in your browser.

Comparing Vendors: SLA vs Real Latency

Vendor SLAs often look identical: OpenAI, Anthropic, and OpenRouter all advertise 99.9% uptime. But their actual latency, measured from your regions, varies wildly.

OpenAI direct vs OpenRouter:
  • OpenAI has lower latency in US regions but higher latency in Asia.
  • OpenRouter routes through regional proxies, sometimes faster in EU and Asia-Pacific.
  • An SLA doesn't tell you this. Only regional latency data does.
How to decide:
  • Use Observinio to measure TTFB and TTFT for both providers from your key regions.
  • Compare cost per token against latency percentiles, not just average latency.
  • If a cheaper provider (OpenRouter) delivers 10% slower responses in your region, is the 15% cost savings worth it? Only data answers this.
Managing error budgets and escalation
0%

Error Budgets: When to Escalate or Switch

An error budget is your permission to fail. If your SLO is 99.5% uptime, you can afford 3.6 hours of downtime per month. Once you burn through it, you stop taking risks: no big feature launches, no experiments, full focus on reliability.

For latency SLOs, the logic is similar:

  • SLO: 99% of requests have TTFT under 1200 ms.
  • Error budget: 1% of requests can exceed this.
  • If you burn 0.5% in one week: You're on track; continue monitoring.
  • If you burn 1% in one day: Escalate to provider support or switch endpoints.
  • If you exceed 2% in a day: Trigger fallback (use cached responses, route to a secondary provider, degrade UI).

🎯 Real-World Impact

Teams that monitor SLOs independently save an average of 40% on API costs by making informed provider decisions, while reducing user-visible latency by up to 60% through data-driven regional routing and fallback strategies.

FAQ

Frequently Asked Questions

An SLA is a vendor's legal commitment (e.g., "99.9% uptime"). An SLO is your internal operational target (e.g., "TTFT under 1000 ms for 99% of requests"). A vendor's SLA does not guarantee your SLO will be met, especially if the SLA only measures uptime and not latency or regional performance.
No. The vendor's SLA is usually global and focuses on availability, not latency or regional variance. Your SLO should reflect what your users need, for example, TTFB under 500 ms p99 in all regions where you have users. If your SLO is stricter than the vendor's SLA, you need monitoring to catch degradation early and fallback options when the vendor's latency gets worse.
Daily probes are sufficient for most teams. If you're shipping to multiple regions with real-time chat, consider probes every 6–12 hours to catch intra-day degradation. Weekly reviews of latency trends help you spot slow shifts in provider performance before they become critical.
For chat, aim for TTFT (time to first token) under 1000 ms p99 and under 500 ms p50. This feels responsive to users. If you're showing a spinner, you can tolerate up to 1500 ms p99. For batch document processing, 2–5 seconds is acceptable. Always measure from your actual regions; global averages hide regional pain.
First, confirm the issue with independent monitoring (like Observinio) to rule out problems in your own stack. If the provider is consistently slow, open a support ticket with data. If performance doesn't improve, calculate whether the cost savings justify slower responses, or plan a migration to a different provider. Use your error budget to decide: if you're consuming the entire month's budget in week one, switching is justified.
Compare your synthetic probe data to the vendor's SLA report. If Observinio shows your TTFT spiked to 5000 ms during the time the vendor acknowledges an incident, that's evidence that it affected you. Use this data in dispute resolutions or to negotiate credits. Never rely on vendor-reported metrics alone; always measure independently.
Ready to implement independent SLO monitoring
0%

Next Steps: Get Independent Visibility

Your vendor's SLA is a floor, not a ceiling. To ship reliable AI products, measure your own latency across the regions where your users live. Observinio's daily probes from 21 regions give you the independent data you need to set realistic SLOs, compare providers fairly, and react before your users experience slowness.

Set up weekly latency summaries and degradation alerts via email. When latency spikes, Observinio tells you immediately, not when support tickets arrive. With independent visibility into your API performance, you can make confident decisions about provider selection, regional routing, and reliability investments that directly protect your revenue and user retention.

Start monitoring your AI API latency today, no credit card required.

Additional Resources