SLOs turn reliability from a general aspiration into an operating decision. This guide explains how to choose user-centered SLIs, set attainable service-level objectives, calculate error budgets, connect burn-rate alerts to action, and run repeatable reviews as traffic, architecture, and product expectations change.
Overview
A service-level indicator (SLI) is a measurement of a user-relevant aspect of service behavior, such as successful requests, latency, or availability. A service-level objective (SLO) is the target you set for that indicator over a defined time window. A service-level agreement (SLA) is a formal commitment, often involving customers or contractual consequences. These terms are related, but they should not be used interchangeably.
An SLO is useful when it changes how a team prioritizes work. For example, an API team might define an availability SLO of 99.9% over a rolling 30-day window. The target creates an error budget: the amount of measured unreliability the team can spend while remaining within the objective. If the service has 43,200 minutes in a 30-day window, a 99.9% target permits approximately 43.2 minutes of unavailable time. The exact interpretation depends on the measurement method and whether the window is calendar-based or rolling.
Error budgets are not permission to cause failures. They are a way to balance delivery and reliability using the same evidence. When the budget is healthy, a team may have room for releases, migrations, or controlled experiments. When the budget is being consumed quickly, the team has a clear reason to investigate, reduce operational risk, or pause particularly risky changes.
| SLI type | What it measures | Useful for | Common caution |
|---|---|---|---|
| Availability | Successful requests divided by eligible requests | APIs, web services, control planes | Define which responses count as success and exclude invalid client requests only when that reflects user impact. |
| Latency | Requests completed below a selected threshold or percentile | Interactive APIs, page loads, synchronous workflows | Averages can hide slow users; use a distribution or threshold-based measurement. |
| Correctness | Requests returning valid, expected results | Search, payments, data processing, recommendations | A successful HTTP response does not necessarily mean the result was correct. |
| Freshness | Data updated within an acceptable age | Feeds, analytics, inventory, replication | Specify the timestamp source and account for expected pauses or batch schedules. |
| Throughput or completion | Jobs completed successfully within a time window | Queues, batch systems, media processing | Measure completed user work, not merely worker activity. |
How to estimate an SLO and error budget
- Start with the user journey. Identify the action that must work: signing in, placing an order, retrieving a dashboard, or completing a deployment. Avoid beginning with whatever metric is easiest to collect.
- Choose one primary SLI. Select the measurement that best represents whether the journey succeeded. Add supporting indicators only when they explain or protect the primary outcome.
- Define the eligible population. Document which requests, jobs, regions, tenants, and dependency failures are included. A denominator that changes without explanation makes the objective difficult to trust.
- Choose the evaluation window. A rolling window is useful for continuously assessing current reliability. A calendar window can align with monthly or quarterly reviews. Record the choice in the SLO definition.
- Set a target using evidence. Examine historical performance, user expectations, operational cost, and the consequences of failure. Do not select a long string of nines simply because it sounds precise. A target should be difficult enough to encourage engineering discipline but attainable enough to guide decisions.
- Calculate the error budget. For an availability-style objective, the allowed failure fraction is
1 - SLO target. The time budget iswindow duration × allowed failure fraction. For event-based SLIs, calculate the allowed bad-event count aseligible events × allowed failure fraction. - Define actions before an incident. State what happens when the budget is healthy, partially consumed, nearly exhausted, or exhausted. Link these states to release review, remediation, capacity work, and risk acceptance.
For latency, the calculation should match the SLI definition. If the objective is “99% of eligible requests complete within 400 milliseconds,” the good-event ratio is the number of requests at or below 400 milliseconds divided by the total eligible requests. This is different from saying “the 99th percentile is 400 milliseconds,” even though the two descriptions may be operationally related.
Burn rate and alerting
Burn rate compares the current rate of error-budget consumption with the rate that would use the entire budget exactly by the end of the SLO window. A burn rate of 1 means the service is consuming budget at the maximum sustainable rate. A burn rate above 1 indicates that the budget will be exhausted early if the current condition continues.
A practical alerting model uses multiple windows. A fast window can identify severe incidents quickly, while a longer window reduces sensitivity to brief noise. The generic calculation is:
burn rate = observed bad-event ratio ÷ allowed bad-event ratio
For example, with a 99.9% objective, the allowed bad-event ratio is 0.001. If 0.01 of eligible requests are currently bad, the burn rate is 10. In a monitoring system, express the numerator and denominator using the same filters, service boundary, and time range. A PromQL-style availability ratio might be represented conceptually as:
sum(rate(requests{service="checkout", outcome="good"}[5m])) / sum(rate(requests{service="checkout"}[5m]))
The exact metric names and labels vary by instrumentation and monitoring platform. Validate the query against known incidents and normal traffic before making it paging policy. A useful alert should identify the affected SLO, current burn rate, window, owner, and a link to the relevant runbook.
Inputs and assumptions
Write the assumptions beside every SLO so another engineer can reproduce the result. At minimum, capture:
- Service boundary: the user-facing capability and the components included in its measurement.
- Event definition: what counts as good, bad, eligible, cancelled, retried, or invalid.
- Target: the percentage, threshold, or completion requirement and its rationale.
- Window: rolling or fixed, including the duration and evaluation cadence.
- Aggregation: global, regional, tenant-level, endpoint-level, or another defined scope.
- Exclusions: planned maintenance, client cancellations, synthetic checks, or dependency failures, with a reason for each exclusion.
- Data quality: instrumentation coverage, missing events, duplicate events, and known sampling behavior.
- Ownership: the team responsible for reviewing the SLO and acting on budget consumption.
Exclusions deserve particular care. Removing every dependency failure can make a service look reliable while users still experience failure. If an exclusion is necessary, track the excluded condition separately and explain who owns the resulting risk.
An SLO policy should also distinguish between an alert and a review threshold. Paging is appropriate for a condition that needs immediate response. A weekly or monthly budget review can handle slower consumption, repeated small regressions, or an objective that no longer reflects the product. Treating every threshold as a page creates noise and weakens the connection between monitoring and action.
Worked examples
Availability example
Suppose an order API has a 99.9% availability objective over 30 days. The allowed bad-event ratio is 0.1%. If the measurement window contains 2,000,000 eligible requests, the error budget is:
2,000,000 × 0.001 = 2,000 bad requests
If 600 bad requests have occurred, 30% of the event budget has been consumed. That percentage alone does not determine urgency: consumption speed matters. If the 600 failures happened gradually over most of the window, the trend may be manageable. If they happened in a short period, the burn rate may justify an incident response and release review.
Latency example
Consider a dashboard objective requiring 99% of eligible page data requests to complete within 800 milliseconds over a rolling window. For 500,000 eligible requests, up to 5,000 may exceed the threshold while the objective remains met. A review should examine whether slow requests cluster by region, endpoint, customer size, or deployment version. The next action might be query optimization, cache tuning, capacity adjustment, or a change to the SLI if the measured journey does not match the user's actual experience.
Reusable SLO review template
- Service and owner: name the user journey and accountable team.
- SLI and query: include the event definitions, filters, and dashboard link.
- Objective and window: record the target, threshold, and evaluation period.
- Budget position: state remaining budget, consumed budget, and current burn rate.
- Top contributors: list endpoints, regions, dependencies, versions, or changes associated with bad events.
- Decision: continue normal delivery, add reliability work, restrict risky changes, or revise the objective.
- Owner and due date: assign each follow-up to a person or team.
- Assumption changes: note traffic mix, architecture, instrumentation, or product changes that affect interpretation.
When to recalculate
Revisit an SLO whenever the inputs or user expectations change, not only after a major outage. Recalculate before launching a new service, changing an API contract, moving regions, introducing a queue, or altering traffic routing. Review it after an incident when the existing SLI failed to represent user impact or when an exclusion hid important risk.
Make SLO review part of a recurring reliability cadence. A monthly operational review can examine burn rates, alert quality, and open actions. A quarterly product or architecture review can reassess the target, window, service boundary, and ownership. Recheck the objective after substantial traffic growth, a new customer segment, a dependency migration, or a change in business-critical workflows.
To keep the process practical, version SLO definitions and monitoring queries alongside service documentation or code. Test alert queries against historical incidents, verify that dashboards use current metric names, and remove alerts that do not lead to a clear response. Teams can use OpenTelemetry-based instrumentation and their existing observability platform, but the tool is secondary to consistent event definitions and ownership. For services deployed on Kubernetes, pair SLO evidence with deployment and autoscaling changes rather than treating application reliability and platform behavior as separate problems. The result is a living reliability agreement: measurable, reviewable, and useful when the next release or incident requires a decision.