AutoMerge is generally available

Last updated:

Create SLOs

Create SLOs in Dash0 in the UI, with Agent0 or manually, or as code with Terraform or the Dash0 CLI.

You can create an SLO in two ways:

  • In the UI: open Monitoring › SLOs and click Create with Agent0 to chat with Agent0, which drafts the SLO for you. To fill out the form yourself, choose Manually from the button's More create options menu.
  • As code: through the Dash0 Terraform provider or the Dash0 CLI, both of which take the same OpenSLO SLO definition (apiVersion: openslo/v1, kind: SLO). See Manage SLOs as Code.

Either way, you make the same two design decisions, and this page walks through both: what to measure (your SLI) and how reliable it must be (your target).

Prerequisites

  • To create in the UI: the Create SLO dataset permission, which Admins and users with the Create Check Rule permission have by default. SLOs cannot be created in demo datasets.
  • To create as code (Terraform or the CLI): an auth token with the All permissions (querying, ingesting and configuring) option. Create and manage tokens in Organization Settings › Auth Tokens (Admins only).

Monitoring › SLOs opens the SLO catalog:

SLO catalog with targets, SLI trends, remaining error budgets, and a breached SLO.

Create in the UI

The form asks for the same things this page describes: the SLI shape (Good / Total, Bad / Total, or Raw) and its queries, whether the source is a counter, the target, and the name, description, labels, and annotations. A live SLI preview shows what the queries select before you save:

SLI configuration with Good / Total selected, the frontend service scope, and good-event criteria excluding spans with ERROR status.

The same form is available on the Settings tab of every SLO you are allowed to edit.

Define your SLI

This guide covers the generally available Service Level Indicator (SLI), a ratioMetric expressed as PromQL over your telemetry. In the UI form, the query builder assembles these queries for you and shows a read-only PromQL preview of each one. A ratio SLI takes exactly one of three shapes:

  • good + total — a ratio of good events to total events. Make the good query a subset of the total query so the ratio is well-defined.
  • bad + total — a ratio of bad events to total events. Dash0 derives the good count as total - bad.
  • raw + rawType — a query that already returns a ratio between 0 and 1. Set rawType to success if the value is a success ratio or failure if it is a failure ratio.

The three SLI shapes. Good plus total: good events are a subset of total events and the ratio is good over total. Bad plus total: bad events are a subset of total events and Dash0 derives good as total minus bad. Raw: one query that already returns a ratio between zero and one.

With the counter flag on (the default), a good, bad, or total query must be a bare vector selector (label matchers only; functions and aggregations are rejected with a 400). With the counter flag off, these queries may be any PromQL expression that returns an instant vector, such as histogram_count(...). A raw query may always be any PromQL expression that returns an instant vector, so functions, aggregations, and operators are allowed there. The full set of supported and rejected capabilities is listed in the supported OpenSLO capabilities matrix. For an availability SLO on server spans, for example:

  • good — server spans whose status is not ERROR
  • total — all matching server spans

A good, bad, or total query may match many series: Dash0 sums all matched series into one event count. Only the raw shape must resolve to exactly one time series, because the sum of several ratios can exceed 1 and the SLI then pins to 100%.

Take good and total from the same metric where you can, for example the same counter with an extra label matcher on good. Two different metrics, such as two counters that are exported separately, are not sampled at the same moment. Within a 5-minute window one can run ahead of the other, so good can exceed total. Over the full SLO window these differences mostly cancel out, but short-window burn rates become noisy, and a window where good exceeds total reads as error-free.

Dash0 handles the aggregation for you and computes the ratio across the SLO window, so there is no need to wrap your queries in sum(...). The counter flag tells Dash0 which math to apply:

  • Counter on (the default): for monotonically increasing counters. Dash0 applies increase() over 5-minute windows to each bare selector, so do not add rate(...) or increase(...) yourself.
  • Counter off: for gauges, values that can go up and down, and expressions that already return the event count of a 5-minute window, such as histogram_count(increase(...[5m])). Dash0 only sums the series the query returns and skips series whose value is not a number.

The counter flag is ignored for raw.

In the UI, the Good events and Bad events cards have a Duration row when the metric is a cumulative histogram, such as the span metric dash0.spans.red or http.server.request.duration. Enter a duration like 300 ms. A good event must then also finish within it, and a bad event must take longer. The form writes the histogram_fraction queries described below. Leave the row empty to ignore duration.

For a latency objective such as "99% of requests finish within 5 seconds", count histogram buckets using the le (less-than-or-equal) label instead of raw durations: the good query selects the bucket at your threshold (le="5") and the total query selects the le="+Inf" bucket. Use a classic histogram _bucket series for this.

Exponential (native) histograms do not expose per-le buckets, so an le matcher against them matches nothing and the SLI returns no data. To build a latency SLI on a native histogram, turn the counter flag off and compute both counts from the same histogram:

  • good: histogram_fraction(0, 0.5, increase(<selector>[5m])) * histogram_count(increase(<selector>[5m])), the requests that took 0.5 or less, in the histogram's unit
  • total: histogram_count(increase(<selector>[5m])), all requests

For example, for requests to a checkout service that finish within 0.5 seconds:

yaml
indicator:
  spec:
    ratioMetric:
      counter: false
      good:
        metricSource:
          type: Prometheus
          spec:
            query: histogram_fraction(0, 0.5, increase({otel_metric_name="http.server.request.duration", service_name="checkout"}[5m])) * histogram_count(increase({otel_metric_name="http.server.request.duration", service_name="checkout"}[5m]))
      total:
        metricSource:
          type: Prometheus
          spec:
            query: histogram_count(increase({otel_metric_name="http.server.request.duration", service_name="checkout"}[5m]))

Every request counts once, so a busy hour weighs more than a quiet one, and good never exceeds total. The threshold can be any value, not only a bucket boundary. Dash0 interpolates inside the bucket that contains it, so the good count is an estimate. The raw shape also works for latency, but each evaluation of a raw query counts the same, so a quiet minute weighs as much as a busy one.

Choose a target

  • Target — the reliability goal as a fraction (for example 0.997 for 99.7%). In an OpenSLO document, alternatively set targetPercent (for example 99.7). Set exactly one of the two; the value must be strictly between 0 and 1 (or 0 and 100).
  • Budgeting method and time window: the generally available configuration uses Occurrences budgeting, counting bad events versus total events over a rolling 28-day window. Threshold-based Timeslices SLOs require separate enablement for your organization; see Supported OpenSLO capabilities.

The gap between the target and 100% is your error budget, and every added nine shrinks it to a tenth:

Bars comparing targets of 99, 99.5, 99.9, and 99.99 percent. The error budget shrinks from 1 percent, or ten thousand of every million events, down to 0.01 percent, or one hundred of every million events.

Some guidance for picking the value:

  • Start from measured reliability, not from a wish. Look at what the service actually achieved over recent weeks and set the target slightly below it. The form's live SLI preview shows the current ratio while you build the queries, and once an SLO exists, its 28d SLI tile on the detail page shows the achieved value. A target the service has never met turns the SLO into a permanent alarm that everyone learns to ignore.
  • Do not pick 100%. A single failed request breaches it, and it leaves no budget for deployments, dependency hiccups, or experiments. The error budget is what pays for change; a target that allows nothing to fail forbids releasing.
  • Add nines only when users notice. Each nine makes the burn-rate alerts ten times as sensitive, so a 99.99% target can page someone over a few minutes of errors. If your users are other services with retries, a looser target is usually the honest one.
  • Match the caller's expectations. An SLO slightly stricter than what your consumers assume gives you room to react before they are affected. Anything far stricter just burns attention.

You can change the target later on the SLO's Settings tab.

The target and description controls follow the SLI configuration:

SLO target as a fraction over a rolling 28-day window, followed by name, description, labels, and annotations.

Create as code

The same SLO is one OpenSLO v1 document, whichever tool applies it:

  • Terraform: the dash0_slo resource takes the document in its slo_yaml attribute.
  • Dash0 CLI: the dash0 slos command group creates, lists, gets, updates, and deletes SLOs, and dash0 apply -f <file> applies OpenSLO YAML documents from files or directories.

Manage SLOs as Code covers the workflow (stable origins, idempotent applies, exporting SLOs built in the UI), and the SLO object reference documents the definition field by field, including the optional spec.service link to a service and the dash0.com/* metadata such as display name and the enabled flag.

Verify

Open Monitoring › SLOs to confirm the SLO appears in the catalog, then open it to see its target, error budget, and burndown. The SLO appears right away with a New badge and empty values; it starts showing data a few minutes later, once the first recording-rule evaluation lands. See View and analyze SLOs.

Supported OpenSLO capabilities

The generally available OpenSLO configuration uses Occurrences budgeting, a single objective, a ratioMetric SLI with inline PromQL, and a rolling 28-day window. The supported OpenSLO capabilities reference distinguishes organization-gated alternatives from unsupported combinations and lists the SLO limit per organization.

Further reading