PromQL, while powerful, can present a learning curve. Dash0 addresses this with a visual Query Builder that handles the most common cases — but understanding the underlying query patterns helps you get the most out of the PromQL tab and build robust dashboards and alerts.

The following queries are written with data from the OpenTelemetry demo application in mind. You can add the OpenTelemetry demo application to your organization via our integration hub.

Working with Gauge Metrics

Gauge metrics represent values that can increase or decrease over time, making them ideal for measuring current states like memory usage or connection counts.

Simple Gauge Query

promql

container_memory_working_set_bytes{service_name="adservice"}

Filtering Multiple Values

promql

container_memory_working_set_bytes{service_name=~"^(cartservice|adservice)$"}

Aggregating Across Services

promql

avg by(service_name) (container_memory_working_set_bytes{service_name=~"^(cartservice|adservice)$"})

Working with Sum Metrics (Counters)

Sum metrics represent a cumulative total that changes over time. When monotonic (the common case), they track values that only increase — such as request counts or bytes transferred. In the OTel API these are produced by the Counter instrument; in Prometheus terminology they are called counters. Use rate() for per-second rates or increase() for total change over an interval.

Calculating Rate per Second

promql

rate({otel_metric_name="app.frontend.requests", otel_metric_type="sum"}[$__rate_interval])

Calculating Total Count over a Time Range

promql

increase({otel_metric_name="app.frontend.requests", otel_metric_type="sum"}[$__rate_interval])

Count over Time Range per Service

promql

sum by(service_name) (increase({otel_metric_name="app.frontend.requests", otel_metric_type="sum"}[$__rate_interval]))

Working with Histogram Metrics

Histograms are particularly valuable for measuring distributions of values like request durations or response sizes.

Observation Count

Query the total number of observations using the _count suffix:

promql

http_request_duration_seconds_count

Sum of Observations

Access the sum of all observed values using the _sum suffix:

promql

http_request_duration_seconds_sum

Bucketed Observations

Examine the distribution of values across predefined buckets using the _bucket suffix. The le label selects specific bucket boundaries. The query below returns the count of observations that fell into buckets with upper bounds ≤ 0.5 seconds:

promql

http_request_duration_seconds_bucket{le="0.5"}

Percentiles with `histogram_quantile`

promql

histogram_quantile(0.9, rate({otel_metric_name = "http_request_duration_seconds", otel_metric_type = "histogram"}[5m]))

Native vs Classic Histograms

When querying via otel_metric_name, Dash0 returns native histograms — histogram_quantile() works directly without sum by (le). When querying via the Prometheus-compatible _bucket name (e.g., http_request_duration_seconds_bucket), classic bucket time series are returned and sum by (le) is required to preserve bucket boundaries.

Working with Summary Metrics

Like histograms, summary metrics track both counts and sums of observations, but calculate quantiles on the client side.

Observation Count

promql

http_request_duration_seconds_count

Sum of Observations

promql

http_request_duration_seconds_sum

Quantiles

promql

{otel_metric_name="http_request_duration_seconds", otel_metric_type="summary", quantile="0.9"}

Average

promql

http_request_duration_seconds_sum / http_request_duration_seconds_count

Synthetic Metrics

Dash0's synthetic metrics provide on-the-fly calculations based on ingested telemetry, enabling dynamic analysis without predefined aggregations. To learn more, see Understand Synthetic Metrics.

Tip

To understand the underlying signals for the synthetic metric examples below in relation to spans and logs, see the Trace Explorer and Log Explorer.

Span Count per Service

promql

sum by(service_name) (rate({otel_metric_name="dash0.spans"}[$__rate_interval]))

P99 Span Duration (in milliseconds)

promql

histogram_quantile(0.99, sum(rate({otel_metric_name="dash0.spans.duration", service_name="productcatalogservice"}[$__interval]))) * 1000

Error Percentage per Kubernetes Pod

promql

(
  sum by(k8s_pod_name) (
    increase({otel_metric_name="dash0.spans", otel_span_status_code="ERROR", service_name="productcatalogservice"}[$__interval])
  )
  > 0
)
/
(
  sum by(k8s_pod_name) (
    increase({otel_metric_name="dash0.spans", service_name="productcatalogservice"}[$__interval])
  )
  > 0
)
> 0

Counting Logs Matching a Pattern

promql

sum(increase({otel_metric_name="dash0.logs", otel_log_body=~"connect"}[$__interval])) > 0

Logs Broken Down by Severity

promql

sum by(otel_log_severity_range) (increase({otel_metric_name="dash0.logs"}[$__interval])) > 0

Error Log Count per Deployment

promql

sum by(k8s_deployment_name) (increase({otel_metric_name="dash0.logs", otel_log_severity_range="ERROR"}[$__interval])) > 0

Pod Names for a Service

promql

sum by(k8s_pod_name) ({otel_metric_name="dash0.resources", service_name="adservice"})

Pod Count per Namespace

promql

sum by(k8s_namespace_name) ({otel_metric_name="dash0.resources", k8s_pod_name!=""})