Telemetry Transformations in the Dash0 Operator

With the latest release of the Dash0 operator, you now have not only one, but two powerful mechanisms to only collect the telemetry you really need in your Kubernetes clusters: Filters and transformations. Both capabilities help to reduce the noise in your telemetry, let you focus on what really matters to keep your services running smoothly, and ultimately also help to save hard cash.

We recently announced the availability of custom filter rules in the operator as well as a similar filter mechanism in our platform, called Spam filters, that lets you discard useless telemetry in the Dash0 backend.

Today these capabilities are complemented by a new operator feature that allows you to transform telemetry collected by the operator with custom rules before it is sent to Dash0. In this context, filtering is about dropping entire objects (spans, log records, metrics) according to specific rules, but transforming telemetry goes one step further and gives you even more flexibility and power. Here are some things that you can do with telemetry transformations:

Remove span attributes without discarding the span as a whole (or metric attributes, log record attributes, …).
Limit the number of attributes on objects such as spans or metric data points.
Limit the length of attributes.
Modify resource attributes.
Add attributes, based on the value of other attributes and conditions.
Change the value of attributes (think regex replacements).
Change the type of metrics (e.g. turn a sum into a gauge)
…and probably a gazillion other things.

As usual, the Dash0 operator allows you to configure these rules per Kubernetes namespace. Here is an example of a Dash0 monitoring resource that configures a set of transformations:

yaml

dash0-monitoring.yaml1234678911121415171819202123242526272829
apiVersion: operator.dash0.com/v1alpha1
kind: Dash0Monitoring
metadata:
  name: dash0-monitoring-resource
spec:
 
  transform:
    trace_statements:
      # limit span attributes generally to 4096 characters
      - 'truncate_all(span.attributes, 4096)'
      # remove sensitive data from collected command line invocations 
      - replace_pattern(span.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
      # remove query parameters (using the advanced config style here)
      - conditions:
        - 'span.attributes["http.target"] != nil'
        statements:
        - 'replace_pattern(span.attributes["http.target"], "customerId=.*", "customerId=xxx")'
    log_statements:
     
      # change the log level based on conditions
      - conditions:
        - 'log.body == "request failed"'
        statements:
        - 'set(log.severity_text, "FAIL")'

The configuration syntax is deliberately chosen to be identical to what the OpenTelemetry transform processor allows, supporting both the basic flat statement style as well as the advanced configuration style that supports adding conditions to a single transformation. This allows you to copy and paste existing transformation rules that you already have, or use examples you find online as-is.

For more details on telemetry transformations, check out the operator documentation.

Back to Changelog