Confluent Cloud

Conluent Cloud Integration in Dash0

Kafka is a powerhouse, and Confluent can run it for you. If you are using their offering, you will still want insights into Kafka's and its clients' behavior. To that extent, Confluent exposes metrics for its customers that you can view via Dash0.

Use Cases

• Understand Kafka consumer lag.
• Analyze topic throughput.
• Inspect storage and usage.
• See latency insights.

Overview

Confluent Cloud exposes metrics through a Prometheus endpoint that you can scrape. We recommend to scrape and forward these metrics via an OpenTelemetry Collector.

Retrieving Credentials

To start, get a hold of your exporter key and secret. The Confluent Cloud documentation explains how to access these. Once available, store these. In the next step, we will assume that these are exposed through environment variables called CONFLUENT_METRICS_EXPORTER_KEY and CONFLUENT_METRICS_EXPORTER_SECRET respectively. We also recommend to define the name of the environment within an environment variable called ENVIRONMENT_NAME.

Collector Configuration

The following code snippet shows an OpenTelemetry Collector configuration file.

The OpenTelemetry Collector pipeline visualized in OTelBin

Collector Deployment

Learn how to deploy the collector within our OpenTelemetry collector integration documentation.

References

• Official Confluent Cloud documentation
• OpenTelemetry Collector configuration in OTelBin