AWS-hosted Kubernetes compute on Fargate
Amazon can operate your Kubernetes clusters for you via its EKS services. It can also offer to run Kubernetes pods on Fargate, which this integration is all about.
EKS on Fargate has several limitations regarding telemetry extraction, but AWS offers a way to extract logs. This guide will teach you how.
You can collect logs from Amazon EKS on Amazon Fargate through AWS-maintained Fluent Bit instances. All it takes is the
magical aws-observability Kubernetes Namespace and ConfigMap.
To start, place the following content in a file called config.yaml.
This configuration enables log collection, activates Kubernetes metadata collection, and maps the metadata to OpenTelemetry's semantic conventions through Fluent Bit's filter capabilities.
The output configuration instructs Fluent Bit to send its data to Dash0's Elasticsearch Bulk endpoint.
You can apply those changes to your Amazon EKS cluster on AWS Fargate. The Fargate admission controller will validate the configuration. Once it is applied, you will not see any additional workloads in your Kubernetes cluster as a result of this configuration.
An unfortunate reality of Amazon EKS on Amazon Fargate is that logging changes only apply to newly scheduled pods. Consequently, we recommend rescheduling a pod to validate that everything works as intended.
To validate that Amazon EKS picks up on the configuration, check the events for the newly scheduled pod, e.g., via
kubectl events or kubectl describe pod -n [namespace] [pod name].
When correctly configured, your pod will have an event similar to the following: