How to Set Up OpenTelemetry in Your Application to Send Data to the Otel Collector on Kubernetes

Introduction

Deploying applications in Kubernetes (K8s) environments adds complexity to observability due to the distributed nature of microservices and dynamic infrastructure. OpenTelemetry (OTel), combined with the OpenTelemetry Collector deployed in Kubernetes, provides a powerful and flexible observability pipeline for collecting, processing, and exporting traces, metrics, and logs.

This guide walks you through setting up OpenTelemetry in your application to send telemetry data to an OpenTelemetry Collector running in Kubernetes.

🚀 Prerequisites

Before you begin, ensure you have:

  • A running Kubernetes cluster (e.g., Minikube, EKS, GKE, or AKS).
  • kubectl configured to interact with your cluster.
  • Docker (for building application images, if needed).
  • Basic knowledge of Kubernetes resources (Deployments, Services, ConfigMaps).
  • Helm (optional, for easier deployment of the OpenTelemetry Collector).

🛠️ Step 1: Deploy the OpenTelemetry Collector in Kubernetes

The OpenTelemetry Collector acts as an agent or gateway to receive telemetry data and export it to backends like Jaeger, Prometheus, or Datadog.

✅ Option A: Deploy Using Helm (Recommended)

  1. Add the OpenTelemetry Helm repository: 
    helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
  2. Install the Collector with a basic configuration: 
    helm install otel-collector open-telemetry/opentelemetry-collector -n observability --create-namespace
  3. Verify the deployment: 
    kubectl get pods -n observability

✅ Option B: Deploy Using Kubernetes Manifests

Create a otel-collector-config.yaml for the Collector configuration:

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  logging:
    loglevel: debug
  jaeger:
    endpoint: "jaeger-collector:14250"
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [logging, jaeger]

Deploy the Collector:

kubectl apply -f otel-collector-config.yaml

⚙️ Step 2: Expose the OpenTelemetry Collector

Create a Kubernetes Service to expose the Collector for your application to send data.

apiVersion: v1
kind: Service
metadata:
  name: otel-collector
  namespace: observability
spec:
  ports:
    - name: grpc
      port: 4317
      targetPort: 4317
    - name: http
      port: 4318
      targetPort: 4318
  selector:
    app.kubernetes.io/name: opentelemetry-collector

Apply the service:

kubectl apply -f otel-collector-service.yaml

📥 Step 3: Instrument Your Application to Send Data

Your application needs to send telemetry data to the exposed otel-collector service using the OTLP protocol.

✅ Example Configuration for Different Languages

Python

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

trace.set_tracer_provider(TracerProvider())
tracer = trace.get_tracer(__name__)

otlp_exporter = OTLPSpanExporter(endpoint="otel-collector.observability.svc.cluster.local:4317")
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(otlp_exporter))

with tracer.start_as_current_span("example-span"):
    print("Sending trace to OpenTelemetry Collector in Kubernetes")

Node.js

const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-base');

const provider = new NodeTracerProvider();
const exporter = new OTLPTraceExporter({ url: 'grpc://otel-collector.observability.svc.cluster.local:4317' });

provider.addSpanProcessor(new BatchSpanProcessor(exporter));
provider.register();

const tracer = provider.getTracer('example-service');
const span = tracer.startSpan('example-span');
console.log('Trace sent to Collector');
span.end();

Java

import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;

SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
    .addSpanProcessor(BatchSpanProcessor.builder(OtlpGrpcSpanExporter.builder()
        .setEndpoint("otel-collector.observability.svc.cluster.local:4317")
        .build())
    .build())
    .build();

Tracer tracer = tracerProvider.get("example-service");
Span span = tracer.spanBuilder("example-span").startSpan();
System.out.println("Sending trace to OpenTelemetry Collector");
span.end();

🖥️ Step 4: Visualize Data Using Jaeger and Prometheus

Deploy backends like Jaeger for tracing and Prometheus for metrics.

✅ Deploy Jaeger

kubectl create namespace tracing
kubectl apply -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/examples/simplest.yaml

Access Jaeger UI:

kubectl port-forward svc/jaeger-query 16686:16686 -n tracing

Visit: http://localhost:16686

✅ Deploy Prometheus (Optional)

helm install prometheus prometheus-community/prometheus -n monitoring --create-namespace

Access Prometheus UI:

kubectl port-forward svc/prometheus-server 9090 -n monitoring

Visit: http://localhost:9090

🔍 Step 5: Verify Data Flow

Once your application and the Collector are running:

  • Traces: Check Jaeger to view spans and trace timelines.
  • Metrics: Query Prometheus to see collected metrics.
  • Logs: Use the Collector’s logging exporter for initial validation.

✅ Tip: Use Grafana to create dashboards combining metrics and traces for comprehensive observability.

Conclusion

Setting up OpenTelemetry to send data from your application to the OpenTelemetry Collector in Kubernetes provides scalable and flexible observability. By following these steps, you gain real-time insights into your application’s performance, enabling faster debugging, improved reliability, and better end-user experiences.

Complete OpenTelemetry Blog Series

Subscribe to the Cloud Native Hero! Newsletter for regular updates.

Join the [Observability India LinkedIn Group] (https://www.linkedin.com/groups/9899111/)

LinkedIn | Twitter | GitHub | Blog