otel-instrumentation

---
name: 'otel-instrumentation'
description: Configures trace spans, defines custom metrics, sets up log exporters, and optimizes sampling strategies for OpenTelemetry instrumentation. Use when instrumenting applications with traces, metrics, or logs. Triggers on requests for observability, telemetry, tracing, metrics collection, logging integration, or OTel setup.
license: Apache-2.0
metadata:
  author: dash0
  version: '2.0.0'
  workflow_type: 'advisory'
  supports-traces: "true"
  supports-metrics: "true"
  supports-logs: "true"
---

# OpenTelemetry Instrumentation Guide

Expert guidance for implementing high-quality, cost-efficient OpenTelemetry telemetry.

## Rules & Quick Reference

| Use Case / Rule | Description |
|-----------------|-------------|
| [telemetry](./rules/telemetry.md) | **Entrypoint** — signal types, correlation, and navigation |
| [resolve-values](./rules/resolve-values.md) | Resolving configuration values from the codebase |
| [verify-dependencies](./rules/verify-dependencies.md) | Verifying instrumentation packages and versions exist before adding them |
| [resources](./rules/resources.md) | Resource attributes — service identity and environment |
| [k8s](./rules/platforms/k8s.md) | Kubernetes deployment — downward API, pod spec |
| [spans](./rules/spans.md) | Spans — naming, kind, status, and hygiene |
| [logs](./rules/logs.md) | Logs — structured logging, severity, trace correlation |
| [metrics](./rules/metrics.md) | Metrics — instrument types, naming, units, cardinality |
| [sensitive-data](./rules/sensitive-data.md) | Sensitive data — PII prevention, sanitization, redaction |
| [capture-database-query-parameters](./rules/capture-database-query-parameters.md) | Prepared-statement parameter capture per language (Java, .NET, Python, Node.js, Go) |
| [validation](./rules/validation.md) | Telemetry validation — post-deployment verification checklist |
| [nodejs](./rules/sdks/nodejs.md) | Node.js instrumentation setup |
| [go](./rules/sdks/go.md) | Go instrumentation setup |
| [python](./rules/sdks/python.md) | Python instrumentation setup |
| [java](./rules/sdks/java.md) | Java instrumentation setup |
| [scala](./rules/sdks/scala.md) | Scala instrumentation setup |
| [dotnet](./rules/sdks/dotnet.md) | .NET instrumentation setup |
| [ruby](./rules/sdks/ruby.md) | Ruby instrumentation setup |
| [php](./rules/sdks/php.md) | PHP instrumentation setup |
| [browser](./rules/sdks/browser.md) | Browser instrumentation setup |
| [nextjs](./rules/sdks/nextjs.md) | Next.js full-stack instrumentation (App Router) |

## Official documentation

- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)
- [Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/)
- [Dash0 Integration Hub](https://www.dash0.com/hub/integrations)

## Getting started

Follow these steps when instrumenting an application from scratch:

1. **Pick your SDK rule** — choose the language-specific rule from the table above (e.g., [nodejs](./rules/sdks/nodejs.md), [python](./rules/sdks/python.md)).
2. **Set up resource attributes** — define service identity and environment per [resources](./rules/resources.md).
3. **Add spans, metrics, and logs** — instrument your code following [spans](./rules/spans.md), [metrics](./rules/metrics.md), and [logs](./rules/logs.md).
4. **Guard sensitive data** — scrub PII before export per [sensitive-data](./rules/sensitive-data.md).
5. **Validate** — confirm telemetry reaches the backend using the checklist in [validation](./rules/validation.md).

The snippet below shows a complete span with attributes and status for Node.js — see [nodejs](./rules/sdks/nodejs.md) for full setup including SDK initialisation, exporter configuration, and auto-instrumentation:

```js
import { trace, SpanStatusCode } from '@opentelemetry/api';
const tracer = trace.getTracer('my-service', '1.0.0');

tracer.startActiveSpan('operation-name', async (span) => {
  try {
    span.setAttribute('user.id', userId);
    span.setAttribute('order.id', orderId);

    const result = await processOrder(orderId);

    span.setAttribute('order.status', result.status);
    span.setStatus({ code: SpanStatusCode.OK });
    return result;
  } catch (err) {
    // Record the exception as a structured log record, not span.recordException — see rules/spans.md
    span.setStatus({ code: SpanStatusCode.ERROR, message: `${err.name}: ${err.message}` });
    const spanContext = span.spanContext();
    logger.error('operation-name.failed', {
      'trace_id': spanContext.traceId,
      'span_id': spanContext.spanId,
      'exception.type': err.name,
      'exception.message': err.message,
      'exception.stacktrace': err.stack,
    });
    throw err;
  } finally {
    span.end();
  }
});
```

## Key principles

### Signal density over volume

Every telemetry item should serve one of three purposes:
- **Detect** - Help identify that something is wrong
- **Localize** - Help pinpoint where the problem is
- **Explain** - Help understand why it happened

If it doesn't serve one of these purposes, don't emit it.

### Sample in the pipeline, not the SDK

Use the `AlwaysOn` sampler (the default) in every SDK.
Do not configure SDK-side samplers — they make irreversible decisions before the outcome of a request is known.
Defer all sampling to the [Collector](../otel-collector/rules/sampling.md), where policies can be changed centrally without redeploying applications.

<!-- eval:skip -->
```
SDK (AlwaysOn)  →  Collector (sampling)  →  Backend (retention)
     ↓                    ↓                       ↓
  All spans         Head or tail            Storage policies
  exported          sampling applied
```

From dash0hq/agent-skills at 51a0324, shown as plain text under its license: Apache-2.0 (permissive). The license text is in skills/otel-instrumentation/SKILL.md.

• SKILL.md5.7 KB · e79c66b5f75c
• README.md4.9 KB · 82f9e47627af
• rules/
• capture-database-query-parameters.md5.1 KB · f1c56b8e7e7c
• logs.md10 KB · e0012694b54d
• metrics.md23 KB · 4c1bfc871c3f
• platforms/
• k8s.md7.9 KB · 3f5921df6de0
• resolve-values.md12 KB · 83992f5bd2c5
• resources.md11 KB · b3d64c524863
• sdks/
• browser.md5.2 KB · d8554f6da8d6
• dotnet.md21 KB · 9f99faa2a9a1
• go.md30 KB · 5f1475233613
• java.md17 KB · 0c8e24e9286f
• nextjs.md16 KB · ac76ff3730b4
• nodejs.md25 KB · 3b6d0b159b81
• php.md16 KB · 6616a21e96a9
• python.md19 KB · 313f6546faa5
• ruby.md15 KB · 8115461eb54a
• scala.md24 KB · 303a4dbe53f8
• sensitive-data.md17 KB · 694d27127409
• spans.md27 KB · dc62254d65c2
• telemetry.md2.2 KB · e5cd46ee7400
• validation.md7.3 KB · 0058050e55b1
• verify-dependencies.md6.8 KB · 80d98a66508f

23 files, 329 KB in total. Each file has a sha256 in the pin's manifest; the first 12 characters are shown.