Tracing

SDL provides distributed tracing using OpenTelemetry.

Instrumentation is automatically injected into supported libraries and frameworks, such as Django, SQLAlchemy, and gRPC. The SDL Kafka consumer / producer is instrumented in accordance with the OpenTelemetry Trace Semantic Conventions specification.

If desired, downstream services can add their own tracing on top of what is provided by SDL by using the OpenTelemetry API.

Configuration

SDL_ environment variables are used to configure some aspects of the tracing feature.

Helm Parameter	Key	Type	Description	Default
`tracing.enabled`	`SDL_TRACING_ENABLED`	`bool`	Enables the tracing feature.	`false`
`tracing.useDeprecatedJaegerExporter`	`SDL_TRACING_USE_DEPRECATED_JAEGER_EXPORTER`	`bool`	Exports spans using the deprecated Jaeger exporter instad of the default OTLP gRPC exporter. This setting only affects the Python SDK.	`false`
N/A	`SDL_TRACING_BASELINE_MODE`	`bool`	Enables the testing baseline mode, which prints spans to the console instead of exporting them.	`false`

All other configuration is given using OTEL_ environment variables, as documented in the OpenTelemetry specification.

A summary of the most important configuration is given here, but the OpenTelemetry documentation contains a comprehensive listing.

Helm Parameter	Key	Type	Description	Default
`tracing.exporter.jaeger.agentHost`	`OTEL_EXPORTER_JAEGER_AGENT_HOST`	`string`	The hostname of the Jaeger agent. This setting has no effect unless `SDL_TRACING_USE_DEPRECATED_JAEGER_EXPORTER` is enabled.	`localhost`
`tracing.exporter.otlp.endpoint`	`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`	`string`	The URL of the collector to send spans to, including a scheme of either `http` or `https`.	`http://localhost:4317`
`tracing.exporter.otlp.compression`	`OTEL_EXPORTER_OTLP_TRACES_COMPRESSION`	`string`	The gRPC compression method to be used by the exporter (either `gzip` or unset).	N/A
`tracing.exporter.otlp.timeout`	`OTEL_EXPORTER_OTLP_TRACES_TIMEOUT`	`number`	The maximum time (in milliseconds) that the exporter will wait for each batch export.	`10000`

Helm

This is the recommended way to configure tracing in a service's Helm chart:

tracing:
    enabled: true
    useDeprecatedJaegerExporter: false

    exporter:
        jaeger:
            agentHost: tempo-tempo-distributed-distributor.tempo

        otlp:
            endpoint: http://tempo-tempo-distributed-distributor.tempo:4317
            compression: gzip
            timeout: 10_000