Observability

The Manager emits OTLP/HTTP protobuf traces, metrics, and structured logs. Local Compose runs an OpenTelemetry Collector that routes signals to Tempo, Prometheus, and Loki, with pre-provisioned Grafana data sources.

Start the local stack

cp docker/.compose.env.sample docker/.compose.env
docker compose up -d otel-collector tempo prometheus loki grafana
SurfaceLocal URLUse
Grafanahttp://localhost:3001Explore logs, traces, and metrics (admin / admin)
Prometheushttp://localhost:9090Query scraped and OTLP-derived metrics
Tempohttp://localhost:3200Trace backend API
Lokihttp://localhost:3100Log backend API
OTLP/HTTPhttp://localhost:4318Collector ingest

Correlate a request

  1. Search Loki in Grafana for encode-flow-manager.
  2. Open a structured request log and inspect trace_id and span_id.
  3. Follow the configured TraceID derived field into Tempo.
  4. Compare the server request span with PostgreSQL, NATS, and application spans.

The logger recursively redacts credentials, tokens, cookies, and other sensitive fields. Do not depend on redaction as permission to log request bodies containing secrets.

Production export

OTEL_EXPORTER_OTLP_ENDPOINT=https://alloy.example.com/otlp
OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer%20REPLACE_ME
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_TRACES_SAMPLER=parentbased_traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1
OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production

Production validation rejects an insecure endpoint while telemetry is enabled. Keep certificate verification active and mount private CA trust into the container when required.

Graceful shutdown

The Manager stops accepting work, closes NestJS resources, and flushes telemetry within its configured shutdown timeout. Give Kubernetes enough terminationGracePeriodSeconds for both application shutdown and OTLP export.