OpenTelemetry logs

Any OpenTelemetry SDK or Collector can send its logs to simpledatalogs over OTLP/HTTP. You change environment variables or exporter settings, not code.

POST https://edge.simpledatalogs.com/v1/logs
Authorization: Bearer <ingest key>
Content-Type: application/x-protobuf | application/json
Content-Encoding: gzip   (recommended)

Logs only

Traces and metrics are not stored: /v1/traces and /v1/metrics always answer 404. Turn those exporters off, or use logs_endpoint in the Collector.

SDKs, with environment variables

export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://edge.simpledatalogs.com/v1/logs
export OTEL_EXPORTER_OTLP_LOGS_HEADERS="authorization=Bearer%20sdl_sec_…"
export OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_LOGS_COMPRESSION=gzip
export OTEL_LOGS_EXPORTER=otlp
export OTEL_TRACES_EXPORTER=none
export OTEL_METRICS_EXPORTER=none
export OTEL_SERVICE_NAME=checkout
# Lower this if records average over ~1 KB, so each export stays under 512 KB.
export OTEL_BLRP_MAX_EXPORT_BATCH_SIZE=512

• The header value is percent-encoded (%20 for the space), as the OpenTelemetry spec requires. The Python SDK also accepts a literal space.
• http/json works too, in SDKs that support it.
• With the base endpoint instead (OTEL_EXPORTER_OTLP_ENDPOINT=https://edge.simpledatalogs.com), keep OTEL_TRACES_EXPORTER=none and OTEL_METRICS_EXPORTER=none.
• A device.id or user.id attribute, on the resource or the record, fills the identity, so per-device views work.

Tested with the Python SDK (opentelemetry-sdk 1.29.0), protobuf and gzip.

The Collector

processors:
  batch:
    send_batch_size: 256
    send_batch_max_size: 512   # a hard cap per export, keeps requests under 512 KB once gzipped
    timeout: 5s

exporters:
  otlphttp/simpledatalogs:
    logs_endpoint: https://edge.simpledatalogs.com/v1/logs
    encoding: proto            # or json
    compression: gzip          # the default; keep it
    headers:
      authorization: Bearer sdl_sec_…
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_interval: 60s
      max_elapsed_time: 0s     # never give up on 429 or 503
    sending_queue:
      enabled: true
      queue_size: 1000

service:
  pipelines:
    logs:
      receivers: [otlp]        # or filelog, journald…
      processors: [batch]
      exporters: [otlphttp/simpledatalogs]

Use logs_endpoint, not endpoint

With endpoint, the exporter also derives /v1/traces and /v1/metrics, which answer 404.

Tested with collector-contrib 0.115.1, fed by telemetrygen.

OTLP/JSON by hand

gzip -c logs.json | curl -sS https://edge.simpledatalogs.com/v1/logs \
  -H "Authorization: Bearer sdl_sec_…" \
  -H "Content-Type: application/json" \
  -H "Content-Encoding: gzip" \
  --data-binary @-
# → 200 {}

The same export sent twice stores each record once.

How records map

Each LogRecord becomes one row.

Field	From the LogRecord
ts	time_unix_nano, else observed_time_unix_nano, else the receive time
level	severity_number 1–8 debug, 9–12 info, 13–16 warning, 17–20 error, 21–24 critical; otherwise severity_text
message	The body. A map body's message or msg key, with its other keys in props.body
event	event_name, else the event.name attribute; without either the record is a log line
identity	device.id, user.id then enduser.id, session.id, from the record, else the resource
event_id	Derived from the record's content when it has a time, so an exporter's retry never counts twice
props	The record's attributes (dotted keys stay flat, such as http.route), common resource attributes such as service.name, host.name and deployment.environment, the scope, and trace_id / span_id as hex

Uncommon resource attributes are left out and counted in props.otel.dropped_resource_attributes, so a large resource never refuses a record. OTLP is stored as sent, without masking.

Keep exports under 512 KB

OTLP exporters drop a batch answered 401 or 413; they retry only 429, 502, 503 and 504. So:

• Check the key before going live.
• Turn on gzip (OTEL_EXPORTER_OTLP_LOGS_COMPRESSION=gzip; the Collector's otlphttp exporter gzips by default).
• The SDK default of 512 records per export fits while records average under ~1 KB uncompressed. Lower OTEL_BLRP_MAX_EXPORT_BATCH_SIZE, or the Collector's send_batch_max_size, for larger records.

A slow body answers 503, not 408, so exporters retry it.

Next steps

• Responses: every OTLP answer and its gRPC code.
• Vector and Fluent Bit: ship log files without OpenTelemetry.