Fly.io

Fly's log shipper is a Vector app that reads your organization's logs. Add one sink file and it posts them to /v1/ingest.

Set it up

1. Create a folder with this Dockerfile:

FROM flyio/log-shipper:latest
COPY simpledatalogs.toml /etc/vector/sinks/simpledatalogs.toml

2. Next to it, simpledatalogs.toml:

[transforms.simpledatalogs_fields]
  type = "remap"
  inputs = ["log_json"]
  source = '''
  .event_id = uuid_v4()
  .ts = .timestamp
  .level = .log.level
  .props.app = .fly.app.name
  .props.instance = .fly.app.instance
  .props.region = .fly.region
  .props.provider = .event.provider
  del(.timestamp)
  del(.log)
  del(.fly)
  del(.event)
  '''

[sinks.simpledatalogs]
  type = "http"
  inputs = ["simpledatalogs_fields"]
  uri = "https://edge.simpledatalogs.com/v1/ingest"
  method = "post"
  compression = "gzip"
  encoding.codec = "json"
  framing.method = "newline_delimited"
  request.headers.Content-Type = "application/x-ndjson"
  request.retry_attempts = 1000000
  auth.strategy = "bearer"
  auth.token = "${SDL_INGEST_KEY}"
  batch.max_bytes = 262144
  batch.timeout_secs = 5

3. Launch it and set its secrets:

fly launch --no-public-ips --no-deploy
fly secrets set ORG=personal ACCESS_TOKEN="$(fly tokens create readonly personal)" SDL_INGEST_KEY=sdl_sec_…
fly deploy

What's stored

• message, level and ts from Fly's log line.
• app, instance, region, provider and host as props.
• The event_id is set before the sink, so Vector's retries never count twice.

The transform matters: Fly's raw line has an event object, which /v1/ingest would refuse as an event name.

Tested with Vector 0.43.1 and the shipper's own log_json transform.

Keep the volume down

Every line from every app in the organization is an event. To ship one app, set SUBJECT=logs.your-app.> with fly secrets set.

Next steps

• Vector and Fluent Bit: the same sink for any host.