Python

The simpledatalogs package sends events and log lines from Python servers, workers and scripts. It uses only the standard library, is typed, and never raises into your app.

• Python 3.9+.
• A logging.Handler for Django, Flask or any logger.
• An AsyncClient for FastAPI and other asyncio apps.

Install

pip install simpledatalogs

Early access

The package isn't on PyPI yet. Until it is, install it from GitHub:

pip install "git+https://github.com/XKD-Labs/simpledatalogs-python.git"

Quick start

import simpledatalogs

sdl = simpledatalogs.init()  # reads SIMPLEDATALOGS_KEY

sdl.track("signup", {"plan": "pro"}, user_id=42)
sdl.log("error", "Payment failed", {"order_id": 7})

A background thread sends every 5 seconds, and whatever is left goes out when the process exits. Use a secret key (sdl_sec_…) on servers.

simpledatalogs.init() creates the process-wide client; simpledatalogs.Client(key, **options) creates one you hold yourself. Without a key the client is off and says so once.

Events

sdl.track("checkout", {"plan": "pro", "seats": 3, "trial": False}, user_id=user.id)
sdl.track("import_done", {"rows": 1200}, timestamp=finished_at)  # a datetime or epoch seconds

• Every line gets a UUID event_id and a ts when you call track() or log(), so a retry never counts twice.
• Props are strings, finite numbers or booleans (enums send their value); anything else is skipped with a warning. At most 100 props.

Logs

SimpleDataLogsHandler is a logging.Handler. Each record becomes one log line with its level and time.

import logging
import simpledatalogs

simpledatalogs.init()
logging.getLogger().addHandler(simpledatalogs.SimpleDataLogsHandler(level=logging.INFO))

logging.getLogger("payments").error("Payment failed", extra={"order_id": 7, "user_id": 42})

• Props: logger, scalar extra= values, and for exceptions exception.type, exception.message and stacktrace (the last 8 KB).
• Levels map to debug, info, warning, error and critical. Pass include_extras=False to send no extras.
• The handler only queues; it never waits on the network.

Django

# settings.py
import simpledatalogs

simpledatalogs.init()

LOGGING = {
    "version": 1,
    "handlers": {"simpledatalogs": {"class": "simpledatalogs.SimpleDataLogsHandler", "level": "WARNING"}},
    "root": {"handlers": ["simpledatalogs"], "level": "INFO"},
}

With gunicorn or uWSGI, each worker process gets its own sending thread after the fork.

Flask

import logging
import simpledatalogs
from flask import Flask

sdl = simpledatalogs.init()
app = Flask(__name__)
app.logger.addHandler(simpledatalogs.SimpleDataLogsHandler(sdl, level=logging.WARNING))

FastAPI

AsyncClient takes the same options. Its sends run in a worker thread, so the event loop never waits on the network.

from contextlib import asynccontextmanager

import simpledatalogs
from fastapi import FastAPI

sdl = simpledatalogs.AsyncClient()


@asynccontextmanager
async def lifespan(app: FastAPI):
    yield
    await sdl.aclose()


app = FastAPI(lifespan=lifespan)

await sdl.flush() sends now.

Users

Identity is per call: user_id, device_id, session_id and anonymous_id, on track() and log(). In log records, the same names in extra= fill the identity fields.

The opt-out

sdl.set_enabled(False)  # deletes every unsent line, from disk too, and ignores new ones
sdl.set_enabled(True)   # collects again

To keep a client silent in tests or local runs, pass enabled=False.

Options

Option	Default	What it does
key	SIMPLEDATALOGS_KEY	The ingest key
endpoint	SIMPLEDATALOGS_ENDPOINT, else https://edge.simpledatalogs.com	The ingest host; /v1/ingest is added
flush_interval_seconds	5	How often the background thread sends
max_batch_events	50	Lines per request
max_request_bytes	256000	Bytes per request before gzip
max_queue_events / max_queue_bytes	10000 / 10000000	The queue's caps
outbox_dir	none	A directory for the disk outbox
default_props	none	Props added to every line, under the call's own
gzip	True	Gzips bodies of 1 KB and more
request_timeout_seconds	15	Per request
flush_at_exit / exit_timeout_seconds	True / 5	Flush at interpreter exit, for at most this long
enabled	True	False keeps the client silent

Privacy

• The SDK sends only what you pass: no IP addresses, hostnames, environment variables or request data.
• extra= values reach ingest as you logged them. Leave out personal data, or pass include_extras=False.
• See Privacy for senders for what belongs in user_id.

How it behaves

Batching

Every 5 seconds, or at once when 50 lines wait. Each request stays under 256 KB before gzip.

Flush and exit

• sdl.flush(timeout=5) sends now and returns True when nothing is left.
• sdl.close() flushes and stops the thread.
• At interpreter exit the client flushes for up to 5 seconds. Call close() yourself in workers that exit through os._exit.

Disk outbox

The queue lives in memory unless you give it a directory:

sdl = simpledatalogs.init(outbox_dir="/var/lib/myapp/simpledatalogs")

• Each line is appended to a file as it is tracked and leaves after a 202, so a crash or a deploy loses nothing.
• Each process keeps its own locked file. A file whose process is gone is picked up by the next client on the same directory.
• It needs POSIX file locks (Linux, macOS); elsewhere the client warns and keeps lines in memory.

Errors and offline

Answer	What the SDK does
202	Removes the lines
401	Keeps every line and re-checks the key after 1 minute, then every 10 minutes
413	Halves the batch; a single line still too large is dropped
415	Drops the batch, with one warning
Anything else, a timeout, offline	Backs off with full jitter from 1 second to 5 minutes, never sooner than Retry-After

Caps

The queue keeps at most 10,000 lines or 10 MB. Past that, the oldest unsent lines are dropped, with a warning.

Troubleshooting

Problems are logged once each on the simpledatalogs logger. For local development, set SIMPLEDATALOGS_ENDPOINT=http://localhost:4001.

Next steps

• Python quickstart: the shortest path.
• Text log lines: how levels and messages are stored.
• Privacy for senders: ids, not emails.