The platform behind this site

A personal site needs none of this. That is rather the point: it is a place to build the things I would otherwise only read about, where the consequences of getting it wrong are my own.

What follows is honest about the failures, because those are the parts worth reading.

The LLM gateway

The backend exposes an OpenAI-compatible /v1/chat/completions endpoint backed by LiteLLM. The interesting work is not the proxying — it is everything around it.

Spend is a correctness problem. An endpoint that forwards whatever a caller sends to a paid provider is not an API, it is a billing incident waiting to happen. Three controls, all server-side:

• A model allowlist. The request carries a model name; anything outside the configured list is rejected before the provider is contacted. Without this, a caller picks the most expensive model available and you find out at the end of the month.
• A token ceiling applied to every request. max_tokens is clamped to a server maximum, and omitting it does not mean unbounded — omitting it means the ceiling.
• Disconnect detection on the streaming path. If the client goes away mid-stream, the server stops consuming. Otherwise you pay for tokens nobody will ever read.

The provider sits behind a seam. The client is a Protocol resolved through dependency injection rather than a module-level singleton:

class ChatCompletions(Protocol):
    async def create(self, **kwargs: Any) -> Any: ...

class ChatClient(Protocol):
    @property
    def chat(self) -> Chat: ...

That shape had two concrete payoffs. Swapping the OpenAI SDK for LiteLLM touched exactly one factory function — no router changed, no service changed, and the frontend never knew. And tests substitute a fake through dependency_overrides, so the suite never reaches the network.

The seam also fixed a startup bug. The original client was built at import time and raised when the API key was missing, which crashed the whole application rather than failing one request. Validation now happens inside the factory.

The bug that had been live for a year

The endpoint returned HTTP 500 on every single request, and had since it was written.

The request body was annotated openai.types.chat.CompletionCreateParams. That looks like a Pydantic model. It is a union of two TypedDict subclasses — and Pydantic validates a TypedDict into a plain dict. So this line:

params.stream = True   # AttributeError: 'dict' object has no attribute 'stream'

…threw on every call, into a bare except that turned it into a 500. The frontend defaulted to stream: true, a shape that had never once worked.

The fix was an explicit Pydantic model, which is also where the allowlist and the token ceiling now live. The lesson is narrower than "write tests": it is that a type annotation which looks like validation is more dangerous than no annotation at all, because it stops you looking.

The streaming pipeline

Binance publishes trade ticks over a WebSocket. The pipeline takes them to browsers:

Binance WSS
  → transform
  → Avro + Confluent Schema Registry
  → Kafka
  → consumer
  → Kafka-to-WebSocket bridge
  → Redis pub/sub
  → connected clients

Why Avro rather than JSON. A schema registry turns "the producer changed a field name" from a runtime mystery into a deploy-time failure. For a pipeline nobody is watching, that is the difference between noticing and not.

Why Redis between Kafka and the socket. The WebSocket manager fans out through Redis pub/sub rather than holding connections in process memory, so the API can run more than one worker. That part scales; the rest of the process state does not yet, which is an honest limitation rather than a feature.

The bridge is source-agnostic. It consumes a topic and publishes to a channel; nothing in it knows what Binance is. Pointing it at a different source is configuration.

Kafka plus Schema Registry is roughly 1–1.5 GB of JVM for a pipeline with no consumer in the UI yet, so it now sits behind a Docker Compose profile and is off by default. Keeping the code and not paying for the memory seemed better than deleting work that functions.

A failure worth keeping

An early version called signal.signal() inside a request handler to install SIGINT and SIGTERM handlers for the producer. This hijacks the web server's own shutdown handling, and raises outright if ever called off the main thread. It is the kind of thing that works perfectly in development and is deeply confusing in production.

Identity

Admin surfaces sit behind Authentik, self-hosted: server, worker, and its own Postgres and Redis, deliberately separate from the application's, so a schema migration during an upgrade cannot disturb the API.

The OAuth2 provider and application are a version-controlled blueprint, not something clicked into a web UI — so the configuration survives a rebuild and is reviewable in a diff.

Two things that cost an evening and are worth writing down:

The published schema was wrong. redirect_uris is now a list of structured objects, not the newline-separated string that most documentation and every blog post still shows. I only caught it by reading the example blueprint shipped inside the running image.

A fresh install deadlocked. The worker owns database migrations, but server and worker started together — so the server crash-looped on missing tables while the worker was still migrating, leaving a half-built schema. A system migration then checked only that django_migrations existed, not that the table it queried existed, so every subsequent start failed the same way. Permanently stuck at ten migrations and seven tables. The fix is ordering: the server waits on the worker's health check. The recovery is wiping the volume, because a partial schema cannot migrate forward.

Infrastructure

The host is reproducible from a single Ansible playbook: users, SSH hardening, firewall, fail2ban, Docker, nginx with TLS, systemd units, swap, and the interactive environment. Running it twice is a no-op.

That mattered immediately. Moving hosting meant the new machine was built by running the playbook rather than by remembering what had been done to the old one — and the measured result was time-to-first-byte from ~0.9 s to ~0.14 s, almost entirely from putting the server on the same continent as its readers.

The firewall that did nothing

The old server published its API port with no interface binding, so Docker bound it to 0.0.0.0. The API was reachable directly, bypassing nginx and every access control in front of it.

The obvious fix — a host firewall — does not work. Docker writes its own DNAT rules into PREROUTING, which is traversed before the INPUT chain where ufw lives. Enabling the firewall closed one port and left the other wide open, while looking like it had secured both.

The real control is binding containers to 127.0.0.1 so nginx is the only route in. The firewall is defence in depth, and it is worth knowing which of your controls is load-bearing — because the comforting one here was not.

What I would do differently

• Build in CI, not on the box. Rebuilding images on a small server makes every deploy a multi-minute outage window. Build once, push to a registry, pull on the host.
• Write the test first when the failure is silent. The 500 persisted because nothing exercised the endpoint. Two tests would have caught it on day one.
• Treat memory as a design constraint from the start. The original host had 407 MB free and no swap, which quietly ruled out options long before anybody noticed.