Redis Infrastructure

Overview

Bun and Go, on Fly and on K8s, all use one Redis running in Docker on the home UGREEN NAS (192.168.0.142). K8s reaches it on the home LAN. Fly reaches it over Fly’s private network: the NAS is a Fly WireGuard peer. The decision and the alternatives that were rejected are in ADR-0074.
The home NAS and its internet link are a single point of failure for all four backends. Nothing fails hard when Redis is down (see Outage behaviour), but rate limits become per process and token revocation stops working until it is back.

What uses Redis

Key prefixes don’t overlap, so sharing one database (db 0) is safe.
Revocation only works when the Bun that writes a marker and the Go that reads it use the same Redis. That is the main reason Fly and K8s share one instance instead of each having their own.
Better Auth uses customStorage, not secondaryStorage: secondaryStorage would move sessions out of Postgres, and Go and the K8s replicas read sessions from Postgres.

Topology

The URL format is redis://:<password>@<host>:<port>. Never commit a password or paste it into chat or a ticket.

Why it looks like this

  • Port 6380, not 6379, on the LAN. UGOS runs its own Redis on 127.0.0.1:6379, so Docker can’t publish 6379 on the NAS. Don’t stop or reconfigure that system Redis.
  • An IPv6 literal for Fly. Fly peer DNS names (gremlin-nas-redis._peer.internal) don’t resolve from the machines. Bun’s client and go-redis both accept the bracketed form.
  • Redis shares the WireGuard container’s network (network_mode: service:wireguard), so it listens on the peer address directly. The LAN port is published on the WireGuard container, because a container in another container’s network can’t publish its own.
  • The NAS dials out to Fly (PersistentKeepalive = 15), so nothing is port-forwarded on the router and a changing home IP doesn’t matter.

Configuration Go requires

Go’s startup check ValidateForRateLimiter (redis_startup_validation.go) needs:
  • maxmemory-policy noeviction, so rate-limit keys are never evicted under memory pressure;
  • persistence: AOF (appendonly yes) or an RDB save schedule;
  • CONFIG GET allowed (don’t rename or disable CONFIG, don’t use an ACL user without it).
If any of these fail, Go logs [startup] Redis rate-limiter validation failed: v1 API is degraded. That line only reports the state at boot. Nothing keeps it: requests work as soon as Redis is reachable. The NAS compose project (password elided):

Outage behaviour

When Redis is unreachable (NAS down, home internet down, tunnel down): No restart is needed anywhere: go-redis redials on demand, and Bun’s BunRedisCache reconnects at most every 5 s and waits for an in-flight connect instead of failing. /health doesn’t depend on Redis (Bun skips the limiter for it; Go’s limiter falls back to memory), so an outage doesn’t fail probes or restart pods and machines. Each command from Fly costs about 13 ms. Bun’s app limiter makes about 3 commands per request; Go’s IP limiters make one, with a 150 ms timeout.

Operations

On macOS, if kubectl says no route to host while nc -z 192.168.0.153 6443 succeeds, your terminal is blocked by Local Network privacy. Allow it in System Settings → Privacy & Security → Local Network, or run kubectl in Docker: docker run --rm -v $HOME/.kube:/kube:ro -e KUBECONFIG=/kube/config bitnami/kubectl:latest …

Check that each deployment can reach Redis

Healthy logs: Go prints Successfully connected to Redis and "redis":true; Bun prints Using BunRedisCache for caching and rate limiting.

Change the Redis password

Every backend must get the new value, or the ones you miss lose Redis:
  1. Update --requirepass in the NAS compose project and redeploy it.
  2. Reseal REDIS_URL in backend/bun/k8s/overlays/production/sealed-secret.yaml and backend/go/k8s/overlays/production/sealed-secret.yaml (namespace-wide scope, the overlay’s pub-cert.pem), commit, then sync bun-prod (manual) and let backend-prod auto-sync. Restart the deployments so the pods read the new Secret.
  3. Set REDIS_URL on both Fly apps through stdin, so the password isn’t in your shell history: printf 'REDIS_URL=%s\n' "$URL" | fly secrets import -a <app>.

The WireGuard key or peer is lost

The private key exists only in wg0.conf on the NAS. To replace the peer:
Delete the DNS = line from the new file, copy it to the WireGuard config folder on the NAS, and redeploy the project. The peer gets a new address: update REDIS_URL on both Fly apps with it.

The NAS’s LAN IP changes

K8s loses Redis (Fly doesn’t, because the tunnel doesn’t depend on the LAN IP). Keep a DHCP reservation for 192.168.0.142. If the IP must change, reseal REDIS_URL in both K8s overlays.