Redis Infrastructure
Overview
Bun and Go, on Fly and on K8s, all use one Redis running in Docker on the home UGREEN NAS (192.168.0.142). K8s reaches it on the home LAN. Fly reaches it over Fly’s private
network: the NAS is a Fly WireGuard peer. The decision and the alternatives that were
rejected are in ADR-0074.
What uses Redis
Key prefixes don’t overlap, so sharing one database (db 0) is safe.Revocation only works when the Bun that writes a marker and the Go that reads it use the
same Redis. That is the main reason Fly and K8s share one instance instead of each having
their own.
customStorage, not secondaryStorage: secondaryStorage would move
sessions out of Postgres, and Go and the K8s replicas read sessions from Postgres.
Topology
The URL format is
redis://:<password>@<host>:<port>. Never commit a password or paste it
into chat or a ticket.
Why it looks like this
- Port 6380, not 6379, on the LAN. UGOS runs its own Redis on
127.0.0.1:6379, so Docker can’t publish 6379 on the NAS. Don’t stop or reconfigure that system Redis. - An IPv6 literal for Fly. Fly peer DNS names (
gremlin-nas-redis._peer.internal) don’t resolve from the machines. Bun’s client and go-redis both accept the bracketed form. - Redis shares the WireGuard container’s network (
network_mode: service:wireguard), so it listens on the peer address directly. The LAN port is published on the WireGuard container, because a container in another container’s network can’t publish its own. - The NAS dials out to Fly (
PersistentKeepalive = 15), so nothing is port-forwarded on the router and a changing home IP doesn’t matter.
Configuration Go requires
Go’s startup checkValidateForRateLimiter (redis_startup_validation.go) needs:
maxmemory-policy noeviction, so rate-limit keys are never evicted under memory pressure;- persistence: AOF (
appendonly yes) or an RDBsaveschedule; CONFIG GETallowed (don’t rename or disableCONFIG, don’t use an ACL user without it).
[startup] Redis rate-limiter validation failed: v1 API is degraded. That line only reports the state at boot. Nothing keeps it: requests work as soon
as Redis is reachable.
The NAS compose project (password elided):
Outage behaviour
When Redis is unreachable (NAS down, home internet down, tunnel down):
No restart is needed anywhere: go-redis redials on demand, and Bun’s
BunRedisCache
reconnects at most every 5 s and waits for an in-flight connect instead of failing. /health
doesn’t depend on Redis (Bun skips the limiter for it; Go’s limiter falls back to memory),
so an outage doesn’t fail probes or restart pods and machines.
Each command from Fly costs about 13 ms. Bun’s app limiter makes about 3 commands per
request; Go’s IP limiters make one, with a 150 ms timeout.
Operations
Check that each deployment can reach Redis
Successfully connected to Redis and "redis":true; Bun prints
Using BunRedisCache for caching and rate limiting.
Change the Redis password
Every backend must get the new value, or the ones you miss lose Redis:- Update
--requirepassin the NAS compose project and redeploy it. - Reseal
REDIS_URLinbackend/bun/k8s/overlays/production/sealed-secret.yamlandbackend/go/k8s/overlays/production/sealed-secret.yaml(namespace-wide scope, the overlay’spub-cert.pem), commit, then syncbun-prod(manual) and letbackend-prodauto-sync. Restart the deployments so the pods read the new Secret. - Set
REDIS_URLon both Fly apps through stdin, so the password isn’t in your shell history:printf 'REDIS_URL=%s\n' "$URL" | fly secrets import -a <app>.
The WireGuard key or peer is lost
The private key exists only inwg0.conf on the NAS. To replace the peer:
DNS = line from the new file, copy it to the WireGuard config folder on the
NAS, and redeploy the project. The peer gets a new address: update REDIS_URL on both
Fly apps with it.
The NAS’s LAN IP changes
K8s loses Redis (Fly doesn’t, because the tunnel doesn’t depend on the LAN IP). Keep a DHCP reservation for192.168.0.142. If the IP must change, reseal REDIS_URL in both K8s
overlays.
Related Files
- ADR-0074: One Redis on the home NAS for Fly and K8s
- ADR-0073: client IP and shared rate-limit counts
- Go rate limiting
backend/bun/packages/infrastructure/cache/bun-redis-cache.ts: fail-fast client and reconnectbackend/go/internal/api/http/middleware/rate_limit_middleware.go: Redis-backed Echo store with memory fallbackbackend/go/internal/shared/infrastructure/cache/redis_startup_validation.go: startup checksbackend/go/k8s/overlays/production/kustomization.yaml,backend/bun/k8s/overlays/production/kustomization.yaml: overlay notes