Running a cluster with a shared cache

One Faber instance is limited by its host’s memory and CPU. To scale out, run several instances behind a load balancer. Requests are independent, so any instance can serve any request. With a shared Redis, a step one instance cached is a hit on every other instance.

                    ┌──────────────┐
  clients ──HTTPS──▶│ load balancer│
                    └──────┬───────┘
             ┌─────────────┼─────────────┐
             ▼             ▼             ▼
         ┌───────┐     ┌───────┐     ┌───────┐
         │ faber │     │ faber │     │ faber │   each: own host (or node), privileged,
         └───┬───┘     └───┬───┘     └───┬───┘   local cache tier in memory
             └─────────────┼─────────────┘
                           ▼
                       ┌───────┐
                       │ redis │   shared cache tier: layers and blobs
                       └───────┘

Configure every instance

CACHE_ENABLED=true
CACHE_BACKEND=redis
CACHE_REDIS_URL=redis://:password@redis:6379/0
CACHE_NAMESPACE=judge-prod
FABER_TOOLCHAINS=c@12,cpp@12,python@3.12
FABER_IMAGE_ID=sha256:4f2a…        # the image digest, or your build ID

Instances share a layer only if they compute the same key. The key covers the architecture, CACHE_NAMESPACE, FABER_TOOLCHAINS, FABER_IMAGE_ID, SANDBOX_READONLY_PATHS, every effective limit and the default sandbox profile. So:

• Run the same image with the same configuration everywhere. An instance with a different limit, or a mixed x86_64/ARM64 fleet, still works; it just does not share layers with the others.
• Set FABER_IMAGE_ID from the image. Use the image digest or your CI build ID, so that after an upgrade the new image never replays layers built by different compilers. The Dockerfile builder sets FABER_TOOLCHAINS in the image for you.
• Use a namespace per environment. Staging and production can share one Redis without sharing layers, and DELETE /cache only clears its own namespace.

Run Redis

Redis is a cache here, not a database:

• No persistence is needed. If Redis restarts empty, the next requests just miss.
• Set maxmemory and an eviction policy (allkeys-lru), so a full Redis evicts old layers instead of refusing writes. Layers and blobs also expire after CACHE_TTL_SECS, refreshed on every hit.
• Require a password and keep it on a private network reachable only from Faber. It stores the contents of workspaces.
• One Redis is enough for most fleets. If it is down or slow, Faber treats every lookup as a miss, logs the error, counts it in the metrics and keeps serving.

Two replicas with Docker Compose

docker/prod/docker-compose.cluster.yaml runs two Faber replicas and one Redis on a single host, which is enough to see sharing work:

export API_KEY="$(openssl rand -hex 32)"
docker compose -f docker/prod/docker-compose.cluster.yaml up -d

See Docker Compose for what the file contains.

Several instances on one kernel

Each Faber process leases sandbox UIDs from its own range and does not know about other instances. Instances that share a kernel (replicas on one host, or pods on one Kubernetes node) must each get a separate SANDBOX_IDENTITY_BASE range, or concurrent requests on different instances can run under the same host UID. Prefer one instance per host or node.

Kubernetes

deploy/kubernetes/ has a Faber Deployment, a Redis Deployment and Service, and a Secret for the API key. Kubernetes deployment explains the privileged pod and the scheduling rules.

Check that sharing works

Send the same compile step to two different instances:

REQUEST='{"steps":[{"cmd":"gcc","args":["-o","main","main.c"],"files":{"main.c":"int main(void){return 0;}\n"},"cache":true}]}'

curl -s http://faber-a:3000/api/v1/execute -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" -d "$REQUEST" | jq .cache
# { "enabled": true, "hits": 0, "stored": 1 }

curl -s http://faber-b:3000/api/v1/execute -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" -d "$REQUEST" | jq .cache
# { "enabled": true, "hits": 1, "stored": 0 }

If the second instance reports hits: 0, compare GET /info on both: a different limit, profile or toolchain list changes the keys. Then check the Faber logs and the shared-tier error counter in /metrics for Redis errors.

Things that are per instance

• Task slots. Each instance admits up to its own MAX_CONCURRENCY. The load balancer should retry 503 capacity_exhausted on another instance, or clients should retry after Retry-After.
• The file store. Uploaded files live on the instance that received them. A request referencing a file_id must reach that instance, or the client must handle 404 file_not_found by re-uploading. See the file store.
• The local cache tier. Each instance keeps recently used layers in memory in front of Redis.

Go deeper

Scaling and deployment topology: what is and is not shared between instances, and why.