Skip to content
Faber Faber

Scaling and deployment topology

Faber keeps almost all of its state in the process: task slots, uploaded files, rate limit buckets, the local cache tier. Scaling out therefore means running independent instances side by side, and the only thing they can share is the step cache, through Redis. Docker, Compose and Kubernetes have the configuration; this page shows the shapes.

One instance

Section titled “One instance”
  • Faber and the kernel (trusted)
  • Submitted code (untrusted)
  • Outside Faber
  • Starts or calls
Everything in the Faber box lives in one process and is lost when it restarts, except files in a filesystem-backed store.

Faber speaks plain HTTP/1.1 only. HTTP/2 detection would mean waiting for a client’s preface with no timeout, so it is not offered; a client must send its headers within HEADER_READ_TIMEOUT_MS and its body within REQUEST_BODY_TIMEOUT_MS. Bind the port to localhost or a private network and let the proxy face clients. GET /api/v1/health needs no key and answers 503 once the instance is shutting down, so the proxy’s health check takes it out of rotation by itself.

One instance runs at most MAX_CONCURRENCY tasks at once and refuses the rest with 503; sizing turns that into memory and CPU.

Several instances and Redis

Section titled “Several instances and Redis”
  • Faber and the kernel (trusted)
  • Outside Faber
  • Starts or calls
  • Carries data
Any instance can serve any request that does not reference an uploaded file. With the cache on, a step one instance computed is a hit on all of them.

What is shared and what is not

Section titled “What is shared and what is not”
Shared between instancesKept per instance
Step cache layers and blobsIn Redis, with CACHE_BACKEND=redis and the same CACHE_NAMESPACEThe local tier in front of it
Computing a missing layer onceA lock in Redis across instancesA wait channel within the process
Clearing the cacheDELETE /cache on any instance clears Redis and bumps a generation; the others drop their local tier at their next lookup
Task slotsMAX_CONCURRENCY each. A request refused with 503 capacity_exhausted ran nothing and can go to another instance
Uploaded filesThe store of the instance that received the upload. Route a client to one instance, or re-upload on 404 file_not_found
Rate limitsOne token bucket per key per instance: three instances allow a key three times RATE_LIMIT_PER_MINUTE
Sandbox identitiesLeased per process. Instances on one kernel need disjoint SANDBOX_IDENTITY_BASE ranges
Logs, metrics, request IDsPer process; a request ID is the process ID and a counter

What must match to share layers

Section titled “What must match to share layers”

Instances share a layer only if they compute the same key, and the first key of every chain starts from a digest of the service: architecture, CACHE_NAMESPACE, FABER_TOOLCHAINS, FABER_IMAGE_ID, the read-only paths, the limits, the default sandbox profile and the sandbox policy version of the Faber build. Run the same image with the same environment everywhere. A mixed fleet (two architectures, a canary with different limits) still works; it just forms separate caches inside the same Redis.

Draining an instance

Section titled “Draining an instance”

SIGTERM kills every running sandbox (those requests get 503 shutting_down), stops the listener, and gives in-flight responses SHUTDOWN_TIMEOUT_MS to drain. To replace an instance without failed requests, take it out of the load balancer first, wait for faber_execution_slots_in_use to reach 0, then stop it.

Several instances on one kernel

Section titled “Several instances on one kernel”

Prefer one instance per host or node: the Faber container is root-equivalent on its host, and instances on one kernel compete for the same memory. If you do run several on one host (the cluster Compose file does, to demonstrate the shared cache):

  • each needs its own SANDBOX_IDENTITY_BASE and SANDBOX_IDENTITY_COUNT range, or two of them can hand the same host UID to concurrent requests;
  • each manages only its own container’s cgroup, so their faber cgroups do not overlap, but their sum must fit the host’s memory.

Where this lives in the code

Section titled “Where this lives in the code”