Cold Start to First Response in 1.3 Seconds

Waystones Cloud deploys ephemeral, scale-to-zero microVMs — one per customer deployment — running pygeoapi against DuckDB querying remote GeoParquet over HTTP. The economics are good. The cold start behavior, initially, was not.

On a 1-core microVM, a cold start would peg CPU at 100% for 40 seconds and sometimes return HTTP 500 to the first user. This is what we found, and what we did about it.

The Thundering Herd and the 0-Byte Trap

Three processes compete at boot on a single core:

Gunicorn spawning workers
pygeoapi CLI importing GDAL/Fiona and generating the OpenAPI document
A warmup script pinging collections to cache GeoParquet footers

The CPU contention was bad but survivable. The race condition was not.

Our boot script cached the OpenAPI YAML by opening the file in write mode — which on POSIX systems truncates it to 0 bytes immediately, before the subprocess has written anything. If Gunicorn read the file at that exact moment, it loaded None into memory. The first request to /openapi would return HTTP 500 with no useful error.

Fix: atomic write. The generator writes to /tmp/temp_openapi.yml, then shutil.move() swaps it into place. On Linux, a rename within the same filesystem is atomic. Gunicorn sees either the old file or the complete new file — never 0 bytes.

That eliminated the crashes. The 40-second spike remained.

The Configuration Paradox and the 200MB Trap

The obvious fix was to pre-generate the OpenAPI document in our background data-processing worker. Two problems:

Image bloat. Our worker is a lightweight Alpine container. Installing pygeoapi pulls in GDAL, Fiona, and Rasterio — over 200MB of spatial C-libraries for the sole purpose of generating a YAML file.
The paradox itself. The worker runs before the user deploys. It doesn't know the final hostname (wsc-castles.fly.dev) or the routing configuration, because those don't exist yet.

The real question was why we were using Python to figure out an API schema we already knew completely. pygeoapi follows OGC API Features, which means the routing is deterministic: /collections/{id}/items. With a known dataset model, the OpenAPI structure is just boilerplate.

Our data processing worker already produces a model.json describing each dataset — collections, fields, CRS, and metadata. That model is the source of truth. We wrote a small TypeScript function in the Waystones Cloud backend that reads model.json and emits standard OGC API Features OpenAPI JSON. It runs in ~2ms at deploy time and uploads the result to Cloudflare R2. On boot, the microVM downloads the pre-baked file in ~100ms and skips Python generation entirely.

The generator logic isn't inherently tied to the hosted platform — it reads a well-defined JSON structure. Porting it back into the open-source worker, or shipping it as a standalone script alongside the Docker image, is a natural next step for users who want to pre-bake their own OpenAPI documents.

CPU spike at boot: eliminated.

The Universal Contract

Waystones is also an open-source project. Users running the stack locally via Docker Compose don't have a pre-baked file waiting on R2. The image needs to work cleanly across all deployment contexts.

We implemented a two-path boot script:

Fast path: Pre-baked file exists on R2 → download → Gunicorn boots with full documentation. Available to any deployment where the hostname is known when the data is processed — Waystones Cloud handles this automatically, but any container host deployment can use the same path.
Slow path (fallback): No pre-baked file → boot with a minimal stub YAML → heavy Python generation runs in a background thread. The default for Docker Compose and any deployment where the hostname isn't set at processing time.

The distinction isn't "hosted vs. open source" — it's "hostname is known at deploy time vs. unknown until runtime." Once the OpenAPI generator is available in the OSS worker, any deployment with a fixed hostname can use the fast path regardless of where it runs.

The remaining problem: how do you reload Gunicorn with fresh documentation once the background generator finishes, without restarting the container or dropping connections?

Because our boot.sh ends with exec gunicorn, Gunicorn inherits PID 1. Once the background generator completes, it calls os.kill(1, signal.SIGHUP). Gunicorn responds by gracefully cycling its workers against the new configuration, with no dropped requests.

The Hairpin and the Deadlock

With OpenAPI solved, the warmup script — which fires dummy queries to prime DuckDB's Parquet footer cache — introduced two new failure modes.

The hairpin: The script originally called the public URL. The request left the VM, traversed the load balancers, and came back. Slow and fragile.

The deadlock: We fixed the hairpin by targeting http://127.0.0.1:5001. With a single Gunicorn worker, the warmup request occupies that worker while waiting for a response from... that same worker. The server deadlocks itself.

Fix: hardcode the warmup script to 127.0.0.1:5001 (internal, bypassing load balancers), and configure Gunicorn with --worker-class gthread --workers 2. The server can always talk to itself.

nice -n 19

Deadlock solved. New problem: the warmup script's footer downloads competed with actual user requests on the same single core.

Adding sleep 60 just relocates the spike to when the user is actively exploring the map.

(sleep 5 && nice -n 19 python3 /warmup.py) &

nice -n 19 is the lowest scheduling priority on Linux. The kernel runs the warmup process on leftover CPU cycles and yields immediately whenever Gunicorn needs the core. The 5-second delay gives the server a breath after boot before cache work begins.

The Frontend Gap

The backend was now fast enough to create a new problem: Waystones Cloud was providing a clickable link at T+2s. Users clicking immediately would hit the server inside its 5-second grace period.

We didn't touch the backend. A synthetic progress sequence in the frontend adds a buffer:

0s: Building data...
2s: Provisioning server...
4s: Warming up map engine...

By the time the button becomes clickable, the grace period has elapsed and DuckDB is running footer downloads in the background on idle cycles.

Result

16:37:53  Machine started in 1.374s
16:38:00  [warmup] pygeoapi ready — waiting 5.0s...
16:38:00  "GET / HTTP/1.1" 200 2340
16:38:01  [cache_openapi] Downloaded openapi.yml from R2.
16:38:05  [warmup] Warming 2 collection(s) × 2 workers (paced)...

The problems were: a POSIX file truncation race, a language-level architectural mismatch, a network hairpin, a self-deadlocking HTTP client, and a scheduling conflict — all competing for a single CPU core.

None of them required more hardware.