onefrontieronefrontier.ai

docs · package

package/README.mdrendered from source at build

The launch kit

onefrontier launch kit

The open-source, self-contained kit for renting a GPU on vast.ai, serving a model on it, reaching it over a mutually-authenticated QUIC tunnel, and fronting every live cell with one OpenAI-compatible endpoint — by code alone, on your own keys. Nothing here is wired to anyone's infrastructure: no built-in hostname, port, or brand, and no personal credential anywhere. The kit is one box's worth of a mini-neocloud: the catalog says what to run and what it costs, the snipe rents it, the tunnel reaches it, the gateway fronts it, and the console legs (landing with their build issues) watch the market and the fleet.

What you need

  • Linux, python3 with pyyaml (pip install pyyaml), and a vast.ai key (https://vast.ai/console/account/) — for renting.
  • Docker, curl, and jq — for the gateway cell and its sync.
  • A Rust toolchain (https://rustup.rs) — for the qqt tunnel and the onefrontier catalog CLI, both built from this repo.

The story, in order

  1. Catalog — what to run, what it costs, what it measured. package/catalog/ is the knowledge base: models with their known-good engines, images and flags (models.yaml), instance classes with observed price bands (hardware.yaml), and measured performance rows (performance.yaml — numbers only from real captures, never estimated). Read it with the CLI:
    cd package/cli && cargo build --locked
./target/debug/onefrontier catalog models
./target/debug/onefrontier best RadixArk/GLM-5.3-NVFP4   # cheapest fitting hardware, ranked
./target/debug/onefrontier fit local-inference-lab/GLM-5.3-Flash-NVFP4 --hardware rtx-pro-6000-x4 --tp 4
./target/debug/onefrontier advise "run GLM-5.3-Flash under $8/h"   # the wizard: verdict + class + launch command
  2. Cell — the boot-ready image (optional). Where the catalog row carries a cell_image, the cell is pre-baked: the weights, the pinned engine, and the validated flags in one image, so a launch is a pull and a run instead of a 100+ GiB download. Build your own from any validated row with package/cell/build.sh <model> <class>; best names the published ref where one exists.
  3. Snipe — rent it. package/launch/snipe.py GETs the vast.ai bundles API once, filters from snipe-params.yaml (the single home for every filter number, geo rule, port law, and model fact), and the very next thing it does after seeing a qualifying offer is the create PUT. The fit gate runs before the paid clock starts. watch.py logs the market keyless or fires the snipe the instant a qualifying offer appears; verify.py waits out provisioning and proves the engine over ssh.
  4. qqt — reach it. The rented cell's engine binds box-side and stays unpublished; the private path is qqt, a QUIC tunnel with mutual TLS and ssh -L semantics, dialed straight at the box's public IP over UDP (no platform relay, no extra round trip per byte). It is its own project and its own repository now; the kit consumes the binary. The snipe's profiles publish the box-side UDP port and bake a guarded qqt server start into the launch; you mint the certs and arm the client-side pair the snipe prints.
  5. Hook — self-report. The always-on side's box leg: a launch contract's onstart calls the hook, and the cell's facts (model, quant, tp, engine, ports) go to your console's ingest endpoint as one JSON shape — every cell reports itself, nothing is hand-entered. This leg lands with the daemon/console build (the onefrontier hook subcommand is reserved for it).
  6. Gateway — one endpoint. package/gateway/ runs the sglang-model-gateway as a docker cell: every live cell answers behind one OpenAI-compatible door, the request's model selecting the right worker. The worker rows live in workers.yaml (yours), and sync.sh keeps them equal to the fleet — always: a cell that vanished drops within one tick, a new one appears the same way.
  7. Watch/serve — the price + performance console. The always-running side: the watch daemon records market prices and cell state into postgres, and serve reads the rows back as views — the fleet overview, per-class price history, the performance table. These legs land with the daemon/console build too (the onefrontier watch and serve subcommands are reserved for it); the gateway's sync already reads the console's cell rows today, so the stack composes when they land.

Quick start

# 1. Your vast.ai key
export VAST_API_KEY=...            # or write ~/.config/vastai/vast_api_key

# 2. What to run (the catalog answers; skip if you already know)
python3 -m pip install pyyaml
cd package/cli && cargo build --locked && cd ../..
package/cli/target/debug/onefrontier catalog models

# 3. Rehearse with no billing, then snipe a cell (one command, one create)
python3 package/launch/snipe.py --dry-run
python3 package/launch/snipe.py
python3 package/launch/verify.py          # once the cell is up: prove the engine

# 4. Reach it over the tunnel (the snipe printed the box's UDP port and
#    your client pair; full walkthrough at https://quickquic.ai/docs/runbook/)
cargo build --release --locked --manifest-path ~/ventures/quickquic/Cargo.toml
QQT=~/ventures/quickquic/target/release/qqt
$QQT gencert -o ./qqt-certs                                 # mint once
# ship qqt-certs/server/ + the binary to the box, then:
$QQT client \
  -l 127.0.0.1:18130:127.0.0.1:30000 \
  -r <box-public-ip>:<t-port> --certs-dir ./qqt-certs/client

# 5. One endpoint: front every live cell behind the gateway
export GATEWAY_API_KEY=...        # any secret; the door and the sync speak it
cd package/gateway
docker compose -f docker-compose.yaml up -d
./sync.sh                          # the registry boots empty; this fills it
curl http://127.0.0.1:8080/v1/models           # every live cell's served name
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" -H 'content-type: application/json' \
  -d '{"model":"GLM-5.3-Flash","messages":[{"role":"user","content":"hello"}]}'

# 6. Keep it always-live (pick one)
./sync.sh --loop 60                        # a long-running loop
# the systemd pair (examples/onefrontier-gateway-sync.{service,timer}): a
# timer fires the one-shot every minute
# the compose sidecar:
#   RECORDS_ROOT=~/.local/state/onefrontier/snipe \
#   docker compose -f docker-compose.yaml -f examples/docker-compose.sync.yaml up -d

The snipe prints the offer, instance id, price, and ssh coords the instant the create returns, and leaves the instance billing — no code path stops or destroys anything. You destroy by hand in the vast console. The gateway's sync.sh never stops or restarts anything either: membership changes land through the gateway's control API, never a bounce.

Configuration

  • package/launch/snipe-params.yaml is the only home for the snipe's knobs: the geo allow/exclude map, the box-name scheme, the port laws, the per-model launch facts, and the profiles. Override with SNIPARAMS=/path/to/params.yaml; CLI flags win for one invocation.
  • The gateway cell reads GATEWAY_HOST/GATEWAY_PORT (default 127.0.0.1:8080) and GATEWAY_API_KEY; sync.sh dials the same door via GATEWAY_BASE — keep the pair equal when you change one. The launch records it reads default to ${XDG_STATE_HOME:-~/.local/state}/onefrontier/snipe (override with RECORDS_ROOT).
  • Your rows live in package/gateway/workers.yaml — create it by copying package/gateway/workers.yaml.example; it is yours, never committed. It carries your hand rows in static: — any OpenAI-compatible endpoint the kit did not launch — and your console sources in console:; the sync-owned workers: block at the bottom is rewritten every tick. The header comments of package/gateway/sync.sh document every env knob.

Layout

package/
  catalog/  hardware.yaml, models.yaml, performance.yaml, README.md
  cell/     Dockerfile, serve-cell.sh, build.sh, README.md (the baked validated-cell image)
  cli/      the onefrontier binary (catalog/fit/perf/best/advise/launch; watch/serve/snapshot/hook)
  launch/   snipe.py, watch.py, verify.py, snipe-params.yaml, tests/
  gateway/  docker-compose.yaml, sync.sh, workers.yaml.example, examples/

Provenance

This kit is the productization of two former standalone repos — the vast.ai launch kit and the QUIC tunnel — folded into one home and sanitized: no personal credential, no built-in endpoint, no vendor lock-in. The gateway cell and the always-live sync descend from a production gateway whose worker registry is reconciled by an on-demand sync script; the productized form (probe the door, drop within a tick, never restart for membership) is the research record's design — the superneocloud dolphin series on the house research shelf: the deployability lab's build order, the launch-pain map, and the vast.ai market-sentiment census. The kit's case law (the fit-gate floor, the market-flicker walk, the UDP-publish-is-not-UDP-path proof) is the same record's paid lessons, encoded.

read the launch kit all docs