Reliquary: a market for the learning frontier

Reliquary is Subnet 81: a decentralized reinforcement-learning training market. Miners compete to find the prompts at a model's learning frontier; ranked candidates are proven top-down, every credited group must pass GRAIL, and healthy selected batches can train in Forge. The result of putting prompt selection on a market, at identical compute:

What we built

Miners generate rollouts on problems with verifiable rewards; the validator verifies and selects training-eligible groups; Forge records train-or-hold, optimizer, checkpoint, and evaluation evidence as separate states. Ranked candidates are proven top-down; every credited group must pass GRAIL verification, which lets the validator confirm the generations came from the declared model weights. That is the infrastructure — the contribution is in how prompts are selected for training.

Prompt selection is the dominant lever

DAPO (Xu et al., 2025) showed that of its four techniques, Dynamic Sampling — discarding rollout groups with zero reward variance — was the largest single gain, exceeding the other three combined. The reason is structural: in group-relative policy optimization the gradient signal is proportional to the reward variance across a group. When every completion succeeds, or every one fails, the advantages collapse to zero. Only prompts at the frontier carry signal.

Only 1–15% of prompts occupy the learning zone at any checkpoint. A reactive filter discards the rest after paying to generate them.

From reactive filter to competitive market

DAPO reacts: generate a group, measure variance, discard below threshold. As the policy improves the frontier shrinks, the rejection rate climbs, and a growing share of inference is spent on groups that are thrown away. Reliquary replaces the filter with a market — each window needs N rollout groups; miners independently select a prompt, generate a group, and race to submit. The first N valid, distinct submissions seal the window.

Two pressures compound. Speed — slots are scarce and awarded in arrival order. Selection accuracy — with only 1–15% in-zone, a miner picking at random wastes 85–99% of its budget; a coarse difficulty estimator lifting hit-rate 5%→30% cuts wasted inference 6×. The supervision is free: every submission is a labeled (prompt, checkpoint, accepted?) point, so the incentive gradient bends toward building difficulty models.

The controlled experiment

To isolate the market, everything is held fixed except the source of rollouts: same base model (Qwen3-4B-Instruct), same training function (imported from the same commit), same hyperparameters, dataset, and step count (300). Reliquary's rollouts come from competing miners selecting at the frontier; the baseline's come from one trainer selecting uniformly at random. Both checkpoints were scored on the same held-out math set with the same seed — a paired comparison.

The Reliquary arm gains +14 pp pass@1 at identical step count (≈2.8 SE from zero, p ≈ 0.01). pass@4 is unchanged (0.73 vs 0.73): training sharpens the single-attempt answer rather than expanding the solvable set. Truncation falls to 62.5% — the market-trained policy converges on answers more efficiently.

Why the efficiency hypothesis needs another run

FIG.05 shows the mechanism that motivates an efficiency hypothesis: as the in-zone rate falls, a reactive filter spends a larger share of inference on groups it later discards. This run did not directly meter end-to-end compute for a centralized DAPO-style filter, so it cannot establish a universal efficiency multiplier.

The next controlled comparison must hold model, tasks, target quality, hardware envelope, and seeds fixed, then record useful groups and wall-clock compute for both selection systems. Until those repetitions exist, +14 pp pass@1 is a single controlled result—not a production guarantee and not proof that an efficiency gap necessarily widens.

Vision

The architecture isn't tied to a model or domain — it's a general-purpose RL inference layer. The goal is a product: a client brings a model and a set of environments, and Reliquary handles the RL inference — rollout generation, prompt selection, and verification — returning an optimized training signal without the client building inference infrastructure.

The gap between naive sampling and informed selection by experienced miners is where both the network and its clients capture value.