Reliquary is Subnet 81: a decentralized reinforcement-learning training market. Miners compete to find the prompts at a model's learning frontier; ranked candidates are proven top-down, every credited group must pass GRAIL, and healthy selected batches can train in Forge. The result of putting prompt selection on a market, at identical compute:
What we built
Miners generate rollouts on problems with verifiable rewards; the validator verifies and selects training-eligible groups; Forge records train-or-hold, optimizer, checkpoint, and evaluation evidence as separate states. Ranked candidates are proven top-down; every credited group must pass GRAIL verification, which lets the validator confirm the generations came from the declared model weights. That is the infrastructure — the contribution is in how prompts are selected for training.
Prompt selection is the dominant lever
DAPO (Xu et al., 2025) showed that of its four techniques, Dynamic Sampling — discarding rollout groups with zero reward variance — was the largest single gain, exceeding the other three combined. The reason is structural: in group-relative policy optimization the gradient signal is proportional to the reward variance across a group. When every completion succeeds, or every one fails, the advantages collapse to zero. Only prompts at the frontier carry signal.
Only 1–15% of prompts occupy the learning zone at any checkpoint. A reactive filter discards the rest after paying to generate them.
From reactive filter to competitive market
DAPO reacts: generate a group, measure variance, discard below threshold. As the policy improves the frontier shrinks, the rejection rate climbs, and a growing share of inference is spent on groups that are thrown away. Reliquary replaces the filter with a market — each window needs N rollout groups; miners independently select a prompt, generate a group, and race to submit. The first N valid, distinct submissions seal the window.
can only react — and waste grows as the frontier narrows.
anticipation gets more valuable as the zone contracts.
Two pressures compound. Speed — slots are scarce and awarded in arrival order. Selection accuracy — with only 1–15% in-zone, a miner picking at random wastes 85–99% of its budget; a coarse difficulty estimator lifting hit-rate 5%→30% cuts wasted inference 6×. The supervision is free: every submission is a labeled (prompt, checkpoint, accepted?) point, so the incentive gradient bends toward building difficulty models.
The controlled experiment
To isolate the market, everything is held fixed except the source of rollouts: same base model (Qwen3-4B-Instruct), same training function (imported from the same commit), same hyperparameters, dataset, and step count (300). Reliquary's rollouts come from competing miners selecting at the frontier; the baseline's come from one trainer selecting uniformly at random. Both checkpoints were scored on the same held-out math set with the same seed — a paired comparison.
| base | vanilla GRPO | Reliquary | |
|---|---|---|---|
| pass@1 | 0.330 | 0.470 | 0.610 |
| pass@4 | 0.700 | 0.730 | 0.730 |
| truncation | 80.5% | 73.2% | 62.5% |
The Reliquary arm gains +14 pp pass@1 at identical step count (≈2.8 SE from zero, p ≈ 0.01). pass@4 is unchanged (0.73 vs 0.73): training sharpens the single-attempt answer rather than expanding the solvable set. Truncation falls to 62.5% — the market-trained policy converges on answers more efficiently.
Why the efficiency hypothesis needs another run
FIG.05 shows the mechanism that motivates an efficiency hypothesis: as the in-zone rate falls, a reactive filter spends a larger share of inference on groups it later discards. This run did not directly meter end-to-end compute for a centralized DAPO-style filter, so it cannot establish a universal efficiency multiplier.
The next controlled comparison must hold model, tasks, target quality, hardware envelope, and seeds fixed, then record useful groups and wall-clock compute for both selection systems. Until those repetitions exist, +14 pp pass@1 is a single controlled result—not a production guarantee and not proof that an efficiency gap necessarily widens.
Vision
The architecture isn't tied to a model or domain — it's a general-purpose RL inference layer. The goal is a product: a client brings a model and a set of environments, and Reliquary handles the RL inference — rollout generation, prompt selection, and verification — returning an optimized training signal without the client building inference infrastructure.
a model + environments
an optimized training signal
M=8 · binary reward geometry
active group · k=2
Discovery peak
Reinforce two rare correct trajectories
2 / 6
correct / wrong
- p
- 0.250
- σ
- 0.433
- 1 − p
- 0.750
- discovery pull
- 0.325
- A correct
- +1.732
- A wrong
- −0.577
where training signal lives
The steady gate keeps k=2…6 where σ ≥ 0.43.
correct rollouts k · eight attempts per group
| k | correct / wrong | p | sigma | 1 − p | discovery pullσ(1−p) | A correct | A wrong | training role |
|---|---|---|---|---|---|---|---|---|
| 0 / 8 | 0.000 | 0.000 | 1.000 | 0.000 | — | — | No correct path Unanimous failure, no contrast | |
| 1 / 7 | 0.125 | 0.331 | 0.875 | 0.289 | +2.646 | −0.378 | Possible discovery Strong positive, but one success may be lucky | |
| 2 / 6 | 0.250 | 0.433 | 0.750 | 0.325 | +1.732 | −0.577 | Discovery peak Reinforce two rare correct trajectories | |
| 3 / 5 | 0.375 | 0.484 | 0.625 | 0.303 | +1.291 | −0.775 | Strong exploration Useful correct paths remain uncommon | |
| 4 / 4 | 0.500 | 0.500 | 0.500 | 0.250 | +1.000 | −1.000 | Balanced frontier Equal positive and negative concentration | |
| 5 / 3 | 0.625 | 0.484 | 0.375 | 0.182 | +0.775 | −1.291 | Early refinement Correct behavior is already common | |
| 6 / 2 | 0.750 | 0.433 | 0.250 | 0.108 | +0.577 | −1.732 | Refinement mirror Suppress two rare failure trajectories | |
| 7 / 1 | 0.875 | 0.331 | 0.125 | 0.041 | +0.378 | −2.646 | Nearly mastered Mostly cleanup of one rare failure | |
| 8 / 0 | 1.000 | 0.000 | 0.000 | 0.000 | — | — | Already mastered Unanimous success, no contrast |
p = k/8 · σ = √p(1−p) · discovery pull = σ(1−p) · A = (r−p)/σ · displayed to 3 decimals
The pull is an asymmetric teaching-signal lens, not protocol variance. Advantages are the idealized ε-free values; an em dash means σ=0 and no within-group contrast.
The gap between naive sampling and informed selection by experienced miners is where both the network and its clients capture value.