GitHub – slee-persis/GVS5H: GVS5H: Five Qwen3.8-27B Fashions Match Claude Fable 5 on ResideCodeBench Onerous | Fable 5 Degree Coding for a Fifth the Worth

Opus 5 ran on an earlier model of the scaffold, in a single go served via OpenRouter quite than 5 on a pinned backend.

Abstract. Frontier coding efficiency is usually purchased with bigger proprietary fashions at excessive price. We introduce ledger-based zero-shot self-orchestration, a training-free methodology by which contemporary situations of 1 mannequin decompose issues and coordinate via a shared filesystem holding a plan, notes and present answer. Across eleven open and closed-weight fashions on the 100 newest laborious ResideCodeBench issues, the tactic yields good points of as much as 25.6 share factors on pinned backends and provides two routes to frontier-level accuracy. Orchestrated Qwen3.8 Flash Next reaches 93.0% go@1 in opposition to Fable 5’s 90.4% at 9% of the price, and domestically served, open-weight Qwen3.8-27B rises from 66.8% to 92.4%. Gains aren’t common: some fashions are unchanged or worse. Transcript evaluation attributes the acquire to decomposition and protracted context. Inference-time group can strategy frontier coding accuracy at a fraction of the price, or barely exceed it on self-hostable weights.

— the paper

Needs uv and an API key for the mannequin you wish to take a look at.

cd codebase/v2-current
export OPENAI_API_KEY=...

LCB_RELEASE=release_v6 
ESCALATION_CLOUD_MAX_TOKENS=128000 
ESCALATION_CLOUD_TIMEOUT=7200 
MULTIAGENT_MODEL=openai:gpt-5.6-terra 
uv run --no-project --python 3.12 --with 'datasets<4' --with numpy --with anthropic 
  python escalation/run_bench.py --engine multiagent --only lcb --lcb 100 --parallel 8

• --engine multiagent runs the supervisor; --engine single is the one-call baseline.
• Other fashions: anthropic:, dashscope:, openrouter:, every with its personal *_API_KEY.
• The go@1 rating prints on the finish. Results are written to runs/outcomes.json, workspaces to runs/ws/.

Code is below the MIT License. The paper, figures and run information are below
CC BY 4.0. The ResideCodeBench fork, the benchmark drawback statements
and the LaTeX template recordsdata preserve their very own licenses. See NOTICE.md for
which license covers which path.