Ornith-1.5 matches Opus 4.8 on coding

Ornith-1.5 lands, matches Opus 4.8 on coding

A US open-source lab called Ornith AI released Ornith-1.5 this week as a family of open-weight models in three sizes — 9B, 35B and 397B — claiming state-of-the-art open-source performance on coding and agentic work. The flagship 397B lands at 86.1 on Terminal-Bench 2.1, a coding-agent benchmark that scores a model on real shell tasks, and 86.0 on SWE-bench Verified, a standard software-engineering test — numbers that put it on par with Anthropic’s Claude Opus 4.8 (85.0 and 85.8). Weights are already on Hugging Face in multiple precisions.

Where it actually ranks

Read past the headline and the picture is more interesting. The 397B’s 56.0 on DeepSWE — a coding benchmark weighted towards harder, real-world repos — trails Claude Opus 4.8 (59.0) and Kimi K3 (67.5), so it isn’t the absolute frontier on the test that matters most right now. But against named open-source rivals of similar scale, it beats GLM-5.2 (46.2) and DeepSeek V4 Flash 0731 (54.4) cleanly.

On Terminal-Bench 2.1, the 397B lands ahead of Claude Opus 4.8 (85.0) and within touching distance of Kimi K3 (88.3). On SWE-bench Pro, the harder version, it scores 65.1 — better than GLM-5.2’s 62.1, below Opus 4.8’s 68.0. On HLE with tools, a reasoning test, it lands at 56.1, almost matching Opus 4.8’s 57.9 and Kimi K3’s 56.0. On Toolathlon-Verified, an agentic tool-use test, the 397B reaches 71.2 against Opus 4.8’s 76.2 — close, but not a tie.

86.1Ornith-1.5-397B on Terminal-Bench 2.1, ahead of Claude Opus 4.8’s 85.0 and well clear of the open-source field

On Frontier-Bench v0.1 — the genuinely hard reasoning test — Ornith-1.5-397B lands at 13.5, against Opus 4.8’s 21.1 and Kimi K3’s 23.0.

The training loop is the story worth tracking

The methodology is the more interesting part. Ornith-1.5 extends an earlier self-scaffolding framework into a full self-improvement loop: the model proposes new tasks, builds its own scaffold — the instructions, tools and decomposition strategy used to approach a problem — then writes solutions that get graded automatically. Better solutions earn more reward, which feeds back into how the model picks and structures its next tasks.

Difficulty is measured against the model’s own success rate — the target is 20% success, which keeps tasks hard but learnable. As the model improves, the tasks automatically get harder. A novelty term stops the loop grinding on slight variants of the same problem.

In plain terms: the model teaches itself harder and harder problems, with no human-curated task list driving the curriculum. If this scales, the cost of frontier training shifts from data labelling to compute — a different bottleneck, and a different moat, than the rest of the field is racing on.

The small version is the sleeper

The 9B lands at 47.0 on Terminal-Bench 2.1 and 70.6 on SWE-bench Verified — numbers that match or beat the much larger Gemma 4-31B (42.1 and 52.0) and Qwen 3.6-35B-A3B (52.5 and 73.4). A compressed version of the 9B runs on iPhone and Android, the lab says. Weights ship in several precisions and formats — full details in the box below.

The 35B-A3B activates only 3B parameters per token yet beats Meta’s Muse Glimmer 30B by wide margins on agentic coding (68.5 vs 51.7 on Terminal-Bench 2.1; 79.0 vs 76.0 on SWE-Bench Verified). For anyone running local — see our earlier Ornith-1.0 coverage — this is a credible upgrade path on a single 24GB card.

What to watch

Three shifts worth flagging:

• Self-improvement is becoming a training paradigm. If Ornith’s loop scales, the cost of frontier training shifts from human-curated tasks to compute — a different bottleneck entirely.
• The hard-reasoning ceiling still sits with Anthropic and Moonshot. Ornith-1.5 ties Opus 4.8 on agentic coding but trails both Opus 4.8 and Kimi K3 on Frontier-Bench v0.1 by a wide margin; the open-weights tier has closed the agentic gap, not the reasoning one.
• Useful coding on a phone is here. The compressed 9B variant, with SWE-Bench Verified above 70, makes on-device coding a real option rather than a demo.

This isn’t a release a UK small team will deploy today — the 397B needs a rack, and the 9B, while capable, is competing in a crowded small-model field. The interest is where this points: open weights at the coding frontier, and a training method that doesn’t need a human task list to keep improving.