Copilot HydraFusion Routes Your Code Through Multiple AI Models and Two of Three Benchmarks Got Worse
GitHub's new Copilot feature picks between AI models per task and claims a 67% cost reduction. We read the benchmark tables: it beat Opus 5 on TerminalBench, lost 1.5 points on DeepSWE, and the internal benchmark was a wash.
The WJS Desk
Sep 12, 2026 · updated 12 days ago · 5 min read
On September 4, GitHub announced Project HydraFusion, a research preview in Copilot CLI that routes coding tasks across multiple AI models instead of making you pick one. The headline number: 67% lower estimated cost than Claude Opus 5 on TerminalBench 2.1, with a 4.9 percentage point quality gain. We read the rest of the benchmark table, and the story is more complicated than the blog post's opening paragraph suggests.
HydraFusion is available now behind the /experimental flag in Copilot CLI on all plan tiers. It is free to try in the sense that you pay per token at each model's standard rate, which means you do not know the cost until the workflow finishes choosing models for you.
How it actually works
Instead of sending every request to one model, HydraFusion evaluates each task against four capability signals: reasoning depth, code generation complexity, debugging requirements, and tool use. Based on that evaluation it picks one of three execution patterns:
- Single: One model handles the whole task.
- Cascade: A cheaper model drafts an answer. A quality gate decides whether to accept it or escalate to a more expensive model.
- Critique: One model drafts, a second model from a different provider reviews, and the first model revises once.
The blog post does not name which models sit in the pool. It references Claude Opus 5 and GPT-5.6 Sol as comparison baselines and notes that new models can be added as they become available. The routing logic, model selection criteria, and quality gate thresholds are not published.
The benchmarks, all three of them
GitHub evaluated HydraFusion against a straight Claude Opus 5 baseline on three benchmarks. The blog post leads with TerminalBench 2.1, the one where HydraFusion won on both axes. Here is the full table:
| Benchmark | Cost vs. Opus 5 | Quality vs. Opus 5 |
|---|---|---|
| TerminalBench 2.1 | 67% lower | +4.9 points |
| DeepSWE | 36% lower | -1.5 points |
| CheckpointBench | 65% lower | -0.1 points |
On DeepSWE, which evaluates complex repository-level engineering tasks, HydraFusion came in 1.5 points behind Opus 5. On CheckpointBench, an internal benchmark derived from real Copilot agentic coding sessions, it was 0.1 points behind. The cost savings are real on all three. The quality improvement is real on one.
GitHub's own footnote is worth quoting: results are "specific to the evaluated benchmark revisions, workflow configurations, model pool, and pricing assumptions, with all models evaluated at the same medium reasoning level." That last clause matters. Running Opus 5 at medium reasoning and then comparing it to an orchestrated workflow that can escalate to higher reasoning is not an apples-to-apples comparison.
What it cannot do yet
The blog post is unusually direct about the current limitations, and we are crediting that.
HydraFusion is optimised for "first-turn, single-prompt coding tasks." Multi-turn sessions, the kind where you iterate on a solution across five or six messages, are not supported yet. The team says multi-turn performance is the next focus.
During execution, HydraFusion holds all intermediate drafts until it returns one final result. You do not see the cascade happening, the critique happening, or the revision happening. GitHub acknowledges this is "a real trade-off for developers" who want to watch progress, but argues it prevents showing unfinished work.
This means debugging a bad result is harder. When a single model gives you a wrong answer, you can adjust your prompt. When an orchestrated workflow gives you a wrong answer, you cannot tell which stage failed or which model was responsible.
Watch out: Pricing is per-token at each model's standard rate. Because you do not choose the model, you do not control which price tier runs. A cascade that escalates to a frontier model on a task you thought was simple could cost more than running the frontier model directly.
The design principles are the interesting part
Underneath the benchmark marketing, the five operating principles GitHub published suggest a team that has actually run multi-model workflows in production and learned from the failures:
- Complete accounting: Aggregate cost and usage across every workflow component, not just the final model.
- Bounded execution: Every stage gets an explicit timeout and cancellation behaviour.
- Isolated review: Critique steps run in tool-less contexts so the reviewer cannot accidentally execute code.
- Fail-safe application: If any stage fails validation, no changes are applied.
- Validated routing: Verify workflow definitions and model availability before execution begins.
Principle 3 is the one that caught our attention. Running the critique model without tool access means it can only reason about the code, not run it. That prevents the reviewer from becoming a second actor in your codebase, which is a real risk in multi-agent setups that most implementations do not guard against.
What the industry is actually reacting to
The coverage from DevOps.com, MarkTechPost, and Gigazine all led with the 67% number. None of them mentioned DeepSWE. Gigazine's headline specifically claimed a "4.9-point score improvement compared to Claude Opus 5" without noting that this was one benchmark out of three.
This is not unusual, but it is worth naming. The 67% figure will become the number people remember, the DeepSWE regression will not, and the next vendor comparison will cite the TerminalBench result as though it represents general performance. We have seen this cycle before with every major model release.
Whether this matters
The underlying idea is sound and probably inevitable. Model selection has been a manual, vibes-based decision for most developers, and automating it with a routing layer that can cascade and critique is a genuine architectural advance. The team behind it has clearly thought about failure modes, which is more than most multi-agent frameworks can claim.
But the current version is a research preview limited to single-turn prompts, with opaque routing, unpredictable pricing, and benchmark results that tell three different stories depending on which one you read. The honest summary is: promising architecture, early execution, marketed as though it is further along than it is.
If HydraFusion's routing logic were open, developers could verify the benchmark conditions themselves. Until then, 67% is a press release number, not a guarantee.
We will revisit this when multi-turn support ships and when someone outside GitHub publishes independent benchmark numbers. Until then, it is worth watching and not worth switching to.
