Repo3 days ago

jev-ultrafast Hit 6,831 Stars on Three Commits and Half Its Speedup Comes From Somewhere Else

browser-use published a browser agent, its raw measurement JSON, and per-run source hashes. We recomputed every claim from the raw data: all of them hold, and the 10x protocol-call reduction buys a 1.8x browser speedup, not 10x.

The WJS Desk

Sep 21, 2026 · 6 min read

Photo by Benni Fish on Pexels

browser-use/jev-ultrafast was created on 16 September 2026 at 21:30 UTC. Our snapshot runs recorded it gaining 4,152 stars in 1.4 days, and it sits at 6,831 stars and 433 forks as we write this. The description is three words: "i. am. speed."

Under that is one contributor, three commits, and 850 lines of core code. What made us actually clone it is not the star count. It is that the repo ships docs/full-speed-measurement.json, the raw per-run data behind every number in its writeup, including SHA-256 hashes of the source files each run executed. Almost nobody does this. So we recomputed the lot.

What it is

It is a browser agent that chooses instead of generating. Each observation produces a numbered table of on-page controls, and a single request to TypeSafe's Jev model returns an operation plus a target index. A small LLM is only invoked when the operation is TYPE_TEXT, and only to produce the string.

from jev_ultrafast import Agent

with Agent(
    "https://www.google.com/travel/flights?hl=en",
    "Find one-way flights from Zurich to London on September 20, 2026, "
    "for one adult in economy. Stop when matching flight options are visible.",
) as agent:
    for state in agent.run():
        print(state["elapsed_ms"], state["status"])

The operation set is eight items: CLICK, TYPE_TEXT, SELECT, SCROLL_UP, SCROLL_DOWN, WAIT, DONE, BLOCKED. Model output never becomes a selector, a coordinate, or executable JavaScript, which is a meaningful security property and one of the few design decisions here we would copy outright.

The whole thing is 743 lines of Python plus a 107-line snapshot.js, and it has two runtime dependencies: browser-harness==0.1.13 and httpx. The model instruction file, questions.py, is 26 lines. You can read the entire policy in a sitting, which is rare enough in this category to be a feature.

We could not run it, and here is exactly why

Being straight about this: we did not execute the agent. It needs a TYPESAFE_API_KEY and an OpenRouter key, both paid, and the project requires Python 3.12 where this machine has 3.9 and no uv. So we did not reproduce the 7.1-second flight search, and nothing below is our timing.

What we could do is check the published artifacts, which turned out to be the more interesting job. We ran node --check over both JavaScript files (clean), counted the source, read the loop, and then recomputed every published claim from the raw measurement JSON.

Every headline number checks out

The file contains six runs, three baseline and three candidate, tagged by arm and pair. Recomputing the medians ourselves:

Metric (median of 3)OriginalOptimizedTheir claim
Task time9,450 ms7,092 msMatches, 24.95% vs "25.0%"
Browser protocol calls1,092101Matches exactly
Model requests2217Matches exactly
Total time inside the browser4,583 ms2,605 msNot published
Runtime.evaluate cost per call9.82 ms30.02 msNot published
Median model latency187.0 ms182.5 msNot published
Verified passes3 of 33 of 3Matches

The bottom three rows are ours, and they are where the story is. The advertised win is "1,092 to 101 browser protocol calls", a 10.8x reduction. Total time spent in the browser fell from 4,583 ms to 2,605 ms, which is 1.8x. Those 828 DOM.resolveNode calls the new snapshot eliminated were numerous and cheap, costing around 1.2 seconds between them.

Meanwhile the calls that survived got much more expensive. Runtime.evaluate went from 158 calls at 9.82 ms each to 75 calls at 30.02 ms each, because a single atomic DOM snapshot does far more work per round trip. That is still a clear net win, and it is a different win from the one the number implies.

The other half of the saving is model round trips. Median requests fell from 22 to 17 while median per-request latency barely moved, 187.0 ms to 182.5 ms. Browser time accounts for 1,978 ms of improvement and model time for 977 ms, against 2,358 ms of total wall clock saved, so the two overlap on the timeline rather than adding up. Roughly speaking: half the gain is a better DOM reader and half is asking the model fewer times.

The source hashes, which is the test we expected to fail

Each run records SHA-256 hashes of the seven source files it executed. We hashed the files at HEAD and compared. Six of seven match byte for byte. The seventh, browser.py, does not.

That is disclosed. performance.md says that after the timed runs, "native-select interruption handling was tightened: uncertain mutation results stop instead of being treated as retryable stale reads." The diff bears this out: the committed version raises RuntimeError("Dropdown execution was interrupted; inspect before retrying.") where the older path retried.

The catch nobody will notice: because that change landed inside the same commit as everything else, the exact browser.py that produced the 7.092-second median is not in the repository at any commit. The disclosure is honest and the reasoning is sound, but the headline run is not reproducible from published source. One extra commit would have fixed it.

What it does not do

The README is unusually forthcoming here, and we are repeating it rather than discovering it:

  • No shadow roots, no iframes, no canvas, no file uploads, no pop-up tabs, no nested scrolling, no arbitrary keyboard widgets.
  • The DOM reader handles common HTML and ARIA controls, not the full accessible-name algorithm.
  • DONE is a model choice, not evidence. Outcome verification is your problem.
  • It runs against your existing Chrome profile, with your cookies and logins.
  • The comparison is three pairs on one task on one profile. Their own sign test gives p = 0.25, which they print.
  • There are no releases and no tags. uv sync against main is the install story.

Who made it, and can you rely on it

One contributor: Gregor Žunič, of browser-use. Three commits, the most recent of which adds a Cloud waitlist banner to the README. The repo links to a hosted product, and the README's first content block is a signup link, so read the velocity accordingly: this is a launch asset for a commercial offering, not a community project that happened to take off.

The ratio worth noting is 6,831 stars against 12 watchers and 43 open issues. Stars are cheap on a well-launched agent repo. Twelve people have subscribed to notifications.

The mitigating factor is size. At 850 lines with two dependencies and an MIT licence, vendoring the snapshot approach into your own agent is a realistic afternoon, and that is probably the right way to consume this.

Verdict

Do not adopt this as a dependency. One maintainer, no releases, and an explicit MVP scope that excludes iframes and shadow DOM rules out anything real. Do read snapshot.js and agent.py if you maintain a browser agent, because the atomic-snapshot idea and the "model picks an index, code owns the selector" boundary are both worth stealing today.

What would change our mind: a second maintainer, a tagged release, and one matched comparison on a task the authors did not choose. The methodology to do that properly is already in the repo, which is more than most projects can say.

Your turn

If you run a browser agent in anger, we want the number you would put in this table: what is your median wall clock on a real multi-field form, and how much of it is model latency versus browser round trips? Our read is that most teams have never split those two apart, and that the split matters more than the total. Tell us if your breakdown looks nothing like 50/50.

For the other half of the star-velocity question, we looked at a repo with a similar profile and a claim we could not verify at all: arc-task-gen hit 11,000 stars on 12 commits and one column we cannot check. Same shape, opposite outcome, and the difference is entirely whether the raw data shipped.

Share

6,831 stars in three days on three commits. browser-use shipped the raw JSON behind its speed claim, so we recomputed it. Every number holds, and 10x fewer browser calls buys 1.8x. #OpenSource #Python #AIAgents #BrowserAutomation

Never miss a ship

The best stuff that shipped this week, delivered every Thursday. Free, no spam. We read all the boring stuff so you get the fun parts.

Keep reading