NVIDIA Announced Rust GPU Kernels With No Benchmarks, So We Went Looking in the Repo

On 8 September NVIDIA published a post announcing two separate ways to write CUDA kernels in Rust. It ran to several thousand words and contained exactly zero performance numbers. For a company whose entire pitch is throughput, that is a strange omission, so we went to look for the numbers ourselves.

They exist. They are sitting in the cutile-rs repository, committed as CSV files under a directory called paper, and they answer the only question that matters about safe GPU programming: what does the safety cost you. The answer, in NVIDIA's own measurements on a B200, is somewhere between nothing and 0.4%.

What actually shipped, and what state it is in

Two projects, both under the NVlabs org, both Apache-2.0, and they are not at the same maturity at all.

cutile-rs is the tile track. You write a kernel inside a #[cutile::module] block, the macro captures the Rust AST into your host binary, and the first time the kernel is needed it JIT compiles through CUDA Tile IR into a cubin. It runs on stable Rust 1.89 or newer, no nightly, and cargo add cutile is genuinely all you need as of 0.3.1. The README describes it as a "research project" and says you should "expect bugs, incomplete features, and API breakage".

cuda-oxide is the SIMT track, and it is the more ambitious of the two: a custom rustc codegen backend that routes ordinary Rust through MIR, the Pliron IR framework and LLVM IR down to PTX. No DSL, no macro, just Rust functions that become kernels. NVIDIA calls it "early alpha". It needs a pinned nightly toolchain, specifically nightly-2026-04-03, plus clang and libclang headers.

Both want Linux and a GPU at compute capability 8.0 or later. cuTile wants CUDA 13.3 for FP4 packing and block-scaled MMA. Anything at sm_70 or sm_75 is explicitly unsupported, so a lot of older Turing hardware is out.

The numbers that were not in the announcement

Under cutile-benchmarks/paper/ the repo carries the artifact bundle for a paper, including committed result CSVs from an NVIDIA DGX B200 running CUDA 13.2 on driver 595.58.03. The section 5.1 experiment tests the claim that safe, bounded, disjoint partition indices cost nothing against raw-pointer persistent kernels. Here is gemm_rust_persistent_safe_results.csv against gemm_rust_persistent_raw_results.csv, square GEMM, TFLOPS:

Matrix size	Raw pointer	Safe partitions	Safe as % of raw
1024	208.54	210.08	100.74%
2048	954.86	958.41	100.37%
4096	1645.78	1638.69	99.57%
8192	2074.08	2068.88	99.75%
16384	2130.74	2127.31	99.84%
32768	2072.37	2121.46	102.37%

The spread is under half a percent in either direction, which is close enough to run-to-run noise that the honest reading is "no measurable difference". That is a real result and it is the whole argument for the project.

The same directory has gemm_cublas_fraction.csv, which is the number people will actually care about: how close does a safe Rust kernel get to cuBLAS. At 1024 and 2048 it edges ahead at 100.3% and 103.7%. From 4096 up it lands between 94.1% and 97.0%. So cuBLAS still wins at the sizes that matter, by three to six percent, and NVIDIA did not say so in the blog post.

The safety is free. The gap to cuBLAS is not, and it is the number the announcement left out.

The caveat is in their own metadata file

The experiment README says the microbenchmarks "lock SM clocks for reproducibility" and MACHINE.md records an 1830 MHz clock target. Then metadata.csv, sitting in the same committed results directory, says this:

gpu_clock_mhz,1830
clock_lock_status,failed

We are not claiming that invalidates the result. The measured spread is tiny and consistent across six sizes, which is not what unstable clocks usually look like. But a benchmark that documents clock locking as its reproducibility mechanism, and then ships a metadata file saying the lock failed, is worth knowing about before you quote the figure at anyone.

The end-to-end numbers are more modest than the microbenchmarks

Section 5.3 puts Grout, the Rust inference engine built on cuTile, against vLLM and SGLang on Qwen3-32B on a single B200. Median generation throughput, 18-token prompt, ten runs per point:

Generated tokens	Grout v2	vLLM	SGLang
36	80.9	78.3	75.5
512	81.6	78.8	77.7
2048	81.2	78.5	77.4
8192	80.1	77.5	76.5

Three to five percent over vLLM. Real, repeatable, and much less dramatic than the GEMM story. It is also worth reading what the benchmark is: an 18-token prompt and a single request stream. That measures single-stream decode latency, which is the case where vLLM's continuous batching contributes least. Nobody runs a 32B model that way in production, and this bundle does not contain the concurrency sweep that would settle it.

Separately, the 0.3.1 changelog claims cache-hit kernel launches got roughly 2.4x faster on the host, 3.9 microseconds down to 1.6 per launch on a Ryzen 9 9950X with an RTX 5090, by cutting allocations per launch from about 108 to single digits.

The adoption claim is true, with an asterisk

The announcement says cuTile is "already used outside NVIDIA" in HuggingFace's Grout inference engine and in mistral.rs. We checked both Cargo.toml files. It is there, and in both cases it is an optional dependency behind a feature flag:

# candle-core/Cargo.toml
cutile = { workspace = true, optional = true }
cutile = ["cuda", "dep:cutile"]

Same shape in mistralrs-quant, where it gates a single nvfp4_bench example. So it is wired in and buildable, not switched on by default for everyone who installs those crates. The download figures agree: cutile on crates.io has 7,026 downloads across its entire life since March, and 691 on the current 0.3.1. That is an early-adopter number, not a production one.

Repo activity tells the same story about which track has people on it. cuda-oxide has 3,434 stars and 1,154 commits from 48 contributors since April, though 48% of those commits come from one person. cutile-rs has 949 stars and 176 commits from 19 contributors, with 69% from a single author. Neither looks like a team of twenty.

What Hacker News actually argued about

The thread ran to 313 comments and 771 points, and a surprising share of it was not about GPUs at all. Commenter claiir flagged the line "The launch is checked rather than trusted" and wrote "Damn even Nvidia is putting out fully Claude-written articles." winwang put it as "it reads like Claude instead of what Nvidia posts have generally been like in the past", and nicebyte drew the harsher conclusion: "what this article tells me is that no one at Nvidia actually cares about this project whatsoever." Given that the post omitted benchmarks its own repo contains, that reading is not unreasonable.

The sharpest technical objection came from rvz, who pointed out that a pre-1.0 release on a pinned nightly adds a layer to every future debugging session: "now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile". jacobgorm made the lock-in argument, that once CUDA is in a codebase you end up "tied to a single vendor or an #ifdef hell, probably both", and argued kernels belong in separate files the way Metal and D3D12 do it. And jauntywundrkind supplied the context the announcement skipped: NVIDIA open sourced CUDA Tile IR about eight months ago, which is what made the cuTile track possible at all.

Why this is bigger than one crate

The interesting claim here is not "Rust on GPUs", which rust-gpu and CubeCL have been doing for years. It is that Rust's ownership model survives the launch boundary. Partition a mutable tensor into disjoint pieces before launch, share the immutable ones, and the compiler can rule out data races in device code the same way it does on the host. If the 99.6% figure holds up on hardware other than a B200, then the usual objection to memory-safe GPU code, that you pay for the checks in throughput, stops being true.

The catch is that the track with the good numbers is the DSL, and the track that would let you write plain Rust is the one still on an April nightly. Those are two different products with one announcement, and only one of them has a paper behind it.

Your turn

If you maintain CUDA kernels today, the useful question is narrower than "is Rust ready": what fraction of your kernel bugs are actually aliasing or race bugs that a partition model would have caught at compile time? If the answer is close to zero, none of this changes your week. Tell us which it is, because that ratio decides whether this is a curiosity or a migration.

And if you want the other side of the Rust rewrite story, we ran Ubuntu 26.10's Rust replacement for rm and segfaulted it on a directory tree, which is a useful reminder that "rewritten in Rust" and "finished" are different claims: Ubuntu 26.10 Finished Moving rm to Rust, and We Segfaulted It.