Why the AI Era Flipped the Economics of Rust

Rust's trade-off, flipped

Relative cost, before and after coding agents

Before agents

With coding agents

Writing Rust

High

Lower

Running a slow tool

Low

High

Cheaper choice

Python & Node tools

Rust

Ask what language the LLM world is written in and the honest answer has three parts. Models are trained in Python. They are served on C++ and CUDA kernels. And almost everything in between, the code that turns text into tokens, checks a model’s output against a schema, installs and type-checks the Python around it, stores its embeddings, sandboxes the code it writes, and runs the agent loop on a developer’s machine, is increasingly written in Rust.

For teams building products on top of models, that third layer is where most engineering effort now goes. To them, a frontier model is an API or a few files of weights and a serving stack. An agent product is a tokenizer, a validator, a retrieval store, a sandbox, a command-line harness, a set of linters and checkers that run on every turn, and a lot of glue. Rust has quietly become the default for that layer.

That wasn’t always the obvious choice. Rust has produced fast, memory-safe programs for a decade, but writing them took more human time than writing Python or TypeScript, and most tools in this layer ran rarely enough that their speed didn’t matter much. Coding agents changed both sides of that trade at once, as Figure 1 shows. They run tools on every turn, which makes a slow tool expensive, and they take on much of the work of writing Rust, which makes Rust cheaper.

After a short primer on Rust itself, this post looks at that shift from three sides: what the ecosystem has actually rebuilt, which of Rust’s benefits matter for this layer, and why the trade-off flipped in the AI era—before closing with the places where Rust is still the wrong call.

What Rust is

Rust is a compiled systems programming language. It began in 2006 as Graydon Hoare’s personal project, was sponsored by Mozilla from 2009, reached its stable 1.0 release in 2015, and has been stewarded by the independent Rust Foundation since 2021. It plays in the same space as C and C++: it compiles to native machine code, runs without a virtual machine, and gives the programmer control over how data is laid out in memory. Linux 6.1 merged initial support for writing kernel code in Rust, making it the first language besides C and assembly accepted there.

What makes Rust unusual is how it manages memory. Most languages pick one of two approaches. C and C++ hand the job to the programmer, who allocates and frees memory by hand and gets crashes and security holes when that goes wrong. Python, JavaScript, Go and Java hand it to a garbage collector, a runtime that periodically finds and frees unused memory, which is safe but costs CPU time, memory headroom and occasional pauses.

Rust takes a third route called ownership. Every value has exactly one owner, the value is freed the moment its owner goes out of scope, and other code can only borrow a value under rules the compiler checks: many readers at once, or a single writer, never both. Those checks happen entirely at compile time, so a whole class of bugs never reaches a running program:

let names = vec![String::from("ada")];
let moved = names;          // ownership of the vector moves to `moved`
println!("{}", names[0]);   // error[E0382]: borrow of moved value: `names`

The program above doesn’t compile, so it can never read memory that has been handed off or freed. The same rules stop two threads from writing to the same data at once, which is why Rust’s documentation calls its concurrency “fearless”.

The benefits

• The compiler checks far more than types. Like C, C++ or Java, Rust catches type errors before the program runs. It also rejects use-after-free, double frees and data races between threads, and safe Rust has no null references at all. Array access is bounds-checked while the program runs, so reading past the end stops the program instead of corrupting memory. Those bugs cause most security holes in C and C++ code.
• Native speed. Rust compiles to machine code through LLVM, the same backend Clang uses, so well-written Rust runs at about the speed of C and C++. On CPU-heavy work it is often 10 to 100 times faster than Python or JavaScript.
• Predictable performance. With no garbage collector, there are no pauses while a runtime cleans up memory, and memory is freed the moment it’s no longer used. Programs use less memory and respond in steady, predictable time.
• Control over memory layout. Data sits where the programmer puts it, packed together in memory, which keeps the CPU’s caches full and hot loops fast.
• Abstractions that cost nothing. Iterators, generics and closures compile down to the same code as hand-written loops, so readable code doesn’t run slower.
• Safe concurrency. The rules that prevent memory bugs also stop two threads from changing the same data at once, so parallel code is much easier to get right.
• Errors that are hard to ignore. Functions that can fail return Result. Callers can’t use the value inside without handling the failure case, and the compiler warns when a Result is ignored, so errors don’t slip past the way an uncaught exception can.
• No runtime to ship. A Rust program starts in milliseconds, needs no interpreter or virtual machine, and ships as a single native binary.
• Embeds anywhere. A Rust library can be called from Python, Node, C or WebAssembly with little overhead, so one fast core can serve many languages.
• One standard toolchain. Cargo builds, tests, documents and publishes every project the same way, and rustfmt and Clippy handle formatting and linting.

The costs

Rust’s advantages come with real costs, and they explain why it took this long to spread:

• A steep learning curve. Ownership, borrowing and lifetimes are unfamiliar even to experienced engineers, and the compiler rejects code that would run fine in other languages until it is restructured.
• Slow compiles. All those checks and optimizations take time. Large Rust projects build noticeably slower than most languages, and a clean build can take minutes.
• Slower prototyping. Rust asks for decisions about data ownership and error handling up front, which is friction when an idea is still changing every hour.
• Gaps in the ecosystem. Libraries for systems, networking and command-line tools are strong. Machine learning, GUIs and many enterprise integrations are thinner than in older, larger ecosystems.
• Async is harder. Asynchronous Rust works well once it is set up, but it adds its own concepts, such as choosing an async runtime and rules about which data may move between threads, that trip up newcomers.
• unsafe doesn’t disappear. Talking to C libraries or hardware still needs unsafe blocks, where the compiler’s guarantees are suspended and humans have to check the code.
• A smaller talent pool. Rust is young, and far fewer engineers know it well than know the mainstream languages.

Most of those costs land on the people writing the code, not on the program that ships. That distinction matters later in this post. The sections below show why Rust’s strengths fit the layer around the model so well, and why the cost of writing it has started to fall.

What has actually moved

The pattern is easiest to see as a list of components that an LLM application touches on a normal request. Each row below links to the project’s own repository or announcement.

Two things stand out. The first is how much of this sits inside Python. A typical LLM application is still a Python program, but its tokenizer, its validator, its dataframe engine, its package installer, its linter and its type checker are now Rust. Python became the interface; Rust became the engine.

The second is that the list isn’t uniform, and it shouldn’t be read as if it were. PyTorch is Python and C++. llama.cpp is C and C++. Ollama is Go. Two of the most widely used coding agents, Claude Code and Gemini CLI, are TypeScript. Hugging Face put its Rust-based Text Generation Inference server into maintenance mode and now recommends vLLM and SGLang, which are mostly Python, C++ and CUDA. And Microsoft’s native rewrite of the TypeScript compiler for TypeScript 7, which it announced as about 10× faster (the companion benchmark measured 4–6× on VS Code), is written in Go, not Rust, for reasons covered under “Where Rust doesn’t pay”. Rust is the default for the layer around the model. It isn’t the only answer, and the exceptions are instructive.

The wider developer population is moving in the same direction more slowly. In the 2025 Stack Overflow Developer Survey, Rust was again the most admired language, at 72%. Admiration isn’t adoption, but it does mean that teams which choose Rust tend to keep it.

Why Rust, specifically

The benefits listed earlier don’t matter equally everywhere. For the layer around the model, four of them decide the choice. None is unique to Rust on its own; what is unusual is getting all four in one language, with no runtime to ship.

Native speed without a garbage collector

For most of an LLM request, that difference doesn’t matter: the request is waiting on a GPU somewhere. It matters a great deal for the work that isn’t waiting. A tokenizer runs on every prompt and every streamed token. A validator runs on every structured response. A linter or type checker runs on every turn of a coding agent, sometimes several times. A vector index answers thousands of queries per second. These are hot loops, and a 10–100x improvement in a hot loop changes what a product can afford to do on every request.

Safety by construction

Rust’s ownership rules turn whole classes of bugs into compile errors: use-after-free, double free, iterator invalidation and data races between threads. Runtime bounds checks then turn an out-of-range access into a controlled stop rather than a buffer overflow. The language marks the exceptions with an explicit unsafe keyword that is easy to search for and review.

This is the reason governments and large vendors have pushed memory-safe languages. The US Office of the National Cyber Director made the case in 2024 that manufacturers can “prevent entire classes of vulnerabilities” by adopting memory-safe languages. Android is the best-documented case. Google reported that memory-safety bugs fell from 76% of Android’s vulnerabilities in 2019 to 24% in 2024 as new code moved to memory-safe languages, and in 2025 reported below 20%, with a roughly 1000x lower memory-safety vulnerability density in Rust than in Android’s C and C++.

The same post reports a result that matters more for teams shipping quickly: Rust changes in Android had a roughly 4x lower rollback rate than comparable C++ changes, needed about 20% fewer revisions, and spent about 25% less time in review. That is a comparison against C++, not against Python or TypeScript, and it should be read that way. But it undercuts the old assumption that Rust’s safety costs delivery speed.

For LLM infrastructure, safety has a specific shape. Agent sandboxes run code that nobody reviewed. Inference servers parse attacker-controlled input at high volume. Tokenizers and file-format readers take bytes from the internet. Those are exactly the places where a memory-safety bug turns into a remote exploit.

A runtime-free core that embeds anywhere

This is the property that explains why Rust won the tooling layer rather than Go. A Rust library has no garbage collector or virtual machine to start, and no scheduler unless it opts into async I/O. It compiles to an ordinary shared library with a C-compatible interface, or to a small WebAssembly module.

That makes it cheap to wrap. PyO3 exposes a Rust crate as a Python module; napi-rs does the same for Node. The result is the shape the table above keeps repeating: one fast, safe core, with thin bindings for each language the ecosystem actually uses. Hugging Face ships one tokenizer implementation to Python, Node and Rust users. Pydantic ships its validation engine to every Python application that uses it, without those applications knowing it is there. A Go library can do this too, but it brings its runtime with it, and crossing the boundary is expensive.

One binary, no environment

The last property is the least technical and possibly the most decisive for agent tools. A Rust program ships as a single native binary with its Rust dependencies compiled in (on Linux it still links the system C library unless built against musl, which makes it fully static). There is no interpreter to install, no virtual environment to activate, no node_modules tree, and no post-install script.

For a coding agent CLI that runs on every kind of developer machine, a CI runner and a stripped-down sandbox image, that removes a whole category of support problems. It also means the tool starts in milliseconds, which matters when the agent invokes it dozens of times in a session.

Why the trade-off flipped

If those four properties fit developer tools and infrastructure so well, a natural question is why Python and Node tools dominated that layer for the previous decade.

Why it was harder to develop in Rust before

The costs listed earlier explain it. Historically, the argument against Rust was never about the program it produced; it was about the human time required to write and change it, and three of those costs mattered most:

1. Upfront ownership and lifetime modeling. In Python, TypeScript or Go, a developer can sketch a graph of objects, pass references freely across functions, and let the garbage collector sort out when memory dies. In Rust, data layout and ownership hierarchy have to be decided upfront. Borrowing across async code, lifetimes on zero-copy parsers, and wrappers for shared ownership and interior mutability (Arc, Mutex, RefCell) force architectural decisions early, when a product’s requirements are still fluid.
2. Strict type and error plumbing. Every fallible operation returns a Result, every optional value is an Option, and every match must be exhaustive. That discipline prevents production crashes, but for years it meant writing substantially more structural boilerplate by hand than an equivalent script in Python or TypeScript.
3. Compile-time iteration cost. Compiling generic code separately for every type it’s used with, expanding macros and running LLVM’s optimizer make clean Rust builds slow compared with an interpreted language or a fast incremental bundler. When a tool only ran once in CI or once before a human opened a pull request, teams rationally optimized for how fast an engineer could write the code rather than whether the tool ran in 200 milliseconds or 10 seconds.

What changed in the LLM and agent era

Two shifts arrived at the same time and inverted that calculus.

First, the LLM stack turned once-a-day tools into inner-loop hot paths. A human runs a linter or type checker occasionally. A coding agent runs the whole verification gate (linter, type checker, formatter and tests) several times per task as it edits, reads diagnostics and repairs its own mistakes, and a serving pipeline runs its tokenizer and validator on every request.

Why inner-loop speed matters to coding agents

Composite per-turn verification gate: Legacy Python/Node vs. Rust/native

Legacy gate (Py / Node)

Rust / native gate

Django · 461k Py

17.4 s

1.94 s

VS Code · 3.6M TS

100.6 s

9.96 s

Hono · 75k TS

5.59 s

0.34 s

In the companion post (Rust in the Loop), that effect was measured across 60 hyperfine repository benchmarks and 120 end-to-end Gemini coding agent sessions. On Django’s 461,000 lines of Python, replacing flake8, mypy and black with ruff, pyrefly and ruff format cut incremental gate time from 17.4 seconds to 1.94 seconds (9.0× faster, or 24.4× faster against Pyright). On VS Code’s 3.64 million lines of TypeScript, replacing eslint, tsc 6 and prettier with oxlint, tsgo and oxfmt cut incremental verification from 100.6 seconds to 9.96 seconds. Inside end-to-end agent sessions, those fast gates cut median task completion time by 13% to 68% in four of the six model-and-language pairs tested, because checks finished synchronously inside the current tool call instead of stalling every edit cycle or spilling into background polling tasks. (The other two pairs were a near tie and a pair where too few sessions passed to compare; the companion post has the details.)

Second, coding agents sharply cut the human cost of writing and migrating to Rust. Much of what made Rust feel taxing to write by hand—satisfying borrow-checker diagnostics, threading explicit error types across modules, writing napi-rs or PyO3 bindings, and filling out exhaustive pattern matches—is well-scoped, compiler-guided work where frontier models excel. Teams now report finishing agent-driven Rust migrations in hours or weeks that would previously have needed multi-year roadmaps. The three examples below are self-reported by the teams that did the work, in vendor blog posts and research papers, and haven’t been independently reproduced here:

• Scientific Fortran and C to #![forbid(unsafe_code)] Rust in 59 minutes (Google Gemini 4 Argon & migrate-to-rust skill). Google reports that Gemini 4 Argon migrated the core solver of FVCOM, a 174,774-line Fortran 90 and C coastal ocean model from the 1980s, into 100% safe Rust in 59 minutes. The workflow, now released as an open-source skill, runs in stages: baseline capture, an architectural blueprint and bug audit, then a #![forbid(unsafe_code)] rewrite with a self-healing test loop, a 100-seed A/B parity harness against the Fortran binary, pre-allocated Struct-of-Arrays (SoA) workspaces and LLVM Profile-Guided Optimization. According to the post, forcing Fortran’s untyped global arrays into Rust’s type system surfaced six decades-old memory and math bugs, and the finished port used 54% less RAM and ran 1.88× faster than gfortran -O3 (883 ms → 469 ms), matching all 100 seeds to within 3.91 × 10⁻¹³.
• TypeScript to 832,000 lines of production Rust (GitHub Copilot Runtime). GitHub reports rewriting its shared Copilot agent runtime, which backs Copilot CLI, VS Code, Visual Studio and six language SDKs, in place from TypeScript and Node.js. The result was 832,378 lines of production Rust and 468,689 lines of Rust unit tests, landed across 128 pull requests in 14.5 weeks, mostly by one engineer directing coding agents. Of the 8,678 rustc error codes logged during the port, ownership, borrowing and lifetime errors together made up just 1.7%. The borrow checker stayed in the background; cargo check mostly caught ordinary name, field and type mismatches across 4,478 incremental runs.
• Systems and aerospace C/C++ to safe Rust (Asari AI, Syzygy, SpeC2Rust, Req2Rust). DARPA’s Translating All C to Rust (TRACTOR) program has made automated C-to-Rust translation a funded research target. Asari AI reports using agents to migrate four C libraries from the TRACTOR set, more than 40,000 lines in total (zlib, gzip, libyaml and libmcs, a math library for spacecraft software built to European Space Agency standards), into safe Rust that passed the original test suites. GTD GmbH, which supplied the libmcs test suite, is quoted in the post saying the agents found “a very subtle double error” in it that years of reviews had missed. Research harnesses such as Syzygy (Google’s Zopfli), SpeC2Rust and Req2Rust (OpenHarmony C++ modules) report similar specification-first, test-validated pipelines.

Where Rust doesn’t pay

The shift to Rust is real, but it has boundaries, and rewrites outside the hot path rarely earn their cost back.

Model training and research. PyTorch, JAX and the CUDA ecosystem are where models are built. A research loop that changes every hour needs an interactive language and the tensor libraries that already exist, and that is Python.

High-level orchestration that waits on the model. A web backend or orchestration service that spends 95% of its wall time waiting for streamed tokens from a model API gains almost nothing from native CPU speed. TypeScript and Python remain faster to iterate on for product workflows, UI state and prompt orchestration.

Clean compile times on large workspaces. Rust compiles slowly on cold builds, and agents that run a full workspace build on every edit can spend significant wall time waiting on cargo. (In GitHub’s Copilot runtime port, when 15 concurrent agent sessions triggered cargo builds on one laptop at the same time, the machine ground to a halt until a coordinator session was set up as a build mutex.) Teams that use Rust effectively with coding agents lean on cargo check, workspace crate splitting, shared build caches and targeted test runs rather than rebuilding release binaries on every turn.

Line-by-line translations that keep the old design. When Microsoft ported the TypeScript compiler to create TypeScript 7 (tsgo), it chose Go rather than Rust. The team’s stated reasons were that Go let the new compiler keep the same structure as the original codebase, so changes could be ported between the two, and that garbage-collection pauses don’t matter for a batch compiler. Similarly, a 2026 study of automatically transpiled C projects found the Rust output more than 3% slower than the original C on 12 of 19 measured operations, by up to 62%, largely because C-style memory patterns were carried straight into Rust. Rust rewards architectures designed around ownership and contiguous data layouts (Struct-of-Arrays, pre-allocated arenas); successful agentic migrations like migrate-to-rust, SpeC2Rust and Req2Rust work precisely because they generate an architectural blueprint first and validate behavior through external A/B parity tests rather than translating line by line.

The shape of the LLM stack

The picture that emerges is not “everything is moving to Rust.” It is more specific than that.

Rust sits around the model

The layer every request and agent turn passes through

Products

TypeScript · Python

Web appsInterfacesOrchestration

Around the model

Rust

TokenizersValidationData enginesVector storesLinters & checkersSandboxesAgent CLIs

The model

Python · C++ · CUDA

TrainingKernelsServing

Models stay in Python and CUDA. Products and their user interfaces stay in TypeScript and Python, where product iteration is fastest. Between them sits a layer that runs on every request and every agent turn: tokenizers, validators, data engines, vector stores, sandboxes, linters, type checkers, package managers and agent CLIs. That layer needs to be fast without garbage-collection pauses, memory-safe on untrusted inputs, embeddable inside Python and Node without a runtime, and shippable as a single binary.

For years, Rust offered all four of those properties, but its steep human authoring curve kept much of the ecosystem on slower tools. The rise of LLMs and coding agents changed both sides of that equation—making slow inner-loop tools far more expensive while making Rust itself much cheaper to write—and that is why the layer around the model is converging on Rust.