Microsoft agentically ports Copilot runtime to Rust for $120K

The software program engine underpinning GitHub Copilot and a rising variety of Microsoft merchandise is now written completely in Rust, with AI brokers doing a lot of the porting work.

The migration price about $120,000 in AI token utilization plus about three weeks of a developer’s time. However, managers additionally needed to grapple with a number of dozen regressions within the ensuing code, pointing to AI’s ongoing challenges in understanding Rust. 

The effort up to date the runtime module-by-module till the job was accomplished, spanning over 135 releases throughout a 14.5-week time interval. Roughly 1.3 port pull requests had been opened per day.

Overall, brokers transformed 430,000 traces of TypeScript into 800,000 traces of manufacturing Rust. To maintain the port so simple as attainable, the port solely changed TypeScript modules on a case-by-case foundation. It didn’t look to optimize the construction of the runtime itself. That work is subsequent. 

And Rust, identified for its lean efficiency, didn’t disappoint.

One benchmark measured how shortly the runtime may full 1,000 one-turn session lifecycles, utilizing a shared consumer and 100 concurrent pipelines. The unique TypeScript implementation accomplished 7.55 of these lifecycles per second, whereas Rust working in-process managed 120 per second – representing a 15.9x speedup on that individual workload.

In phrases of reminiscence, a 10-client batch of brokers consumed 1,383 MB with TypeScript whereas the Rust rewrite consumed solely 126 MB serving the identical swarm. Within Rust, the work remained in-process as an alternative of spawning exterior background processes for completion, a requirement for TypeScript.

Copilot from VS Code to Microsoft Office

At first look, most customers could not understand how pervasive the Copilot runtime is. It backs the GitHub Copilot command-line interface (CLI), the Copilot app, the SDK and the GitHub Copilot cloud agent. It exhibits up in VS Code, Visual Studio, Excel, Outlook, PowerPoint, and innumerable different Microsoft cloud companies.

Originally, the runtime was written in TypeScript and used Node.js because the framework and V8 for the execution engine. TypeScript and Node had been ace for fast improvement, however when used at scale, they suffered by way of offering quick start-up and server density. 

“This is under no circumstances a declare that each massive TypeScript program ought to change into Rust. Our necessities emphasised embedding by a C ABI, low startup and steady-state overhead, and predictable useful resource use. Rust made these objectives attainable,” wrote Microsoft Distinguished Engineer Stephen Toub in a publish that explained the entire process.

The undertaking used Copilot to rewrite Copilot, which in flip used a number of LLMs – GPT-5.6 Sol and Claude Opus 4.8 had been each namechecked – to execute totally different elements of the job, relying on every LLM’s pure energy.  

Well-read however chatty brokers

Toub assessed using brokers as largely profitable. Indeed, this undertaking would have taken years and price hundreds of thousands if performed by hand. 

Agents confirmed a number of shocking emergent behaviors, Toub famous. For one, they spent much more time gathering data than really writing code.  

“The widespread picture of AI spewing code is nearly backwards; at this scale, the work appeared far more like iterative investigation, inspecting the present state, forming a speculation, making a focused change, rinsing and repeating,” he wrote. 

Another shock was how regularly classes interacted with different classes, both these they spawned or different ones completely. 

One of the thorniest conversions was the session.ts file, which was over 30,000 traces of TypeScript that touched all features of the runtime. The porting session, which ended up taking 25 hours to finish, started by spending 56 minutes studying the documentation and making 122 instrument requires clarification. It then spawned 15 youngster classes, every creating its personal worktree and agent. Then, they began speaking with one another.

Using a built-in orchestration ability, one session discovered each different energetic session and despatched messages to these whose missions overlapped, requesting coordination.

The compiler is a trainer, not an oracle 

Given its efficiency advantages, Rust has confirmed to be a preferred language for rewriting functions. Bun creator Jarred Sumner, as an illustration, not too long ago ported the Anthropic-owned JavaScript runtime and toolkit, which contained roughly 535,000 traces of Zig code, over to Rust, nearly completely utilizing Claude brokers. As of July 30, the experimental Rust port was passing 99.8 percent of Bun’s current exams on Linux x64 glibc, with the steady releases nonetheless transport from the Zig codebase.

That job price $165,000 in tokens. 

But there are hidden risks in utilizing Rust as effectively, as Toub has discovered. 

For an LLM and a lazy programmer alike, if the code compiles, then it’s legitimate Rust.

But all through the method, the undertaking encountered dozens of regressions throughout the code. A regression is one thing that used to work, however not does after an replace. 

The compiler has no thought, as an illustration, whether or not capabilities are in the proper order, whether or not the job completes at a suitable price, whether or not it meets the developer’s unwritten necessities, or whether or not it’s internally coherent in any respect, for that matter.

At RustConf, held in Montréal final week, marketing consultant Lisa Crossman warned concerning the observe of treating the compiler as an “oracle,” or the final phrase on whether or not some Rust code is legitimate or not. “Rust stops the agent writing reminiscence unsafe code; it doesn’t cease the agent writing the fallacious program accurately,” she stated. 

If they’re smart, a human developer treats the compiler as a trainer, decoding the errors as a path to a greater understanding of the language’s area. An LLM, alternatively, simply makes use of the compiler as a black field, one it will possibly batter with blunt, inefficient workarounds till one passes muster (maybe that is what Zig creator Andrew Kelley meant when he known as Bun’s Rust code “unreviewed slop”). 

In Toub’s case, he had discovered that compiler-approved regressions may come from ambiguous semantics and behaviors, department drift, lacking options that weren’t ported over, and differing behaviors from the substitute code. 

“That’s under no circumstances an argument in opposition to Rust’s compiler,” Toub wrote. “But ‘if it compiles, it’s right’ is helpful solely as a joke.” ®