Muse Wonder | Meta AI Analysis
Scaling Toward Personal Superintelligence
Fourteen months in the past, we started a ground-up overhaul of our synthetic intelligence efforts. In that point, we’ve gone from rebuilding our stack from first ideas to pushing the frontier.
When we launched Muse Spark, we outlined our strategy to predictable, rigorous scaling. Today’s launch builds on that work throughout pretraining and reinforcement studying, delivering our most succesful fashions but. Trained at our gigawatt-scale Prometheus knowledge heart, Muse Wonder and Muse Wonder Full deliver collectively the work of groups throughout Meta to ship frontier capabilities for advanced duties at work and in on a regular basis life — one other step towards bringing private superintelligence to the billions of people that use our merchandise.
Frontier Capabilities
Model Highlights
The Muse Wonder collection achieves state-of-the-art outcomes throughout key benchmarks for private {and professional} brokers, reminiscence, and coding.
The full outcomes under examine Muse Wonder and Muse Wonder Full with different frontier fashions throughout a broad analysis suite.
For extra particulars about our evaluations, see our report.
The outcomes above present a broad view of the fashions’ efficiency. We now take a better have a look at 4 areas central to their work on real-world duties: area experience, Active Memory, Active Focus, and frontier coding.
Domain Expertise
Muse Wonder and Muse Wonder Full apply specialised data to issues folks encounter at work and in day by day life, from understanding a lease to creating sense of monetary or well being data. Our evaluations embody duties in regulation, finance, and well being that check this skill. Harvey’s Legal Agent Benchmark (LAB), for instance, requires fashions to work with supply paperwork and comply with detailed directions to provide authorized work merchandise.
Active Memory
One of probably the most crucial steps in direction of helpful private {and professional} brokers is dependable reminiscence. Models want to recollect what issues and acknowledge when earlier data turns into related. We name this Active Memory: carrying vital context about you throughout conversations and actively retrieving the small print related to the duty at hand.
Active Focus
Using that context to finish a activity additionally requires sustained effort. Active Focus describes how Muse Wonder continues working towards a difficult aim: attempting different approaches, working by blockers, and adapting to new data. Along the best way, it asks for clarification when wanted and checks in earlier than taking vital actions. We consider focus by FocusBench and sustained execution by SWE-Marathon, which checks work on long-running software program initiatives.
Frontier Coding
Coding brings these capabilities collectively. Building software program requires monitoring necessities, making modifications throughout a codebase, and dealing by repeated cycles of testing and revision. Muse Wonder and Muse Wonder Full ship frontier coding efficiency, supporting initiatives from small purposes to advanced software program. We educated each fashions on numerous, long-running duties to assist them maintain progress throughout this course of.
Muse Wonder builds a playable 3D kitchen recreation from customizable picture inputs.
Capabilities That Generalize
Two fashions typically have comparable benchmark scores, but one appears to deal with a a lot wider vary of unfamiliar duties. That’s what complaints of “benchmaxxing” and reward for “huge mannequin scent” each get at: a breadth of functionality that in style benchmarks in a couple of domains can’t totally seize.
To quantify this mismatch, we examine how efficiency on duties focused throughout post-training pertains to efficiency on unfamiliar duties as we scale test-time compute. The plot under compares Muse Spark 1.3 and Muse Wonder Full, with every level denoting efficiency at a definite reasoning effort. The horizontal axis exhibits the mannequin’s rating on a composite consultant of the duties we centered on throughout post-training (“in-distribution”). The vertical axis exhibits the common rating on a set of duties we didn’t goal throughout post-training (“out-of-distribution”).
Both of our fashions enhance equally on out-of-distribution duties as we scale test-time compute. But given its in-distribution efficiency, Muse Wonder Full performs even higher on unfamiliar duties than the Muse Spark 1.3 development would predict. This upward shift is as a result of Muse Wonder and Muse Wonder Full construct on bigger, stronger base fashions: a stronger base yields post-training positive aspects that carry over extra successfully to unfamiliar duties. That versatility particularly issues for private brokers, which must deal with many combos of targets, instruments, and constraints — not all of which we will anticipate throughout coaching.
Working instantly in a browser and throughout apps, Muse Wonder builds a operating plan (Map knowledge © OpenStreetMap contributors) based mostly on private preferences and circumstances.
Safety on the Frontier
Delivering private superintelligence to everybody requires a major financial backing in alignment and security. A private agent takes actions on a person’s behalf and may deal with context reminiscent of messages, calendars, and personal paperwork. That stage of entry calls for a mannequin that workout routines discretion with non-public data and resists misuse. We educated the Muse Wonder household to guide on each dimensions, with robustness to adversarial directions and cautious dealing with of personal data.
As fashions develop into extra succesful, our aim is to assist folks harness these capabilities safely and responsibly. Muse Wonder affords our strongest safeguards to this point towards CBRN (chemical, organic, radiological, and nuclear) and cyber misuse, whereas preserving entry to frontier cyber capabilities for defensive use, per our Meta Superintelligence Scaling Framework.
Muse Wonder is our most aligned mannequin to this point, and alignment is central to how we construct private superintelligence. A full report with further security evaluations can be out there quickly.
We hope that Muse Wonder and Muse Wonder Full give extra folks the ability to pursue their goals and construct the international stage they need. This is a serious technical step towards private superintelligence for everybody, with far more to return.
