Xbox announces ‘a generative AI model for gameplay ideation’ called Muse, but don’t get too excited: Machines aren’t about to make games for you just yet

“In nearly every corner of our lives, the buzz about AI is impossible to ignore,” Microsoft said in today’s announcement of its new generative AI model called Muse, and that is absolutely true. “It’s destined to revolutionize how we work, learn, and play,” it continued, and well, I’m not so sure about that part. But Microsoft seems determined to roll with it, saying, “At Xbox, we’re all about using AI to make things better (and more fun!) for players and game creators.”

Created in a partnership between Microsoft Research and Xbox studio Ninja Theory, developer of the Hellblade games, Muse is a “world and human action model (WHAM)” trained on Ninja Theory’s Bleeding Edge.

“What’s groundbreaking about Muse is its detailed understanding of the 3D game world, including game physics and how the game reacts to players’ controller actions,” Microsoft corporate vice president of gaming AI Fatima Kardar wrote. “This allows the model to create consistent and diverse gameplay rendered by AI, demonstrating a major step toward generative AI models that can empower game creators.”

Microsoft acknowledged that “it’s still early,” but predicted that Muse could be used for “faster creative ideation,” presumably by making it easier for developers to chuck in new stuff for the machine to play to see how it works. Interestingly, though, an even greater focus was put on the system’s potential for game preservation.

“Today, countless classic games tied to aging hardware are no longer playable by most people,” Kardar wrote. “Thanks to this breakthrough, we are exploring the potential for Muse to take older back catalog games from our studios and optimize them for any device. We believe this could radically change how we preserve and experience classic games in the future and make them accessible to more players.”

Xbox boss Phil Spencer dove more deeply into that aspect of Microsoft’s ambitions for Muse in a supplementary video, saying the potential impact on game preservation is “one of the things that I get excited about.”

“Time will erode the amount of hardware that’s out there that can actually play a game,” Spencer said. “But you could imagine a world where, from gameplay data and video, that a model could learn old games and really make them portable to any platform where these models could run.

“I think that’s really exciting. We’ve talked about game preservation as an activity for us, and these models and their ability to learn completely how a game plays without the necessity of the original engine running on the original hardware I think opens up a ton of opportunity.”

Amidst all the excitement about what its generative AI systems can do, Spencer and company predictably took pains to explain that its embrace of AI isn’t meant to replace human beings, but to—as mentioned earlier—”empower” them, saying that “game creators will always be the center of our overall AI efforts.” Ninja Theory studio chief Dom Matthews echoed that in his own comments.

“It’s not about using AI to generate content, but it’s actually about creating workflows and approaches that allow our team here of 100 creative experts to do more, to go further, to iterate quicker, to ideate quicker, to bring the ideas in their heads to life in a tangible form for others to see,” Matthews said.

“Although this is technology that we haven’t used in the creation of our games and we don’t intend to use this technology for the creation of content, I think the interesting aspect for us that is exciting is how can we use technology like this to make the process of making games quicker and easier for our talented team so that they can really focus on the thing that I think is really special about games, which is that human creativity.”

For a closer look at what Muse actually is, you’re probably better off diving into the full report published on Nature. It’s dry but detailed, and also more clear that this is all preliminary work—we’re not yet at a point where we can tell a computer to make Arx Fatalis 2 and then come back in an hour and it’s done. From the conclusion:

As we navigate the unfolding role of generative AI in the creative industries, there are ways to direct its development to ensure human agency over the creative process. We have presented a user study with diverse game creatives through which we identified three model capabilities that should be given priority when developing AI systems that aim to support creative ideation through iterative practice and divergent thinking: consistency, diversity and persistency. We have also shown that it is possible to develop generative AI models that exhibit these capabilities when trained on appropriate datasets.

Microsoft CEO Satya Nadella was a little more forward-looking on the whole thing, however:

Dr. Michael Cook, an AI researcher, game designer, and senior lecturer at King’s College London, threw some cold water on the Muse hype in a blog post, writing that the research paper “is not really about ‘generating gameplay’ or ‘ideas’,” but rather “a study of how human designers think about working with generative AI tools.” Cook also took issue with the idea that Muse could serve as a preservation tool: While AI models could have uses for reconstruction (with some limitations), there’s more to game preservation than simply mimicking what’s on the screen at any given moment.

Cook clarified, however, that he’s not against using AI as a design tool—in fact, he thinks it’s cool. “I think systems like this can inspire people to make interesting stuff, I think they can make alien games that humans would not, I think they can be fun art pieces and interesting engineering challenges,” he wrote. “I think when we feel we have ownership over technology, that we can build things ourselves on small scales where we feel in control of what it does, and in charge of its scope and potential harms, then we feel a lot happier opening ourselves up to ideas like ‘what if I used algorithms to generate game levels for me’.”

I do have to wonder who this Microsoft report is for. It’s clearly high-level research, and it seems equally clear to me that we’re a long way from seeing this sort of thing deployed for any sort of practical use. I have no doubt that the continued growth of generative AI will have a major impact on the videogame industry, but that’s always been the way of it: Technology advances, tools evolve, and games get—well, not necessarily better, but certainly bigger and prettier. Game development also becomes more accessible, which is a big deal in its own right.

I can’t shake the feeling that Microsoft is trumpeting its AI advances because everybody is trumpeting their AI advances these days, and there’s literally billions of dollars at stake. But there are real advancements underpinning the hype, and Microsoft said we’ll be seeing more “about how we are using AI to empower game creators and provide the best possible player experience across all devices” in future announcements coming this year. “New immersive AI gaming experiences” powered by Muse are also “coming soon to Copilot Labs.”