Microsoft and Hellblade Developer Ninja Theory Built an AI Called Muse Designed to Generate Ideas for a Game's Design — and It’s Already Making 'Complex' Gameplay Sequences
Microsoft has announced a new generative AI model designed for gameplay ideation.
The company detailed what it calls the first World and Human Action Model (WHAM). The WHAM, Katja Hofmann, Senior Principal Research Manager and lead of the Microsoft Research Game Intelligence team, said in a blog post, is a generative AI model of a video game that can generate game visuals, controller actions, or both.
Microsoft calls this generative AI model Muse, which was developed by the Microsoft Research Game Intelligence and Teachable AI Experiences (Tai X) teams in collaboration with Hellblade developer Ninja Theory. It’s open sourcing the weights and sample data and making the executable available for the WHAM Demonstrator — a concept prototype that provides a visual interface for interacting with WHAM models and multiple ways of prompting the models.
The company provided a number of gameplay clips showing what Muse is capable of. Currently, the model can generate “complex gameplay sequences that are consistent over several minutes” just by prompting the model with 10 initial frames (one second) of human gameplay and the controller actions of the whole play sequence.
The game used to train Muse was Ninja Theory’s 2020 multiplayer game Bleeding Edge. “We worked closely with our colleagues at Ninja Theory and with Microsoft compliance teams to ensure that the data was collected ethically and used responsibly for research purposes,” Hofmann insisted.
“It’s been amazing to see the variety of ways Microsoft Research has used the Bleeding Edge environment and data to explore novel techniques in a rapidly moving AI industry,” said Gavin Costello, technical director at Ninja Theory.
“From the hackathon that started it all, where we first integrated AI into Bleeding Edge, to building AI agents that could behave more like human players, to the World and Human Action Model being able to dream up entirely new sequences of Bleeding Edge gameplay under human guidance, it’s been eye-opening to see the potential this type of technology has.”
Muse is used in “world model mode” meaning it is used to predict how the game will evolve from the initial prompt sequence. The more closely the generated gameplay sequence resembles the actual game, the more accurately Muse has captured the dynamics of that game.
Generative AI is one of the hottest and most controversial topics in the creative industries. As video game development costs rise, publishers are increasingly looking to AI tools to speed up work and cut costs. Call of Duty reportedly sold an “AI-generated cosmetic” for Call of Duty: Modern Warfare 3 in late 2023, and fans accused Activision of using generative AI again for a loading screen last year. EA said in September that AI was “the very core” of its business, and just last month, Capcom said it was experimenting with generative AI to create the “hundreds of thousands” of ideas needed for in-game environments.
Head of PlayStation Productions and head of product at PlayStation Studios Asad Qizilbash weighed in on AI to say its use in video games is important to Gen Z and Gen Alpha gamers who seek “personalization across everything.”
“For instance, non-player characters in games could interact with players based on their actions, making it feel more personal,” Qizilbash said. “This is important for the younger Gen Z and Gen Alpha audiences, who are the first generations that grew up digitally and are looking for personalization across everything, as well as looking for experiences to have more meaning.”
Microsoft is all-in on generative AI, as anyone with even a cursory knowledge of ChatGPT and OpenAI will be aware of, and so this development with Ninja Theory comes as little surprise. Still, the company will face tough questions from some within the video game development community who are worried models like Muse will put them out of a job.
In the blog post, Hofmann insisted that Microsoft’s team “focus on exploring the capabilities that models like Muse need to effectively support human creatives.”
“I look forward to seeing the many ways in which the community will explore these models and build on our research,” Hofmann continued. “I cannot wait to see all the ways that these models and subsequent research will help shape and increase our understanding of how generative AI models of human gameplay may support gameplay ideation and pave the way for future, novel, AI-based game experiences, including the use cases that our colleagues at Xbox have already started to explore.”
Wesley is the UK News Editor for IGN. Find him on Twitter at @wyp100. You can reach Wesley at [email protected] or confidentially at [email protected].
About Post Author
