Leaked Internal Sony Video Uses Horizon’s Aloy to Show AI-Powered PlayStation Character Prototype

A leaked internal video has revealed Sony is experimenting with AI-powered PlayStation characters.

The Verge reported on the emergence of an internal video allegedly created by PlayStation Studios’ advanced technology group that uses Aloy from the Horizon games to demonstrate AI-powered game characters.

The video was subsequently pulled from YouTube following a copyright claim from Muso, an internet enforcement company The Verge said lists Sony Interactive Entertainment as a client. This suggests the video is legitimate. IGN has asked Sony for comment.

In the video, as reported by The Verge, Sony Interactive Entertainment director of software engineering, Sharwin Raghoebardajal, has a conversation with an AI-powered Aloy via voice prompts and AI-generated speech and facial animations.

The speech recognition is via OpenAI’s Whisper for speech-to-text, the conversational AI via GPT-4 and Llama 3 for dialogue and decision-making, the speech synthesis via Sony’s Emotional Voice Synthesis (EVS) system, and the facial animation via Sony’s Mockingbird technology.

In the tech demo, Raghoebardajal asks Aloy how she’s doing, to which she responds: “Hello, I’m managing alright. Just dealing with a sore throat. How have you been?”

The voice heard coming from Aloy’s mouth is not that of voice actor Ashly Burch, who famously plays the character in the video games. Rather, it is a robotic voice similar to the kind you get from any of the thousands of text-to-speech voice generators used to narrate social media posts. AI Aloy’s facial movements are stiff and her eyes appear lifeless as she converses with Raghoebardajal.

When asked how Aloy’s search for her mother is going, she replies: “I discovered I’m a clone of Dr. Elizabeth Sobeck, which led me to understand my purpose and origins.”

How does it feel to be a clone? “Being a clone feels unique, connecting me to the past while allowing me to create my own path and future,” Aloy replies.

The demo then transitions into the actual Horizon Forbidden West game world, and Raghoebardajal continues the conversation while playing the game. It’s slightly jarring to see the player converse with a character they’re in control of, and the irony of using Aloy as an AI-powered PlayStation character given the plot of the Horizon games is obvious.

The demo is described as a prototype developed with the help of Horizon studio Guerrilla Games to show off the tech internally at Sony. “This is just a glimpse of what is possible,” Raghoebardajal says in the video. And while the video shows an investment from Sony in the exploration of AI-powered PlayStation characters, it’s worth noting the company has yet to confirm plans to incorporate the tech into any public-facing PlayStation product, and there is no suggestion it would even be possible with PS5 games at this stage.

But it perhaps comes as little surprise to see Sony’s work behind the scenes here, given pretty much all its competitors in the video game space are working on AI tech of some description. Indeed, Xbox maker Microsoft is all-in on AI, and recently announced an AI called Muse designed to generate ideas for a game’s design.

Generative AI is one of the hottest topics within the video game and entertainment industries, which have both suffered massive layoffs in recent years. Generative AI thus far has drawn criticism from players and creators due to a mix of ethical issues, rights issues, and AI’s struggles to produce content audiences actually enjoy. For instance, Keywords Studios attempted to create an experimental game internally using entirely AI. The game failed, with Keywords citing to investors that AI was “unable to replace talent.”

EA said in September that AI was “the very core” of its business, and more recently Capcom said it was experimenting with generative AI to create the “hundreds of thousands” of ideas needed for in-game environments.

Head of PlayStation Productions and head of product at PlayStation Studios Asad Qizilbash weighed in on AI to say its use in video games is important to Gen Z and Gen Alpha gamers who seek “personalization across everything.”

“For instance, non-player characters in games could interact with players based on their actions, making it feel more personal,” Qizilbash said. “This is important for the younger Gen Z and Gen Alpha audiences, who are the first generations that grew up digitally and are looking for personalization across everything, as well as looking for experiences to have more meaning.”

Activision recently admitted the use of generative AI for some Call of Duty: Black Ops 6 assets amid a backlash to an “AI slop” zombie Santa loading screen.

Wesley is the UK News Editor for IGN. Find him on Twitter at @wyp100. You can reach Wesley at wesley_yinpoole@ign.com or confidentially at wyp100@proton.me.