How Datamining Has Changed the Landscape of Video Game Puzzle and Secret Design

Last year, Blizzard launched a new event in World of Warcraft: Classic called Season of Discovery. The game mode offered players the chance to explore an old-school version of the MMORPG as it existed back in 2004, but with a twist. Players would be able to collect “runes” that would transform their characters into wacky new variations like mage healers or shaman tanks. New quests and other secrets were said to pepper the world, and certain dungeons were converted into larger, more challenging raids with new bosses and new loot. It was an exciting event, one that hinged on surprise and, as the title suggested, discovery.

The problem? Datamining. Through datamining, curious players were able to “discover” all the new runes, bosses, spells, and loot before they had even set foot in Azeroth.

True, not everything was spoiled out of the gate. Datamining, at its simplest, enables players to view basically any file in the game: images, text strings, locations. But none of those files are available in context. In Season of Discovery, players could see the names and uses of all the runes, but still had to find the runes in game, which often required solving puzzles or exploring areas off the beaten path. But the prevalence of spilled secrets before the event even started did prompt some players to quip on the irony of datamining in a “Season of Discovery.” Prominent World of Warcraft guide site Wowhead ended up backing off publishing datamined info in response to community feedback. Even developers commented on the issue.

While Season of Discovery is a particularly poignant example of World of Warcraft’s ongoing relationship with dataminers, the truth is that the entire game has been like this for years. Because Blizzard uses Public Test Realms (PTR) to test content before it goes live, dataminers have a ready supply of new tidbits of information to dump onto websites, guides, forums, and social media sites before the vast majority of players encounter them in regular gameplay. And Blizzard isn’t alone, either. In recent years, a number of games centering concepts such as discovery and secret-hunting have been forced to reckon with the challenges posed by dataminers. Live service games like Destiny 2 and Helldivers 2 face similar challenges as World of Warcraft, while even single-player games centered around secrets like Tunic and Animal Well have found creative ways to hide their biggest, most interesting twists.

Working against dataminers to hide secrets is a losing battle, which is why some developers are taking a different approach. Increasingly, creators are designing their games’ biggest surprises around the expectation that someone, somewhere, will crack them open and learn everything about them instantly.

Flavor Crystals

Jeremy Feasel is an associate game director on World of Warcraft, but he quips to me that his actual job is to add “flavor crystals” to World of Warcraft: that is, secrets or other fun little diversions. He started in the Cataclysm expansion working on rare spawns, which usually meant mounts that were difficult to obtain. But whenever he had a little time, he says, he would get up to some wacky stuff.

Which is how World of Warcraft players were introduced to Dormus, the Camel Hoarder.

For those unfamiliar, Dormus is part of a fun little secret chain that ultimately rewards players with a rideable camel. To start, players come across a “Mysterious Camel Figurine” which is a very small statuette that can be found randomly in one of 50 different spots in the large, sandy region of Uldum. Most of the time, finding a figurine and clicking on it will simply cause it to crumble to dust. Only very rarely will it teleport players to Dormus to receive the camel, making the mount itself very rare, random, and time-consuming to find – even if you know exactly how to find it.

I got WoW access and was like, what can I do with this?

“I got WoW access and was like, what can I do with this?” Feasel recalls. “I’m going to try adding 50 spawn points, will it let me? And then I tried to put the 51st spawn point in and I got an error and that’s when I discovered there were limits to the things that I could do. So then I put it on a crazy timer so that it would still be really difficult to get. And then I was thinking, you know, what else can I do with this? I’ll teleport you up to the steam pools and make you fight a guy. Why not have him carrying around camels and throwing camels at you? Because, of course, why not?

“I sent the email off to the WoW directors at that point. So proud of myself though. Very first thing that I had added to the game and I’ll never forget, I got an email back from, I think our lead class designer at the time, Greg Street, saying, ‘Hey, this is exactly the kind of stuff that makes World Warcraft a great world to explore.’ And I think that was probably the thing that kicked off my desire to do more of those.”

Dormus and his camels were one of World of Warcraft’s earliest efforts at deliberately hiding these kinds of secrets. Since then, Feasel and his colleagues have tried to add more secrets, and specifically ones that dataminers couldn’t find. Feasel tells me they “tried every trick in the book”, recalling hiding pages around the world in Battle for Azeroth with riddles on them. Dataminers could easily datamine all the riddles, but the riddles themselves would still have to be solved.

“We discovered that 50,000 people can go and look at everything in World of Warcraft in a week pretty effectively,” Feasel says. “It doesn’t actually take them that much time to canvas the entire universe. And that was my first learning of, ‘Oh, I’m going to have to do better than this. I’m going to have to try harder to make each individual piece more difficult to figure out.’”

Over time, WoW’s penchant for riddles sparked a vibrant secret-finding community within the game. There’s a secret-finding Discord server, and communities like Wowhead routinely track and celebrate potential secrets, hints, and their eventual solutions. With so many people working constantly to solve the riddles Feasel creates, he’s had to get increasingly creative. For instance, when trying to conceal the Lucid Nightmare mount from players, he “added a bunch of items and a bunch of quests that just went nowhere and did nothing.” And more recently, Feasel worked on an in-game event focused on treasure hunting, entitled Secrets of Azeroth. He hopes the event served as “an on-ramp” for players who loved the idea of secrets but didn’t usually have the time or interest in joining secret-finding community groups.

Like Feasel, Ben Cureton has embraced the chaos of player secret hunting in his work on Remnant 2. As the principal designer on the franchise, Cureton says he and his team’s job is to make “every toy that the player plays with,” including guns, archetypes, rings, and so forth. While Cureton loves to hide goofy breadcrumbs for players, he has also struggled with dataminers over the years. He recalls a situation years ago in the first Remnant, where dataminers uncovered an item called “Toxic Juju” that they were certain had to be some secret. In reality, Toxic Juju was just a leftover test item that didn’t lead to anything and never was intended to. But the brief fervor over its inclusion gave Cureton an idea. What if he hid something in Remnant 2 that only dataminers could find?

Which is how Cureton and his teammates ended up hiding an entire class – the Archon – behind datamining.

Cureton recalls sitting in the “Cult of the Door” Discord, the group of secret finders trying to crack the mysterious red door that hid the class, and watching them unravel it together. It took them roughly 28 hours, he says, and he personally felt accomplished when they didn’t get it within the first hour. But he was also glad it didn’t take them days or weeks. “Our goal is never to antagonize the player,” he says. “We’re not trying to do anything other than be a game, one of the games that have ridiculous secrets.”

In Remnant 2’s case, Cureton and his team’s experiment with datamining paid off. Players had a great time, and the developers were inspired to find even more creative ways to surprise their community in the future just by watching the players progress.

“We get to participate in watching somebody else play the game and watching the community come together to try to solve puzzles because we never get to experience Remnant like a player,” he says.

Rabbit Holes

Andrew Shouldice tells me he wasn’t thinking too hard about how to hide secrets from dataminers when he first started working on his single-player, secret-stuffed game: Tunic.

“If someone is interested in playing your game, they’re probably not going to look stuff up,” he says. “Think of Ace Attorney, right? You’re playing an Ace Attorney game, you start the game and you think, ‘Well, I should look up the whole script and read the solution to everything.’ No, you probably don’t want to do that. If the way you want to enjoy the game is to play it and figure things out yourself, then you’re probably going to do that.”

But even with that belief, Shouldice inadvertently designed a game that turned out to be pretty hard to crack. He tells me he used “a piece of technology that’s part of Unity called IL2CPP, which takes the intermediate language that C# is compiled into and then mangles it a bunch and turns it into C++ code.” As a result, he says, the whole game “is a bit more opaque.”

Beyond the code, Tunic’s very design naturally evades reading files for solutions or secrets. Much of its puzzles rely on collecting manual pages throughout the game, and then solving a secret within the images themselves. A dataminer could conceivably see all the pages without collecting them, but they still needed to understand the manual’s cryptic notes and esoteric sketches. And still further beyond that, some of Tunic’s deepest puzzles rely on understanding the game’s mysterious, in-universe written language, and its even more mysterious musical language.

“Those assets are in the game, so people will probably find that first by blowing it open,” Shouldice says. “You have access to the music just fine, but digging around isn’t going to give you what you’re looking for. The puzzle is in figuring out the information that you have.”

Taken together, Shouldice and his colleagues managed to design Tunic in such a way where players could, in theory, datamine out all the scenes in the game and “experience probably some amount of joy, but I’ve never heard of someone doing that because that’s not fun.”

Still, Shouldice wanted to tease players who were determined to go the extra mile for secrets. There are a number of locations in Tunic that cannot be reached by most people playing normally, but can be snuck into via various exploits. There, Shouldice planted little signposts that produced a single word in Tunic’s in-universe text. When translated, the signpost simply read “egg.”

Billy Basso’s game, Animal Well, is often mentioned in the same breath as Tunic due to its penchant for secrets. But while Shouldice wasn’t worried about datamining in his early development, Basso recalls hearing lots of defeatist attitudes from other developers about datamining over the years that colored his impression for a while. He says he was told by others to not even bother trying to conceal elements of games he made from players. Everyone would just find it anyway.

I wanted to recreate that old school experience pre-internet of people word of mouth trading tips about a game.

So, naturally, Basso found a way to program his game not to allow datamining at all.

“I kind of wanted to push back against that and see what it would take to recreate that old school experience pre-internet of people word of mouth trading tips and their experiences about a game, and to recreate that sense of mystery and wonder that I feel like I remember games having growing up,” he says.

Basso explains that he programmed Animal Well in C++ in his own custom engine, for which there are no existing tools to unpack the files. “You have to get serious about really reverse engineering the machine code,” he says. “For my assets, I don’t use any text or strings. If you go through the animations or the code, there’s nothing you can read to make sense of it. It’s all just numbers and machine instructions.

“Then for the really serious stuff, the files are actually encrypted with AES encryption, which should in theory be unbreakable. It’s what you would use to encrypt network traffic or something. Typically that still shouldn’t be that secure because you need the key to decrypt the files, and you have to put the key in the code so it knows how to decrypt it. But instead, the key is actually input through your actions in the game. You the player have to indirectly enter the key into the world, and then it will be able to unlock things. It’s easier to just play the game and figure out the puzzles than to hack it.”

But Basso didn’t just stop at trying to prevent dataminers. Animal Well is fundamentally a game about discovering secrets, and he knew players would use any means necessary to crack its most challenging puzzles. Guides, speedrunning tricks, you name it, Basso tried to prepare for it.

For instance, one of Animal Well’s late-game treasure hunts involves finding 16 well-hidden rabbits. But Basso hid additional bonus bunnies in completely inaccessible areas that, if the player manages to collect by tricking the game somehow, will lock them out of the actual 16-bunny reward. Elsewhere, Basso included a room that’s visible in a normal playthrough, but isn’t accessible without clipping through walls or using other tricks. When you enter, you’re rewarded with a “Cheater’s Ring” that allows your character to wallclip freely.

Basso’s efforts to prevent datamining largely succeeded, but his secrets were still cracked within a matter of days thanks to the power of jolly cooperation. When I interviewed him, Animal Well was on the cusp of release, and Basso believed his most difficult puzzles might elude the community for weeks, months, maybe even years. Then, Animal Well launched, and Basso’s weirdest secret was found roughly within a week thanks to a diligent, secret-hungry community working round the clock. He’s taking it in stride, though.

”It does let me know that, in a future game, there’s kind of no limit to how obscure you can make something,” he told Game File.

Inscrutable Quantum Devices

Jeff Hamilton is currently a game designer at Riot Games, though when I spoke to him he was clear that we were only speaking about his past experiences working on both World of Warcraft and League of Legends: Wild Rift. Hamilton is a veteran dataminer dodger, and he’s got some theories, from a developer perspective, about why players love datamining so much. Especially in live service games, players “want to know everything as soon as they possibly can so that they can make all the best decisions about all their resources.” For them, “information is power.”

“They want to understand how they’re all put together, and there’s this conflict sometimes also with the games being so permanent that they want to feel like they don’t make any mistakes,” he explains. “There’s a need for datamining to…comfort themselves with a sort of security blanket, like an assurance that this guess they have about how they want to build their character is actually correct, defensible, playable. And that’s a pattern that we’ve seen across many, many games over the years.”

Hamilton tells me that during his time working on World of Warcraft, there was a good rule of thumb for everyone: if you’re making something on your machine and you hit “Save,” players are going to see it. That’s been true for years, he says, going all the way back to the heyday of Everquest where fansites would just datamine and list every single thing in the entire game: every item, every spell, every location, everything. Now, those databases still exist, but players often rely on influences and theorycrafters to help them make sense of the massive dumps of information, which in turn helps it spread even further.

Even with players combing over everything, Hamilton says he never felt datamining took the wind out of his sails on something he was actively making. “But I do think probably that the existence of data mining changed what I would or wouldn’t make,” he adds. He tells me about working on Patch 8.2, Mechagon, and wanting to make a particular puzzle. But he recalls deciding not to do it, because the only way he could make it fun for the entire playerbase would require “six times as much time as I would have needed if I didn’t have to worry about all the pieces being datamined.”

But in another case, Hamilton was actively able to use datamining to playfully confuse secret hunters. He recalls working on an item called the “Inscrutable Quantum Device,” a trinket with a use listed in-game as “???” His hope was that players would simply try it out and see what happens, without expectations.

“So the way that I made this was to make one spell that had this tooltip with the question marks and then to make a bunch of other spells that it would determine, by some logic, which to actually cast. I was able to give it 13 different effects that kind of hinted at what the logic might be because I knew that players’ first step in figuring out how this would work would be to look up the item. And then their second step would be to look up the same name and they would see that there are 14 spells that are all named Inscrutable Quantum Device. Interesting. So we could get away with this sort of obtuseness that might otherwise be super frustrating because I knew that players had the tools to put the puzzle together.”

World of Warcraft is big enough and well-resourced enough to carefully walk the line between supporting the player desire to know everything while still concealing its biggest secrets. Over the years, the team has used a number of tactics to conceal secrets, such as waiting to implement certain items or cutscenes until launch, using techniques to mask specific dialogue from curious dataminers, or even just making statements around PTR patches indicating that content was not final. But not every game has that luxury, says Hamilton. When he worked on Wild Rift, for instance, there were only 40 people working on it, and there wasn’t time to build an engineering solution to datamining. They were too busy making sure matchmaking worked. Many other games are in this same predicament.

I don’t think there’s any point in getting mad at people for following the impulses of what is interesting to them.

While Hamilton thinks datamining is cool and fundamentally comes from players enjoying and wanting to understand a game better, it does sometimes come into conflict with what developers want to do with said game. He describes players as effectively split into two camps: those who want to understand everything, and those who want to explore and experience new things. Sometimes, the desires of the first group conflict with the desires of the second group, especially when it comes to datamining. That’s just a reality of game development, he says, and one that game developers have to take into account.

“I don’t think there’s any point in getting mad at people for following the impulses of what is interesting to them and stuff like that,” he says. “…Sorry, you’re making games for humans and that’s just part of the deal. But I do think it would be interesting for dataminers to sort of reflect on or be introspective about the times where the thing they’re making for the audience that wants categorization can ruin the experience for the audience that wants novelty and wants exploration.”

Datamining can be a frustrating reality that game developers often have to work around, especially when trying to hide interesting secrets or puzzle solutions that are intended to take extra effort to solve. And it can present interesting challenges to developers of live service games in particular, who may need to field community feedback on changes that aren’t ready for primetime.

“The Internet moves so quickly nowadays,” Feasel says. “There’s a couple of things that I think that we’ve changed up in just our general process that I think have helped make it more of a fundamentally positive thing. Something really exciting to see is [the day a new patch gets datamined], where all this stuff shows up and all the really cool models show up and they get to see all these interesting changes and interesting systems and stuff like that. That’s like one of my Christmases. I get to read the comments and see what people are liking about everything. I think it’s same thing for just about everybody on the team.”

Like Feasel, all those I spoke to gracefully accepted datamining and its challenges as a sign of a community that loves their games and wants to engage with them. That, they all say, is ultimately a positive.

“We made something for them,” Cureton says. “We love that the community enjoys finding stuff and we love people working together. So that’s all we really care about in the end.”

Rebekah Valentine is a senior reporter for IGN. Got a story tip? Send it to [email protected].

About Post Author