Creating a video game demands hard, repetitive work. How could it not? Developers are in the business of building world, so it’s easy to understand why the games industry would be excited about generative AI. With computers doing the boring stuff, a small team could whip up a map the size of San Andreas. Crunch becomes a thing of the past; games release in a finished state. A new age beckons.
There are, at the very least, two interrelated problems with this narrative. First, there’s the logic of the hype itself—reminiscent of the frenzied gold rush over crypto/Web3/the metaverse—that, consciously or not, seems to consider automating artists’ jobs a form of progress.
Second, there’s the gap between these pronouncements and reality. Back in November, when DALL-E was seemingly everywhere, venture capital firm Andreessen Horowitz posted a a long analysis on their website touting a “generative AI revolution in games” that would do everything from shorten development time to change the kinds of titles being made. The following month, Andreessen partner Jonathan Lai posted a Twitter thread expounding on a “Cyberpunk where much of the world/text was generated, enabling devs to shift from asset production to higher-order tasks like storytelling and innovation” and theorizing that AI could enable “good + fast + affordable” game-making. Eventually, Lai’s mentions filled with so many irritated replies that he posted a second thread acknowledging “there are definitely lots of challenges to be solved.”
“I have seen some, frankly, ludicrous claims about stuff that’s supposedly just around the corner,” says Patrick Mills, the acting franchise content strategy lead at CD Projekt Red, the developer of Cyberpunk 2077. “I saw people suggesting that AI would be able to build out Night City, for example. I think we’re a ways off from that.”
Even those advocating for generative AI in video games think a lot of the excited talk about machine learning in the industry is getting out of hand. It’s “ridiculous,” says Julian Togelius, codirector of the NYU Game Innovation Lab, who has authored dozens of papers on the topic. “Sometimes it feels like the worst kind of crypto bros left the crypto ship as it was sinking, and then they came over here and were like, ‘Generative AI: Start the hype machine.’”
It’s not that generative AI can’t or shouldn’t be used in game development, Togelius explains. It’s that people aren’t being realistic about what it could do. Sure, AI could design some generic weapons or write some dialog, but compared to text or image generation, level design is fiendish. You can forgive generators that produce a face with wonky ears or some lines of gibberish text. But a broken game level, no matter how magical it looks, is useless. “It is bullshit,” he says, “You need to throw it out or fix it manually.”
Basically—and Togelius has had this conversation with multiple developers—no one wants level generators that work less than 100 percent of the time. They render games unplayable, destroying whole titles. “That’s why it’s so hard to take generative AI that is so hard to control and just put it in there,” he says.
A technique analogous to generative AI will be familiar to a lot of gamers: procedural generation. Togelius says, for argument’s sake, that he would be happy to say procedural generation is the same as generative AI (he describes their connection as “kind of a sliding scale”). But procedural generation typically doesn’t use machine learning. Rather than an AI model, it runs on predetermined equations, generating, for example, the gargantuan cosmos of No Man’s Sky. Developers also use software like SpeedTree, which, as its name suggests, conjures forests. The point is that procedural generation systems still require massive human supervision; developers must keep vigilant for unscalable crevasses or monstrous trees. And it’s not even clear that replacing procedural generation with generative AI right now would make a noticeable difference.
“These things already exist,” says Togelius. “And it works because this content doesn’t really need to function: It doesn’t have functionality constraints. Maybe you can replace them with deep-learning-based stuff. But I don’t think it’s going to make a big difference. Perhaps it will make some difference in the long run.”
There is a general misunderstanding of where the tech is at, explains Mills. “A fundamental reason why these generative AIs can’t make something like Night City is because these tools are designed to produce specific outcomes,” says Mills. “A lot of people seem to be under the impression that these are somehow close to general intelligences. But that’s not how it works. You’d need to custom-build an AI that could build Night City, or open world cities in general.”
There’s also a failure to take into account the corporate landscape. Games still employ systems that grew from early technological limitations, like dialog or behavior trees. You can’t just drop fancy machine learning into game franchises that have developed without generative AI in mind. Games—in an industry with huge budgets and tight margins—would need total redesigns to accommodate and take advantage of this technology.
Take, for example, non-player characters. Text-based generative AI tools seem like a great way to deepen conversation, and Togelius has been advising developers intrigued by this very idea. But it’s not that simple. Characters based on these language models are liable to go off on tangents, discussing topics outside of the game’s world. “This is super interesting, but it’s also super hard,” says Togelius. “You can’t just drop it in there. It’s not going to work. You can’t expect the NPCs to behave in Skyrim or Elden Ring or Grand Theft Auto or your typical RPG. You have to design around the fact that they are, in some sense, uncontrollable.”
Nevertheless, there are some peripheral uses for generative AI right now. A good rule of thumb—one that applies to procedural generation too—is that the less crucial the content is, the more likely deep learning methods could be helpful. “For things like text generation, I could use this today to help generate filler for assets that aren’t really meant to be the focus of the player’s attention, like prop newspapers and such,” says Mills.
Another appeal is these tools’ low barrier of entry, says Adrian Hon, the CEO and founder of independent games developer Six to Start and the co-creator of Zombies, Run! Procedural generation, at least as the term is typically understood, requires a coder; anyone can use tools like Midjourney and Stable Diffusion. He can see how they could help with prototyping or mood-boarding during a game’s early concept phase.
But, Hon notes, many artists are skeptical of AI. Part of the backlash to the generative AI hype has been that these tools are modeling their output on the work of human creators. Some are even suing Stable Diffusion and Midjourney, claiming that Stable Diffusion, which powers Midjourney, was trained on images used without permission. “Obviously, there’s a whole copyright question. We know about all these suits going on,” he says. “But even if they get resolved, I think that there’ll be some real upset among artists, which is understandable.”
As with so many discussions about automation, the hype here is detached from current reality (debates over automation usually arise during times of “deep anxiety about the functioning of the labor market,” writes sociologist Aaron Benanav). But, leaving reality for a second, it’s notable that much of the conversation around generative AI seems almost to revel at the prospect of replacing humans. Even an innocuous statement promising a boon for indie developers—“A small team can make a world the size of Red Dead’s,” for example—contains a kernel of this logic, explains Raphael van Lierop, the founder and creative director of independent studio Hinterland. It’s reductive, suggesting the work of a large part of that large team is mindlessly robotic.
“The focus on generative AI is another facet of what feels like an attack on creators and the act of creation, one that is expressed in a lot of different ways in our society right now,” he says. Reflecting a prevailing mood among artists across mediums, he sees nothing interesting about art made by an AI. “It’s a dead end,” he says.
There’s definitely an unsettlingly inhuman element to all of this, one that you could imagine manifesting as a torrent of AI-generated shovelware run on predatory monetary systems. But at the higher echelons of game development, games created entirely by machines—ones worth playing, at least—are some way off. “The way some people say it’s going to be used, to just suddenly replace people and do the whole job by itself, is bullshit,” says Togelius. “You need humans.”
Read More
Will Bedingfield