Modders are using AI to create chatty companions in Skyrim and Stardew

In Stardew Valley, you can usually only speak to characters a few times per day. This limitation has led to a number of mods that add things for them to say so that players can spend more time with their virtual pals. But like many games, Stardew recently turned to AI, resulting in the addition of a mod with the ability to have theoretically endless conversations with your favorite farm companion.

“The reception has been a lot better than I initially thought [it would be],” says modder DualityOfSoul. In the comments on Nexus Mods, users have called it “brilliant” and “one of the best mods out this year.”

To function, the mod plugs into OpenAI’s API. The implementation is impressive but limited in its own way. Every character takes on a similar jovial personality that I’ve come to associate with the built-in parameters of these large language models. Grumpy camper Linus first tells me bluntly to leave him alone; this is his prewritten dialogue since he hasn’t warmed up to me yet. But click again, and I get the OpenAI dialogue — three long text boxes run one after the other. He calls me his friend and says he hopes I’ve “been finding peace in the beauty of nature.” Next, Pam complains that “every day is the same routine” before changing her tune to say she hopes “you’re enjoying the season as much as I am.”

The responses are somewhat tailored to the characters, and it works better when I’m not talking to grouchy NPCs. But the mod can’t easily escape that underlying voice. Stardew is a largely cheery game, especially after you’ve befriended everyone, so I can see how it can fit into the vibe for some players. But it’s a noticeable limitation.

That limitation is less present in Herika, an AI companion mod for Skyrim. Herika acts like most Skyrim companions, following the player and helping out in combat and elsewhere. But she will also respond to conversation, both written and spoken. The LLM can draw from an understanding of the game’s map, quests, and key features and be assigned a personality. This means that she can be prompted to use a different style from the standard LLM voice, at least to a point.

Reece Meakings joined the Herika project after seeing the first iteration created by his now-coworker who goes by Tylermaister. The pair have expanded the mod from a tool to summarize the many books of Skyrim into a fully-fledged follower and intend to expand it again into a framework for any NPC to become an AI companion.

Nonetheless, Meakings doesn’t think that AI is ready to be used in game development more broadly. “It’s not going to work right now,” he says. For starters, using Herika or AI Stardew costs money. Both use OpenAI’s API, which charges the user a fraction of a cent per generated dialogue line. From the comments on Nexus, that low-level cost has put off a lot of players who are used to mods being entirely free. (Herika does offer a free option, but this requires running the LLM on your own setup, which is resource-intensive.) On a larger scale, any company will have to deal with the price of connecting to an API or running their own servers, multiplied across every player.

Then there’s the open-ended nature of how people interact with LLMs. “That completely changes how you [would] have to design the game,” says Meakings. Developers would lose linear control of dialogue trees and script triggers, quickly opening up possibilities that they didn’t account for in the narrative or worldbuilding. LLMs might be able to help NPCs keep up, but the rest of the game’s scripting wouldn’t have any way of adapting. Plus, he says, despite the safeguards companies attempt to use, it’s “very easy, if you know what you’re doing” to make these LLMs say things that are NSFW, potentially causing PR problems for companies that modders don’t have to consider.

At the moment, these larger-scale problems mean Meakings sees the most potential for AI in NPCs like Herika, particularly in the mod space. There are also AI NPC mods for games like Hogwarts Legacy, Cyberpunk 2077, and Garry’s Mod. Mantella, another Skyrim mod that gives LLM interactions to every Skyrim NPC, has 30,000 unique downloads. (Herika has 25,000.) All of them have positive feedback — limitations aside, people are enjoying playing with AI-generated content.

Meakings’ own interactions with Herika give an indication as to why. In his experience, “you’ll really end up having meaningful conversations with something which isn’t human.” Although he understands that Herika is just a program, he’s clearly engaged by the fact that he’s been able to interact with her on a different level than previous characters. You don’t need to make small talk with Herika or stick to “ritualistic” social niceties. You can jump right to debating the philosophy and theology of Skyrim. In fact, you might need to because she can judge your actions.

He gives a specific example. Playing as an Argonian, Meakings heard a rude comment about the lizard-like race from a standard NPC. “I [said,] ‘Hey, Herika, can you kill him? Please? He’s being racist to me.’” Herika refused, apparently either because of her own morality or because they were in a place with guards watching, which could have landed them in jail. Then, she started wandering away, talking about visiting the catacombs under the settlement.

At first, Meakings thought it was a bug. “But then I realized, ‘oh, I know what she did.’ …She realized, ‘oh, Reece is very angry. We can’t kill this person because we’re in the middle of a city — you can’t just kill people. I need to get him out of the situation… what if I lure him to his catacomb for treasures where we can get some space away from the situation?’” She seemed to be trying to diffuse things based on the tools available to her.

This kind of experience is not unique to this form of AI. Procedurally generated content creating emergent narratives through a combination of interesting behavior and player-led meaning-making is well documented. But AI mods may open up more possibilities for these kinds of moments.

For that to happen, though, the LLMs need material to work with. Meakings is clear that Herika would not function without the wider framework of Skyrim since much of how she communicates relies on the writing of the original team. In a scenario in which LLMs are used as NPCs in a produced video game, they still could not exist without writers.

The video game performers strike raises crucial questions about how these technologies will deal with digital replications of voice actors, something Nexus Mods treads carefully around. AI mods are allowed, but affected creators, including voice actors, can ask for them to be removed if they feel the “work is damaging to them.” (Herika has multiple options for generating her voice.)

The rollout of AI on the web has been fast and messy, and how it might go in the game industry is still an open question. But thanks to modders, tens of thousands of players are already experimenting with it.