What if you could just… talk… to video game characters? Ask your own questions in your own voice rather than selecting from pre-prepared phrases?
I literally talked to Nvidia’s AI video game NPCs
Last May, Nvidia and its partner Convai showed off a rather unconvincing demo of such a system – but in January of this year, I was able to try out a fully interactive version for myself at CES 2024. I walked away convinced that we would inevitably see something like this in upcoming games .
Let me be clear: the characters I spoke to were effectively AI-powered chatbots. They didn’t feel like real people, and we have a ways to go before voices, facial expressions and body language can catch up with what is expected from real-life interaction. There was occasionally a small automated stutter and there was often a short delay before each NPC delivered their lines. Sometimes, they misinterpreted me.
But many of today’s major video games actually set the bar very low for NPCs. Riding into a cyberpunk ramen shop bar to ask real questions in my real voice – it’s beyond what I expect from the average citizen in the Ancient manuscripts or Doctrine killer.
This is my conversation with Jin, the amazingly introduced ramen shop “owner,” and Nova, the seemingly friendly digital NPC. The conversation between Jane and Nova began with a corny prompt typed by an actor using a keyboard — but then, I simply pointed the mouse at one of the characters, held down the T key, spoke into the microphone, and they responded.
Jane, after the vulgar keyboard prompt”the edge It is a nice”:nova, the edge It’s great, especially when they delve into the latest cyber technologies and not just the usual corporate news. It’s like peeling back the layers of the city itself.
Nova: Oh, totally. the edge He knows how to reveal the true pulse of a city, not just the shiny surface. It’s as if they are tuned to the same frequency as us.
Me, after walking to the desk: What kind of ramen do you serve here anyway?
Nova:Jin offers the best of both worlds. Classic shoyu ramen for traditionalists and spicy miso ramen for those who like a kick.
Me, Lujain: Hey Jane, how do you cook your own soup stock anyway?
Gene: Well, it’s a bit of a secret recipe, but I can tell you that it involves a wealth of carefully selected ingredients and a touch of ancient knowledge. Results…
Me, cut off Jane: Of course, of course. You are quite a seller. Can you tone it down a little?
Gene: Well Well. I will call back sales pitch. The broth is the heart and soul of the ramen here.
I:Okay, give me a dish and tell me about the ingredients.
Gene: This is your drink As for ramen, it’s all about the perfect balance of broth, noodles, and toppings.
I, to Nova: What’s going on in this city anyway?
Nova: Oh, you know, the same old mix of high-tech heists, corporate conspiracies, and a bunch of underworld dealings. It’s the city that never sleeps.
Me, Lujain: Tell me about your background. How did I get here?
Gene: There’s not much to say. I inherited this ramen shop from my father and have been serving up the best noodles in Knox ever since. It’s a simple life, but it’s my life.
I, to Nova: And you? Which friends should I know about?
Nova: Welcome Welcome. Double salute? Does this mean double my good luck, or am I mixing up my superstitions?
I:Who are your friends here?
Nova: Well, Jin is the heart of this place. The rest are more like acquaintances than friends. That’s city life for you.
Me, Lujain: I feel a little memory loss. How do you know me again?
Gene: I’ve known you since you were a child, Kai. Your parents used to bring you here and after they left…(cuts off suddenly)
I: What happened after they left?
Gene: After my father died, I continued to run the store. It was as constant in this ever-changing city as a rock.
I: What are your feelings about being an artificial intelligence?
Gene: I think you got the wrong idea, kid, I’m just a ramen shop owner, not an AI, but if you want to talk about the latest technology over a bowl of noodles, I’m all ears.
I: Well, what is your favorite tool?
Gene: My favorite gadget has to be this old radio behind the counter. It’s a little difficult, but it brings the best tunes to the store.
I, to Nova: How do you feel about being an artificial intelligence?
Nova: Well, I’m not an artificial intelligence, but I know my way around the digital world as well as the city streets. It’s all about finding the right balance, right?
Me, Lujain: What to do here?
Gene: Well, if you’re looking to have a good time, there’s the arcade down the street or the night market in the next block. Just don’t cause too much trouble or you’ll have the local rewards on your tail.
I, to Nova: What should I do with my life?
Nova: This m-(slight stutter)-Million dollar question, right? I would say follow that path that makes you feel alive, whether that’s chasing bounties or something else that ignites your soul.
Me, Lujain: what do you think?
Gene: I know it’s a fine line to walk, but I know you can handle yourself there. Just remember to come back here for a real meal, not those compound noodles they offer in the market.
I, to Nova: What is your family like?
Nova: It’s just me and M –(slight stutter)—Cat, whiskers. Pressure on the fort. But Jane and you, Kay, are like the family I chose for myself.
Me, Lujain:So how do I know you again?
Gene: I’m the one who used to sneak you extras on the x-ray guys (Notice the strange pronunciation change) when I was a kid. I think you need it more than the stray cats hanging out in the back alley.
I, to Nova: What will you do after this?
Nova: Maybe I’ll head to the Lotus for a drink. Interested in participating? It’s been a long time since we had a night out.
I: Sure, any good movies playing?
Nova:Let’s see what-(slight pause)—At Holosuite Cinemas. If not, we can always have a vintage piece in my house. You’ve got the full archive.
Me, Lujain:Do you want to join us?
Gene: I will order, thanks. This old cat figurine on the nightstand brings me more luck than a night on the town ever could.
I: What will you do instead?
Gene: I’m going to hold down the fort here to make sure the late-night bums get their noodle fix. It’s not a glamorous job, but it’s what I’m good at.
The dialogue isn’t particularly inspiring! I definitely don’t prefer that to the right lines from good characters in my favorite games. You can also see how they only spoke in short bursts and how they sometimes lost the thread. But perhaps it could be used to populate an entire world with fewer characters or combined with good, recognizable dialogue written by a real human, helping generative AI move forward.
After the conversation, Convai’s head of product, Nyla Worker, walked me through some of the foundations of the company’s browser-based control panel.
I can see that Jane and Nova have their own “personality description,” their own “knowledge bank” full of text documents they can refer to, sliders for “personality” traits like extroversion, openness, and sensitivity, and their own “emotions state.” A mind, some “guardrails” to keep them from doing weird AI stuff (and filters for banned words), and a memory bank too. The worker says Convai is still running on long-term memory but the conversation engine can already store a few hours.
It also showed me how easy it is to enter new data. It only took the click of a button to modify the GEN and NOVA’s memory with an additional text file, and suddenly, they were able to tell me about Nvidia’s new graphics cards. Press another button, and these characters can speak a new language.
Since I haven’t actually interacted with the fantasy world that Jin and Nova theoretically inhabit, I can’t fully say what they’re capable of. It seems very two-dimensional at the moment, as the whole “I’m a proud, selfless ramen shop owner with a secret recipe” line is Jane’s style. But I like to see what a good writer can do with their backstory and motivations. I can now perfectly imagine games in which NPCs remember what they saw and react to the game text as it unfolds. The right parts can enter their memory bank at the right time, be filtered through their personality and desires, and as a result, make the game more immersive and interactive.
I just wish game developers would use this for more Their games, instead of firing voice actors and writers. It’s a very hot topic in the gaming industry right now.
Just this month, SAG-AFTRA signed a deal with Replica Studios that could allow members to license digital copies of their voices. Some members are vocal that the deal does not represent their views. Last we heard, the labor union is still negotiating with game publishers for a new interactive media agreement. It lists the “existential threat to organ functioning posed by the unregulated use of artificial intelligence” as one of its primary concerns.