Mailbag: Academic literature on modeling conversations

hello :) I was reading some of your stuff about conversation systems, was wondering if you’d have any links/pointers to academic literature on modelling conversations (can be more theoretical/non-game-related), or stuff relevant to people who might be trying to do it?

This one came in via Twitter. I’ve covered some adjacent topics before on mailbag, including

  • Games that do complex conversational mechanics
  • Dialogue and story generation techniques (Parts 1, 2, 3)
  • Dialogue filtering to apply personality and emotion to existing text — this includes links to some academic research into how personality traits affect people’s utterances
  • And back in 2009 I wrote this on conversational analysis and how it applied to my work at the time, including going through a number of dialogue situations recorded in literature and talking about how the conversation model I was using at the time would address or fail to address those

But this question is asking something a little different, specifically about how conversation is modeled in the abstract, not necessarily in games and not necessarily for AI production purposes. What academic literature is out there to help us understand how people talk to one another? What types of approaches exist for modeling conversation in general?

Unsurprisingly, this is a huge field of study, so this is not remotely a literature review; instead, it’s a tour of a few pieces of terminology and resources that might be useful in digging deeper.

Also, I am not approaching it primarily from the perspective of a trained linguist (I’ve taken a few classes, but it’s not my field) and instead from the perspective of a person trying to model things for interactive conversation purposes.

So, with those caveats:

Continue reading

Emotional Mechanics

[Most months, I devote the second Tuesday of the month to a mailbag post that answers a question someone’s sent to me. I’m still doing one of those in March, but it will come out in two weeks’ time.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

restless2

In January, I published a talk about conversation as gameplay, and in particular my ECTOCOMP game jam piece Restless. There, I talked about using Character Engine to give the player more expressive ability. They can tune their dialogue to be open, scary, vulnerable, aggressive, playful; they can pick input topics that they want to focus on.

I’m now revisiting Restless to address some of the flaws I identified in that piece, to give it a better tutorial introduction, polish its use of dynamic text, and add some depth to the content. Tea-Powered has created some additional art, so we can better signal gradations of character emotion. And I’m also refining some of the mechanics under the hood, particularly those that deal with gradual changes in character emotion.

What I’m going for in terms of the design:

  • perceivable consequence:
    • all of the player’s actions should have some effect on the outcome of the story, even if just by moving stats in one direction or another
    • the NPC’s emotional state should be apparent to the player, communicated through art, words, and actions
    • some areas of the NPC’s emotional state space should trigger narrative progression, by having the character get angry, leave, reveal new information, go back to sleep, etc
  • intentionality:
    • the NPC’s reactions should be consistent and predictable enough that the player can choose what to do and be rewarded by an expected outcome, even if there is an additional unexpected consequence
    • the player should have some warning when the NPC is approaching a state that will trigger a significant reaction
  • hysteresis:
    • the NPC should not rapidly cycle back and forth between two emotional states in response to opposed actions from the player; the history of interactions should matter
  • varied narrative intensity and pacing
    • not all player actions should be of equivalent efficacy; some actions should be more important, memorable, or high-stakes
    • it should not be possible to achieve an important result by grinding repetition of an unimportant action

That last point possibly deserves a little bit of expansion. One of the issues with stat-based relationship tracking in games is that it can mean that you could do a lot of banal minor services for a character, and work your way to a point where you’ve maxed out your affinity and now they want to marry you.

This is bad for multiple reasons. One, it’s unrealistic; humans don’t work like that. Two, it’s boring gameplay and boring story. Doing minor favors over and over feels like a chore, from a gameplay perspective. And narratively, there’s no sense of rising stakes, or risk, or the thrill of the relationship becoming more intense.

One way to resolve this would be to throw away the consequences of player action after a certain point. If you’re already at 50% affinity, paying one more compliment to the character does nothing at all. Now you can’t grind your way to romance through compliments: problem solved! Except that this goes against the earlier design principle that all player actions should have consequences of some kind.

So, instead, what we have to do is turn off the player’s ability to do minor actions when we’re in major consequence territory. Compliment away… while your relationship is still in trivial flirt zone. But work your way up to where things are getting serious, and the available affinity moves also get more serious, more freighted. And just having those “bigger” moves popping up in the choice menu should communicate to the player, along with the character art, that we’re in a more important situation.

Fortunately, all of that is pretty doable, by restricting dialogue lines to particular ranges of the emotional space.

This work is still in progress — I’ll post when the new version of Restless is generally available.

Natural Language Understanding for Characters

[This post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Character Engine is designed for two kinds of interaction: it can either generate input options for the player (as seen in Restless), or it can take natural language input, where the user is typing or using a speech-to-text system.

screen shot 2019-01-20 at 1.02.02 pm

That natural language approach becomes useful for games in AR or VR where more conventional controls would really get in the way. It’s also good for games that are using a chatbot style of interaction, as though you were chatting to a friend on Facebook; and a lot of interactive audio projects deployed on platforms like Alexa use spoken input.

There are plenty of business applications for natural language understanding, and a lot of the big tech players — IBM, Google, Amazon, Microsoft, Facebook — offer some sort of API that will take a sentence or two of user input (like “book a plane to Geneva on March 1”) and return a breakdown of recognized intents (like “book air travel”) and entities (“March 1”, “Geneva”). If you prefer an open source approach over going to one of those companies, RASA is worth a look.

For people who are used to parser interactive fiction, there’s some conceptual overlap with how you might set up parsing for an action: we’re still essentially identifying what the command might be and which in-scope nouns plug into that system. In contrast with a parser, though, these systems typically use machine learning and pre-trained language models to allow them to guess what the user means, rather than requiring the user to say exactly the right string of words. There are trade-offs to this. Parsers are frustrating for a lot of users. Intent recognition systems sometimes produce unexpected answers if they think they’ve understood the player, but really haven’t. In both cases, there’s an art to refining the system to make it as friendly as it can be, detect and guide the user when they say something that doesn’t make sense, and so on.

Some natural language services also let the developer design entire dialogue flows to collect information from the user and feed it into responses — usually interchanges of a few tens of sentences, designed to serve a repetitive, predictable transaction, and lighten the load on human customer service agents.

Those approaches can struggle a bit more when it comes to building a character who is part of an ongoing story, who has a developing relationship with the player or user, and who uses vocabulary differently each time.

That’s exactly where Character Engine can naturally provide a bit more support. We’re tracking character knowledge and personality traits. We’re doing text generation to build output that varies both randomly and in response to lots of details of the simulated world. And because dialogue in Character Engine is expressed in terms of scenes of interaction, making it easier to maintain large and complex flows where the same input should be understood differently at different times.

We don’t want to be in a race with the providers of existing NLU solutions, though. What we want to do is facilitate author-friendly, narrative-rich writing options that tie into whatever happens to be the current state of the art in language understanding. So here’s a bit about how we do that:

Continue reading

Memory and Knowledge for Characters

[For a couple of years now, I’ve reserved the first Tuesday of the month for a review of a book on writing or game design that might be of interest to IF folks. I’m still doing one of those in March, but it will come out the 19th, while I’m at GDC.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Knowledge and memory are a somewhat vexed area for game characters. It’s easy to think of characters who don’t remember the last fifty times you asked them the same exact lore question, or are strangely forgetful about the ways you’ve harmed them, or who aren’t equipped to answer common-sense questions about the world they live in.

So why is this a problem? Simply recalling that something has happened is not the main challenge. We can set flags; we can assign variables; we can check on quest journals to see what the player has already done. We can refer back to whatever data store is otherwise tracking world state in this game.

The hard part is building a system where

  • everything important to remember is stored in a reasonably systematic way
  • differences between world truth and character knowledge are handled as much as (and no more than) useful
  • there’s a way to track and author for the combinations of possible state so that the NPCs always have something to say about what they remember and know

There are quite a few technical, design, and writing challenges packed into those three bullet points.

Continue reading

Conversation as Gameplay (Talk)

[Yesterday I gave a talk at the Oxford/London IF Meetup. The session was about conversation as gameplay, and also featured Flo Minuzzi of Tea-Powered Games, speaking about their released game Dialogue and their upcoming Elemental Flow. There’s a nice livetweeted thread version of my talk available on Twitter thanks to Florence Smith Nicholls, but I promised also to make a blog post about what I said.

Because the talk was written for an audience that included students, game designers from other parts of the industry, and newcomers to interactive fiction, I included some history of my own work that may be redundant for readers of this blog; there’s also some overlap with a talk I gave in Warsaw last September. However, the material towards the end of this talk is largely new.]

*

The Problem Statement

I want more games to be about human interaction, about the nuances of how people deal with one another, about the kinds of topics that appear in dramatic movies. That’s partly because I’d like to play more games about conversation and social interaction. I’m not as interested in action as a topic, and to be honest I often fall asleep during superhero movies these days.

Meanwhile, as an artist, part of the reason I write games is to explore and interrogate things I don’t yet fully understand. Building procedural systems and seeing how they perform is a great way to explore whether our mental models are correct. How people understand each other (or don’t), how they connect and why, are topics of enduring fascination for me.

So I want more conversation-rich games. For that to work as I’d like, the conversation needs to be rewarding as gameplay — not just bolted on around gameplay, as it so often is.

When it comes to my own work, I have a few more ambitions and requirements as well:

Screen Shot 2019-01-19 at 10.31.44 PM.png

First, I want it to allow the player to act with intentionality: to lay plans and carry them out. That means that we need some systematic mechanics that the player can learn and manipulate.

For the purposes of this talk, I’m not spending much time on things that are pure branching dialogue trees without ongoing state or clear mechanics. I’ve sometimes written work in that space, and if you’re interested in how to get the most out of a relatively state-light dialogue presentation, I recommend having a look at Jon Ingold’s AdventureX talk about writing sparkling interactive dialogue. But that’s not what we’re looking at today.

[I’ve written more about world model and systematic mechanics for conversation elsewhere.]

Second, I want the resulting mechanic to have good pacing and dramatic qualities — so a mechanic that systematizes conversation but makes it feel very slow, stilted, metaphorical, or hard to manipulate is not what I’m looking for. Some of these can be cool to play, but I myself tend to be looking to write something that has a bit more fluidity.

Continue reading

Restless

AnnaScreenshot.png

So I released a new game! Here’s its blurb:

You’ve been haunting old Mrs Fagles for decades. Now she’s sold the house, and the new owner’s moved in. Sylvie’s broke, bad at plumbing, and anxious about everything.  And with a living, breathing, fretting roommate, how are you supposed to rest in peace?

Drink blood. Set fires. Tell lies. Give advice, loan out a wedding dress, reclaim your true name.  Remix your dialogue options to reflect your mood or dig deeper into the topics that interest you.

I mentioned this briefly in yesterday’s link round up, but I wanted to give a little more background on it than a link round up typically allows for.

Restless is a game written for ECTOCOMP, a venerable Halloween-themed IF competition. There are six endings, if you’re counting — though some of those endings mean different things depending on how you get to them.

It’s a purely conversation game. As in a lot of choice-based games, you have up to three options, and you can pick one. But in contrast to the typical dialogue situation, you can do something about it if you don’t like your current menu. Click a mood, and your options will shift to reflect that new attitude. Turn on moods individually or in combinations. Discover conversation topics and you can set your dialogue to explore those too.

Continue reading