Mailbag: Knowledge-driven Dialogue in Inform

I am doing my first steps in Interactive Fiction and your work has helped me a lot. I have been working on an idea, that requires dialogue based on “knowledge”, in other words, the character and the NPCs will initiate dialogue in order to fill out their gaps in knowing the other person. Firstly, I was wondering if Inform7 can do something like that, and if it can which dialogue system would be the best to serve as a basis. Secondly, I was wondering if Inform7 can implement AI, without falling back to Inform6. Thank you in advance and most importantly, for your work in the community! Sorry for the annoyance of my question but I was kind of lost among the many different dialogue systems that are out there…
[and then on confirmation that the asker was okay with a mailbag post]:

First of all let me clarify that I am not asking for mentorship, with this question. More like pointing to the right direction, if there is such a direction. There is a strong chance that there is not something similar implemented, so in this case, don’t let me take your time.

My question is if there is a dialogue system in Inform7 (or in some other framework) , that is based on knowledge of a predefined set of data. For instance the protagonist to be able to query for any one of that data and to store that information in such a way, that the next answer to reveal even more data. Or the knowledge itself to enable the protagonist to make more specific questions. The answer I would like, is not how to implement such a system, of course, but only a reference of the type  “have a look at this extension of Inform7” or “there is no such thing implemented” or “there is no such thing, but you could draw inspiration from this”. Nothing more than that! I have thought of a potential implementation, treating data as things that are visible or not, and the knowledge to be treated as “possession” of those things, but I am not certain it is the right approach.

First of all, re. the Inform 6/7 question: Inform 7 is a full programming language, and you do not need to drop to Inform 6 to code behavior. In the early years there were things that were hard to express in Inform 7, especially mathematical things or elements that accessed files or manipulated on-screen behavior, but most of those elements do now have an Inform 7 wrapper available. Occasionally people still choose to insert Inform 6 chunks inside an Inform 7 program for various reasons, but it isn’t required.

Likewise, when you say “to implement AI,” this is such a big and fuzzy question that it’s hard to answer without more of a breakdown.

Inform 7 is good — and indeed much better than Inform 6 — for handling rule-based decision-making and firing off character interactions within the model world; the main issue here is performance if you’re driving a large number of characters or asking them to plan over complex world state. The rule-based aspects of Inform 7 in fact have influenced other approaches to game AI in larger game applications, as Elan Ruskin discusses in one of his GDC talks.

For other AI approaches, you’d have to do quite a bit more work; for instance, it isn’t really designed to make use of any natural language processing methods outside of its own rich and complex parsing mechanism, and if you wanted to do something that for instance tried to guess what the player meant by words that weren’t in the game’s dictionary but might be similar to ones that were, you’d more likely use some kind of special pre-processing layer or a call out to an external script, because it doesn’t provide ways to e.g. access WordNet or a word2vec model. Likewise, it’s not designed to plug together with SpaCy or Google NLU or any of the external tools that have come into being over the past decade+ to help interpret the semantic structure of a piece of input. It might be interesting to explore how that would work, but that doesn’t exist currently.

Inform plus machine learning is a slightly more interesting point of conversation, because the TextWorld project exists, and there are researchers who are exploring how to use an Inform-based world model as a sandbox environment for training ML agents to solve a text-based game.

That’s different again from the idea of an ML agent designed not to solve an IF game but to be a companion or competitor within an IF game, for the sake of enhancing the player’s experience. There are relatively few IF games in which it would even really be meaningful to talk about a competitor character within a game, because most IF doesn’t have mechanics designed for competitive play (though Kerkerkruip might work). My old game When in Rome 2 also featured an NPC with dynamically selected characteristics who might work counter to the intentions of the player, and it was possible in some cases to be bested or even killed by this creature, if you went up against a clever one and it got resources ready faster than you could.

Now, on to the main question.

Continue reading “Mailbag: Knowledge-driven Dialogue in Inform”

Mailbag: Academic literature on modeling conversations

hello :) I was reading some of your stuff about conversation systems, was wondering if you’d have any links/pointers to academic literature on modelling conversations (can be more theoretical/non-game-related), or stuff relevant to people who might be trying to do it?

This one came in via Twitter. I’ve covered some adjacent topics before on mailbag, including

  • Games that do complex conversational mechanics
  • Dialogue and story generation techniques (Parts 1, 2, 3)
  • Dialogue filtering to apply personality and emotion to existing text — this includes links to some academic research into how personality traits affect people’s utterances
  • And back in 2009 I wrote this on conversational analysis and how it applied to my work at the time, including going through a number of dialogue situations recorded in literature and talking about how the conversation model I was using at the time would address or fail to address those

But this question is asking something a little different, specifically about how conversation is modeled in the abstract, not necessarily in games and not necessarily for AI production purposes. What academic literature is out there to help us understand how people talk to one another? What types of approaches exist for modeling conversation in general?

Unsurprisingly, this is a huge field of study, so this is not remotely a literature review; instead, it’s a tour of a few pieces of terminology and resources that might be useful in digging deeper.

Also, I am not approaching it primarily from the perspective of a trained linguist (I’ve taken a few classes, but it’s not my field) and instead from the perspective of a person trying to model things for interactive conversation purposes.

So, with those caveats:

Continue reading “Mailbag: Academic literature on modeling conversations”

Emotional Mechanics

[Most months, I devote the second Tuesday of the month to a mailbag post that answers a question someone’s sent to me. I’m still doing one of those in March, but it will come out in two weeks’ time.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

restless2

In January, I published a talk about conversation as gameplay, and in particular my ECTOCOMP game jam piece Restless. There, I talked about using Character Engine to give the player more expressive ability. They can tune their dialogue to be open, scary, vulnerable, aggressive, playful; they can pick input topics that they want to focus on.

I’m now revisiting Restless to address some of the flaws I identified in that piece, to give it a better tutorial introduction, polish its use of dynamic text, and add some depth to the content. Tea-Powered has created some additional art, so we can better signal gradations of character emotion. And I’m also refining some of the mechanics under the hood, particularly those that deal with gradual changes in character emotion.

What I’m going for in terms of the design:

  • perceivable consequence:
    • all of the player’s actions should have some effect on the outcome of the story, even if just by moving stats in one direction or another
    • the NPC’s emotional state should be apparent to the player, communicated through art, words, and actions
    • some areas of the NPC’s emotional state space should trigger narrative progression, by having the character get angry, leave, reveal new information, go back to sleep, etc
  • intentionality:
    • the NPC’s reactions should be consistent and predictable enough that the player can choose what to do and be rewarded by an expected outcome, even if there is an additional unexpected consequence
    • the player should have some warning when the NPC is approaching a state that will trigger a significant reaction
  • hysteresis:
    • the NPC should not rapidly cycle back and forth between two emotional states in response to opposed actions from the player; the history of interactions should matter
  • varied narrative intensity and pacing
    • not all player actions should be of equivalent efficacy; some actions should be more important, memorable, or high-stakes
    • it should not be possible to achieve an important result by grinding repetition of an unimportant action

That last point possibly deserves a little bit of expansion. One of the issues with stat-based relationship tracking in games is that it can mean that you could do a lot of banal minor services for a character, and work your way to a point where you’ve maxed out your affinity and now they want to marry you.

This is bad for multiple reasons. One, it’s unrealistic; humans don’t work like that. Two, it’s boring gameplay and boring story. Doing minor favors over and over feels like a chore, from a gameplay perspective. And narratively, there’s no sense of rising stakes, or risk, or the thrill of the relationship becoming more intense.

One way to resolve this would be to throw away the consequences of player action after a certain point. If you’re already at 50% affinity, paying one more compliment to the character does nothing at all. Now you can’t grind your way to romance through compliments: problem solved! Except that this goes against the earlier design principle that all player actions should have consequences of some kind.

So, instead, what we have to do is turn off the player’s ability to do minor actions when we’re in major consequence territory. Compliment away… while your relationship is still in trivial flirt zone. But work your way up to where things are getting serious, and the available affinity moves also get more serious, more freighted. And just having those “bigger” moves popping up in the choice menu should communicate to the player, along with the character art, that we’re in a more important situation.

Fortunately, all of that is pretty doable, by restricting dialogue lines to particular ranges of the emotional space.

This work is still in progress — I’ll post when the new version of Restless is generally available.

Natural Language Understanding for Characters

[This post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Character Engine is designed for two kinds of interaction: it can either generate input options for the player (as seen in Restless), or it can take natural language input, where the user is typing or using a speech-to-text system.

screen shot 2019-01-20 at 1.02.02 pm

That natural language approach becomes useful for games in AR or VR where more conventional controls would really get in the way. It’s also good for games that are using a chatbot style of interaction, as though you were chatting to a friend on Facebook; and a lot of interactive audio projects deployed on platforms like Alexa use spoken input.

There are plenty of business applications for natural language understanding, and a lot of the big tech players — IBM, Google, Amazon, Microsoft, Facebook — offer some sort of API that will take a sentence or two of user input (like “book a plane to Geneva on March 1”) and return a breakdown of recognized intents (like “book air travel”) and entities (“March 1”, “Geneva”). If you prefer an open source approach over going to one of those companies, RASA is worth a look.

For people who are used to parser interactive fiction, there’s some conceptual overlap with how you might set up parsing for an action: we’re still essentially identifying what the command might be and which in-scope nouns plug into that system. In contrast with a parser, though, these systems typically use machine learning and pre-trained language models to allow them to guess what the user means, rather than requiring the user to say exactly the right string of words. There are trade-offs to this. Parsers are frustrating for a lot of users. Intent recognition systems sometimes produce unexpected answers if they think they’ve understood the player, but really haven’t. In both cases, there’s an art to refining the system to make it as friendly as it can be, detect and guide the user when they say something that doesn’t make sense, and so on.

Some natural language services also let the developer design entire dialogue flows to collect information from the user and feed it into responses — usually interchanges of a few tens of sentences, designed to serve a repetitive, predictable transaction, and lighten the load on human customer service agents.

Those approaches can struggle a bit more when it comes to building a character who is part of an ongoing story, who has a developing relationship with the player or user, and who uses vocabulary differently each time.

That’s exactly where Character Engine can naturally provide a bit more support. We’re tracking character knowledge and personality traits. We’re doing text generation to build output that varies both randomly and in response to lots of details of the simulated world. And because dialogue in Character Engine is expressed in terms of scenes of interaction, making it easier to maintain large and complex flows where the same input should be understood differently at different times.

We don’t want to be in a race with the providers of existing NLU solutions, though. What we want to do is facilitate author-friendly, narrative-rich writing options that tie into whatever happens to be the current state of the art in language understanding. So here’s a bit about how we do that:

Continue reading “Natural Language Understanding for Characters”

Memory and Knowledge for Characters

[For a couple of years now, I’ve reserved the first Tuesday of the month for a review of a book on writing or game design that might be of interest to IF folks. I’m still doing one of those in March, but it will come out the 19th, while I’m at GDC.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Knowledge and memory are a somewhat vexed area for game characters. It’s easy to think of characters who don’t remember the last fifty times you asked them the same exact lore question, or are strangely forgetful about the ways you’ve harmed them, or who aren’t equipped to answer common-sense questions about the world they live in.

So why is this a problem? Simply recalling that something has happened is not the main challenge. We can set flags; we can assign variables; we can check on quest journals to see what the player has already done. We can refer back to whatever data store is otherwise tracking world state in this game.

The hard part is building a system where

  • everything important to remember is stored in a reasonably systematic way
  • differences between world truth and character knowledge are handled as much as (and no more than) useful
  • there’s a way to track and author for the combinations of possible state so that the NPCs always have something to say about what they remember and know

There are quite a few technical, design, and writing challenges packed into those three bullet points.

Continue reading “Memory and Knowledge for Characters”

Conversation as Gameplay (Talk)

[Yesterday I gave a talk at the Oxford/London IF Meetup. The session was about conversation as gameplay, and also featured Flo Minuzzi of Tea-Powered Games, speaking about their released game Dialogue and their upcoming Elemental Flow. There’s a nice livetweeted thread version of my talk available on Twitter thanks to Florence Smith Nicholls, but I promised also to make a blog post about what I said.

Because the talk was written for an audience that included students, game designers from other parts of the industry, and newcomers to interactive fiction, I included some history of my own work that may be redundant for readers of this blog; there’s also some overlap with a talk I gave in Warsaw last September. However, the material towards the end of this talk is largely new.]

*

The Problem Statement

I want more games to be about human interaction, about the nuances of how people deal with one another, about the kinds of topics that appear in dramatic movies. That’s partly because I’d like to play more games about conversation and social interaction. I’m not as interested in action as a topic, and to be honest I often fall asleep during superhero movies these days.

Meanwhile, as an artist, part of the reason I write games is to explore and interrogate things I don’t yet fully understand. Building procedural systems and seeing how they perform is a great way to explore whether our mental models are correct. How people understand each other (or don’t), how they connect and why, are topics of enduring fascination for me.

So I want more conversation-rich games. For that to work as I’d like, the conversation needs to be rewarding as gameplay — not just bolted on around gameplay, as it so often is.

When it comes to my own work, I have a few more ambitions and requirements as well:

Screen Shot 2019-01-19 at 10.31.44 PM.png

First, I want it to allow the player to act with intentionality: to lay plans and carry them out. That means that we need some systematic mechanics that the player can learn and manipulate.

For the purposes of this talk, I’m not spending much time on things that are pure branching dialogue trees without ongoing state or clear mechanics. I’ve sometimes written work in that space, and if you’re interested in how to get the most out of a relatively state-light dialogue presentation, I recommend having a look at Jon Ingold’s AdventureX talk about writing sparkling interactive dialogue. But that’s not what we’re looking at today.

[I’ve written more about world model and systematic mechanics for conversation elsewhere.]

Second, I want the resulting mechanic to have good pacing and dramatic qualities — so a mechanic that systematizes conversation but makes it feel very slow, stilted, metaphorical, or hard to manipulate is not what I’m looking for. Some of these can be cool to play, but I myself tend to be looking to write something that has a bit more fluidity.

Continue reading “Conversation as Gameplay (Talk)”