Mailbag: Multimedia in Spanish Text Parser IF

Hi Emily,

I thought you might be able to shed some light on this question:

Text parser IF tends to rely heavily on the text for narrative, and uses little by way of multimedia.  Until you get to Spanish parser IF… here, multimedia is much more common. Spanish-language games often incorporate video, pictures, or sound effects.  Is there a reason behind this (possibly due to Spanish-language games using different engines better suited to multimedia?).  Or is there another reason?  Can Inform and similar platforms support these elements as well?

[Ed note: at the request of the asker, the original question has been re-written from a longer, less anonymous format.]

Several points here. One: for a lot of English-speaking IF fans, the defining IF of the commercial age came from Infocom in the early to mid 1980s, and almost all of their work was without illustration. There were a handful of late exceptions, but they were generally not considered Infocom’s best work.

arthur-the-quest-for-excalibur_3.png
Arthur: The Quest for Excalibur (Infocom / Bob Bates, 1989)

In Spain, by contrast, the golden age of commercial IF came just a few years later, on different hardware. Adventuras AD was publishing illustrated interactive fiction and setting expectations somewhat differently for hobbyist fans to follow. So most likely there was a certain amount of founder effect at work, in terms of what interactive fiction fans wanted to build.

Perhaps as a result of this, or perhaps coincidentally, Spanish language IF games have been written with an overlapping set of tools to Inform. Superglus for instance is a tool that compiles to the Glulx virtual machine, but uses a different, non-Inform parser.

And, in fact, the French and Italian IF communities have also traditionally done more with multimedia parser games than the Anglophone community — I’ve put a few links about this below as well.

Can Inform and similar platforms support these elements as well?

Yes, they can, though historically it was quite a bit of effort to get them set up. That’s less true now.

Continue reading “Mailbag: Multimedia in Spanish Text Parser IF”

Emotional Mechanics

[Most months, I devote the second Tuesday of the month to a mailbag post that answers a question someone’s sent to me. I’m still doing one of those in March, but it will come out in two weeks’ time.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

restless2

In January, I published a talk about conversation as gameplay, and in particular my ECTOCOMP game jam piece Restless. There, I talked about using Character Engine to give the player more expressive ability. They can tune their dialogue to be open, scary, vulnerable, aggressive, playful; they can pick input topics that they want to focus on.

I’m now revisiting Restless to address some of the flaws I identified in that piece, to give it a better tutorial introduction, polish its use of dynamic text, and add some depth to the content. Tea-Powered has created some additional art, so we can better signal gradations of character emotion. And I’m also refining some of the mechanics under the hood, particularly those that deal with gradual changes in character emotion.

What I’m going for in terms of the design:

  • perceivable consequence:
    • all of the player’s actions should have some effect on the outcome of the story, even if just by moving stats in one direction or another
    • the NPC’s emotional state should be apparent to the player, communicated through art, words, and actions
    • some areas of the NPC’s emotional state space should trigger narrative progression, by having the character get angry, leave, reveal new information, go back to sleep, etc
  • intentionality:
    • the NPC’s reactions should be consistent and predictable enough that the player can choose what to do and be rewarded by an expected outcome, even if there is an additional unexpected consequence
    • the player should have some warning when the NPC is approaching a state that will trigger a significant reaction
  • hysteresis:
    • the NPC should not rapidly cycle back and forth between two emotional states in response to opposed actions from the player; the history of interactions should matter
  • varied narrative intensity and pacing
    • not all player actions should be of equivalent efficacy; some actions should be more important, memorable, or high-stakes
    • it should not be possible to achieve an important result by grinding repetition of an unimportant action

That last point possibly deserves a little bit of expansion. One of the issues with stat-based relationship tracking in games is that it can mean that you could do a lot of banal minor services for a character, and work your way to a point where you’ve maxed out your affinity and now they want to marry you.

This is bad for multiple reasons. One, it’s unrealistic; humans don’t work like that. Two, it’s boring gameplay and boring story. Doing minor favors over and over feels like a chore, from a gameplay perspective. And narratively, there’s no sense of rising stakes, or risk, or the thrill of the relationship becoming more intense.

One way to resolve this would be to throw away the consequences of player action after a certain point. If you’re already at 50% affinity, paying one more compliment to the character does nothing at all. Now you can’t grind your way to romance through compliments: problem solved! Except that this goes against the earlier design principle that all player actions should have consequences of some kind.

So, instead, what we have to do is turn off the player’s ability to do minor actions when we’re in major consequence territory. Compliment away… while your relationship is still in trivial flirt zone. But work your way up to where things are getting serious, and the available affinity moves also get more serious, more freighted. And just having those “bigger” moves popping up in the choice menu should communicate to the player, along with the character art, that we’re in a more important situation.

Fortunately, all of that is pretty doable, by restricting dialogue lines to particular ranges of the emotional space.

This work is still in progress — I’ll post when the new version of Restless is generally available.

Natural Language Understanding for Characters

[This post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Character Engine is designed for two kinds of interaction: it can either generate input options for the player (as seen in Restless), or it can take natural language input, where the user is typing or using a speech-to-text system.

screen shot 2019-01-20 at 1.02.02 pm

That natural language approach becomes useful for games in AR or VR where more conventional controls would really get in the way. It’s also good for games that are using a chatbot style of interaction, as though you were chatting to a friend on Facebook; and a lot of interactive audio projects deployed on platforms like Alexa use spoken input.

There are plenty of business applications for natural language understanding, and a lot of the big tech players — IBM, Google, Amazon, Microsoft, Facebook — offer some sort of API that will take a sentence or two of user input (like “book a plane to Geneva on March 1”) and return a breakdown of recognized intents (like “book air travel”) and entities (“March 1”, “Geneva”). If you prefer an open source approach over going to one of those companies, RASA is worth a look.

For people who are used to parser interactive fiction, there’s some conceptual overlap with how you might set up parsing for an action: we’re still essentially identifying what the command might be and which in-scope nouns plug into that system. In contrast with a parser, though, these systems typically use machine learning and pre-trained language models to allow them to guess what the user means, rather than requiring the user to say exactly the right string of words. There are trade-offs to this. Parsers are frustrating for a lot of users. Intent recognition systems sometimes produce unexpected answers if they think they’ve understood the player, but really haven’t. In both cases, there’s an art to refining the system to make it as friendly as it can be, detect and guide the user when they say something that doesn’t make sense, and so on.

Some natural language services also let the developer design entire dialogue flows to collect information from the user and feed it into responses — usually interchanges of a few tens of sentences, designed to serve a repetitive, predictable transaction, and lighten the load on human customer service agents.

Those approaches can struggle a bit more when it comes to building a character who is part of an ongoing story, who has a developing relationship with the player or user, and who uses vocabulary differently each time.

That’s exactly where Character Engine can naturally provide a bit more support. We’re tracking character knowledge and personality traits. We’re doing text generation to build output that varies both randomly and in response to lots of details of the simulated world. And because dialogue in Character Engine is expressed in terms of scenes of interaction, making it easier to maintain large and complex flows where the same input should be understood differently at different times.

We don’t want to be in a race with the providers of existing NLU solutions, though. What we want to do is facilitate author-friendly, narrative-rich writing options that tie into whatever happens to be the current state of the art in language understanding. So here’s a bit about how we do that:

Continue reading “Natural Language Understanding for Characters”

Memory and Knowledge for Characters

[For a couple of years now, I’ve reserved the first Tuesday of the month for a review of a book on writing or game design that might be of interest to IF folks. I’m still doing one of those in March, but it will come out the 19th, while I’m at GDC.

Instead, this post is part of a short series on Character Engine and what we’re doing at Spirit AI. I’m writing these posts with IF and interactive narrative folks in mind, but more general-audience versions of the same content are also appearing on Spirit’s Medium account. Follow us there if you’re interested in hearing regularly about what Spirit is up to.]

Knowledge and memory are a somewhat vexed area for game characters. It’s easy to think of characters who don’t remember the last fifty times you asked them the same exact lore question, or are strangely forgetful about the ways you’ve harmed them, or who aren’t equipped to answer common-sense questions about the world they live in.

So why is this a problem? Simply recalling that something has happened is not the main challenge. We can set flags; we can assign variables; we can check on quest journals to see what the player has already done. We can refer back to whatever data store is otherwise tracking world state in this game.

The hard part is building a system where

  • everything important to remember is stored in a reasonably systematic way
  • differences between world truth and character knowledge are handled as much as (and no more than) useful
  • there’s a way to track and author for the combinations of possible state so that the NPCs always have something to say about what they remember and know

There are quite a few technical, design, and writing challenges packed into those three bullet points.

Continue reading “Memory and Knowledge for Characters”

Mailbag: Research on Dialogue and Story Generation (Part 2)

This is a continuation of an earlier mailbag answer about research that touches on dialogue and story generation. As before, I’m picking a few points of interest, summarizing highlights, and then linking through to the detailed research. In this section, I’m mostly looking at authoring tools and at academic theoretical work on interactive narrative.

This will not be comprehensive.

Authoring Tools for Dynamic or Procedural Storytelling

Several academic projects focus on building authoring tools for various types of dynamic or procedural storytelling, whether or not those are heavily augmented by AI. Many of these don’t rely on machine learning per se but do explore some other aspect of  the problem; in particular, several attempt to furnish the author with the means to build content for a planner-based storytelling system. But there’s a whole range of functionality here (and this is not a complete list):

Screen Shot 2017-04-23 at 9.26.07 PM

Andrew Gordon has done quite a bit of work around tools designed to assist authors with story creation ideas based on large corpora. I’ve written elsewhere about DINE, his interactive story authoring tool. DINE allows authors to describe the sorts of prompts that they want to understand, but uses its own models of language to determine whether a player’s input qualifies as matching a prompt. The result is less controllable but sometimes more robust than a standard interactive fiction parser. (“Sometimes” is the key word in that sentence.)

Screen Shot 2018-08-26 at 9.39.39 PM.png

Emma’s Journey is a project out of UCSC that combines fragments of choice-based narrative with a planner to create dynamic scenes. Individual pieces feel like they could have been done in Twine, but the selection and ordering of pieces is very dependent on current stats; and there is a distracting minigame for the player that also affects what options are available. This is built with the experimental StoryAssembler tool. There are also several associated research papers.

Continue reading “Mailbag: Research on Dialogue and Story Generation (Part 2)”

Restless

AnnaScreenshot.png

So I released a new game! Here’s its blurb:

You’ve been haunting old Mrs Fagles for decades. Now she’s sold the house, and the new owner’s moved in. Sylvie’s broke, bad at plumbing, and anxious about everything.  And with a living, breathing, fretting roommate, how are you supposed to rest in peace?

Drink blood. Set fires. Tell lies. Give advice, loan out a wedding dress, reclaim your true name.  Remix your dialogue options to reflect your mood or dig deeper into the topics that interest you.

I mentioned this briefly in yesterday’s link round up, but I wanted to give a little more background on it than a link round up typically allows for.

Restless is a game written for ECTOCOMP, a venerable Halloween-themed IF competition. There are six endings, if you’re counting — though some of those endings mean different things depending on how you get to them.

It’s a purely conversation game. As in a lot of choice-based games, you have up to three options, and you can pick one. But in contrast to the typical dialogue situation, you can do something about it if you don’t like your current menu. Click a mood, and your options will shift to reflect that new attitude. Turn on moods individually or in combinations. Discover conversation topics and you can set your dialogue to explore those too.

Continue reading “Restless”