Scenario FST

FST format is a primitive format for writing interaction with MMDAgent

Dialog / interaction FST is a low-level scenario description for built-in dialogue / interaction manager. A dialog / interaction can be defined as a finite state transducer (FST). All the messages thrown by MMDAgent-EX or its plugins to the global message queue will be fed into the FST, and if a message was accepted at the current state, state transition will occur and the output text will be thrown to the queue.

A simple example of FST scenario is as follows. Assume the state to start with state number 0. The first line defines that “When the current state number is 0 and a message RECOG_EVENT_STOP|hello comes, it outputs nothing (<eps> means no output) and go to state number 10”. In state number 10, since its accepting message is <eps>, it immediately outputs a message MOTION_ADD|mei|greet|greet.vmd and go to state 11. The state 11 also immediately outputs SYNTH_START|mei|normal|hi, and go to state 12. In the state 12, it will wait until a message SYNTH_EVENT_STOP|mei comes, and when it comes, it outputs no message and go to state 0.

 0     10    RECOG_EVENT_STOP|hello   <eps>
10     11    <eps>                    MOTION_ADD|mei|greet|greet.vmd
11     12    <eps>                    SYNTH_START|mei|normal|hi
12      0    SYNTH_EVENT_STOP|mei     <eps>

MMDAgent-EX and its plugin modules will output various messages while running, and also accepts messages thrown by other modules. See the All messages section to know what kind of messages exist.

Basic format

.fst file is a text file that contains arc definitions each per line:

state to input output

where state is a state number where the arc exists, to is the target state number of the arc, input is the input message to be matched, and output is the message to be output when this transition occurs. The initial state number is fixed to 0.

Fields should be separated by a sequence of space or tab. To include a space within an argument, embrace the field with "".

100 110  RECOG_EVENT_STOP|weather SYNTH_START|mei|normal|"Hi, how are you"

<eps> is a special string for epsilon (“void”) symbol. <eps> at input means that the arc will be executed immediately just after current state has moved to the state. <eps> at output means no output.

110 120  <eps>  SYNTH_START|mei|normal|"Hi, how are you"

When specifying multiple items in a message with comma, all the items in the field shoul match.

#### for input "RECOG_EVENT_STOP|today,is,sunday,"
# this matches
100 110  RECOG_EVENT_STOP|sunday SYNTH_START|mei|normal|hi
# this does not match
100 110  RECOG_EVENT_STOP|like,sunday SYNTH_START|mei|normal|hi

When defining multiple arcs on a state, order is important! At run time, matching will be checked for all arcs in the order of the definition in fst, and the first matched one will be chosen. For example,

100 120 MESSAGE|foo,bar <eps>
100 130 MESSAGE|bar <eps>

an input message “MESSAGE|foo,bar” will be captured at the first arc, and “MESSAGE|bar “will be captured at the second arc.

Launching FST

At least one FST file (primary FST) is required to run a content in MMDAgent-EX. The starting FST file should have a name of the same basename as the startup .mdf file. For example, when a content has foobar.mdf at the top directory of the content to start, the fst file name should be foobar.fst.

  foobar.mdf
  foobar.fst

Concurrent scenario management with sub-FST

You can run multiple FSTs in parallel. They are called “sub-FST”. The primary FST and sub-FSTs will run simultaneously. All FSTs capture all messages and can output messages. This sub-FST function enables us to handle complex scenario where multiple concurrent event should be handled at the same time.

You can define sub-FST by having the primary FST’s file name as prefix. Below is an example. A primary FST (FooBar.fst in this example) as two sub-FSTs.

  • FooBar.fst
  • FooBar.fst.1
  • FooBar.fst.hogehoge.fst

At startup, after MMDAgent-EX finds the primary FST, it will also search for any files that has the primary FST as prefix, and start all the files in different threads to run concurrently.

Using local variable

You can also define additional field to enter a value to “local variables”.

state to input output ${varname}=string

The local variables are internal that can be used inside the FST. The format should be ${varname}=string. No space around =. Only alphabets, numerics and underbar character can be used as variable names. Here is an example:

0 10  <eps> <eps> ${place}=Nagoya

Use comma to set multiple variables:

210 220  <eps> <eps> ${src}=Nara,${dst}=Tokyo,${pref}=nozomi

The variables can be referred as ${varname} in the message field of the FST. The string ${varname} in FST will be replaced to the current value if the variable at run time.

100 110  RECOG_EVENT_STOP|${place} ...

You can also use a variable to set another variable:

10 20 <eps> <eps> ${target}=${place}

At run time you can see the list of current local variables at FST log window (Shift+F in desktop). Also the values will be also output to log.

Using regular expression

Regular expression is supported at the third field (input message). Embracing with @ to treat the field text as regular expression, and the input messages are checked if it matches the expression.

Here is an example that captures any recognition result that contains “station” or “Station”.

180 200  @RECOG_EVENT_STOP\|.*[Ss]tation.*@  MOTION_START|mei|base|greeting.vmd
  • The regular expression should be in Google RE2 format
  • The regular expression should be full match
  • As in the example, message delimiter | should be escaped inside regular expression

The matched part can be extracted as local variable ${1}, ${2} and so on. The group or sub-match region defined by () in the expression will be stored automatically. Here is a modification of the example above that sets the station name (i.e, recognized word before “station”) to the variable ${station}.

180 200  @RECOG_EVENT_STOP\|.*?([^,]*),?[Ss]tation.*@  MOTION_START|mei|base|greeting.vmd  ${station}=${1}

Another examples:

# Store the whole recognition result to ${s}
0 1 @RECOG_EVENT_STOP\|(.*)@ <eps> ${s}=${1}
# Capture motion target model name to ${model} and motion name to ${motion}
0 1 @MOTION_EVENT_ADD\|(.*)\|(.*)@ <eps> ${model}=${1},${motion}=${2}