All messages

  • ”()” is a literal description of the value.
  • “x,y,z” is a position coordinate.
  • “rx,ry,rz” is rotation degrees.
  • ”(A or B..)” shows that valid values are A and B, and the first value (‘A’ in this case) is the default value when not specified.

Model

Show, change, delete and configure 3-D model in the scene.

MODEL_ADD|(model alias)|(model file name)
MODEL_ADD|(model alias)|(model file name)|x,y,z
MODEL_ADD|(model alias)|(model file name)|x,y,z|rx,ry,rz
MODEL_ADD|(model alias)|(model file name)|x,y,z|rx,ry,rz|(ON or OFF for cartoon)
MODEL_ADD|(model alias)|(model file name)|x,y,z|rx,ry,rz|(ON or OFF for cartoon)|(parent model alias)
MODEL_ADD|(model alias)|(model file name)|x,y,z|rx,ry,rz|(ON or OFF for cartoon)|(parent model alias)|(parent bone name)
MODEL_CHANGE|(model alias)|(model file name)
MODEL_DELETE|(model alias)
MODEL_BINDBONE|(key name)|(min)|(max)|(model alias)|(bone name)|x,y,z|rx,ry,rz|x,y,z|rx,ry,rz
MODEL_BINDFACE|(key name)|(min)|(max)|(model alias)|(face name)|r1|r2

MODEL_EVENT_ADD|(model alias)
MODEL_EVENT_CHANGE|(model alias)
MODEL_EVENT_DELETE|(model alias)
MODEL_EVENT_SELECT|(model alias)

Model-to-bone mapping

Let a KeyValue dominates and controls a bone of a model.

MODEL_EVENT_BINDBONE|(key name)|(model alias)|(bone name)
MODEL_EVENT_BINDFACE|(key name)|(model alias)|(face name)

Motion

Start, stop, delete and configure motions onto a model.

MOTION_ADD|(model alias)|(motion alias)|(motion file name)
MOTION_ADD|(model alias)|(motion alias)|(motion file name)|(FULL or PART)
MOTION_ADD|(model alias)|(motion alias)|(motion file name)|(FULL or PART)|(ONCE or LOOP)
MOTION_ADD|(model alias)|(motion alias)|(motion file name)|(FULL or PART)|(ONCE or LOOP)|(ON or OFF for smooth)
MOTION_ADD|(model alias)|(motion alias)|(motion file name)|(FULL or PART)|(ONCE or LOOP)|(ON or OFF for smooth)|(ON or OFF for reposition)
MOTION_ADD|(model alias)|(motion alias)|(motion file name)|(FULL or PART)|(ONCE or LOOP)|(ON or OFF for smooth)|(ON or OFF for reposition)|priority
MOTION_ACCELERATE|(model alias)|(motion alias)|(speed)|(duration)|(specified time for end)
MOTION_CHANGE|(model alias)|(motion alias)|(motion file name)
MOTION_DELETE|(model alias)|(model alias)
MOTION_EVENT_ADD|(model alias)|(motion alias)
MOTION_EVENT_ACCELERATE|(model alias)|(motion alias)
MOTION_EVENT_CHANGE|(model alias)|(model alias)
MOTION_EVENT_DELETE|(model alias)|(motion alias)

Motion override configuration

Set special bone / face motion override rule: replace (=overwrite), add(=additive) or none(=ignored).

MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_BONE_REPLACE|bonename[,bonename,..]
MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_BONE_ADD|bonename[,bonename,..]
MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_BONE_NONE|bonename[,bonename,..]
MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_FACE_REPLACE|facename[,facename,..]
MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_FACE_ADD|facename[,facename,..]
MOTION_CONFIGURE|(model alias)|(motion alias)|MODE_FACE_NONE|bonename[,bonename,..]

Moves

Let the model move, turn or rotate. GLOBAL means that the position or rotations are treated at global coordinates, and LOCAL means they are local to each model.

MOVE_START|(model alias)|x,y,z
MOVE_START|(model alias)|x,y,z|(GLOBAL or LOCAL position)
MOVE_START|(model alias)|x,y,z|(GLOBAL or LOCAL position)|(move speed)
MOVE_STOP|(model alias)
MOVE_EVENT_START|(model alias)
MOVE_EVENT_STOP|(model alias)
TURN_START|(model alias)|x,y,z
TURN_START|(model alias)|x,y,z|(GLOBAL or LOCAL position)
TURN_START|(model alias)|x,y,z|(GLOBAL or LOCAL position)|(rotation speed)
TURN_STOP|(model alias)
TURN_EVENT_START|(model alias)
TURN_EVENT_STOP|(model alias)
ROTATE_START|(model alias)|rx,ry,rz
ROTATE_START|(model alias)|rx,ry,rz|(GLOBAL or LOCAL rotation)
ROTATE_START|(model alias)|rx,ry,rz|(GLOBAL or LOCAL rotation)|(rotation speed)
ROTATE_STOP|(model alias)
ROTATE_EVENT_START|(model alias)
ROTATE_EVENT_STOP|(model alias)

Stage

Make a stage.

STAGE|(stage file name, .xpmd)
STAGE|(bitmap file name for floor),(bitmap file name for back)

Lighting

Light color and direction.

LIGHTCOLOR|(Red,Green,Blue) # [0..1]
LIGHTDIRECTION|x,y,z

Camera

Move the camera view. The parameter values and orders are the same as displayed on the screen at log view, so once you get a nice camera position at the app, apply the screen values to the message as is.

Transition time period is a duration to accomplish the move, set 0 to change immediately, and set -1 to do smooth move (default).

By giving model alias, you can “mount” the camera on the model; moving or motioning the model also moves the mounted camera. Be careful of motion sickness!

Camera motion can also be given by “camera motion” as .vmd file.

CAMERA|x,y,z|rx,ry,rz|(distance)|(fovy)
CAMERA|x,y,z|rx,ry,rz|(distance)|(fovy)|(transition time period)
CAMERA|x,y,z|rx,ry,rz|(distance)|(fovy)|(transition time period)|(model alias)
CAMERA|x,y,z|rx,ry,rz|(distance)|(fovy)|(transition time period)|(model alias)|(bone name)
CAMERA|(camera motion file name)

User Input to message

Some user action will also generates message.

xxxxx_yyyyy_wwwww_hhhhh shows information about the position where user has long-pressed or released. (x,y) is the screen position, and (w,h) is the screen width and height, both in pixel.

DRAGANDDROP|(file name)
KEY|(key name)
SCREEN_EVENT_LONGPRESSED|xxxxx_yyyyy_wwwww_hhhhh
SCREEN_EVENT_LONGRELEASED|xxxxx_yyyyy_wwwww_hhhhh

You can add a menu via message. It gives users a choise to issue some predefined messages by menu tap. alias corresponds to a page of a menu you are going to create. You can add up to 20 menu pages in the app. id is the vertical position of the item in a page from 0 to 29. The type and argN are the message body to be generated when the menu item is selected.

MENU|ADD|(alias)
MENU|ADD|(alias)|backgroundImagePath
MENU|DELETE|(alias)
MENU|SETITEM|(alias)|(id)|(label)|(type)|(arg1)|(arg2)|...
MENU|DELETEITEM|(alias)|(id)

MENU_EVENT|ADD|(alias)
MENU_EVENT|DELETE|(alias)
MENU_EVENT|SETITEM|(alias)|(id)
MENU_EVENT|DELETEITEM|(alias)|(id)

Prompt

Display message dialog to user in front of the scene, and get response.

PROMPT_SHOW|(main text)|(select text 0)|(select text 1)|...
PROMPT_EVENT_SELECTED|(selected number)

KeyValue

You can set any KeyValue by message.

KEYVALUE_SET|(key name)|(value)

InfoText

Display document at full screen.

buttonLabels is a text label to display at the bottom of the text area. When user taps the button, the window will close. Giving multiple button labels like “Yes,No,Cancel” will show users multiple buttons for each, and the button user tapped is returned by INFOTEXT_EVENT_CLOSE. The colors are hex style like “ff00cc”. Alpha value can be set like “ff00ccaa”.

INFOTEXT_FILE|(filepath)|(titleLabel)|(buttonLabels)
INFOTEXT_FILE|(filepath)|(titleLabel)|(buttonLabels)|(scale)
INFOTEXT_FILE|(filepath)|(titleLabel)|(buttonLabels)|(scale)|(BACKGROUNDCOLOR)|(TEXTCOLOR)
INFOTEXT_STRING|textbody|(titleLabel)|(buttonLabels)
INFOTEXT_STRING|textbody|(titleLabel)|(buttonLabels)|(scale)
INFOTEXT_STRING|textbody|(titleLabel)|(buttonLabels)|(scale)|(BACKGROUNDCOLOR)|(TEXTCOLOR)

INFOTEXT_EVENT_SHOW
INFOTEXT_EVENT_CLOSE|(selecteDButtonLabel)

Logging

Start user input logging and send it to server. User inputs and system actions between LOG_START and LOG_FINISH will be stored in local, and all the logs are sent to server by LOG_UPLOAD.

These messages are just for controlling the logging timings inside interaction. The kind of data and server are managed in content package.

LOG_START
LOG_FINISH
LOG_UPLOAD

Plugin_Julius

Speech recognition engine.

  • Default system jconf file is jconf_ja_JP.txt for desktop, jconf_ja_JP_for_mobile.txt for mobile devices.
  • Also load “xxx.jconf” in the content folder, where xxx is the base name of the .mdf file.
  • User dictionary “xxx.dic” will be also loaded.
  • If a model has a morph named “volume”, its rate will be dynamically changed by the input volume.
  • If a model has a morph named “trigger”, its rate will be switched to 1.0 while performing recognition, and 0.0 while not.
  • This plugin can be enabled and disabled either by message or by menu. When disabled, RECOG_EVENT_AWAY|ON event will be issued and audio input will be muted input by setting audio scale to 0.0. When enabled, RECOG_EVENT_AWAY|OFF event will be issued and audio input will be restored.

Recognition result notification:

RECOG_EVENT_START
RECOG_EVENT_STOP|(word1,word2,...)
RECOG_EVENT_OVERFLOW
RECOG_EVENT_GMM

Away notification:

RECOG_EVENT_AWAY|ON
RECOG_EVENT_AWAY|OFF

Set Audio scaling (1.0 = read as is):

RECOG_MODIFY|GAIN|scaling_value
RECOG_EVENT_MODIFY|GAIN

Switch jconf and re-start engine. jconf_file path should be relative to AppData/Julius, not the content folder:

RECOG_MODIFY|CHANGE_CONF|jconf_file

Swap user dictionary:

RECOG_MODIFY|USERDICT_SET|(dictionary file path)
RECOG_MODIFY|USERDICT_UNSET
RECOG_EVENT_MODIFY|USERDICT_SET
RECOG_EVENT_MODIFY|USERDICT_UNSET

Word prediction (internal use):

RECOG_MODIFY|PREDICTWORD_FACTOR|float
RECOG_MODIFY|PREDICTWORD|word1,word2,word3,...

Enable / disable:

PLUGIN_ENABLE|Julius
PLUGIN_DISABLE|Julius
PLUGIN_EVENT_ENABLE|Julius
PLUGIN_EVENT_DISABLE|Julius

Plugin_Open_JTalk

Japanese speech synthesis engine.

SYNTH_START|(model alias)|(voice alias)|(synthesized text)
SYNTH_STOP|(model alias)
SYNTH_EVENT_START|(model alias)
SYNTH_EVENT_STOP|(model alias)
LIPSYNC_START|(model alias)|(phoneme and millisecond pair sequence)
LIPSYNC_STOP|(model alias)
LIPSYNC_EVENT_START|(model alias)
LIPSYNC_EVENT_STOP|(model alias)
PLUGIN_ENABLE|Open_JTalk
PLUGIN_DISABLE|Open_JTalk
PLUGIN_EVENT_ENABLE|Open_JTalk
PLUGIN_EVENT_DISABLE|Open_JTalk

Plugin_Audio

Playing audio file. Supported formats are mp3 and wav.

SOUND_START|(sound alias)|(sound file name)
SOUND_STOP|(sound alias)
SOUND_EVENT_START|(sound alias)
SOUND_EVENT_STOP|(sound alias)
PLUGIN_ENABLE|Audio
PLUGIN_DISABLE|Audio
PLUGIN_EVENT_ENABLE|Audio
PLUGIN_EVENT_DISABLE|Audio

Plugin_LookAt

Eye gazing to mouse (desktop).

PLUGIN_ENABLE|LookAt
PLUGIN_DISABLE|LookAt
PLUGIN_EVENT_ENABLE|LookAt
PLUGIN_EVENT_DISABLE|LookAt

Plugin_Variables

Common variable and count-down timer.

Be careful for “TIMER_EVENT_STOP“: when “TIMER_STARToverrides existing count down timer, MMDAgent-EX issues “TIMER_EVENT_CANCELLED”, whereas Normal MMDAgent issues “TIMER_EVENT_STOP”. This is non-compatible point, so take care when moving dialogue contents into MMDAgent-EX.

VALUE_SET|(variable alias)|(value)
VALUE_SET|(variable alias)|(minimum value for random)|(maximum value for random)
VALUE_UNSET|(variable alias)
VALUE_EVAL|(variable alias)|(EQ or NE or LE or LT or GE or GT for evaluation)|(value)
VALUE_EVENT_SET|(variable alias)
VALUE_EVENT_UNSET|(variable alias)
VALUE_EVENT_EVAL|(variable alias)|(EQ or NE or LE or LT or GE or GT for evaluation)|(value)|(TRUE or FALSE)
TIMER_START|(count down alias)|(value)
TIMER_STOP|(count down alias)
TIMER_EVENT_START|(count down alias)
TIMER_EVENT_STOP|(count down alias)
TIMER_EVENT_CANCELLED|(count down alias)
PLUGIN_ENABLE|Variables
PLUGIN_DISABLE|Variables
PLUGIN_EVENT_ENABLE|Variables
PLUGIN_EVENT_DISABLE|Variables

Plugin_VIManager

PLUGIN_ENABLE|VIManager
PLUGIN_DISABLE|VIManager
PLUGIN_EVENT_ENABLE|VIManager
PLUGIN_EVENT_DISABLE|VIManager
FST_LOAD|(.fst file path)
FST_LOAD|(.fst file path)|(initial state number)

Plugin_WindowController

EXECUTE|(file name)
KEY_POST|(window class name)|(key name)|(ON or OFF for shift-key)|(ON or OFF for ctrl-key)|(On or OFF for alt-key)
PLUGIN_ENABLE|WindowController
PLUGIN_DISABLE|WindowController
PLUGIN_EVENT_ENABLE|WindowController
PLUGIN_EVENT_DISABLE|WindowController

Plugin_TextArea

TEXTAREA_ADD|(textarea alias)|(width,height)|(size,margin,exlinespace)|r,g,b,a|r,g,b,a|x,y,z
TEXTAREA_ADD|(textarea alias)|(width,height)|(size,margin,exlinespace)|r,g,b,a|r,g,b,a|x,y,z|rx,ry,rz
TEXTAREA_ADD|(textarea alias)|(width,height)|(size,margin,exlinespace)|r,g,b,a|r,g,b,a|x,y,z|rx,ry,rz|(parent model alias)
TEXTAREA_ADD|(textarea alias)|(width,height)|(size,margin,exlinespace)|r,g,b,a|r,g,b,a|x,y,z|rx,ry,rz|(parent model alias)|(parent bone name)
TEXTAREA_SET|(textarea alias)|(text)
TEXTAREA_DELETE|(textarea alias)

TEXTAREA_EVENT_ADD|alias
TEXTAREA_EVENT_SET|alias
TEXTAREA_EVENT_DELETE|alias
PLUGIN_ENABLE|TextArea
PLUGIN_DISABLE|TextArea
PLUGIN_EVENT_ENABLE|TextArea
PLUGIN_EVENT_DISABLE|TextArea

Explanation of TEXTAREA_ADD:

TEXTAREA_ADD
  |(textarea alias)
      a new alias name
  |(width,height)
      text area box parameter
          >0 to use fixed size (longer text will shrink to fit within the box)
          <=0 to use variable size (varies according to the text)
      default: 0,0
  |(scale,margin,exlinespace)
      font scale, margin, additional linespace
      default: 1.0, 1.0, 0.0
  |r,g,b,a
      background color (0 ~ 1), set 0 to 'a' for disable background drawing
  |r,g,b,a
      text color (0 ~ 1)
  |x,y,z
      offset coordinate of the center of this area
  |rx,ry,rz
      (optional) rotation in angles
  |(parent model alias)
      (optional) model alias name to be put on
  |(parent bone name)
      (optional) bone of the model to be put on

Explanation of TEXTAREA_SET:

TEXTAREA_SET|(textarea alias)|(text)
  text: text to be displayed. should be braced by "" to handle white space. Use "\n" (backslash + 'n') for multiple line text

Plugin_Remote

PLUGIN_ENABLE|Remote
PLUGIN_DISABLE|Remote
PLUGIN_EVENT_ENABLE|Remote
PLUGIN_EVENT_DISABLE|Remote

Plugin_ErrorBlock

Plugin_Network

NETWORK_GET|(net alias)|(URI)|(save file name)
NETWORK_EVENT_GET|(net alias)
PLUGIN_ENABLE|Network
PLUGIN_DISABLE|Network
PLUGIN_EVENT_ENABLE|Network
PLUGIN_EVENT_DISABLE|Network

Plugin_Flite_plus_hts_engine

Plugin_Kafka