Layering Motions #
To achieve high levels of interactivity in dialogue, it is necessary to dynamically perform complex actions in response to events that arise during conversation, such as “bowing while speaking”, “stopping waving when spoken to and listening”, or “noticing something while speaking and continuing the conversation”. MMDAgent-EX has a mechanism to parallel play multiple partial motions combined for this purpose.
The following is a schematic diagram of layering motions. In the diagram, Glance.vmd is a motion that only moves the head and face, and Response.vmd is a motion (partial motion) that moves the fingers and facial expressions. All motions are calculated for posture in parallel, and the motion with higher priority is applied to the overlapping parts (blending or addition is also possible depending on the settings). With this mechanism, motions can be combined and played on the spot according to user input and situations, realizing complex movements in real time.
Below, we will introduce and explain how to use this mechanism through the demonstration of layering motions in the Example.
Running the Motion Overlay Demo in Example #
There’s a motion overlay demo located in the
example_motion folder of Example. Let’s give it a try.
|- example_motion.fst FST script
|- example_motion.mdf .mdf file
|- gaze.vmd motion: gaze
|- onehandwave.vmd motion: waving a hand
+- walk.vmd motion: walking
example_motion.mdf in MMDAgent-EX. It will start in a state where the
walk.vmd motion is looping as shown below.
In this state, pressing the
1 key will make Gene look at us. This is the “gaze.vmd” motion that only involves looking to the left, overlaid on the walking motion.
Furthermore, pressing the
2 key will overlay the hand-waving motion
onehandwave.vmd motion only defines a smiling face and the movement of the right arm. By overlaying this on the walking and looking-left motions, you can observe how to control actions such as “looking at us and greeting at any time”.
Motion Aliases and List Display #
Looking at the
example_motion.fst from the demo above, you can see that after the script is launched, pressing the
1 key triggers
gaze.vmd, and pressing the
2 key triggers
onehandwave.vmd to play, each under
walk.vmd is given the alias “base”,
gaze.vmd is “look”, and
onehandwave.vmd is “greet”. These different motion aliases are managed independently and can be individually controlled with
While in operation, you can press the
b key to view information about the currently running motion. An example display is shown below (press
b again to hide it). The model name is displayed in green, and the motion alias names are shown in light blue, listed in order of priority. The pink line is a simplified display of the motion of each bone being played.
Detailed Overlay Playback #
This section explains the detailed specifications of the
MOTION_ADD message related to overlay playback.
Specifying Partial Motions #
When overlapping and playing motions, the motion to be overlaid later needs to start as a “partial motion”, controlling only the moving parts rather than the full-body motion.
To play it as a partial motion, specify “PART” as the fourth argument in
MOTION_ADD as shown below:
The motion started as a partial motion will ignore bones/morphs that are not defined to move (i.e., only defined at the 0th frame), and only the movements of bones/morphs that are defined to move (i.e., keyframes exist from the 1st frame onwards) will be applied.
If you want to treat it as a regular full-body motion, specify FULL. If omitted, the default is FULL.
Specifying Loop Playback #
In regular motion playback, the motion automatically disappears after playing up to the last frame. You can specify this to return to the first frame and loop playback.
If you want to loop the motion, specify LOOP as the fifth argument in
MOTION_ADD. The loop-specified motion will return to the 0th frame after playing up to the last frame and will continue to play. Loop playback stops with
MOTION_DELETE or by deleting the model.
To explicitly specify playback only once, specify ONCE. If omitted, the default is ONCE.
Switching Smoothing #
To avoid motion jumps at the seams during overlapping playback, MMDAgent-EX automatically adds motion smoothing at the start and end of the motion. If you want to turn this off for some reason, specify the sixth argument as OFF.