Live2D Cubism Editor 5.1 alpha3

User Manual


04/25/2024  r3


Table of Contents

Table of Contents

Revision History

Precautions for Use

Purpose of alpha version

Setting information for alpha version of the Editor

About language switching

About the English and Chinese editors

New Feature

Enhanced template functions

External Application Integration

Apply 3D Expression

[AI function] Auto Generation of Deformer

[AI function] Auto Generation of Sway Motion

Limitations

Modeling

Animation

Common

About AI Technology

AI on-board functions

Learning data


Revision History

Version

Revision date

Revision details

r3

04/25/2024

Updated in alpha3

・Added “Apply Mask, Culling, and Blend Mode” to “Enhanced template functions.”

・Added “Apply Physics Group” to “Enhanced template functions.”

・Updated the UI of “External Application Integration.”

・Updated the “Screen explanations” and the “Operating procedure” of “Apply 3D Expression.”

・Updated the “Screen explanations” and the “Operating procedure” of “[AI function] Auto Generation of Sway Motion.”

・Fixed the bug in “Apply 3D Expression” limitations where objects could disappear if you set the angle to 0.

・Updated “[AI function] Auto Generation of Sway Motion” limitations.

・Updated “External Application Integration” limitations.

r2

04/11/2024

Updated in alpha2

・Added "Display Deformers of the Template Model" in "Enhanced Template Feature."

・Added "Delete Empty Parameter Groups, Parameters, Parts, and Deformers" in "Enhanced Template Feature."

・Added "Insert Key into the Timeline Using the Integration Function" in "External Application Integration."

・Fixed a bug in the "Apply 3D Expression" limitations where the variation with the perspective strength depends on target size.

・Reported a bug that an object may disappear if the angle range is set to 0 in the "Apply 3D Expression" limitations.

・Added a note that BlendShapes are not supported in the "[AI function] Auto Generation of Sway Motion" limitations.

・Reported a bug that keys are not inserted into the Timeline according to the correct frame rate in the "External Application Integration" limitations.

r1

03/28/2024

First edition

Precautions for Use

Purpose of alpha version

This alpha version is intended to evaluate the usability and performance of the new features and improvements planned for Cubism Editor 5.1. Therefore, you will be able to try out new features before they are available. However, some functions may not yet be implemented, or the operation may be unstable and behave unexpectedly.

The UI is based on specifications currently under development and may change in the future. In addition, there is a possibility that data such as model files (.cmo3) and animation files (.can3) that you have created may no longer be opened (incompatible) due to bugs or specification changes.

While it is possible to create works using the alpha version, we strongly recommend that you use it to “evaluate” the usability and performance of the new features.

Please ensure you fully understand the purpose of the alpha version before using it.
Please send your requests for new features and bug reports below!

Japanese

Live2D Creators Forum

We also accept requests and bug reports on X (formerly Twitter)!

Post with the hashtag #cubism51_alpha!

English

Live2D Creators Forum

Live2D Community Discord

We also accept requests and bug reports on X (formerly Twitter)!

Post with the hashtag #cubism51_alpha!

Chinese

bilibili

Provide your feedback in the comments section!

Setting information for alpha version of the Editor

If you are already using the full version of Cubism on your PC, Editor settings such as keyboard shortcuts will be inherited when you start Cubism 5.1 alpha. At this time, the settings for the alpha version will be saved in a different location than the full version and will not be shared thereafter. This is to preserve the operating environment of the full version.

Please note that due to this fact, setting items changed in Cubism 5.1 alpha will not be carried over to Cubism 5.1 beta and the official version.

Note: Setting items changed in 3.x and 4.x will be carried over to the 5.1 beta and the full version.

The settings files for Cubism 5.1 alpha are saved in the following folders. Note that these settings files are not used in beta and later versions, so there is no problem with deleting them.

Windows

C:\Users\[USER_NAME]\AppData\Roaming\Live2D\Cubism5.1alpha_Editor

C:\Users\[USER_NAME]\AppData\Roaming\Live2D\Cubism5.1alpha_Viewer

Mac

/Users/[USER_NAME]/Library/Live2D/Cubism5.1alpha_Editor

/Users/[USER_NAME]/Library/Live2D/Cubism5.1alpha_Viewer

Note: Please be careful when handling these folders as they are hidden folders.

Note: If Cubism 5.1 alpha does not work properly, it may help to delete the above folders. Once the folders are deleted, if the settings files for the full version Editor (3.x /4.x) are already on your PC, they will be transferred over again.

About language switching

The display language can be switched in [Environment settings] starting from Cubism 5.1 beta.

In Cubism 5.1 alpha, you cannot switch from [Environment settings], so please download and use the editor for each language.

About the English and Chinese editors

Please note that some wording newly added in Cubism 5.1 alpha may be of lower translation quality. We will provide high-quality translations from Cubism 5.1 beta.

New Feature

Enhanced template functions

Summary

Enhanced model template functionality.

The previous model template function could be used only once at the start of model creation, but the function has been greatly expanded so that it can be applied to models during creation.

This document describes enhancements from the previous model template functionality.

For new ways to use the enhanced model templates, see the “User Tutorial.

 

Model template file

In the alpha version, in the Model template selection dialog box described below, the model template is empty by default.

To try the model template function, the following files must be unzipped and placed in the model template folder as described below.

ZIP: https://cubism.live2d.com/editor-alpha/doc/template-model/Template_Model_en.zip

See the [User Tutorial.] for details.

Apply template

Re-application is now possible

Previously, application of templates was limited to models with only an ArtMesh on the canvas, such as PSD imports.

With this enhancement, templates can now be applied to models that have been edited, such as after deformers or parameters have been added.

Partial application is now possible

Templates can now be applied only to selected ArtMeshes.

Specifically, when executing “Apply Template,” only the selected ArtMesh is targeted.

ArtMeshes that are not targeted will not appear in the Template dialog box and the template will not be applied to those ArtMeshes.

If you apply the template without selecting anything, the entire model will be applied.

Adding elements to apply templates

Other improvements


Model Template Selection

In the modeling process, select [File] menu -> [Apply Template...] to display the [Model template] dialog box.

In this dialog box, you can select the template to be applied.

There are two ways to select a template: [Select from model template folder] or [Select from .cmo3 file].

[Select from model template folder] displays a list of model templates. You can select a template by clicking on the thumbnail.

In [Select from .cmo3 file], you can directly specify a model file as a template using the [...] button on the right side.

Model Template Folder

Select [Edit] menu -> [Open model template folder] to open the model template folder.

Model templates placed in this folder will appear in the [Select from model template folder] dialog box.

Search Filter

The [Search filter] allows you to narrow down the templates that appear in the [Select from model template folder].

Specifically, only templates that include the string you entered in either [Name] or [Details] will be displayed.

Apply Settings

After clicking OK in [Preview], the [Apply Settings] dialog box appears.

Applying a template adds new parameters, deformers, parts, and other objects to the model to which it is applied, and changes the selected ArtMesh.

In this dialog box, you can adjust the properties of the object being added and the ArtMesh being modified when the template is applied.

Same Parameters

[Same parameters] should be used when the template model and the model to which it will be applied contain the same parameters.

When [Add] is selected, if the template and the destination have parameters with the same ID, the parameter on the template side will be added as a parameter with a different ID.

Selecting [Overwrite] will not add the parameter as a separate new parameter, even if the template and the destination have parameters with the same ID.

An exception is if one of the same parameters is a blend shape and the other is a regular parameter, the process is the same as if you had chosen [Add].

Label Color

[Label color] is the label color setting for the object to which the template will be applied.

The label color is set for each of the following four items.

You can choose one label color setting for parameter groups, parts, and deformers from [Use template color] and [Use specified color].

Use template color

The label color of the template is used as is.

Change to specified color

Changes the label color to the specified color.

You can select one of three settings for the label color of the ArtMesh: [Do not change], [Change to template color], or [Change to specified color].

Do not change

The label color is not changed from the destination label color.

Change to template color

Change the color to the same as the label color of the template.

Change to specified color

Changes the label color to the specified color.


Export Template

In modeling mode, execute [File] menu -> [Export Template...] to display the [Export Template] dialog box.

This dialog box allows you to export the model you are editing as a model template.

[Name] is the name that appears in the Model Template Selection dialog box.

Note: Do not enter characters that cannot be used for file names, as they will also be used as folder names and file names.

[Details] is text that appears as a tooltip for the thumbnail of the Model Template Selection dialog box. It is also a target of the [Search filter] in the Model Template Selection dialog box.


Model template information (.template.json)

Model template information is represented in JSON format.

Precautions

Model template information created in the alpha editor cannot be used in the editors other than the alpha version.

Model template information for alpha1 consists of the following seven properties.

Number

Property

Description

1

FormatVersion

The version number related to the format of the model template information.

Only referenced internally in the system and not displayed.

2

ModelVersion

The version number related to the model of the model template.

Referenced internally in the system and not displayed.

3

Name

The name of the model template.

It appears in the [Model template selection] dialog box.

4

BaseModel

Model files (.cmo3) of the model template.

This can be loaded from the [Model template selection] dialog box and displayed as a template model.

5

Thumbnail

Image files (.png/.gif) identifying the model template.

They are displayed as thumbnails in the [Model template selection] dialog box.

When executing “Export Template,” a still image of the entire model to be registered as [BaseModel] is registered.

If you want a preview of how the model template will work, for example, replace it with a GIF.

6

Description

Explanation of the model template.

It appears as a tooltip in the [Model template selection] dialog box.

7

ReferenceFiles

Reference files for the model template.

You can register model template description files (.txt), source image files (.psd), etc.

The files appear items on the [Reference] button menu in the [Model template selection] dialog box, and when selected, a duplicate of that file can be created.

Display Deformers of the Template Model

You can display the deformers of the template model by enabling [Display Rotation Deformer] and [Display Warp Deformer] in [Layout].


Delete Empty Parameter Groups, Parameters, Parts, and Deformers

You can delete empty elements all at once from each palette's menu.

Delete all empty Parameter groups:

Deletes all parameter groups of the editing model that do not contain any parameters.

You can execute this from the Parameter Palette menu.

Delete all empty Parameters:

Deletes all parameters of the editing model that have no keys in any object.

You can execute this from the Parameter Palette menu.

Delete all empty parts:

Deletes all parts of the model currently being edited that have no child elements.

Any parts that become empty due to the above will also be deleted.

You can execute this from the Part Palette menu.

Delete empty parts in the below child:

Deletes all parts under a single selected part that has no child elements.

Any parts that become empty due to the above will also be deleted.

You can execute this from the Part Palette's right-click menu.

Delete all empty deformers:

Deletes all deformers of the editing model that are not set as parent deformers of any object.

Any deformers that become empty due to the above will also be deleted.

You can execute this from the Part Palette menu.

Delete empty deformers below child:

Deletes all child deformers under a single selected deformer that are not set as parent deformers from any object.

Any deformers that become empty due to the above will also be deleted.

You can execute this from the Deformer Palette's right-click menu.

Apply Mask, Culling, and Blend Mode

Applies [Mask], [Culling], or [Blend Mode] if it is enabled in [Preview].

Note: Blend modes other than "Normal" may not match the appearance of the Modeling View.

Apply Physics Group

Applies the physics group of the template.

Input and output settings will be added if the input or output parameters are applied.

If there is already an input or output setting with the same name, another setting with the same name will be added.

Please note that Calculate FPS will not be applied.

External Application Integration

Summary

The external application integration function allows for communication with external applications to interact with the editor. You can set and retrieve model parameters.

For information on implementing external applications, see the “Developer’s Manual.”

Sample

External application samples are published on GitHub.

https://github.com/Live2D-Garage/CubismExternalAppPluginSamples

Settings

You can set up the connection with external applications from the main menu [File] -> [External Application Integration settings…] or the Physics Settings window's menu [Preview] -> [External Application Integration settings…].

Number

Name

Details

1

Port number

Specifies the port number to connect to.

The default value is “22033.”

If another application is using the same port number, it cannot be used, so please set it so that there are no duplicates.

2

Start/Stop connection button

Start: Enables connection to external apps.

Stop: Disconnects and disables all connected external apps.

3

State of connectivity

Allows you to check the status of connections with all external applications.

4

Allow remote connections

When this checkbox is checked, remote connections with external applications are allowed.

5

External applications

Displays information on currently connected external applications.

Check the [Allow] checkbox to enable the integration.

6

Disconnect

You can disconnect a connected application from the right-click menu.

Enable the integration function

Set the port number.

Next, click the Start connection button.

When it is shown as enabled, connections with external applications will be possible.

Once you have connected the external application you wish to use, check the [Allow] checkbox.

Permission is required to use the external application integration function.

If an unexpected external application appears, do not allow it.

Once a port number is connected, it can be selected from the drop-down list.

To reset the list of connected port numbers, select [Setting] > [Reset port number history].

Remote connection

If you want to make a connection from a different machine, check the [Allow remote connections] checkbox.

Checking it may result in malicious attacks from third parties via the internet.

Please be sure to check your router and local network settings and use it with caution.

Disable the integration function

Click the Stop connection button.

All external applications that are connected will be disconnected.

Insert Key into the Timeline Using the Integration Function

You can insert a key into the model track's timeline with the parameter value obtained from the external application.

Number

Name

Details

1

Still image button

Inserts a key into the timeline using the parameter value obtained from the external application. It is reflected in real-time while holding down the button.

2

Video button

Inserts a key into the timeline with the parameter value obtained from the external application. You can advance to the next frame while holding down the button.

Apply 3D Expression

Summary

Apply 3D Expression is a function that assists in the generation of 3D-like motion shapes.

It targets drawable objects (ArtMesh, ArtPath) and deformers (warp deformer, rotational deformer) and estimates their shapes in 3D space to generate three-dimensional expressions.

How to use

Apply 3D Expression is available in the Modeling View.

Select [Modeling] menu -> [Apply 3D Expression] -> [3D Rotation Expression] to open the [Apply 3D Expression] dialog box.

This dialog box is not modal, so other modeling operations can be performed while it is open.

Screen explanations

Number

Item

Description

1

List

Objects registered under [Register objects] will appear in this list.

Right-clicking in the list will bring up a submenu.

2

Register Objects

Creates a setting for “Apply 3D Expression” to the object selected in the Modeling View.

3

Mapping to Parameters

Sets the parameters for rotation in the X and Y directions.

Click [Change the mapping] to open the window for selecting parameters.
If you want to rotate only in the X or Y direction, select the corresponding parameter and leave the other blank.

4

3D Shape

Configures settings regarding the shape in 3D space.

5

3D Rotation

Configures settings regarding the rotation in 3D space.

6

Target for update

Sets the values to specify which keyforms will be updated.

The four possible values are [All Keyforms], [X direction], [Y direction], and [Corners].

7

Automatically update keyforms when setting values are changed

When checked, changes to [Mapping to Parameters], [3D Shape], [3D Rotation], and [Target for update] will automatically update the keyform.

8

Update keyform

Updates the keyform based on the settings.

You must click this button to estimate the 3D shape after editing the keyform.

Number

Item

Description

1

Flattening

Flattens the shape of the target in 3D space.

Used as a preliminary step to estimate the 3D shape to generate orientation in 3D space as a foundation

2

Depth Estimation

Estimates the 3D shape from the displayed keyforms.

The 3D shape can only be estimated from views other than the front. Because a keyform's default value is the front-view, you must use a non-default keyform value.

3

Make Z symmetrical horizontally at each point (only warp deformers)

Makes the Z position of each point in the warp deformer by depth estimation horizontally symmetrical.

4

Make Z symmetrical vertically at each point (only warp deformers)

Makes the Z position of each point in the warp deformer by depth estimation vertically symmetrical.

Number

Item

Description

1

Camera position

Specifies the position of the camera that captures the object in 3D space.

Specifies XY coordinates in canvas coordinates (Marker will appear in the Modeling View depending on the settings).

Z is automatically changed according to the perspective strength.

2

Center of rotation

Specifies the center position of the object when it is rotated in 3D space.

Specifies XY coordinates in canvas coordinates (Marker will appear in the Modeling View depending on the settings).

Z is the same as the canvas.

3

Offset in Z direction

Specifies how much the depth is shifted in the Z direction.

Specifies the scale to be the same as the canvas coordinates, with the front of the screen as the positive side.

The procedure for shifting in the Z direction is performed before rotation.

If depth estimation is used, it is performed on the estimated position.

4

Perspective strength

Specifies the perspective after the image is taken.

If set to 0, parallel projection is used.

If set to a value greater than 0, perspective projection is used.

5

Angle range

Specifies the limit angle of rotation.

The default value of the parameter is the front, and the minimum and maximum values, whichever is farther from the default value, is the limit of rotation possible.

6

Rotation method

Specifies the rotation calculation method.

This setting mainly affects the appearance of the four corners.

Below is a submenu that appears when right-clicking on the list.

Item

Description

Synchronize selections with views

If checked, the same object will be selected in the View and Settings dialog box.

Display

Changes the display type of the list.

By checking the checkbox, you can set whether the object is displayed by Name or ID.

Copy settings

Copies the settings in [Ties to parameters] and [Rotation settings] of the selected object.

Paste settings

Copied settings with [Copy settings] will be applied to the selected object.

Delete settings

Deletes settings related to the Apply 3D Expression of the selected object.

Objects whose settings have been deleted disappear from the list.

Operating procedure

Create an expression of 3D rotation by completing following three phases.

Each phase is intended to be done in sequence, but may be skipped or redone one step earlier if necessary.

  1. Registration of target objects
  2. 3D rotation in a plane
  3. 3D rotation with depth estimation

Registration of target objects

At this stage, the objects that will be the subject of subsequent operations are registered.

By registering the objects, a configuration is created, and “Apply 3D Expression” can be used.

The operating procedure for this phase is as follows.

  1. Display the Apply 3D Expression settings dialog box.

  1. Select an object (ArtMesh, ArtPath, warp deformer, or rotation deformer) in the Modeling View.

    Using the Apply 3D Expression, any existing keyform edits will eventually be replaced with auto-generated content. Therefore, it is recommended that objects selected here are those that have not yet undergone keyform editing.

  1. Return to the settings dialog box and click the [Register objects] button.
    Once registered, the settings of the selected object in the Modeling View are added to the list.

3D rotation in a plane

At this stage, the keyform is created as a foundation for creating expressions with depth.

Assign a 3D rotation expression to the parameter and set it up so that it can be oriented to add depth.

The operating procedure for this phase is as follows.

  1. Select the object from the list for which you created the keyform.

  1. Select the parameters for rotation in the X and Y directions in [Mapping to Parameters].
    Click [Change the mapping], and a window for selecting parameters will open. Select the parameters for each direction and click OK to update the mapping.
    If you want to rotate only in the X or Y direction, select the corresponding parameter and leave the other blank.

  1. Select [Flattening] in [3D Shape].

  1. Set each item under [Rotation settings]. See the previous screen explanations for details on each item.

  1. Click the [Apply plane form to keyform] button to create a keyform for the 3D rotation expression. The keyform created here is a 3D rotation of the front keyform as a plane.
    If the [Automatically update keyforms when setting values are changed] checkbox is checked, the keyforms will update each time you perform steps 1 through 4. There will be no need to click [Update keyform].

    The keyform created will have the default value as the front, with the minimum value facing left (down) and the maximum value facing right (up). The keyform is created with the one that is farther away from the default value as the maximum angle.


If the parameter has not yet been keyed:

A message box appears asking if you want to add the three key points.

Clicking the OK button will key in the default, minimum, and maximum values for the parameter.


If the parameter has already been keyed:

The keyform of that key’s position is overwritten by the keyform of the 3D rotation expression.

If there is no key in the default value, the creation of the keyform will result in an error and the process will be aborted because the shape that will be the front cannot be determined.

  1. In the Parameter palette, move the values of the parameters to see the keyforms created.
    If there are any corrections you wish to make, return to step 4.

3D rotation with depth estimation

At this stage, depth is added to the keyform created in “3D rotation in a plane” to create a three-dimensional rotation.

The operating procedure for this phase is as follows.

  1. In the list, select the object for which you created the keyform with “3D rotation in a plane.”

  1. Select the parameter specified in [Ties to parameters] in the Parameter palette and select a key that is not the default value.

Since the default value is considered the front, depth estimation is not possible with the default value keyform.

OK: Non-default value

Not OK: Default value (Angle X 0.0, Angle Y 0.0)


The next step is to select the orientation for adding depth. It is best to select areas where it is easy to get a sense of three-dimensionality.

Edit keyforms with three-dimensional awareness in the Modeling View.

  1. Return to the settings window and select [Depth Estimation] in [3D Shape].

  1. Click [Update keyform] to create keyforms for 3D rotation expressions.
    The keyforms will not automatically update after editing even if [Automatically update keyforms when setting values are changed] is checked. Please be sure to update the keyforms manually after editing them.


When you click the button, depth estimation is performed based on the keyform currently displayed in the Modeling View.

A keyform of 3D rotation expression is then created, similar to “3D rotation in a plane.”

  1. In the Parameter palette, move the values of the parameters to see the keyforms created.
    If there are any corrections you wish to make, return to step 3.

To achieve a better look, it is effective to arrange the keyforms in multiple orientations.

Supplementation of each setting item

This section provides supplemental explanation of each setting item and how changing it will affect the results.

Camera position (Rotation settings)

The [Camera position] setting determines how the image will look when rotated in 3D space. It is affected by the perspective strength described elsewhere. The greater the perspective strength, the greater the change in appearance due to changes in camera position.

The figure below shows the change in the rightward keyform when the camera position is changed (the X symbol is the camera position). Moving the camera position to the left gives the appearance of going around to the side of the subject.

Change in keyform facing right when camera position (X) is changed

Center of rotation (Rotation settings)

The [Center of rotation] setting is used to set the center of rotation of the object in 3D space. Specify X and Y in canvas coordinates. Z is on the canvas (fixed at 0).

The figure below shows the movement when the center of rotation is placed on the right and left sides of the canvas. The further away a point is from the center of rotation, the greater the

Center of rotation on right side of canvas

Center of rotation on left side of canvas

Offset in Z direction (Rotation settings)

The [Offset in Z direction] setting specifies how much the depth of the target should be shifted in the Z direction. The shifting procedure is performed before rotation.

If depth estimation is used, it is performed on the estimated depth. By changing this setting, the rotation can be expressed as floating forward or retracting backward from the center.

The figure below shows the motion when the center of rotation is set to the center of the object and the Offset in Z direction is positive and negative.

A positive Offset in the Z direction gives the object the appearance of floating in the foreground. Conversely, a negative Offset in the Z direction gives the object the appearance of being pulled back.

Center of rotation set to center of object and Offset in Z direction is positive

Center of rotation set to center of object and Offset in Z direction is negative

Perspective strength (Rotation settings)

The [Perspective strength] setting specifies the perspective after the image is taken. Increasing the perspective value increases the distortion after rotation and gives the appearance of a closer camera. A perspective value of 0 gives the appearance of a parallel projection.

The figure below shows the change in the rightward keyform as the perspective strength is increased from 0. Increasing the perspective strength will emphasize the difference in appearance between the foreground and background.

Change in rightward keyforms as perspective strength is increased from 0

The figure below shows the difference in appearance of the rotation with different perspective strengths. When perspective strength is 0, the rotational movement is modest, but when perspective strength is 100, the movement is more dynamic.

Perspective strength is 0

Perspective strength is 100

Angle range (Rotation settings)

The [Angle range] setting specifies the limit range of rotation.

Larger values give the appearance of greater rotation. The value specified here is the angle between the minimum and maximum values, whichever is farther from the default value, with the default value of the parameter as the front.

The figure below shows the change in the rightward keyform as the angle range is changed. Larger values give the appearance of greater rotation.

Change in rightward keyform when changing the Angle range

Rotation method

The [Rotation method] setting specifies the rotation calculation method. This setting mainly affects the appearance of the four corners.

The figure below shows the difference in appearance of rotation for each value of [Rotate in X-Y order], [Rotate in Y-X order], and [Rotate on any axis].

Rotate in X-Y order

Rotate in Y-X order

Rotate on any axis

Bilaterally symmetrical/Vertically symmetrical (Depth Estimation)

The “Bilaterally symmetrical” and “Vertically symmetrical” settings for Depth Estimation can be used to automatically make the estimated depth of warp deformers symmetrical. Simply shape one side with keyform editing to create a bilaterally symmetrical or vertically symmetrical 3D curved surface.

The figure below shows the result of Depth Estimation with [Bilaterally symmetrical] enabled. If you make a keyform edit on the right (left) side of the warp deformer, the other side will have the same depth.

Result of Depth Estimation with [Bilaterally symmetrical] enabled

[AI function] Auto Generation of Deformer

Summary

Auto Generation of Deformer is a function that automatically generates deformer structures for the entire body of a humanoid model.

It creates a deformer structure of a typical human figure and automatically estimates where it should be placed in the deformer structure based on the placement of the ArtMesh.

It is possible to quickly create the initial deformer structure from an ArtMesh only, such as immediately after PSD import.

Supported models

This function is intended for the following models.

The following models can be used, but the accuracy of auto generation will be reduced.

This function is not intended for use with the following models.

Deformer structure to be generated

The generated deformer has the following structure.

This structure is fixed regardless of the model on which the function is executed.

However, deformers with no child objects are not generated.

When the function is executed, the ArtMesh in the model is automatically assigned to the children of the deformer in this structure.

The position and size of the generated deformer are determined by the position and size of the child ArtMesh.

How to use

This function is used in the Modeling View. The operating procedure is as follows.

  1. Select [Modeling] menu -> [Deformer] -> [Auto Generation of Deformer] to open the [Auto Generation of Deformer] dialog box.

  1. The dialog box displays a preview of the deformer structure that will be created.
    Here you can see which deformer the ArtMesh will be a child of.

  1. The user can make changes to the displayed deformer structure, such as moving an ArtMesh to a different deformer or removing it from the deformer structure.

  1. After confirming the deformer structure to be created, click the [OK] button to execute the creation of the deformer structure.

Screen explanations

When the function is executed from the menu, the following dialog box appears.

This dialog box allows you to review and partially modify the deformer structure to be created.

Number

Name

Details

1

Deformer structure to be generated

The deformer structure that will be created is displayed here.

When the dialog box is launched, the ArtMesh in the model is automatically assigned to a child of a deformer here, and you can see which deformer the ArtMesh will be a child of when the function is executed.

You can change the parent deformer by dragging and dropping the ArtMesh.

Deformers cannot be moved.

Selected ArtMeshes and deformers are also selected in the View, and their position and size can be confirmed.

2

ArtMesh not used

A list of ArtMeshes not included in the deformer structure can be found here.

ArtMeshes shown here will not change their parent deformer after deformer creation.
When the dialog box is launched, hidden ArtMeshes are categorized here.

3

Remove ArtMesh

This button moves an ArtMesh selected in the Deformer structure to be generated (1) to the ArtMesh not used (2) list.

Deformers cannot be moved.

4

Add to deformer structure

This button moves an ArtMesh selected in the ArtMesh not used (2) to the Deformer structure to be generated (1).

It is moved to the location selected in Deformer structure to be generated (1).

5

OK

Performs deformer generation as per the edited result.

A deformer is created and placed directly under the root of the [Deformer] palette with the same structure as shown in the dialog box, and the ArtMesh is placed in the same manner as shown in the dialog box.

The existing deformer structure remains intact.

6

Cancel

Cancels the editing result and closes the dialog box.

No deformer will be added, and the existing ArtMesh and deformer structure will remain as it was before the dialog box was launched.

Linkage with Auto generation of facial motion

This section describes the linkage between a similar function, Auto generation of facial deformer, and its companion function, Auto generation of facial motion.

This function creates a deformer for the entire body, which includes the facial deformer structure that can be generated by “Auto generation of facial deformer.”

If “Auto generation of facial deformer” is performed after deformers are generated by this function, or if this function is performed after “Auto generation of facial deformer” is performed, unintended generation results may be obtained.

However, it is possible to use this function and Auto generation of facial motion in combination.

When executing this function on a model that has already undergone “Auto generation of facial deformer,” delete the facial deformers generated by “Auto generation of facial deformer” before executing the function.

[AI function] Auto Generation of Sway Motion

Summary

This function automatically generates the movement of swaying objects such as small objects and hair.

The degree of swaying can be adjusted from the dialog box while viewing the generated motion, streamlining the process of adding motion to swaying objects.

Scope of functionality

The following two types of motion attachments are supported.

Generate swaying:

This generates movement such as hair swaying from side to side in the wind.

Generate droop by angle Z:

This generates a downward motion such that when the head is tilted, the hair hangs downward due to gravity.

Creating complex sway movements that do not fall into these categories may not result in the intended movement.

This function also applies deformation to the warp deformer containing the ArtMesh of the area to be subjected to motion.

Motion attachment of sway using the rotation deformer is not covered by this function.

Screen explanations

Number

Name

Details

1

Deformer List

The warp deformer selected here becomes the target of the sway deformation.

2

Register Warp Deformer

Registers the warp deformer selected in the Modeling View to the Deformer List.

3

Parameter

Selects the parameter to become the target of the sway deformation.

4

Synchronize with parameter palette selection

When checked, the setting window will synchronize with parameter selections in the Parameter Palette.

5

Apply parameter palette selection

Selects the parameter selected in the Parameter Palette.

6

Preset

You can reset sway settings to default values with two settings: "Hair Sway" and "Drooping by Angle Z."

7

Root Location

Change the root position of the swaying.

You can choose from "Top," "Bottom," "Right," or "Left.”

8

Reflect sway horizontally

When checked, the generated sway direction reflects horizontally.

9

Adjustment of sway deformation

Adjusts the setting values that define the degree of sway deformation.

Descriptions of each setting value are given below.

10

Synchronize left and right settings

When checked, the setting values of the keyform (left side) and keyform (right side) of Adjustment of sway deformation are synchronized.

It is checked by default. Uncheck this checkbox if you want to create asymmetrical sway deformations.

11

Automatically update keyforms when setting values are changed

When checked, changes to the setting value will automatically update keyforms.

It is checked by default.

12

Update keyform

Click the [Update keyform] button to start calculating the shape of the sway deformation and update keyforms.

Below is a submenu that appears when you right-click on the Deformer List.

Item

Details

Synchronize selection with View

The same object is selected in the Modeling View and settings window if checked.

Display

Change the display type of the list.

Enter a check to set whether to display the objects by "Name" or "ID."

Copy settings

Copy the settings in the [Sway Settings] of the selected object.

Paste settings

Copied settings with [Copy settings] will be applied to the selected object.

Delete settings

Delete the settings related to the selected object's Auto Generation of Sway Motion.

Objects with deleted settings will disappear from the list.

Effect of each setting value in adjusting sway deformation

Number

Name

Details

1

Sway strength (horizontal)

Sets the strength of the horizontal oscillation.

2

Sway strength (vertical)

Sets the strength of the vertical oscillation.

If the value is set lower than 0, movement can be applied as if it is being pulled downward.

3

Attenuation level

Adjusts swaying of the upper and lower parts of the deformer.

If the value is increased, the upper part will almost stop moving and only the lower part will sway.

4

Zoom In/Zoom Out

Deformations can be made to increase or decrease the width of the deformer. Larger than 0 means expansion, and smaller than 0 means contraction.

Note: For sway strength (horizontal) and sway strength (vertical), reasonable values are automatically estimated based on the target deformer information (size, etc.) and used as initial values.

Note: If you reapply Auto Generation of Sway Motion to the deformers and parameters with sway motion, the values from the previously created sway motion setting will become the default values. The model will retain these settings even when reloaded.

Operating procedure

This function is used in the Modeling View. The operating procedure is as follows.

From the menu bar, select [Modeling] -> [Parameter] -> [Auto Generation of Sway Motion...].

Once selected, a window like the one shown in the screen explanations will appear.

  1. Select the deformer you want to add sway motion to from the Modeling View. Click [Register Warp Deformer] in the window to add it to the Deformer List.
  2. Select the parameter you want to apply sway motion to from the parameter selection box.
  3. Change the setting values as desired and click [Update keyform] to apply the sway deformation to the keyform.

The settings will automatically update the keyform when you check the [Automatically update keyforms when setting values are changed] checkbox. This checkbox is checked by default when the window opens.

You can adjust the sway deformation in this secondary window while checking the movements with the parameters and Modeling View in the main window.

  1. When you have completed the motion application, close the dialog box by clicking the [×] button.

Automatic key generation

When the sway deformation is reflected in step 2, a process is executed to check if there are three keys for the target parameter.

Selecting [OK] deletes the existing keys and creates three new keys.

If [Cancel] is selected, the existing keys will remain in place and the sway deformation will not be reflected.

Limitations

This alpha version includes some unimplemented features, known defects, and other items of concern that should be specifically noted. Normally, these limitations will be dealt with when upgrading from the alpha version, but this is not guaranteed.

Modeling

Enhanced template functions

Model template information compatibility

Model template information (.template.json) created with the alpha editor cannot be used with the beta editor.

Default model template

In the alpha version, the model template will initially be empty, but in the beta version, multiple model templates will be available by default.

Apply 3D Expression

Deleting all of the keyforms will cause inconsistencies

If you delete all the keyforms of the edited object in Apply 3D Expression, the 3D rotation expression will be lost and the object will be fixed in the shape it was at the time it was displayed.

From this point on, if the parameters are rekeyed and the keyform is created again with depth estimation, the results will be inconsistent with the previous depth estimation results and correct estimation will not be possible.

If you delete all the keyforms, you will need to start over from 3D rotation in a plane (preferably after restoring the original shape, if possible).

Variation with perspective strength depends on target size

In alpha1, there is a problem in the way the perspective strength is calculated. The larger the target size, the more extreme the change.

This issue will be fixed in alpha2 or later.

Note: Fixed in 5.1 alpha2

Setting the Angle Range to 0 May Cause the Object to Disappear

As a known bug, an object may not display if the rotation method is set to [Rotate on any axis] and the angle range is set to 0. This bug will be fixed in alpha3 or later.

Note: Fixed in 5.1 alpha3

[AI function] Auto Generation of Deformer

Cannot run if there is an ArtMesh outside of the canvas

When an ArtMesh is outside of the canvas and Auto Generation of Deformer is executed, the Auto Generation of Deformer dialog box does not appear.

Make sure all ArtMeshes are within the canvas before performing this function.

This issue will be fixed in alpha2 or later.

[AI function] Auto Generation of Sway Motion

Sway deformation adjustment settings are not saved

The setting values for the sway deformation adjustment are not saved. Closing and redisplaying the dialog box returns the settings to their default values.

Note: Fixed in 5.1 alpha3

Target warp deformer and parameters cannot be changed

Target warp deformer and parameters cannot be changed while the dialog box is displayed. To make changes, close the dialog box once.

Note: Fixed in 5.1 alpha3

Cannot be used on Blend Shapes

Parameters for Blend Shapes are not supported.

This will be addressed in beta or later.

"Undo" and "Redo" actions in the window box may not work correctly

Repeatedly executing "Undo" and "Redo" may result in unstable behavior.

Errors when used with extended interpolation

Running Auto Generation of Sway Motion with parameters that have extended interpolation enabled may result in an error.

Deformers do not retain sway settings when transferred to other models

A warp deformer created by Auto Generation of Sway Motion and copied or exported into another model does not retain the sway settings applied in the source model. When you open the Auto Generation of Sway Motion window again, it will display the warp deformer's default values.

Animation

External Application Integration

Keys are Not Inserted into the Timeline According to the Correct Frame rate of the Scene When Inserted with the Video Button

Timeline-related stabilization will be addressed in alpha3 or later.

Note: Fixed in 5.1 alpha3

Unable to obtain model information in the physics settings window when connected to an external application in the physics setting on the first try.

This will be addressed in beta or later.

Common

About AI Technology

AI on-board functions

The following functions use AI technology.

Auto Generation of Deformer:

AI technology is used to estimate which part of the model is which from the image information of each object.

Auto Generation of Sway Motion:

AI technology is used to estimate a reasonable amount of deformation based on information such as the position and size of the target deformer.

Learning data

Among the works submitted to nizima, those that have given us permission for use in AI research are used as learning data. For more information, see the Live2D AI Research Policy and Request for Cooperation.