In this 4 part tutorial series, Marko documents the process of creating a fully-rigged character in Character Creator 3, animating the character with iPhone facial mocap and the Reallusion motion library in iClone, and importing character and motion into Blender. Marko showcases two methods in working with motion data in Blender, one is using Auto-Rig Pro, another is using Non-linear Action Editor (NLA).
Marko Matosevic is an enthusiast with Blender for over 8 years, runs his own YouTube channel (YouTube.com/Markom3D) and is now a multi award winner of several short films using the Reallusion suite and Blender.
PART #4 – COMBINING FACIAL SHAPE KEYS AND ARMATURE ANIMATION IN BLENDER WITH AUTO-RIG PRO
Exporting character from either iClone or Character Creator 3 and importing them straight into Blender, will only import the default rig. If you want to be able to refine your motions, but would like more controls, such as IK and FK tools, then using Auto-Rig Pro will resolve this issue. Auto-Rig Pro as well will speed up the process of applying motion capture data from other sources onto your characters, increasing workflow times. There are some tricks to getting this flow working correctly, but once it is set up, you won’t have to do it again for that character.
In this article, we will go through the process of exporting your character from Character Creator 3, importing it in Blender. From here we will use Auto-Rig Pro to create a new rig that allows more control over the default armature, whilst still being able to apply facial motion capture to the character. This is the last part of the series where we will go through the full process of exporting an animation with both body and facial motion capture, importing it into Blender, and then rigging the character with Auto-Rig Pro.
Export Character from Character Creator
Once your character is complete, it is time to export your character from Character Creator 3. Simply select File > Export > FBX (Fully Clothed). This will bring up the panel below. In the panel, you will see the setup for exporting to Blender.
Set up the Target Tool Preset to Blender, then change the FBX options to Mesh as we only export the character, and not any motions. The next step is a personal preference, but unticking the Embed Textures box as this allows for the .fbx to be smaller in size, and for all the textures to be placed into the one main folder. The bind pose of CC3 character base (CC3+) is A pose. What this means is that if the T-pose armature is removed at any point, the mesh of the character will revert to the A-Pose.
Import Character in Blender
From here we jump into Blender and select File > Import > FBX, we then find our fbx file and simply double click on it. The first thing that we need to do is select the armature properties, expand Viewport Display, and enable In Front. This will bring the armature to the front of the mesh and will come apparent later when we use the previous rig as reference for the Auto-Rig Pro rig.
What we need to do now is to lock all the vertex groups on all the objects. This is because part of the face is controlled by bones that Auto-Rig Pro is not set up for. To do this, we select each object, and in the vertex tab, select the drop down, and Lock All. The most important objects that this needs to be done for are the Body, Eyes, Teeth and Tongue.
Rig the Character with Auto-Rig Pro
From here we will select all the objects of the character minus the armature, and in the Auto-Rig Pro panel, we will expand Auto-Rig Pro: Smart, and select Get Selected Objects.
Once we press Get Selected Object, the view of the scene will change to just the objects that we selected previously (The character). We will then need to unhide the armature by going into the scene collection, which is normally at the top right, and clicking the closed eye to unhide.
From here we can go back to the Auto-Rig Pro: Smart tab ,and click Add neck. A green circle will appear over the mesh and left click to set the marker on the character, using the previous rig as a guide. Do the same for the chin, shoulders, wrists, spine and ankles. Once they have all been set, click Go. This will then generate a rig. From here ensure that all the objects are selected once again and select the new rig last. Under the Auto-Rig Pro tab, select Skin, and then Bind.
Merge Armatures
Now for facial motion capture to work, we need to combine the two rigs. Select the original rig and go into Edit Mode. We need to select all the bones from Neck.01 and down, and delete them. There is a little catch here and that at the base of the Facial bone, there is a tiny bone called Head. We will also need to keep that one.
Once we have done that, go back into Object Mode, select the two rigs, and press Ctrl+J to join them. Now we can go back into Edit Mode for the single armature and select the head bone that was well hidden at the root of the facial bone. Now we can fix the relationship of this bone and attach it to the whole rig by selecting the Bone Tab and setting the parent to c_head.x.
Apply Body Motions
Now it comes time to add motions to the character. Select the Animation tab at the top of Blender, and change the editor type of the top left window to “Nonlinear Animation”.
We will start off with the easy part and that is adding the shape key animations (Facial Animation) to the Body, Tearline and Eye Occlusion. If you imported your character from iClone, then the Shape Key actions will already be assigned. If you are transferring over shape keys from one character to another, take note of the source objects Shape Key action names. In the NLA Editor, select a track, and press Shift+A, and add the corresponding shape key. If nothing happens when you have added the Shape Key Action, there is a good chance that you have selected the wrong action.
To apply Body Motion, from another source where the names of the bones do not match up, you will need to retarget the armature. From here we need to select Auto-Rig Pro: Remap, select our imported armature as the Source Armature, and the Auto-Rig Pro armature as the Target Armature. From here we can select the Build Bones list and this will create the list of bones that we need to match.
Fast Way to Map Bones with Preset Template
If you are using a character from iClone, Reallusion has already created this mapping for the Bone list, and its download link is available on the video description. Once you have downloaded it go down to Mapping Pre-set and select the folder, open the file, and click import. Everything should now line up. If we were to press retarget now, we would have a lot of issues.
First thing we need to do is select the armature, click search (Spacebar or F3 or if on a laptop fn+F3) and search for Make Local, and select All. Next select the Auto-Rig Pro Rig and click Redefine Rest Pose.
Select the ARP Rig, press Tab to go into Pose Mode, select everything, and then under Redefine Source Rest Pose, select Copy Selected Bones rotation. From here go back into Object Mode, select the source armature, Tab to do into Pose Mode, select everything, and then once again under Redefine Source Rest Pose, select Apply. Now you can select Retarget and everything should just work.
We have now successfully setup a character with Auto-Rig Pro in Blender, whilst keeping all the important data and features from iClone and Character Creator 3.
PART #1 – Character Creator to Blender with Auto-Rig Pro
PART #2 – Animating Characters in Blender
PART #3 – Facial Motion Capture for Blender
