Saxon Rix | Head of Animation & Tech – Stada Media
Saxon Rix
Saxon Rix heads up the animation department of a video production company and discusses how Cartoon Animator (CTA) is his 2D animation software of choice when he needs to meet tight deadlines for clients.
Cartoon Animator’s power and flexibility allow Saxon and his team to rig and animate custom characters in record time, saving valuable time and money. In this interview, Saxon also gives advice to other artists, animators and developers who work with or are considering Cartoon Animator for their business projects.
.
“Cartoon Animator gives us the freedom to create our own characters and animations from scratch, all within a streamlined workflow.”
Saxon Rix | Head of Animation & Tech – Stada Media
The “Winning Tips & Tricks” series covers practical workflows and techniques shared by winners from the “2021 iClone Lip Sync Animation Contest”. To let users see the full spectrum of the iClone facial animation pipeline, we are introducing projects that are rendered with different 3D engines including; Unreal, Unity, iClone, Blender, and Cinema4D. Let’s now take a look at Jason Taylor’s: “Animal Farm”, combining Character Creator (CC3) / iClone workflow and Unity render.
Jason Taylor
My name is Jason Taylor. I’m a filmmaker and multimedia artist, running my own digital media studio, Jumping Guy Productions in Ontario, Canada.
I’m the creator and producer of a puppet-based video series for kids which I created using real-life puppets in a 3D animated world via Unity in 2016. Since then I have been working on developing professional 3D character animation skills in an effort to diversify my skillset and tell stories that I couldn’t with puppets. Reallusion’s Character Creator and iClone have been a real game-changer for me in this regard.
Why I created this showcase
Working part-time over a few evenings or so, I was able to bring a scene from Animal Farm to life. The story needed a variety of characters from talking pig characters, a talking horse to a variety of rough-looking farmers. With relatively little effort I was able to create eleven original AAA characters for my video in a matter of hours using Character Creator. From there, once I exported the characters over to iClone, I was able to easily add facial expressions, animate the head and easily add high-quality lip-syncing to my characters before I exported them over to Unity for composition and final render.
How I do it with iClone
Step 1: Building Five Animal Characters
To create the characters I started with CC3 compatible base characters from the Reallusion Marketplace and content stores. Rather than reinventing the wheel, I was able to take a simple clean looking pig character and add facial hair, eyebrows, clothing and morph the facial features to create the human-like pigs that the story called for.
Step 2: Building Five Human Characters
The process for creating the human characters was similar to the pigs. I started with simple base characters similar to what I had in mind for the story and then I morphed their faces, added facial hair and unique hairstyles and the proper clothing. For one of the characters, a farmer character, I was even able to add dirt on his face like he just came in from working in the field.
Step 3: Morphing Horse Character from Deer
Blender helped with pretty much everything else. Using simple shapes, I was able to sculpt them into jewelry, pants and even a braid-looking thing. And Blender has for the creation of the horse character — this one stumped me at first as I couldn’t find the style of horse I was looking for in any of the Reallusion stores. I had a specific style in mind and my horse also needed to be compatible with CC3’s facial morphs system for animating the mouth using AccuLips. My solution was to purchase a compatible deer model that came with an isolated head model that I was able to stretch and morph and shape into the horse base that I had in mind. I then added custom textures/materials that I photoshopped together from an actual image of a horse. For the final touch, I added some stylish hair that I purchased from the marketplace.
Step 4: Animating the Body and Hair
The facial emotions and lip-syncing abilities, paired with the live facial mocap abilities with LIVE FACE the iPhone mocap plugin for iClone 7 have really blown me away and have allowed me to take my animations to the next level, imbuing my characters with personality and emotion. As a general rule for myself, I start with lip-syncing and then add emotions afterward. To create the lip-syncing, I simply import the audio of my character speaking as well as the text to go along with the audio and iClone’s AccuLips give me a solid starting point that I am able to tweak to my liking, adding more or less jaw motion, more or less lip motion and the ability to add, remove or move details. There are also presets I can use like “Singing” or “Whispering”, that I can use as-is or tweak to my heart’s desire.
The scenes I picked to recreate in 3D had little to no character animation. Nevertheless, I took advantage of the Motion Puppet tool to create some simple movements and avoid having the character stand in place like a statue the whole time. In the first sequence, for example, I positioned his hands above his head and once I am committed to the finalized result of the lip-sync, I’ll add emotions using iClone’s library of emotion, e.g., “Happy” or “Sad” or “Angry” etc. I can keyframe these emotions in, blend them together from one to the other, adjust the intensity in terms of how happy or how sad, and so on, to bring my character and story to life.
On top of my keyframed emotions and lip-syncing from AccuLips, I like to add facial mocap data to the mix to give things a more human-like realistic look and feel. I go subtle here though. Using my iPhone connected to LIVE FACE in iClone, I can add a layer of head rotations and another layer of facial expressions from the mocap data that is blended together with the original.
Step 6: Animate the Head
For this project, in addition to the head movements from the mocap data, I also animated some of the larger head movements using the “Look at” option in iClone. This allowed me to give the character something to look at directly, which in the end gave the character more of a believable performance. To do this, I animated a cube to the various points I wanted my character to be looking at. I then tweaked and keyframed the options of how much the character’s eyes were looking at the cube without the neck and head moving versus how much the character turned his full head to look at the object.
Final Step: Tips to do Composition in Unity
And finally, it was time to add all the cool effects, such as smoke, lighting, magical butterflies, and of course, floating dust particles. I had already bought as a Unity user, I was happy with how great the characters looked in Unity once I exported them from CC3 and imported them into Unity. I used the Unity / CC3 Auto Setup Addon which helped to give me good results on the materials in Unity HDRP shader mode. After I did a bit of custom tweaking to the materials and hair and I was off to the races. If something came up in Unity with one of the characters that I wasn’t happy with given how my scene was building out, e.g., a character’s nose, head shape, eyes or arms, etc. It was no trouble to re-export the character from CC3 and then re-import to Unity without losing references to 3D character animations etc. The same went for the facial animation and lip-syncing. Once I saw things come to life in Unity, I was able to go back to iClone, make some adjustments accordingly to timing and expressions and lip-syncing and I could re-import things back into Unity seamlessly.
Final Thoughts and Conclusions
Working with 11 different characters on a 30-second clip, this process could have taken me weeks or months to do. And, it’s not a stretch to think that this would be the kind of endeavor that a studio team would take on. But no, I was able to do this on my own in a very timely manner, working part-time in between my main project. Overall, I’m glad I gave this project a shot and I’m happy it was even possible thanks to Reallusion’s professional tools. I’m excited to see what comes next from Reallusion!
In the prior WIP I, we envisioned a future of efficient and enjoyable process of 3D character animation, beginning with the unveiling of Motion Director and cutting-edge interactive motion editing functionalities.
For WIP II, we’d like to highlight newer features built into iClone 8 that will help pro studios and indie developers maximize their productivity. Not only are these time-saving features essential to producing high-quality 3D character animations, but they can also greatly reduce the cost of production.
Rotoscope : pose-to-pose editing
Rotoscope is the easiest way to start learning pose-to-pose 3D character animation. Anyone can find video references to set keyframes and repeat this process to complete whole motion cycles.
• Apply live-action footage (MP4) for reference to a 3D plane, and pose your character accordingly.
• Set 5 keys for half of the animated sequences. Copy them and click ’Full body mirror’ to flip the poses.
• Use offset keys to create an overlapping effect, and use Transition Curve to refine the timing, as shown below.
Pose Mixer : pose-driven motion synthesis
Pose Mixer is a handy tool that generates new poses from existing sources with just a few clicks: select a source pose, define the mixing area (from the body mask), and mix with the target pose.
• Replace selected body parts with other poses you like and mix them in different ways.
• Choose the body parts which you like to Flip Directions, Mirror Copy, or build Symmetrical Poses.
Mix-and-matching motions can be a brilliant way to come up with new ideas and happen upon creative possibilities. You can think of this method of working like the cut-up technique employed by Dadaist poets to the songwriting of David Bowie, where “happy accidents” are left to flourish.
Expandable Character-centric Timeline
Most 3D tools provide timeline controls for character bones and blendshapes, but the sheer number of tracks can make editing stunningly hard. iClone 8 introduced Expandable Tracks on both body motions (2 levels) and facial expressions (3 levels), which allows creators to focus at primary editing first, and later use the advanced timeline controls for the details of the facial and body animation.
• Body-Motion Tracks (2 levels):
Part level can be used intuitively by selecting general body parts to edit.
Bone level provides advanced editing to all body bones and all the way to fingers.
Users can fully control the interpolation between FK/IK through the FK/IK Blend track.
• Facial-Expression Tracks (3 levels):
❶ Basic : Eyeblink track is freshly added, in addition to pre-existing Muscles, Eye and Head tracks.
❷ Feature: Access 16 prominent facial feature groups from the brow, jaw to ear.
❸ Morph: 60 to 140 morphs are provided to refine the expression details, the blendshape number is determined by the selected level of expression profile (Standard or Extended).
Curve Editor : Professional Cartoon Performances
The Curve Editor is now built seamlessly into iClone 8 with no additional purchases necessary! In addition to bone and blendshape support, iClone 8’s Curve Editor comes with FK/IK effector controls and body/facial 3D animation curve tools.
• Greatly reduce time and effort to create Cartoon Performances.
• Create curve animation for Arcs, Follow Through, Exaggeration, and sudden movement with Steps.
You can check the 3D Character Creation & Animation webpage for more information about the free upgrade offer, feature comparison, and FAQs. If you’re interested in the Play-to-Animate feature of the Motion Director system, you can refer to the article ‘Working in Progress: Part One‘. For more technical detail, please have a look at our latest New Features Introduction, which will be updated continuously in the coming months.
Animation production as we know it — the type practiced by Hollywood studios — is a laborious process. But with the help of the latest animation technology from Character Creator 4 and iClone 8, a good performer can now create professional-quality animation in a fraction of the time, thanks to real-time animation.
A good example comes from Enoc Burgos, the Director of Partnership Marketing at Reallusion, who was invited by the art team to test out the new character setup for the upcoming Character Creator 4. His spontaneous performance ended up getting more than 400,000 views on his personal LinkedIn account, with more than 17,000 likes and 500 shares.
Read on to understand the workflow behind creating and animating a coyote character using Character Creator 4 and iClone 8.
Step 1: Set up characters the way a concept artist usually does
The process starts with a character expression sheet for style setup.
Step 2: Streamline process to customize facial expressions
A base character in Character Creator supports 70-plus facial expression templates and 100-plus custom templates. Users can choose to customize partial blendshapes or replace all blendshapes at once.
Character Creator 4 includes a GoZ function that lets the user send a character to ZBrush to sculpt expressions in an intuitive and efficient way. To present iconic expressions for this coyote, a total of 114 blendshapes were customized.
Once sculpted, users can update the results to Character Creator 4 by filling in the target expressions into the expression templates.
Step 3: Test perform using face key / puppet
To review the results, animators can use the Face Key editor in Character Creator 4, or send the character to iClone 8 to test facial animations with the Face Puppet tool.
Step 4: Live perform using iPhone
iClone 8 is compatible with several mocap devices, including an iPhone as the character blendshapes have already been one-to-one mapped to the iPhone ARKit signals for precise results.
In this particular case, a Reallusion artist converted a basic-rigged coyote, purchased from Sketchfab, into a mocap performance-ready character. This greatly simplified process proves to be at least five times faster than any custom facial rig, which usually requires a team of artists with different skill sets for modeling, rigging, animation, and more.
The Character Creator 4 Work in Progress video highlights the complete process from characterizing any character rig by mapping bones to a HIK system, to customizing the facial rig. This makes characters fully compatible with iClone for lip-syncing, facial/body animation, and ActorCore mocap animations to save time in production.
Try It Out For Yourself
Character Creator 4 and iClone 8 are scheduled to be launched in the spring of 2022. Artists who haven’t had the chance to use Character Creator and iClone before can try then now: 30-day trials are available for Character Creator 3 and iClone 7 with over 1,000 ready-made content assets (click links for more information).
Reallusion provides free upgrade offers and combo specials for Character Creator 4 and iClone 8. See the 3D Character Creation & Animation webpage for more information.
Character creation and animation are two distinct disciplines that demand the skill of well-trained artists with specialized background knowledge. These domains can be difficult and frustrating for artists who come from non-related backgrounds with different skill sets, and it’s a pain point that Character Creator and iClone were specifically created to resolve.
Furthermore, Character Creator is well-positioned as the go-to solution for creating realistic and stylized characters with robust pipelines for mainstream tools like ZBrush, Substance, and Blender. Combined with the ability to export FBX with LODs (levels of detail), digital human shaders, and a rich collection of motion assets, iClone stands out as an animation editor without a steep learning curve.
The launch of NVIDIA Omniverse in 2020 was a momentous occasion that attracted our attention. Omniverse represents the next-generation 3D virtual collaboration and real-time simulation platform that connects people and applications for broad-based collaboration. Reallusion software suite combined with a massive library of high-quality character assets and motions would play a crucial role in this ecosystem, while Omniverse provides the path-traced rendering and AI technology that makes for a powerful synergy.
Where to start?
To build the connector, we start by referencing the “Connector Sample code” available for download on the NVIDIA launcher application.
iClone and Character Creator’s 3D scene consists of nodes with basic transforms consisting of translation, rotation, and scale values. Characters, meshes, lights and cameras are all attached under these nodes. Furthermore, the bones of a character skeleton are also represented by these nodes. Nodes that only represent transforms are exported as USD Xform and nodes that represent the body and facial bones are exported as USD skeleton joints. Additional bones and skinning are added to accessory nodes that are attached to bones before converting to USD format.
Scene graph of transforms (Xforms) and joints
USD Xform scaling works in fundamentally different ways to iClone. In iClone, a node can be made to inherit or ignore the parent scale, whereas, in Omniverse, the node scale is always inherited from its parent node. Under such conditions, bone node scale inheritance must be removed with its values reset before exporting to Omniverse, in order for the scale values to match.
Most of iClone’s facial expressions are composed of morph animations which are exported as USD blend-shapes. In contrast to FBX blend-shapes which are stored in local-space positions, USD blend-shapes store positional offsets. Since iClone also stores positional offsets (in combination with strength multipliers), it is completely compatible with Omniverse, allowing for direct conversion to USD format. It should be noted that Omniverse requires a skeleton root for props with blend-shapes attached and additional processing may be required.
Within the USD file, MDLs are specified using info:mdl:sourceAsset with info:mdl:sourceAsset:subIdentifier as the entry function. The new subIndentifier attribute was introduced by NVIDIA and PIXAR. input: is later called to feed the texture and material parameters. OmniPBR.mdl and OmniSurface(Base).mdl provided with Omniverse were used as starting points.
OmniPBR.mdl was chosen from the start because it works well in both RTX Real-time and RTX Path-traced mode, on the other hand, OmniSurface and OmniHairare photo-realistic MDLs for RTX Path-traced mode. The existing PBR, Skin, Head, and SSS shaders were then rewritten from HLSL to MDL.
Besides the above mentioned built-in MDLs, there is also a base.mdl on GITHUB with some reusable functions that can be deployed in a jiffy.
Light Conversion
Point lights and spotlights use UsdLuxSphereLight with adjusted cone angles. Tube and rectangle lights use UsdLuxCylinderLight and UsdLuxRectLight respectively. Light IES profile file is also one of the shaping attributes. Light intensity in USD is similar to Luminous intensity per unit surface area. The USD intensity of a spherical light with radius (r in meters) can be approximated with the following formula:
USD intensity = candela * 1000 * / (4PI r*r)
The following formula is used when radius is in centimeters:
Radius is a significant attribute in Omniverse Renderer. We impose a minimum radius of 2 cm as suggested by NVIDIA. See our lighting tutorial for more examples.
What’s next
A prototype of a one-way live sync connector is under development. Since iClone’s undo/redo system is similar to the memento pattern, a table for keeping track of live objects with universal IDs is used. This table will update after performing an undo and redo function.
Weien Chen is a senior software engineer on the Graphics team at Reallusion. He participated in shipping iClone 7/Character Creator 3 and their Omniverse Connector. He also worked on Reallusion PopVideo enhancement, VXGI and PopcornFX integration and Reallusion skin, SSS, and hair shader development.
Benjamin Sokomba Dazhi (Benny Dee) – Founder of Just Art Animation Studios
Benjamin Sokomba Dazhi
Benjamin Sokomba Dazhi (Benny Dee) is a renowned Nigerian animator, film producer and director. He hails from the north-central Nigeria, Niger state. Benjamin started his artistic craft as a rapper back in the Federal University of Technology, Minna as part of a hip-hop duo called “Hot Mics”.
Following his love for video production, Benjamin gradually ventured into the production of music videos for artists. In 2014 during a year long academic staff strike, Benjamin decided to improve his skills in 3D cartoon animation. He found his breakthrough in animation by establishing his own Nigerian animation company, then called Night Light 3D Works and now renamed Just Art Animation Studios that specializes in character modeling, character animation, music video producing and special effects.
In late 2020, Benny was approached by the team at Turner Media to help create a series of dance animation challenges – CN Dance Challenge, for Cartoon Network Africa where Benny used iClone 8, Character Creator 4 and Rokoko SmartSuit PRO motion capture suits. Benjamin was recently announced as the African Head Animator of a Cartoon Network dance instructional series, “Dance Challenge”. This deal according to Cartoon Network is to reiterate Cartoon Network’s commitment to creating contents that will make its African audience feel at home.
“Reallusion’s iClone and Character Creator are breakthrough solutions for the animation industry, as they have brought convenience for network studios and small companies to bring their ideas to life in the most professional way. Reallusion simplifies professional tools for even novice users trying to pursue a career in animation. “
Benjamin Sokomba Dazhi (Benny Dee) | Founder of Just Art Animation Studios
Founded in 2000, Reallusion software simplifies character creation and animation like nothing before. Major products include Character Creator (CC) for character creation, iClone (iC) for facial and body animation, and ActorCore an asset store dedicated to scan-quality characters and AAA mocap animations.
In early 2021, Reallusion introduced the Blender Character Pipeline, a streamlined workflow between iClone, Character Creator and Blender. Now, Reallusion is commiting to the Blender Development Fund and becoming a Corporate Silver partner while planning a string of new features that will harness the best of both programs to unlock entire new synergies.
