The Making of Ancestories XR Cinematic Game Trailer

This article is featured on Creativebloq.

José Antonio Tijerín

José Tijerín is a digital illustrator, 3D sculptor, and creator of video games such as “Dear Althea” available on Steam. His content pack “We’re Besties” is currently for sale in the Reallusion content store.

In this tutorial, Tijerín explains how Character Creator (CC) and iClone helped to make a fund-seeking cinematic trailer for the Ancestories XR video game. Read on to discover the possibilities of Reallusion software to attain professional and industry-standard results sans expensive equipment and enormous financial support.

The Ancestories XR Project

José Tijerín: This promotional trailer was made to bring awareness to AncestoriesXR.com. We are telling stories about American history and stories of individual families. We are building a company that uses narrative video game techniques to create connections between younger and ‘digital-first’ generations to their family history and their combined legacy to American history. Millions of Americans can trace their family history back to the time of the Civil War, and this promotional video makes that history come alive. We don’t stop at animation; we also add interactivity so people can learn and explore via modern game techniques.

Planning the Project

The first thing to do, in a really big project like this, was to spend weeks visualizing the result. Then we developed a plan that would allow us to get there in the most efficient way possible, to not waste time and resources.

The most important aspect was to define a storyboard and main character according to the narrative and audio files provided to us. In order to emphasize the aim of the project, it was decided that an old family photograph was to come to life to tell a story—specifically, an old African-American Union soldier in army uniform.

Creating the Character

To create the central character, I used Character Creator 4 together with Reallusion’s Human Anatomy pack, and after modifying the base character to resemble the references that I have gathered, I took it to Zbrush for retouching until I achieved the desired look. When sculpting, I usually do so without perspective in order to keep all polygons in sight, as lens distortions can augment the actual shape of the character. Modeling should not be approached in the same fashion as drawing: It’s important to constantly check all angles of the character and the shadows that form on its 3D face.

After that, I got to work on the character’s add-ons, most importantly, his beard. This element will be responsible for giving shape to his face and personality, so it was crucial that it was exactly the beard I had in mind.

For this, I chose one of the beards that comes with a mustache which was also offered as a base in Character Creator, and took it into Zbrush for reshaping. Even then I added more hair cards with transparent textures of thick and kinky hair for a more fluffy and realistic look.

You can see the beard layers that were progressively applied in the image below. And thanks to Character Creator’s efficient system, the beard automatically adopts the weight maps of the 3D face (when the program is kept open). If we close Character Creator during this procedure, then we’ll have to manually synchronize the morphs of the face and beard when it comes time to animate facial expressions and perform lip-sync. It’s also important to test out 3D character animations, and in this case, we had to make sure the beard didn’t deform, due to its length, while the animated character turns his head.

To make the clothes and objects of the scene, I had to do a lot of research and ask for help from historian Retha Hill of the AncestoriesXR group. I usually use, as was this case, Maya and Zbrush for modeling and Substance Painter for texturing. While Character Creator makes clothing setup fast and adaptable, we still had to make sure the clothes were error-free while the animated character moves about.

CC 4 lets us apply an animation test to check for possible errors with clothing and accessories, and allows us to correct them and prevent further complications later down the road. In the same way, it is now also possible to test, modify, and add facial expressions.

Animating the Character

After having done all of the necessary checks, including correcting the position of the teeth and tongue—which is often overlooked—it’s time to export the character for animation in iClone, and after having set up the scene with props, I then applied lip-syncing. Adding audio to the character as the base for the rest of the 3D character animation made the most sense, as it gave us a clear guide to match the voice actor’s performance with the character’s movements and, ultimately, a much more realistic animation.

iClone also offers manual and easy ways to correct the lip-sync and match them to the words pronounced by the character, as well as intensity and emphasis of the syllables, which helped to get the animation just right.

After having completed this process, it was time to use the revolutionary Digital Soul content pack available in the latest version of iClone. With this tool, we were able to apply subtle facial animations that contributed to the ‘presence’ of the character and enhanced his performance with the lip-sync already applied.

Simply by adding one of the many animations available in the package, the character came to life in a solidly realistic way with subtle and natural movements that made the character feel more human. This became a great base animation, on top of which, we applied additional motion keys with tools such as the real-time Face Key and Face Puppet tools, to perfect and customize the performance until we had what we were looking for. We were also able to layer on additional animations from Digital Soul without a loss in animation quality. It’s important to keep trying out several options and retouching the final animation, to give the performance a ‘genuine’ touch, fitted for the specific scenario.

After having defined the facial expressions, it was necessary to add an animation to the body to enhance the performance of the character. This should be done even if the camera shots do not capture the character completely, for example, even if the character’s hips are not shown, their movement will inevitably affect the head and make their movement less stiff and more natural.

Fortunately one of iClone’s default animations worked perfectly for this task and only a couple of adjustments had to be made with the Edit Motion Layer window to make the character fit the chair. While applying this animation, I noticed that there was a moment when the character changed his posture and raised his hand. Combining these movements with a change in the voice acting gave intention to the character and made more sense and gave more power to the performance.

Exporting work to Omniverse

After having made sure the whole cinematic sequence looked exactly as planned with camera animation which were mostly still-shots, we finished the project in iClone and exported it to Omniverse for final rendering.

It’s a good idea to leave the scene lighting to Omniverse and check the result in RTX Path-Traced mode. Otherwise, the lighting can give unpredictable results in the final render, if we don’t preview the light bounces.

As shown in the image above, there was a general light thanks to the HDR image that encompassed the entire scene (not visible in the shot). Another two sets of lights were placed to illuminate the 3D face. I then added a cube as a back-wall that created soft-shadows on the entire character and darker shadows for the lower parts of the character. In my opinion, this created a more intimate effect for the scene. Finally, I placed a yellowish cloth to bounce lighting onto the opposite side of the character’s face.

Now that everything was complete, the only remaining step was to configure the Movie Capture window and render for as many hours as necessary. In this case, rendering at 4K in RTX Path-Traced mode with 60 frames per second on my computer took more than 40 hours. After assembling the video together with the music and audio, I was able to deliver the Ancestories XR cinematic advert to the team, which was then used to promote the project. Remember to visit ancestoriesxr.com to support the project and I recommend you to watch more related videos of this article on the official Reallusion Youtube channel.