Layering different shots and media have been around for a long time. Many beginners don’t realize the extent to which some productions rely on compositing and not filming the entire scene as it happens. After Effects is a compositing program that elevated this type of production by making it available to the masses. There is a reason Adobe identified After Effects as an effective software solution way back in 1994 when it purchased the company, CoSA – Company of Science and Art, to make it part of Adobe.

According to Wikipedia:

Compositing is the process or technique of combining visual elements from separate sources into single images, often to create the illusion that all those elements are parts of the same scene. 

To be clear, iClone is NOT compositing software. However, it can be used as such when you consider we have the workspace background, image planes, image billboards and image layers to take the place of actual layers like we have in After Effects, Photoshop and similar applications. In fact, I use the workspace background and image planes more than anything and it can add a dramatic and cinematic touch to a simple scene.

The Workspace Background

We can access the workspace background by going into the Project tab and looking for background which can be a solid color, image or video. All you have to do is drag and drop the AI generated video onto the project background area in the Project tab or even faster just drag and drop it visually to workspace background. It should map the background space and play with the regular playback controls.

It is recommended you line up the 3D workspace with the background where the grid matches the ground plane of the video. This starts the process for a smooth working experience. This doesn’t mean you need to angle the grid to match a road, which you can do, but it’s more important to line up the grid to the viewport at background horizon level.

Dragon Slayer Breakdown

Let’s take this simple dragon slayer example and break it down. It only has one character (the Knight), five particle effects, both legacy and PopcornFX, along with an AI-generated background. This background is from the new video generation service from Midjourney based on an image of a Dragon challenging what is in front of it.

To be clear, the background was first generated as an image with Midjourney then animated by pressing a button. You can have Midjourney choose what to animate in the image and how it will animate which isn’t always right the first time. There are ways to control this with manual prompting versus the automatic animate feature. We can also set the beginning and ending frame to the same image in Midjourney to stop most camera movement.

This type of scene can draw a viewer deeper into the story with the visuals even when it only lasts a few seconds. It can immediately set the tone for whatever follows, and it is beginners level work in this example. While there is a lot of movement in the scene it boils down to those canned motions dragged and dropped onto the Knight character while the animated background does the rest.

The particle FX are optional but are easy to use and add more flair to the scene. In a manner of minutes, you can put together a scene like this. It will most likely take more time in Midjourney or the AI video generator of your choice than you will spend in iClone.

Dust Off

 This example like the Dragon slayer above is a simple scene that only involves a few more characters (all duplicated from the original character) with different poses. The two standing characters watch the aircraft lift off and depart by looking at it and following an invisible box dummy that moves with the aircraft to mimic that character following the aircraft lift off with their heads.

Don’t overcomplicate things.

As beginners we already have a lot to keep up with when we work with the feet or lower legs of the characters in full view. This can turn into a chase-our-tails task where we fight one little problem only to create another trying to meld the 2D video background to the 3D assets. Keep the ground plane contact with character’s feet OFF SCREEN. Place the feet or props in positions that don’t show the full contact point of 2D and 3D assets. Problem solved and no chasing our tail.

The goal is to have the viewer not even realize you have used a video or where that video may start compared to the 3D work in the scene. The tutorial also shows how to hide parts of the background footage with props to cover up any strange AI-related happenings like a person walking in the background with their head backwards. Hiding and eventually masking these anomalies will become a common tool or skill if it doesn’t improve.

Both the demo and tutorial videos have one thing in common. Without the animated background they are not usable. They are not even a scene.

Quick and Dirty Color Grading (Color Matching)

A big problem with using different media in iClone is the sharp contrast in the color pallets of the media. We aren’t always able to use or find assets in the same color pallet, so we use Color Grading which on the pro level is a skill in itself. We don’t need pro level color grading so much as we need blending of colors along with color overlays like image layers that match the main color of the scene.

This is an old-school work-around that has been around since digital art existed and can come close to color grading which didn’t exist back then. It’s a simple concept to tie disparate media together for continuity. I’ve written about this method before in various contexts over other articles.

For example, we take a stock piece of footage for the animated background, and it has an orange tint to it. Then we add iClone characters that may have a brown tint to them. Adding props makes this even more complicated as each prop could have a different colored “look”. A purplish piece of furniture along with green tinted lamps or floors.

I don’t mean the lamps or floors are green. I mean they have normal colors but have a slightly green tint or palette compared to assets that are around them.   These assets all need to have the same overall tint to them so we can use overlays as a quick and dirty method of achieving this effect.

LUTs are another important piece to the puzzle as they keep the image layer color from bleaching or washing out the image. These are effects that use images to project a color or combination of colors. While a color based image layer can work in most cases, lighter colors can washout the scene if dialed up too much.

In the color grading example below, I show you what it looks like in native iClone lighting, what it looks like with just an image overlay and the final look with is a combination of three LUTs and the image overlay to achieve the dusty look I wanted. The demo also shows some variations between the LUT layers as they are turned on.

While true color grading is an advanced skill, we can use this much simpler method to get more continuity of color. Color grading can be a hard a concept to explain as all props and characters have their own colors so the short tutorial below should help to visualize this.

The tutorial covers this quick and dirty color grading/matching process using the same scene as in the above tutorial.

New AI Render Beta Plugin Now Available

Not only can we use AI-Generated and animated backgrounds we can also try out the new AI Render plugin now in open beta. I will cover this exciting new plugin in a future article but for now the future looks very bright for iClone rendering with the power of AI. Like other iClone features this plugin can be as simple as just using it right out the box following directions or you can jump off into the deep end of data and customize your own look.

This is an integration of ComfyUI for the ability to use almost any model with the AI Render. The potential is huge. If you have no idea what I’m going on about… don’t worry. There will be tutorials and discussions to get all of us up to speed on AI rendering.

AI models included with the plugin are:

Reallusion has automated the ComfyUI install to the point where almost anyone can do it without AI experience. Its all packaged up in a nice little installer that takes care of all the tech nerd stuff that makes the magic inside our little boxes. While that is a bit flowery it is the truth as Reallusion has made the plugin ridiculously easy to install. You’ll need about 80 Gigs of space if you download all the models provided.

Summary

One thing I’ve covered many times is the little bits of easy-to-produce eye candy like these AI-animated background scenes that can make a lot of difference in getting and keeping the attention of viewers. Whether we are just starting out as animators or are seasoned pros… A few easy seconds of eye candy can go a long way to filling in gaps while maintaining the focus of the viewer.

It doesn’t have to be Midjourney as they are new to video generation for consumers as of this writing. Any AI video generator will work if they have compatible resolutions or close enough to look smooth.

Compositing is still the lifeblood of moviemaking. Nothing may be what it seems after it comes out of the hands of a good compositor. Might as well start picking up on this technique now so you will be prepared for it in the future as your animation skills grow.

MD McCallum – WarLord

Digital Artist MD “Mike” McCallum, aka WarLord, is a longtime iClone user. Having authored free tutorials for iClone in its early years and selected to write the iClone Beginners Guide from Packt Publishing in 2011, he was fortunate enough to meet and exchange tricks and tips with users from all over the world and loves to share this information with other users. He has authored hundreds of articles on iClone and digital art in general while reviewing some of the most popular software and hardware in the world. He has been published in many of the leading 3D online and print magazines while staying true to his biggest passion, 3D animation. For more information click here.

WarLord’s Workshop