Image credit: Image courtesy of Shangyu Wang
This article was authored by Autodesk.
While the adoption of AI in media and entertainment has raised concerns about job security and diminished creativity, some innovators and early adopters are seeing the opposite. During Autodesk’s Vision Series at SIGGRAPH 2024, VFX studios Rising Sun Pictures and Crafty Apes shared their experiences harnessing AI and machine learning to augment creative workflows, improve production timelines, and ultimately help teams produce better VFX.
AI in Media and Entertainment Unlocks Artists’ Full Potential
Artists are already under a lot of pressure to deliver based on creative directives while pushing their own boundaries. On top of that, they often spend hours sifting through software windows, waiting on file transfers, and troubleshooting.
“We’ve come into this industry as artists to create beautiful work and present pieces for everybody to enjoy,” Petar Tsonev, head of machine learning and technologies at Crafty Apes, says. “Yet most of the time we’re technicians, and that’s just no fun for anybody.”
AI has the potential to automate repetitive tasks, reducing creative downtime and freeing artists to focus on bringing their visions to life rather than worrying about the technological aspects. In the case of Tsonev’s team, AI has allowed them to bring stories to life in new and inventive ways.
“Art is just such a dynamic process full of individual creativity, historical influences, and cultural inspirations,” Tsonev says. “AI is not going to be able to substitute human input.”
He compares concerns about AI in media and entertainment to the introduction of 3D animation. “When 3D was coming about, there were a lot of concerns about how it’d affect the market,” Tsonev recalls. “As we can see, it has become a crucial part of the industry, allowing people to express themselves in different ways and providing a lot of new opportunities.”
Machine Learning Leads to Superior Visual Effects
Head of Technology at Rising Sun Pictures, Alex Meddick, attests that embracing AI has led to job creation and the ability to enhance filmmakers’ storytelling vision.
The studio behind the effects for the Marvel Cinematic Universe, “Game of Thrones”, and recently, “Furiosa: A Mad Max Saga”, uses an in-house machine learning toolkit called REVIZETM. “Behind RSP’s REVIZETM is a very talented group of people who spend every day problem-solving their way through creative and technical challenges,” Meddick says. “Combining traditional VFX and cutting-edge machine learning, our teams deliver seamless feature film quality results.”
Meddick continues, “We can swap and modify the look and performance of entire heads, hair included, as well as bodies, with both artist-driven and synthetic data.” This often leads to the creation of custom ML models and solutions that are only used in a few frames of a shot to deal with issues like motion-blur, occlusion, and lighting variation.
A growing number of companies are incorporating AI and machine learning tools into their software, which will make the technologies even more ubiquitous.
For example, Maya’s ML Deformer can quickly approximate complex deformation systems to speed up character rig posing and playback. Artists can seamlessly switch between the AI representation and the original to develop a polished final render.
Similarly, Flame now offers ML Timewarp, which uses AI to generate intermediate frames for retiming clips.
Say So Long to Scheduling Headaches
The smallest changes can sometimes cause scheduling chaos, but AI tools are alleviating some of these challenges.
For example, teams can compare multiple schedule scenarios and evaluate trade-offs with Flow Generative Scheduling. They can then publish revised schedules directly into Flow Production Tracking to reduce disruptions and allow everyone to focus on their core roles.
The Crafty Apes team is employing AI tools from the beginning of their pipeline, laying the groundwork for smoother production schedules.
“As a post house, you’re usually stuck at the back of the line, waiting for earlier creative decisions to be made before you can truly stretch your legs and shine,” Tsonev says. “Now, AI has allowed for a lot of new opportunities in this field for us to jump into the process earlier and be part of the creative solutions from the start.”
Learn more about how AI is augmenting artist workflows in the media and entertainment industry, and watch recordings of Rising Sun Pictures and Crafty Apes’ Autodesk Vision Series sessions here.
