Join SIGGRAPH

In Real Time: Behind the Scenes of ‘The Cavern’

by | 12 April 2023 | Conferences, Interactive Techniques, Real-Time

Image Credit © Microsoft 2022 The Cavern UE5 Cinematic Visual Tech Test Created by The Coalition

Technology is constantly evolving, and there’s still so much left to explore. Not many projects have been created using UE5 and running on an Xbox such as the SIGGRAPH 2022 Real-Time Live! demo “Behind the Scenes of ‘The Cavern’ UE5 Cinematic Visual Tech Test.” To get the inside scoop on how this project was developed and what the future looks like for this technology, SIGGRAPH caught up with The Coalition’s Colin Penty to learn all about it, including his experience presenting it in front of thousands of people.

SIGGRAPH: Share some background about this project. What inspired this demo?

Colin Penty (CP): We were deep into evaluating a preview version of Unreal Engine 5, and we had just built the Alpha Point environment demo and shown that at GDC 2021. We were looking for a more character-focused demo we could build to test out those aspects of the engine. Our cinematic director at the time wanted to create a cinematic that had some tension and intrigue, so that’s what pushed us down the path of a darker visual tone and mood. We combined the Alpha Point environment with our next-gen character and then had our talented animation, lighting, and tech-art teams create the cinematic with some support from Cubic Motion.

SIGGRAPH: Not many projects have been created using UE5 and running on an Xbox. Why did you choose to create this project using this technology?

CP: The Coalition is using UE5 for our future projects, so we knew we needed to get up to speed with the technology — the best way to do that is almost always rolling up your sleeves and creating something. We use Xbox as one of our primary platforms at The Coalition, so getting UE5 running on Xbox quickly to help ourselves and Epic Games identify any potential Xbox issues with the engine was a priority. We also wanted to see how the engine would perform on Xbox, what visual features we could afford to run, and at what quality on Xbox Hardware.

SIGGRAPH: How do you imagine this technology will transform in the future?

CP: UE5 continues to evolve rapidly with Epic’s continuous releases, as well as modifications The Coalition makes to the engine to suit our needs. I believe the engine will continue to improve visually — eventually matching feature-film quality — as well as improving from a workflow and performance point of view. If you extrapolate the widespread adoption from UE4 to UE5 with this current generation of consoles, you can see that engines like UE5 will become ubiquitous for interactive and non-interactive content creation in the future.

SIGGRAPH: Did you encounter any challenges when creating this demo?

CP: Absolutely! There were plenty of challenges we’ve endured while creating the demo. Initially, we had quite a few issues getting our character’s hair to light correctly while attaching properly to his head. I recall the groom asset eyelashes wouldn’t attach properly to the face while using blend shapes, so we had to replace them with geometry eyelashes. We had a lot of creative back-and-forth with the “smoke monster.” Initially, it was going to be a more elaborate creature formed out of smoke, but we simply ran out of VFX time and had to go for a simpler smoke effect in the end. 

The dynamic camera system that we added for the SIGGRAPH demo that allowed me to pause the sequence and fly around the world was quite tricky to achieve seamless blending while also pausing and resuming audio at the correct time. Our tech art and audio teams had to do some last-minute fixes to get that together for the demo.

Finally, we had to get the demo running in Unreal Engine at 30fps on Xbox Series X. While today this would probably be much easier with the more recent versions of Unreal and our own local optimizations and knowledge of the engine, back when we made “The Cavern” demo, we didn’t have all those optimizations and knowledge, as it was one of our first times getting a cinematic running on console with UE5. I had to lower Lumen lighting quality and disable some larger shadow casters among other things to get the demo running at 30fps at the time.

SIGGRAPH: What was your favorite part of participating in Real-Time Live! at SIGGRAPH 2022?

CP: It was a great experience. First off, I loved being able to show a technical demo that The Coalition had initially just built for internal testing purposes. You don’t usually get the opportunity to do that, especially in real-time on an Xbox in front of thousands of people! Secondly, it was a great experience working with the SIGGRAPH Real-Time Live! team, gathering their feedback, and iterating on “The Cavern” interactive demo prior to SIGGRAPH. Finally, getting to meet the Real-Time Live! team in person at SIGGRAPH and see the other presenters and their excellent real-time demos was simply awesome.

SIGGRAPH: What advice do you have for those submitting to Real-Time Live! in the future?

CP: It was quite the learning experience for me going through this process, as I hadn’t submitted any work to Real-Time Live! prior. I would recommend having something that is either visually excellent or visually novel — and don’t forget to make your demo as interactive as possible. For “The Cavern” demo, we added dozens of interactive features that weren’t initially there — like the free-camera I mentioned earlier — as well as various toggles for lights and debug modes to help the audience better understand the technology and demonstrate that it was truly real time and interactive.

Real-Time Live! is such a lively and energetic atmosphere. Experience it in person by registering for SIGGRAPH 2023 and joining us 6-10 August in Los Angeles.

Colin Penty has worked at The Coalition for 12 years, focusing on visual features, performance optimization, and content creation workflows. Most recently Colin supported Epic with shipping “The Matrix Awakens: A UE5 Experience” on Xbox. Prior to that, Colin was the studio technical art director for “Gears of War 4” and “Gears 5.” Colin was a CG supervisor at EA for seven years working on the FIFA, Skate, and Need for Speed franchises before joining Microsoft.

|

Related Posts