Image credit: © Filmakademie Baden-Württemberg GmbH, Finn-Oliver Walter
By Julian Dünser and Justus Blönnigen
This article was provided by the SIGGRAPH 2026 Media Trade Partner Animationsinstitut Filmakademie Baden-Württemberg.
How It All Started
DOSENMUTTER (ROLLING MOTHER) is a fictional diploma short film directed by Alexander Peskador at Filmakademie Baden-Württemberg. The story follows a son clearing out his late mother’s house, only to discover that she now inhabits an old coffee can, killing anyone who tries to throw away her belongings.
To tell this story, we needed to create the illusion that the audience was inside the can with the actor while his childhood bedroom, in which the can rolled around, appeared enormous beyond its reflective metal walls. The actor had to feel physically present inside the can, making virtual production an obvious choice. Besides avoiding difficult green-screen reflections, the LED wall provided realistic lighting and allowed us to experiment with new production techniques.
The project was developed by Julian Dünser, a visual effects student, and Justus Blönnigen, a technical directing student, at the Animationsinstitut of Filmakademie Baden-Württemberg.
Widely recognized as one of the world’s leading institutions for animation, visual effects, technical directing, animation/effects producing, and interactive media, Animationsinstitut brings together filmmakers, artists, and creative technologists in an environment that combines visual storytelling with technological innovation. DOSENMUTTER provided the perfect opportunity to apply that interdisciplinary approach to a real production challenge.
Planning the Workflow
Our production schedule was tight. The set of the childhood bedroom was only available for a short period of time, before principal photography started, so we had to capture the environment early.
We prepared two workflows. The primary approach was to reconstruct the room as a Gaussian Splat from photographs, allowing real-time rendering inside Unreal Engine. As a backup, we also recorded video plates that could be synchronized with the actor’s movement. While both approaches made it into production, the Gaussian Splat ultimately became our primary solution.
Image credit: © Filmakademie Baden-Württemberg GmbH, Julian Dünser
Building the Virtual Environment
Instead of creating the entire room as traditional CG, we decided to capture the practical set using Gaussian Splatting. Building and lighting a complete digital replica simply wasn’t realistic within our production schedule, while photogrammetric capture allowed us to preserve the look of the original set with comparatively little manual work.
The biggest challenge was lighting. Gaussian Splatting performs best with bright, clean images, but our set represented a dimly lit bedroom at nighttime. After several tests, we found that increasing ISO quickly reduced reconstruction quality because image noise affected feature matching between photographs.
To maximize image quality, we chose interval-based still photography instead of video capture. This allowed us to keep ISO lower, maintain sharp images, and move efficiently through the set without manually triggering every frame. In total, we captured roughly 4,000 photographs.
The resulting dataset required AI-assisted denoising and some selective focus correction before generating the final Gaussian Splat, but the processed environment imported cleanly into Unreal Engine and provided a reliable production asset.
One important lesson emerged immediately: Overshoot your capture. Directors inevitably want to frame areas that were never part of the original plan, and additional coverage is always easier to discard than recreate later.
Synchronizing the Performance
Creating the environment solved only half the problem. Since the actor performed inside a rotating hamster-wheel version of the coffee can, the background had to respond naturally to changes in speed.
Image credit: © Filmakademie Baden-Württemberg GmbH, Finn-Oliver Walter
Our first prototype used magnets and a Hall sensor attached to the rotating wheel. While functional, the system only generated discrete position updates, introducing noticeable lag whenever the actor accelerated or slowed down.
We therefore switched to a gyroscope connected to an Arduino, streaming real-time angular velocity into Unreal Engine via OSC. A custom Blueprint processed and smoothed the incoming data while allowing us to adjust the relationship between wheel rotation and background movement during shooting. Once calibrated, the system remained stable throughout the production day.
Why Gaussian Splats Won
Alongside the Gaussian Splat workflow, we also experimented with sensor-driven video playback. The idea was simple: The wheel’s speed would directly control playback of pre-recorded background plates.
Although the concept worked in principle, Unreal struggled to reliably cache and play high-resolution, high-frame-rate EXR sequences at variable speeds. Once playback deviated significantly from real time, visible stuttering became unavoidable.
The Gaussian Splat environment avoided these limitations entirely. Instead of driving recorded footage, the actor controlled movement through a fully three-dimensional scene, preserving smooth motion regardless of speed changes while maintaining complete freedom for timing adjustments during production.
For this project we used the XGrids Gaussian Splat plugin for Unreal Engine, which offered the best balance between performance and functionality at the time.
Image credit: © Filmakademie Baden-Württemberg GmbH, Julian Dünser
Camera Tracking — or Rather, the Lack of It
Unlike many virtual production shoots, we did not use camera tracking. Since every shot looked directly into the rotating can, external tracking systems would have been blocked by the set itself.
Instead, we manually adjusted the frustum for each setup and relied on locked-off cameras with subtle handheld movement. For our shot design, this proved to be both practical and entirely sufficient.
An Unexpected Benefit
One of the biggest advantages of the workflow became apparent during production.
While reviewing footage, the editor noticed that an important shot was missing. Rather than returning to a traditional post-production workflow, we captured a single additional photograph on location, integrated it into the Unreal setup, and immediately filmed the missing take.
That flexibility became one of the most valuable — and least expected — benefits of using a live virtual production pipeline.
Image credit: © Filmakademie Baden-Württemberg GmbH, Finn-Oliver Walter
Key Takeaways
Our biggest takeaway is simple: Always capture more than you think you need. Camera angles change, creative decisions evolve, and extra coverage provides valuable flexibility later in production.
Equally important, physically demonstrating the system to the director and cinematographer proved far more effective than explaining the workflow with diagrams or laptops. Once the team understood how the system responded in real time, communication became significantly easier.
Final Thoughts
DOSENMUTTER combined a demanding production schedule with several technical challenges, but Gaussian Splatting and virtual production allowed us to solve a very specific storytelling problem while keeping the actor physically connected to the environment.
Rather than serving as technology for its own sake, the workflow became a practical filmmaking tool that reduced post-production complexity, preserved realistic reflections, and offered unexpected creative flexibility during the shoot.
Projects like DOSENMUTTER reflect the interdisciplinary approach encouraged at the Animationsinstitut, where students combine filmmaking, visual effects, and technical innovation to develop production-ready workflows for real-world projects.
Justus Blönnigen will attend SIGGRAPH 2026 and present the project LICHTUNG at the Immersive Pavilion. If you would like to learn more about DOSENMUTTER, our virtual production workflow, or the Gaussian Splat pipeline used on set, feel free to reach out during the conference.
Julian Dünser is VP Supervisor / Operator | Visual Effects student, at Animationsinstitut, Filmakademie Baden-Württemberg.
Justus Blönnigen is Technical Director | Technical Directing student, at Animationsinstitut, Filmakademie Baden-Württemberg.
