Image Credit: Goki Muramoto
Explore the sensation of real-time slow motion through a layered video display that challenges our perception of time. By manipulating temporal delay across visual planes, this SIGGRAPH 2024 Emerging Technologies project “Demonstrating Real-time Slow-motion Experience Through Parallel Video Presentation” invites viewers to experience time as both fragmented and continuous. Read on as SIGGRAPH sits down with Goki Muramoto to discuss his work.
SIGGRAPH: What inspired the development of a system that combines real-time and slow-motion experiences?
Goki Muramoto (GM): It is at the intersection of two major interests. The first was in my first undergraduate class, when I learned about the concept of Cyberbetics. In that class, we learned that the basic attitude of “steering” was described as a feedback mechanism under the integration of communication and control. We learned that the body system is flexible and editable on a mechanism. In extreme cases, if you reconnect the nerves in your arm, you may be able to control it again once you adapt to that arm.
What is interesting, however, is that there emerges a slightly different body than before. Designing this “difference” from the original body in advance and constructing the edit into the body to achieve this difference is what I understood as “body editing” as an engineering concept. Then, I realized that this “editing” here is spatial, and wondered if it would be possible to simply edit the body in time and adapt to it, so that a body that is different in time would emerge. The idea is not to edit the spatial structure of the system, but to edit the temporal structure and have it adapt to it. Even so, I think a body editing protocol is possible.
Of course, if we seriously situate my experiment within the framework of cybernetics, it is not as simple as the replacement from space to time that existed at the time of its conception, but is more complex. It involves a processual representation of time, which can be described by contrasting it with Newtonian representations of time.
Second, I am interested in the relationship between film and perception. How are the apparatuses and processes of film and the body, including sensory organs or cognitive process, similar and different, and how can the duality of these be understood in the act of “watching or perceiving film”?
The impossibility of coexistence of real-time and slow motion was a good motif to think about this idea. Does the playback speed of the present have to be 1.0x? Of course, these questions have been considered repeatedly over the understanding of time and film, but I thought that by forcing the coexistence of the two, I could make progress on the temporal relationship between recording/playback and perception.
Image Credit: Goki Muramoto
SIGGRAPH: What kind of technology is used to stretch out these time layers and synchronize them for presentation?
GM: The technical basis is very simple. A camera and a multilayered display are connected to create a lens-like situation. In this setup, the temporal relationship between the camera and each layer of the display — essentially, the delay — is controlled by a set of rules to form a temporal lens. The rules we adopted this time are easy to understand if you imagine making a croissant: folding while stretching.
At first glance, this may seem like wordplay, but it produces a state where all outputs are slowed down, all outputs fall within a certain delay (the subjective present), and all inputs are directed to some layer. Then, the folded time axis is gradually unfolded as the viewer’s mind unconsciously restores the sense of temporal continuity.
In the version presented at the Augmented Humans conference, we created a goggle-type device with two layers, each displayed to the left and right eyes. In this case, the multiple layers do not explicitly overlap. This creates the illusion that the scene in front of you is slowing down — although, in reality, it tends to cause nausea.
Image Credit: Goki Muramoto
SIGGRAPH: What is the user experience like when interacting with this technique through see-through goggles or digital mirrors?
GM: Normally, when the scenery in front of you slows down, it becomes more and more like the past, and you can no longer interact with it, right? But with this configuration, it follows your body in real time — even as it slows down. It’s an unpleasant experience. I remember one subject commented that it felt like “time was saturated.” Another subject compared it to a Shepard tone.
The effect is strongest when your own body is interacting with the laws of physics. The act of throwing and catching an apple is especially symbolic.
Image Credit: Goki Muramoto
SIGGRAPH: Where do you see this project heading in the future? Are there plans to improve or expand this framework for broader uses?
GM: As for practical applications, the device was originally an experiment for thinking about time. In terms of its contribution to engineering, I would consider it a success if it offered the possibility that “the human body can be edited in terms of time” and helped make the approaches of engineers and artists to time more vivid.
At the same time, it has recently become clear that the device has practical benefits for counselors and movement instructors — including dancers — who require sensitivity to fine motor movements. Lately, I’ve also been exploring how this research fits into the broader lineage of efforts to realize “images”, such as sequential photography.
SIGGRAPH: What do you hope will be the most significant impact of your research in the world of digital media or interactive technology?
GM: I would be happy if our research could give readers the possibility that “the human body may be editable in time” and make the approach of engineers and artists to time more vivid. We already recognize, thanks to previous philosophers, artists, and other engaged people, that the idea of time is an hard problem that interacts with the idea of the world and the idea of the self. In recent years, it has been linked to fundamental knowledge of the brain and the universe — but in the hands of artists and engineers, who shape our perception of time through creation, does it also offer a kind of freedom?
To hear more from Goki’s perspective, listen to his conversation about SIGGRAPH 2022 Art Gallery contribution “Imagraph” on the SIGGRAPH Spotlight podcast. Experience even more groundbreaking work like this when you register now for SIGGRAPH 2025.
Goki Muramoto is an Artist who was born in Yamaguchi, Japan and lives and works in Tokyo. He explores “mediation” encompassing perception, communication, and movement through the process of inventing and sculpting his own media. His major works include the Imagraph series, a medium that projects a video onto closed eyelids; the Lived Montage series, a medium that implements a fictional perception in which we share our vision when we share an object of our consciousness; and the dictionary/sculpture, Media of Langue, that depicts the chain of word translations. His work has been exhibited nationally and internationally.
