Photo caption [in-headset screenshot]: The visitor is positioned in the back seat of a car
within intimate proximity of Depthkit-rendered, volumetric characters
Mary-Helen (Hadley Boyd), Sebastian’s mother, in the front passenger seat
and Ed (Drew Moore), Sebastian’s father, who drives. // Photo courtesy of Illya Szilak and Cyril Tsiboulski.
Everyone’s love story has its challenges and obstacles, but what if the challenge faced was telling your estranged family about your love story? “Queerskins: A Love Story” explores the journey of a devoutly Catholic mother living in rural Missouri in 1990 as she gets a chance to know her estranged, gay son, Sebastian, through a box of his belongings.
In celebration of Pride Month (June), we sat down with creators of the SIGGRAPH 2018 VR experience “Queerskins: A Love Story” to discuss the journey of this moving, special experience with its creators Illya Szilak and Cyril Tsiboulski. The four-episode cinematic was recently recognized with a Peabody Futures of Media Award.
SIGGRAPH: Making an experience interactive, of course, hikes up not only the time required to bring it to life, but also the cost. Why was it crucial to make “Queerskins: A Love Story” interactive and how did you get funding?
Illya Szilak (IS): “Queerskins: A Love Story” is inspired by the online, interactive, multimedia narrative we put out in 2013. It tells the story of Sebastian, a young gay physician estranged from his rural Catholic Missouri family who dies of AIDS in 1990. For the online component, Cyril created a multimedia scrapbook, which included 40,000 words of Sebastian’s diary, two hours of audio monologues from five people who knew him, photos curated from Flickr Creative Commons, and crowdsourced and commissioned videos—which the audience could navigate through in any way they chose. Even so, online you feel more like a reader than an active participant. Because the story deals with AIDS and homosexuality in a very intimate way, we were excited by the power of VR, a spatial and interactive medium, to allow visitors to become integral to the storytelling. Not only did we want them to feel like they were really “there” in the backseat of the car behind the man’s grieving parents, but we also wanted to give them the opportunity to construct the main character themselves—from a box of belongings, photos, and a diary. Our goal was to procedurally generate a sense of intimacy and connectedness with the man who died. Rather than have you pretend for five or 10 minutes to be someone who you aren’t, we wanted you to be exactly who you are and leave the experience a little more open and a little more tolerant of other people’s stories of love and loss—stories that everyone experiences.
I began writing the script with help from Oscar Raby (VRTOV), who is an award-winning VR artist. His organization [helped get us a small grant from The Sundance Institute and Arcus Foundation, and his encouragement really gave me the courage to make this project. It was difficult to find funding, though. Eventually, I rewrote the script entirely and The Tribeca Film Institute – MacArthur Foundation gave us a Tribeca All-Access Hybrid Non-Fiction Prototype Grant. It was more money than we had ever seen, so we just took it and ran with it. We also created a Kickstarter, raising $25,000 dollars. And we were incredibly fortunate that so many talented people and studios, including Scatter, Skywalker Sound, our 360-degree videographer Richard Hammer, and so many others, saw the value of our story and vision and contributed their talents.
SIGGRAPH: Talk a bit about the creation process for “Queerskins: A Love Story.” Technologically, what do you find exciting about the final experience you presented to SIGGRAPH 2018 participants?
IS: For us, technology is a tool for storytelling. We aren’t interested in the newest thing for its own sake. We ended up combining a lot of tech that no one has put into the same experience—360-degree video shot on location in rural Missouri from a moving SUV, 3D volumetric live-action video, 3D scanned archival objects, photogrammetry, 3D CG modeling and animation, and spatial sound—all because we really wanted to create a magical, realist aesthetic. We needed to generate a dynamic tension between “real” historical material reality and the very human desire to transcend those limits through art and imagination. We think of our stories as little machines.
Aesthetics are an important cog, but in VR, for us, the most exciting storytelling possibilities are procedural: How does the visitor discover information? How are they involved in the storytelling? In “Queerskins: A Love Story,” you, the visitor, are charged with reconstructing the man who has died by interacting with a box of his belongings. We realized that physically connecting you to the task would be emotionally powerful—just this simple act of sifting through the box and picking up objects connects you to Sebastian in a very different way than passively listening to the fraught conversation between the parents in the front seat.
SIGGRAPH: In addition to being set more than 20 years ago, in 1990, what were some of the key challenges you encountered during the development process?
IS: In order to achieve the emotional impact we wanted, we knew that the mother and father had to seem real and present to the visitor who sits in such close proximity to them. There really was no choice but to shoot the actors with 3D volumetric video. We were incredibly fortunate to be able to work with Scatter, developers of Depthkit, a 3D volumetric toolkit, and Cory Allen, our DP, an experienced and talented 3D volumetric videographer. The actors were shot individually with two cameras in front of a green screen. They sat in a made-to-measure mockup of the front seat and dashboard of the vintage 1986 Cadillac that our collaborator Pat Goodwin had captured with photogrammetry. This was a very draining scene for the actors, and sometimes the Kinect would glitch and we’d have to scrap a take almost at the end. Also, because the lighting on the actors could not be changed much in post, Cory had to program the lights on the actors in accordance with the landscape that would appear outside the car. So, we set up a monitor for the actors and Cory so they could see where they were in the landscape and stay on schedule with the dialogue.
Lastly, the actors had to be careful not to cross over each other in the space and to avoid artifacts. It was a challenge for everyone. We did a full tech rehearsal and shoot, then Cyril had 48 hours to put that rough depth data into a rough 3D model of the car to make sure all the parameters were working before we went ahead and did the final shoot. So much could have gone wrong, but it didn’t. We are still kind of amazed.
SIGGRAPH: What, if any, were some of the VR inspirations you looked to when making “Queerskins: A Love Story”?
IS: We are most interested in stories that use the special affordances of VR. I think “Notes on Blindness”really brought home for us that sound is truly a spatial medium (as is VR), one which powerfully activates the visitor’s imagination. Our work is inspired by a lot of non-VR art. Aesthetically, we drew inspiration from the 1950s melodramas of Douglas Sirk; you might notice that the landscape outside the car window looks like an old-fashioned rear-screen projection and the colors are Kodachrome bright. Our colorist, Juan Salvo, also added a little grain to the landscape video, so it almost looks like you are driving through a home movie. I don’t play a lot of video games, but certainly “Gone Home” would be an obvious parallel in the way it uses objects to recreate the main character. We took it even further, however, making it more open for interpretation. Also, the kinds of objects we chose were all vintage and historically accurate. A couple of examples of this are the 1970s plastic kids’ Hulk Halloween mask and the homoerotic 1970s Tom of Finland “physique pictorial.” The great thing is that how the visitor relates to those objects influences how they conceive of the main character, and, in this way, it really opened up the possibilities for the visitor’s imagination and personal history to be activated.
Lastly, I was inspired by Pina Bausch’s dance theater work. We sensed that visitors would “read” the body language of the two parents seated in close proximity viscerally, through their own bodies. For this reason, we worked with choreographer Dawn Saito to choreograph the actors’ gestures, movements, and postures, creating a kind of stilted, painful missed call and response between the two. We think that the possibility of learning from and incorporating techniques from other spatial art forms into VR is really exciting.
SIGGRAPH: Can you talk a bit about the importance of “Queerskins: A Love Story” as an LGBTQA+ narrative?
IS: We did not go into this wanting to make an AIDS story or a “gay” story. We didn’t want it to be pigeonholed (or dismissed) in that way, or, even worse, create a story that suggests “gay people are just like straight people,” ignoring the political, social, and economic realities of being gay.
What we really wanted was to give the visitor the freedom to be themselves and to genuinely connect with someone who might be quite different from them. The title is purposefully provocative. We wanted people to say, “queer” and “skin” together, and maybe feel a little queer in their own skins. The title is also a beacon, a “welcome home” sign for queers of all types.
SIGGRAPH: Angela Watercutter recently wrote in WIRED that “programmers are looking to make interactive experiences fun for the whole family,” in a story on VR at film festivals. In developing future VR chapters for “Queerskins: A Love Story” is this something you are taking into account?
Cyril Tsiboulski: We are driven to tell emotional, complex stories that explore the potential of new technologies. We think that good art should make people uncomfortable because that is one way we can test our boundaries to understand what we really care about. In “Queerskins: A Love Story,”we create a world which offers visitors the opportunity to connect with an urgent social message in a non-didactic, emotionally powerful way. Through story and technology, the experience puts visitors in the position of living through the intimate, interior worlds of others. It is our hope that this will lead to an emotional engagement with the characters and themes, and, ultimately, an empathy for the characters’ personal experiences and, by extension, for “real” persons who grapple with love, illness, and loss. Is it “fun for the whole family?” That’s a decision and choice best left to the visitor, not the artist.
SIGGRAPH: Since showcasing “Queerskins: A Love Story” at SIGGRAPH 2018 the project has enjoyed success at other festivals and events, including winning the 2019 Breakthroughs in Storytelling Special Jury Prize from the Columbia University School of the Arts. What’s next for the experience? Is there anything else you have been working on in the realm of VR?
IS: We are working on several projects, but this summer and fall are devoted to “Queerskins: Ark,” the second of four planned chapters of the “Queerskins” story. In this second chapter, Mary-Helen, Sebastian’s mother, does something incredibly brave. Reading her son’s diary, she allows herself to imagine him alive and in love. The most important scene, an intimate dance between Sebastian and his lover, will be shot on a full 360-degree, 3D volumetric capture stage. In this scene, we offer the visitor 6DoF, each person will decide how to respond emotionally and physically to the two men. Wherever and however the visitor moves, their hands and body leave a beautiful trace of light that forms a landscape for the dance. We are currently doing prototyping experiments to optimize an interface, which promotes a kind of physical dialogue between the live visitor and the digital dancers. We are working with a wonderful, up-and-coming choreographer, Brandon Powers, who is creating the dance as a series of phrases, the sequencing of which will be determined, in part, by how the visitor uses the space. In essence, each visitor acts as co-choreographer and each dance experience will be unique. We think it is going to be truly groundbreaking storytelling. As with “Queerskins: A Love Story,” we want to house the VR in an interactive installation that extends storytelling into the physical space, and which will eventually incorporate a live dance performance.
SIGGRAPH is an inclusive, exciting environment for all to explore content like “Queerskins: A Love Story” and other stories and experiences. Jump in to all things virtual reality at SIGGRAPH 2019 when you register and visit the Immersive Pavilion and VR Theater this summer in L.A.!
Meet the Creators
Illya Szilak is a writer, director, and interactive storyteller. Shaped by her experiences as a physician, her artistic practice explores mortality, embodiment, identity, and belief in a media-inundated and increasingly virtual world. Cyril Tsiboulski is her longtime artistic partner. Their online works “Reconstructing Mayakovsky” and “Queerskins: A Love Story” were selected for the Electronic Literature Collection. They continue to be taught on a university level as important examples of digital literature and narrative game, and are the recipients of grants from The Sundance Institute, The Tribeca Film Institute/MacArthur Foundation, and The Peter Reed Foundation. Illya has written about and speaks frequently on new media storytelling, her book chapter “Do Cyborgs Dream of iPhone Apps? The Body and Storytelling in the Digital Imaginary” will be published in Structures of Thought and the Digital Imaginary: On The Emerging Shapes Of Immersive, Narrative and Database Arts through Bloomsbury Academic Press. Illya teaches immersive storytelling at The IFP Made in NY Media Center, and also works part-time as a physician providing care at Rikers Island Correctional Facility.
Cyril Tsiboulski co-founder of Cloudred, an interactive design studio in NYC. Cloudred clients include The NYC AIDS Memorial, The Museum of Digital Heritage, The United Nations, and UNICEF, and related work has been recognized by The Webbys, NYC Open Data, AIGA, SXSW, IxDA, Communication Arts, and TIME. He teaches interactive design in New York Univeristy’s Digital Communications and Media Department. Cyril and Illya’s first VR project “Queerskins: A Love Story,” combined VR, interactive installation, and performance photography, premiered at The Tribeca Film Festival in 2018, and was shown at festivals around the world. “Queerskins: Ark” has been selected for the Venice Biennale College Cinema VR Lab in 2019.
