Image Credit: Illustrating a viewer experiencing the artwork with a head-mounted display © Lingfei Yang 2023
Chinese art is submerged in culture. But did you know there’s a place where poetry, calligraphy, and painting meet? Dating all the way back to the Song Dynasty, this art form combines different mediums to expand viewers’ interpretations. We sat down with three experts and SIGGRAPH 2023 contributors who taught us a little bit more about this cultural approach to art.
SIGGRAPH: What inspired you to undertake this project and explore the intersection of technology and cultural expression?
Lingfei Yang (LY): The fusion of poetry, calligraphy, and painting is a defining characteristic of Chinese literati painting. While previous works have emphasized reconstructing the painting, there have been limited attempts to integrate it with the other two elements. By gaining a deeper understanding of this characteristic, viewers can truly appreciate the profound meaning embedded in the artistic conception of these paintings.
SIGGRAPH: Tell us more about the methodology employed in your project. How does this methodology contribute to creating a more impactful experience for viewers and a deeper focus on the paintings and their meaning?
LY: In order to recreate the scenes depicted in the original painting, we must transform the plane, static, paper-based artwork into a dynamic, three-dimensional, immersive multimedia experience. Drawing inspiration from touring famous scenic spots, where visitors follow tour guides to gain a deeper understanding of the stories behind them, we have incorporated an audio-visual combination in this artwork.
SIGGRAPH: The project aims to open doors to a vast cultural context through its exploration of 3D representation. Share some insights into how cultural studies have been incorporated to empower the technological tools utilized in your project.
CC: We have delved into the political, cultural, and artistic milieu of the Song Dynasty, the era in which the artwork was originally created. This comprehensive understanding has allowed us to ensure that the reconstructed experience remains faithful to the characteristics of that time. For instance, we have taken into account people’s fondness of sightseeing by boat in the Song Dynasty, and thus, we have set the visual perspective on a boat, enabling viewers to immerse themselves in the authentic atmosphere of the era.
Rather than selecting from the vast array of renowned artists and their celebrated works throughout the history of Chinese painting, we have chosen Zhang Xian’s “Illustrating Ten Poems” as our focal point. This particular painting serves as a poignant tribute to his father, evoking a wealth of emotions within its original composition. Our objective is to recreate the scenes depicted in his father’s poetry, allowing the audience to delve into the narrative behind the artwork while simultaneously admiring its aesthetic beauty. Through this experience, viewers will gain insight into Zhang Xian’s profound yearning for his father.
SIGGRAPH: Were there any notable findings or surprises that emerged during your research and development process? How did these insights shape the final outcome of your project?
CC: We have explored various techniques for modeling the scenery and characters, and we have infused the sky, lake, and characters with dynamic elements. By artfully combining simple planes, we can achieve extraordinary animated effects that not only surpass traditional rendering methods but also authentically capture the essence of Chinese painting. This accomplishment truly showcases the magic that can be achieved through the application of innovative approaches.
SIGGRAPH: In today’s fast-paced media consumption, it can be challenging for people to truly appreciate and understand artwork rooted in specific cultural codes. How does your project address this issue and provide access to the poetics of Chinese paintings?
LY: Traditional Chinese paintings can be difficult for the general public to comprehend. Many people may simply glance at the content of the artwork without taking the time to read the intricately written words. By incorporating more contemporary and popular forms of display, such as the immersive experience used in our work, we can ignite their curiosity and encourage them to delve deeper into the story behind the original painting and appreciate the intertwined poetry.
SIGGRAPH: How do you envision the potential impact of your work in terms of fostering cross-cultural understanding and appreciation for Chinese paintings, and what are your hopes for the broader reception and engagement with this project at SIGGRAPH 2023?
CC: In this paper, we have introduced some simple technical details in the realm of artistic display for Chinese painting. These innovative approaches not only make the immersive experience of Chinese paintings more accessible but also lower the threshold for individuals to create immersive artworks. We believe that such innovative endeavors can push the boundaries of traditional art consumption and fostering a profound connection with the artistic world at SIGGRAPH 2023.
SIGGRAPH: Looking ahead, what are your future plans or aspirations in utilizing 3D representation and technological advancements to further explore cultural expressions through art?
Xukun Shen (XS): On one hand, we hope to investigate and develop simpler and more effective technical methods to advance 3D cultural display projects. On the other hand, we strive to emphasize the audience’s experience and provide them with a more immersive and enjoyable encounter within the realm of artistic appreciation. This dual focus reflects our commitment to both technological innovation and the human-centered approach, aiming to enhance the overall quality of artistic engagement.
If this article piqued your interest in research and art, check out all of the innovative Art Papers content you will discover in Los Angeles this August to submerge deeper into the world of SIGGRAPH research! Register now.
Chong Cao is currently an associate professor at the School of New Media Art and Design at Beihang University. Her research interest includes virtual reality, interactive media, and interaction design.
Lingfei Yang is currently an undergraduate student at the School of New Media Art and Design at Beihang University. She has a deep-rooted passion for traditional Chinese culture and aesthetics and hopes to not only comprehend but also truly appreciate the timeless allure of Chinese art in a manner that resonates with modern audiences.
Xukun Shen is dean of the School of New Media Art and Design, a full professor, and doctoral supervisor at the State Key Laboratory of Virtual Reality Technology and Systems at Beihang University. His research interests are concentrated in digital media and art design, focusing to the interdisciplinary studies of art and VR technologies.
