Rendering of the western hemisphere of “Terra Mars” generation 65, centered at the enormous canyon system Valles Marineris, also featuring some or Mar’s tallest mountains. (© Weili Shi. Clouds texture credit: NASA)
“Terra Mars: When Earth Shines on Mars through AI’s Imagination,” Weili Shi’s first SIGGRAPH publication, shows artistic renderings of Mars with visual reference to planet Earth. We chatted with Weili to learn more about his SIGGRAPH 2019 Art Paper, how he came up with the idea for it, and the outcomes he sees in audiences interacting with “Terra Mars.” This interactive project questions our sentimental connection to our planet, asking viewers what emotions an alien planet that resembles Earth would arouse. Would they be similar or different to the feelings aroused by Earth?
SIGGRAPH: Can you explain what “Terra Mars“ offers and how it does so?
Weili Shi (WS): “Terra Mars” presents artistic renderings of Mars with visual reference to our very own planet Earth. To do so, I have trained an artificial neural network with topographical data and satellite imagery of Earth so that it can learn the relationship between them. I then applied the trained model to topographical data of Mars to generate images that resemble satellite imagery of Earth to be used for rendering. Therefore, as its name suggests, “Terra Mars” is a fusion of Mars topography and the machine’s understanding of Earth geography.
SIGGRAPH: How did you come up with the idea for “Terra Mars?”
WS: I’ve been interested in machine learning for the purpose of art creation, and one basic observation is that what the machine learns is a function mapping its domain to its range. From this perspective, how to select the domain and range to make the mapping meaningful and poetic becomes a critical question. On the other hand, I have created artworks using geographical data, such as transforming New York’s building information into shanshui (landscape) paintings, so I knew that geodata had great artistic potential. Therefore, when this idea of mapping between Earth and Mars came across me in 2017, it felt like both a natural outcome of my thinking and a eureka moment that I was lucky to have.
SIGGRAPH: You make the claim that “AI is the new imagination.” What do you mean by this?
WS: More specifically, I’d like to claim that AI is a new vehicle for artistic imagination. It enables us to fulfill creative ideas that weren’t feasible for human artists alone. Take “Terra Mars” for example. It could be an intimidating task for a human artist to form a thorough understanding of the Earth’s terrain and apply it to all the topographical details of Mars but a perfect task for a neural network.
We have seen at least two promising approaches to AI-powered imagination. One is using the famous GAN (generative adversarial network) models to generate unseen images from infinite possibilities learned from a training set, and I call this approach enumeration within a domain. “Terra Mars” is my exploration of an alternative approach — using AI to remap data between domains in order to create unexpected but meaningful and hopefully poetic connections. I believe this approach has more potential to make profound reference to the world we live in and, therefore, to reverberate deeper in the audience’s heart.
SIGGRAPH: How did you first hear about SIGGRAPH, and what made you decide to submit to the conference?
WS: I first heard about SIGGRAPH when I was a master’s student in computer science. Because my domain wasn’t computer graphics, I never thought I’d publish a SIGGRAPH paper. Then, I switched to the field of creative technology and started to create visual and interactive projects. I am no longer a computer scientist, but I actually feel more related to the conference now. How amazing is it that my first SIGGRAPH publication will be an Art Paper!
SIGGRAPH: Can you share a bit about your background?
WS: I have an educational background in both computer science and design. Currently, I work at a digital agency developing interactive experiences for museums and cultural institutes. At the same time, I make personal efforts to create artworks that inspire the audience through poetic use of technology.
SIGGRAPH: What were some of the technological challenges you needed to overcome when creating “Terra Mars?”
WS: The distinct challenge of “Terra Mars” came from the need for generating seamlessly stitchable color tiles for the global color map of Mars. A typical neural network is designed to treat each run independently, but in this way, all the tiles of Mars’ topographical data will generate color tiles that make sense individually but don’t get along with each other. I was lucky to find the solution in a paper, but still had to figure out through experiments why it doesn’t work immediately in many popular model architectures and make changes accordingly. Then, there were long iterations of model configuration and data enhancement to generate satisfactory outcomes. Like many deep learning-based art projects, in order to produce “Terra Mars” I had to push the limits of the computational power available to me.
SIGGRAPH: How do you envision people interacting with “Terra Mars?” What do you think are some of the positive outcomes of people interacting with “Terra Mars?”
WS: The project asks the audience a question: “If images of Earth arouse people’s poetic emotions, would an alien planet that resembles our own, arouse the same feelings or different ones?” Beyond that, it invites the audience to contemplate the contrast between the imaginary and actual look of Mars, and reflect on our sentimental connection to the image of our own planet.
You are welcome to interpret “Terra Mars” from various perspectives. To me, the most intriguing aspect of this AI-based creative approach is the seemingly contradicting assessments we can make of the generated Mars imagery — the generated look is, by no means, scientific. On the other hand, it is created by inferring solid logic learned from precise data by a meticulous and tireless artificial intelligence. I hope this additional layer of complexity provides the audience with a bit of space for contemplation.
SIGGRAPH: What’s next? Do you hope to make more AI-powered imaginative works?
WS: I’m still iterating the model to generate more detailed and convincing color maps so that I can present interactive close-up views of the imaginative Mars. Beyond this project, I want to continue pursuing the above-mentioned domain-remapping approach to AI-powered imagination.
Weili Shi creates unconventional experiences with the aim of provoking people’s consciousness. Although most of his works involve substantial use of technology, he always tries to base them on design thinking and infuse them with a dose of Zen mindfulness. Currently, Weili works as a multimedia designer/developer at digital agency Bluecadet in New York. Weili holds an MFA in design and technology from Parsons School of Design and an M.Eng. in computer science and technology from Tsinghua University.
