Image credit: Copyright Gaëtan Robillard
Step into the world of the “Critical Climate Machine”, an installation that explores climate misinformation through machine learning. Catch up with Gaëtan Robillard and Jérôme Nika about their upcoming SIGGRAPH 2024 Art Papers presentation.
SIGGRAPH: Share some background about your SIGGRAPH 2024 Art Paper, “Critical Climate Machine: A Visual and Musical Exploration of Climate Misinformation Through Machine Learning.” What inspired this installation?
Gaëtan and Jérôme: In the last few years, we’ve studied in depth early computer art in Europe, gaining significant insights from the works of pioneers like Frieder Nake, Georg Nees, Véra Molnar, and others, who were instrumental in the development of informational aesthetics. This rich historical context provided a conceptual scaffold for “Critical Climate Machine”, reflecting on how data and information become aesthetic experiences.
We also ventured into the intricacies of cognitive and political sciences, specifically examining climate communication and techno cognition. These inquiries illuminated the complexities of how one perceives and interacts with the threat of climate change through the lens of social media and machine learning technologies. Inevitably, this inquiry in the spread of misleading claims led to considerations on the sharing of collected data in open frameworks, contributing to knowledge on misinformation.
In addition, American minimalism inspired us. The minimalist ethos guided us in neutralizing the form, allowing content to resonate in the exhibition space. The creation of the sound piece is rooted in a reflection initiated when we, Gaëtan and Jerôme, first collaborated on a project with philosopher Bernard Stiegler. It echoes his concept of technical individuation in the use of generative tools whose specification process is intertwined with that of formalizing the artistic intent. In this synthesis of influences and research, we aimed to create a work that is not merely to be viewed but experienced, fostering a new kind of dialogue between society and technology.
SIGGRAPH: What is a common piece of climate change misinformation you have encountered, and how does “Critical Climate Machine” expose it? How did you develop those machine learning algorithms to process climate-skeptic discourse and science-based rebuttals?
Gaëtan and Jérôme: A common piece of climate change misinformation we have encountered includes the denial of the anthropogenic origin of climate change and the denial of science itself. “Critical Climate Machine” exposes these types of misinformation first by using a machine learning algorithm trained on labeled data to classify such misleading claims. A total of 17 categories were encoded. The method and dataset were originally developed in cognitive sciences and climate communication studies and were then adapted to process a livestream of online data. As such, the artwork visually exposes the computational process by scanning through thousands of tweets published by climate skeptics on social network X.
We also structured a large set of short dialogs between climate advocates and climate skeptics based on the same 17 categories. Developed in the context of agent-based musical composition at Ircam, our second machine learning algorithm generates new dialogs by navigating through this memory matching composed abstract scenarios. Finally, the sound is spatialized to create a musical composition that embodies the dynamics of the misinformation and the rebuttals, sometimes becoming more abstract. This multifaceted approach not only highlights the spread and nature of climate misinformation but also transforms it into an engaging visual and auditory experience, remediating scientific knowledge.
SIGGRAPH: Did any challenges take place during the creation of and research for this project and installation?
Gaëtan and Jérôme: One of the most significant challenges in creating this artwork was dealing with Twitter (now X) data, particularly misinformation data originating in climate denial. With the recent significant changes in social network policies, such as the restriction of data access through the Twitter/X API, “Critical Climate Machine” captures the end of a certain era for the network. It’s like portraying a shifting digital landscape, coming with new frontiers.
Another major challenge was developing a complete production platform that involves researchers, engineers, students, educators, and data experts. A central focus was fostering conditions for contradictory debates, which later became crucial in producing the sound work. An early, pivotal part of this process was designing the “Refutation Game”, which creatively bridged computable and non-computable approaches. Overcoming such challenges owes much to the support from major institutes like Ircam Centre Pompidou and Université Gustave Eiffel.
SIGGRAPH: How does “Critical Climate Machine” contribute to the greater conversation around climate? What impact does it have?
Gaëtan and Jérôme: One important contribution is the production of new data on climate misinformation. Another is the dissemination of the conceptual approach, resulting in a framework that enables greater social engagement on climate issues. The project has been especially acclaimed for this facet, receiving the British Computer Society Award (Lumen Prize for Art and Technology). Additionally, it has evolved into participatory exhibitions where the audience is invited to play the “Refutation Game”, fostering an educational experience and opening new dialogs within the civil society.
SIGGRAPH: What other industries would benefit from the information that you gather from the “Critical Climate Machine”?
Gaëtan and Jérôme: Most probably the creative and the media industries.
SIGGRAPH: What do you hope SIGGRAPH 2024 participants take away from your Art Papers presentation?
Gaëtan and Jérôme: In this project, we aimed to uncover the mechanisms behind climate misinformation by combining machine learning algorithms. Therefore, our research highlights the relation between natural language processing and generative musical research, especially when they share the same initial data. While computational logic in cognitive sciences aids in discerning truth, ML-based musical approach ventures into exploring norms around climate mediation. This implies a greater focus on both data and interactivity, and less on the automation of creativity itself. We notably hope to inspire participants to consider the significant perspectives in this area, particularly how generative processes involve new creative tools and practices. Instead of relying solely on autonomous generative agents, we advocate for practices that keep the compositional gesture fundamental. In “Critical Climate Machine”, concepts such as “memory” and “navigation” are valued as artistic leverage for experiencing timely issues in novel ways, encouraging both formalization and reflexivity throughout the process.
SIGGRAPH 2024 is almost here! Register now to access the latest in computer graphics and interactive techniques! Join us in Denver or online for this incredible opportunity.
Gaëtan Robillard (FR) is a visual artist and a researcher living and working in Greater Paris. He produces data art and media-based installations engaging with cognitive sciences and climatology. His work has been exhibited in venues such as Palais de Tokyo and Ircam Centre Pompidou (Paris), Pearl Art Museum (Shanghai), Akbank Sanat (Istanbul), and ZKM Center for Art and Media (Karlsruhe). In 2022, he obtained a Ph.D. in art and technology from the University of Paris 8, and already published nearly 20 papers on early computer art aesthetics, education, and contemporary algorithmic artifacts. He also regularly lectures on the history of visual techniques and artificial intelligence.
Jérôme Nika is a researcher in interactive and generative technologies for musical creation in Ircam’s ISMM team and an electronic musician. His work revolves around the metaphor of “musical memory”, from its modeling to its mobilization in a creative context. This research has led to the creation of the Dicy2 environment and over 60 artistic productions in which he is involved as an electronic musician or computer music designer. Latest production: Creation of the generative electronics for Ex Machina (Pi Recordings / L’Autre Distribution) by the French National Jazz Orchestra, featuring Steve Lehman and Frédéric Maurin, released in September 2023.
