William Chyr Studio, 2020
In August, Arthur Brussee gave a Talk to SIGGRAPH 2020 participants exploring his graphics development work on the popular indie video game “Manifold Garden.” We caught up with Arthur to learn more about his process for developing the virtual Talk, “That’s a Wrap: A Manifold Garden Rendering Retrospective,” and to dig deeper on his team’s work creating a GPU rendering pipeline, the challenges of building a pipeline from scratch, and more.
SIGGRAPH: Talk a bit about the process for developing your virtual Talk.
Arthur Brussee (AB): I wasn’t quite sure where to begin with the Talk. Games are made up of thousands of little decisions that individually may not be of interest. Instead of focusing on each and every one, I tried to identify a few of the bigger “systems” that were relatively self-contained and had some interesting tidbits to explore. My goal was to turn those pieces into a few “chapters” with which I would build the Talk. Some of these systems are novel — like our edge detection or the world wrapping — others just felt nice to highlight in the context of how the look of “Manifold Garden” was created. I was also originally worried I would not fill the time with just the systems I picked, but it turns out there’s always more details to discuss than you’d initially think.
I started by writing a two-page abstract as a condensed version of the Talk’s content. Translating this abstract into a video turned out to be a bit of a challenge, as it was surprisingly difficult to talk without an audience while recording from my room. Without any feedback, it felt very odd to keep going, especially after stumbling over something. In the end, I definitely found myself splicing together a few takes.
SIGGRAPH: As a veteran of the graphics world, what was unique and exciting about working on “Manifold Garden”?
AB: I’d known about the “Manifold Garden” game a few years before I joined the team. The graphical style of it really struck me — it’s so focused, yet recognizable and clean. I had previously worked on “Ori and the Blind Forest“, a 2.5D platformer with ghibli-esque graphics. What I learned working on Ori is that games with peculiar graphic styles also end up with peculiar, and interesting, graphics problems. This is, of course, not to say that games striving for photorealism have it easy, but when you leave that behind, you end up with challenges that are off the beaten path and often have no references to aid in troubleshooting.
When I saw “Manifold Garden” had an opening for a graphics programmer, I went for it. My intuition was right about unique challenges: The world wrapping interacts with a lot of different systems and creates cool problems, and our work on the edge detection shader is another example of this. We can now likely render post-processed, anti-aliased outlines like no one else because no one else has had the need to go this far!
SIGGRAPH: Is there one game in particular that inspired you to pursue your career path in the industry? If yes, talk a bit about how it led you to where you are today.
AB: I honestly don’t believe I’ve ever considered that! Though I (used to) play a lot of games, I don’t think any of my games of choice leaned particularly hard on their graphics technology. If I had to point to a single origin of becoming interested in graphics, I can probably attribute this to a forum post by Aras Pranckevičius, a Unity developer. Somewhere around the time of Unity 4, he posted an example showing off the new-at-the-time DirectX 11 features in Unity by rendering a million floating dots. I remember being just baffled that this was possible on my machine at all and had to know how all of it worked. That was about 9 years ago, and now here we are…
SIGGRAPH: Let’s get technical. How many people were involved? How long did the project take?
AB: The “Manifold Garden” team was rather small, which is often the case for indie games. We had two people mainly focused on the graphics tech, and William Chyr on design, but that didn’t mean we were the only ones to touch the shader code. Indie game job titles are rather fluid. I worked on the project for about two years, and “Manifold Garden” was in overall development for seven years.
SIGGRAPH: Can you discuss more about your GPU-driven approach for efficient instancing used to create the seemingly infinite world in “Manifold Garden”?
AB: The world wrapping was a challenging system. Put simply, we had to duplicate the entire world in a grid until it reached the maximum view distance. For static objects, this works surprisingly well with a few tricks, but for dynamic objects, there’s no getting around the fact that you have to update hundreds of instances when changing the main instance. To solve this, we created a GPU rendering pipeline. This way, only one “real” object is controlled by the CPU, and the GPU can then render many instances of it directly. The GPU can decide which wrapped instances are visible, where to place them, etc.
This is not a groundbreaking GPU pipeline, but I do think it’s encouraging that even indie games can ship with some GPU-driven rendering, and there’s a lot of potential to do more rendering work on your GPU. Sending commands from a CPU can sometimes feel like trying to drive a car by whispering suggestions to the driver, rather than using the pedals yourself.
SIGGRAPH: What did you learn from creating the portals to other levels? Was this a big challenge?
AB: The portals created quite a few additional challenges as well, as they create lots of complicated situations for the sound system, physics, and character controls. The basics of rendering the portals worked within a week, but as the game developed, we kept finding new issues and tweaks that needed to be made. The portals only really started working without flaws or limitations when we rewrote our rendering pipeline. “Manifold Garden” is built in Unity, which by default has a rendering pipeline you can plug into, but not really control. We rewrote it using their API to create a pipeline from scratch, and it immediately benefited a lot of our systems.
The biggest challenge for “Manifold Garden”, overall, was probably the fact we had nowhere to hide. Usual techniques to cover up artifacts weren’t possible and would stand out immediately in the otherwise sterile look of the game. That leaves little room to cheat. When PlayDead’s “Inside” came out, they shared their techniques to prevent many types of artifacts, and we used this as inspiration to solve a few of ours.
I’m also happy I got to talk about our performance scaling system [during SIGGRAPH 2020]. I really dislike the games culture that expects users to change 20 settings to “art direct” their performance vs. quality tradeoff. We did this in a data-oriented way for “Manifold Garden” that turned out to be much more flexible and delivered better results, allowing us to scale to old iOS devices later in production.
SIGGRAPH: You mention that, though the approach is unique, these insights into rendering possibilities and approaches can be applied to multiple contexts. What is the most important concept you want viewers to walk away with from your Talk?
AB: There are a few very concrete tidbits that could certainly apply to more contexts — for example, don’t rely on a standard MSAA resolve, but do consider if the position of the subsamples can help you out further. Another is our double-stencil buffer trick, which allowed us to create a much more robust portal system. Other systems that need to recursively render something into the stencil buffer could use this, too.
Another application is to look for signed distance fields (SDF) in unexpected places. Signed distance fields have lots of solid literature behind them, so when I recognized the edge detection problem as an SDF, it all became a lot easier.
A more general point that I hope smaller games can take away is that there’s a high return on investment in taking control of a rendering pipeline. Even for a relative simple look, like in “Manifold Garden”, this has a good amount of benefits. When creating such pipelines, keep them data oriented and maintain a clear view on inputs and outputs.
SIGGRAPH: Given the complexity of the technology behind “Manifold Garden”, how did you narrow down the main topics in which to focus your Talk?
AB: We did have to leave out a few details. I think mainly, as it usually happens, there are attempts that don’t work out. For example, we spent quite some time trying to improve our geometry pipeline with CSG techniques, and we actually got it to work for 95% of the cases, but the improvements were much smaller than we thought, so fixing the last 5% was just never deemed worth it. We also tried taking advantage of the repetitive nature of the world for our shadow maps. Similarly, this only ended up slightly better, and broke a few times, so it was canned.
I think it’d be great to talk about why we tried some of these in the first place, and why they didn’t work, but in the end, that didn’t seem as compelling as other stories discussed in the Talk.
SIGGRAPH: What did you enjoy most about your first SIGGRAPH?
AB: In a sense this was my first SIGGRAPH; however, I’d already made a habit of following along with the Talks and Technical Papers that are released to the public post-conference. “Advances in Real-time Rendering” is a highlight every year, and actually attending this year made me feel more a part of the conversation; I was able to witness these discussions in real time. It felt nice to be included in the hype on Twitter this time around, too!
SIGGRAPH: What advice do you have for someone looking to submit to Talks for a future SIGGRAPH conference?
AB: Don’t be scared to submit; just go for it! I honestly didn’t think SIGGRAPH would be interested in a Talk about a smaller indie game with relatively obscure rendering problems. I’d also like to give a huge shoutout to Carlos Gonzalez-Ochoa, SIGGRAPH 2020 [and 2021] Games Chair, who encouraged me to give it a go, as I don’t think I would have gone so far as to make a submission without his encouragement.
SIGGRAPH 2021 submissions are open now! Visit the SIGGRAPH 2021 website for the latest information and open programs.
Arthur Brussee has been working in graphics for about seven years now and has won a BAFTA for the graphics technology used in “Ori and the Blind Forest”. He worked with William Chyr on the development of “Manifold Garden” to push the look of the game. He enjoys graphics programming as a unique space to find effective collaboration with artists, in addition to working on difficult engineering problems. Recently he joined DeepMind to pursue the intersection of games and RL agents.
