This all-new episode of SIGGRAPH Spotlight features Mark Billinghurst, professor of virtual and augmented reality at the University of South Australia and the University of Auckland. Mark sits down with Bektur Ryskeldiev (ACM SIGGRAPH International Resources Committee member; and, Ph.D. candidate, University of Tsukuba) to talk about the current and future state of virtual and augmented reality. The two discuss cultural differences of international collaboration, what’s next in the space, and how students can get involved. Learn more about Billinghurst and Ryskeldiev.
Listen on SoundCloud below or subscribe to the podcast on iTunes or Stitcher.
FULL PODCAST TRANSCRIPT
Mark Billinghurst (MB)
I’m Mark Billinghurst, I’m currently a Professor at The University of South Australia, although I’m going to be spending some time moving to New Zealand later this year and having a position at The University of Auckland, as well. My background is in human-computer interaction, and for the last 20 years I have been doing research in augmented reality and virtual reality. Originally starting at The University of Washington, where I did my Ph.D., and then I moved to New Zealand where I worked as the director of the HIT lab – New Zealand – until a few years ago, and three years ago I moved to Australia.
Bektour Ryskeldiev (BR)
How did you first get involved in SIGGRAPH?
MB
I was involved in SIGGRAPH a long time ago, my first SIGGRAPH was in 1992 and I went there because I was an intern student at The University of Washington as a masters student. At the time, The University of Washington was one of the only places in the world to do virtual reality, so I came for three or four months. While I was there, it happened to be during the summertime, SIGGRAPH was happening in Los Angeles, so we took a road trip, we drove down to Los Angeles and went to SIGGRAPH. I was really impressed because they had amazing computer graphics. For example, I saw the first time anybody had a million polygons per second on the screen and they had this very big hardware for making this run. Since then I’ve been to SIGGRAPH every two or three years and also SIGGRAPH Asia now, as well.
BR
SIGGRAPH is a big, international conference, but I was also wondering if you had worked with international communities or international projects before coming to SIGGRAPH.
MB
Yes, a lot of my work was international communities and projects. I went to SIGGRAPH in 1992, but after that we collaborated with research groups in Japan and also in Europe, and now I have a collaboration with researchers in China as well. Most of these are separate from SIGGRAPH, so I normally try and find some joint projects and we have a student exchange who will work on the project. Sometimes that will end up in a SIGGRAPH demonstration or paper, but most of the time it’s separate from SIGGRAPH.
BR
How can you compare the experience of working with different international teams? For instance, as you mentioned, you worked with many teams from different countries.
MB
It’s interesting, every country has its own culture, so, for example, when we’re working with teams in Japan, the students in Japan are all very, very talented students and they do very good research, but oftentimes in Japan the research is very hierarchical, so when the professor says something, everybody does it. Whereas in New Zealand and Australia it’s a more flat structure, so there is more discussion and negotiation around the teams. So it’s just a different style of working. But in each of the countries I have worked with you can always find excellent people that are doing computer graphics or interface design work, so it’s very nice to work—and they provide a different perspective; different labs have different strengths. The lab in Japan we were working with is Nara Institute of Science and Technology, they were very strong in computer vision, so they were able to do some augmented reality work with us in computer vision. We try and find groups that have complementary skills to us.
BR
The Nara group, is that the one that developed the AR toolkit?
MB
That’s right; well, actually the AR toolkit was developed with Professor Kato, and he and I were both at University of Washington together. He had not started the group at Nara yet, he came from University of Hiroshima to the University of Washington and he spent a year there. We started working on the AR toolkit and he went to Nara—to Osaka first and then to Nara. But he is still there, and he is still working on computer vision for augmented reality. It’s a very famous, very big augmented reality lab.
BR
Something that I noticed when I started going to SIGGRAPH is that Japanese laboratories are very well known for demonstrations.
MB
Yes.
BR
For instance, emerging technologies, VR showcase. Why do you think that is? My theory is that the Japanese laboratories are willing to try different ideas even if they might not seem practical at first. What is your take on that?
MB
Yes, that’s a very good point. At SIGGRAPH Asia and at SIGGRAPH in the U.S., often almost half of the demonstrations in Emerging Technologies come from Japan. I think it’s a number of reasons. One, like you said, the Japanese are very willing to try new things; also, they have very, very good engineers, so they are very good, especially at building new types of haptic interfaces or hardware or sensor systems. And then, the Japanese labs tend to have good funding as well, so they have the money and the talent to build, and they’re not afraid to put their technology for demonstration practice. Sometimes the demonstrations seem quite silly, but oftentimes when you look at a piece of it, it has other good practical uses that may be different from ones that they’re thinking of. So, for example, there was a lot of research being done in Japan on systems that can generate artificial scent for virtual reality, so you can smell smells that are not there. The systems that they built were very bulky and big, but now—because the technology is advanced—now you have small systems that become almost usable by general consumers. The Japanese are very, very excellent at building technology demonstrations like that.
BR
Very interesting. Perhaps it’s also one of the motivating factors for me to start working in Japan, because I’m more interested in the demonstrative aspect of technology, and I want to try different interfaces and things. For people who want to work in, for instance, your area of research, or perhaps in your region, what are the specific topics of discipline they should look out for? For instance, I understand AR research is a really big topic, so are there any particular things?
MB
In my lab in Australia and in my lab in New Zealand, in both cases we are interested in having students come from all over the world to work with us. What we’re looking for is students that have—good talented students—but not necessarily in one area. So, for example, if one student is very talented in computer programming that’s very good, or if they’re talented in engineering, electrical engineering, or maybe they’re very good at doing animation and graphics content. Because the research we do is very multi-disciplinary, we can take students from all different backgrounds. We try to find students who are very good in at least one area and then students who are comfortable working with a team of different people, because most of our projects involve several people working together. And then, we typically take students as interns, and some of them decide they very much like doing internship and they want to stay for the Ph.D. or for further research. So, a good way to get started is to email me and to say, “This is my skill, this is what I’m interested in.” And we’ll see if we can make a good match.
BR
I should email you!
MB
Yes, do.
BR
And, aside from research, are there any other notable companies, or maybe other notable labs in your region, that we should keep an eye on? Anything that people should know about?
MB
Oh, there are many, so, in Australia, one of the labs, or one of the companies that is very interesting is a company called Zero Latency. And they’re doing large, warehouse-scale virtual reality installations, in particular for gaming. So, you can go into the warehouse, put the VR headset and backpack on, and then run around and be shooting each other and playing a game. They started with one location and they’re spreading out, I think they’ve five locations now around the world, but they originally started from Australia. In New Zealand, of course, everyone knows Weta Workshop and Weta Digital, who do the special effects for movies, but they are also having some spin-outs now in the augmented reality space, they are collaborating with Apple, and there is the Magic Leap company, which also has a branch office in New Zealand as well. So there is a lot happening in both New Zealand and Australia in both the VR and AR spaces.
BR
Actually, I think I tried a Zero Latency demo in Tokyo.
MB
Oh, yes, of course. That’s one of theirs, I think they have two locations in the U.S., I think two in Australia, and one in Japan, in Tokyo.
BR
It was interesting, I think it was two or three years ago…
MB
Ah, yes, one of the first ones they brought through. I think it’s improved since then, but people who have tried it have said they’re very impressed with it. It’s the first time they can walk around in a very big area wearing the VR, and also can get the feedback from collaborating with a team of people in the same, collaborative VR environment. Which is very impressive.
BR
What do you think is going to become popular next in your area of research?
MB
With augmented reality, and also virtual reality, I think there are a number of areas that will become popular. One is including more eye tracking. There are a number of companies now trying to put eye tracking devices into the head-mounted display, and in Japan, actually, there is a company, FOVE, that makes the head-mounted display with eye tracking already built-in. This is a very interesting input modality which has not really been tried very much before with virtual reality. It provides a lot of abilities to support new types of collaboration or gaming experiences. So that’s one area I think is very important. People also are starting to look at other types of interfaces for VR and AR, so not just eye tracking, but also brain-computer interfaces, so you can wear electrodes on your head that responds to your concentration level or activity, so I think we will start seeing interfaces that move away just from hand-held controller to other types of input mechanisms as well. The other area I think is very exciting is working towards large-scale, VR shared experiences. Companies like High Fidelity are developing a multi-user, persistent virtual environment. In the future, you will have to put your head-mounted display on and be online with hundreds, or thousands of people at the same time. So that is very exciting, but it’s still taking some time to get started as well.
BR
I have heard some studies where people are trying to keep their VR helmet on for a prolonged time, I wonder if that’s actually needed?
MB
I think there’s a lot of studies that have been done with that, for what happens if you’re in the VR environment for six or eight hours—do you start feeling sick? Do you start having some other ill-effects, but I think it has to be studied a lot more, definitely.
BR
How do you see the future of international collaboration in your region?
MB
I think it’s just increasing. International collaboration has always been very important, but sometimes it’s hard to maintain because it’s quite expensive, you have to travel a lot to do collaboration. The lucky thing for us now is that it’s much, much easier than it was 20 years ago when we were starting, because in those days we basically just had the email and the fax machine, and we really could not have reliable video conferencing, so it’s actually quite difficult to maintain a good collaboration. But now, you know, with very good, reliable video conferencing, it’s very easy to have connections across the world and I think in the coming years you will have a lot of virtual meeting spaces, where you can go into the VR environment together and have some meeting as well. So, it becomes easier and easier to collaborate. One other important thing too is that the technology costs keep coming down, so, when I first started doing VR, it cost you about half a million dollars to setup a good VR system. So, because it was so expensive, there were only a few universities in the world who could afford this, mostly in America and Europe. But now, the cost of a VR system is maybe three- or four-thousand dollars, or even less money. So, most universities who want to can start doing, developing VR systems like this. What we’re seeing now is hundreds of universities around the world all doing VR research, which makes much more opportunities for collaboration. So, I think both the tools for collaboration are much better and also the ability and access to the research hardware is much better as well, both together work to help increase the amount of international collaboration.
BR
That’s very interesting, it’s actually something that I’m looking into myself, because I’m originally from Kyrgyzstan and even though my university was in the top ranking of universities, we still didn’t have enough funds to purchase something that’s highly expensive as virtual reality, so I’m looking forward to the hardware…
MB
Well, let me know, you can buy the Vive system for maybe $600 (USD) and put that with a $1500 computer, so for $2000 to $2500 you’ve got the VR system, so maybe it’s a low enough cost now that your university can afford. And, of course, people are doing a lot of mobile VR on their phones now anyway, as well, also Augmented Reality, so you know, you can easily build AR applications for iPhone or for Android devices. So now we are seeing a lot of research coming from China, from Asia, Southeast Asia, and other countries that, 10 or 20 years ago, would not have had the budget to do AR or VR.
BR
Aside from research, is there a reason why I should visit your laboratory or the city where your laboratory is located?
MB
Oh, lots of reasons, so I have two laboratories, one in Adelaide, Australia, and one in Auckland in New Zealand. Both are very beautiful cities to live. Adelaide is always in the top 10 cities in the world in terms of quality of life, and New Zealand is a fantastic country with very beautiful beaches and green mountains and things like this, so, definitely it’s very good to visit, not just for the research but also for the tourism. Also, I think it’s always very good to try to experience different cultures as well, because it helps to give you a different perspective on your own research or your own life too, so there’s many, many reasons to come. I think it’s also very interesting why SIGGRAPH moves the conference around year by year, so people get to experience different countries and cultures. This year, for example, SIGGRAPH Asia will be in Tokyo and next year SIGGRAPH Asia will be in Brisbane, Australia. Now, Tokyo is very different from Brisbane, and if you go to both conferences, you get to see very different cultures which is really great.
BR
Are there any other comments or recommendations you can give to our listeners?
MB
Well, I think it’s a very great time to be involved with augmented and virtual reality. If they’re interested in those research areas, they should start trying for themselves. You know, it’s very easy to learn how to develop VR and AR systems—the tools are cheap and easily accessible and the devices, like the mobile phones or the headset displays are easily accessible, so if you’re interested in those areas please try. And, also please feel willing to contact me with questions or asking for help on how to get started, I’m very happy to help people to get started in this area. I’m always looking for collaborators, so if anybody wants to collaborate, please contact me as well.
BR
Great, thank you very much.
MB
Thank you for your time.
