I had the pleasure of speaking at the International Investment Forum in Virtual, Augmented and Mixed Reality (#IIFVAR 2017) at Technopark Zurich on April 20th, 2017 alongside a wonderful lineup of speakers. The event was organize by the Swiss Society of Virtual and Augmented Reality (SSVAR). My talk centered around the design of immersive experiences, with a focus on Virtual, Augmented, and Mixed Reality experiences. With the explosion of devices and platforms for VR/AR/MR, it’s important to look at the design process for how to go from idea to final product without getting lost in the technology traps and app template cliches. This is part of an evolving design approach for immersive products, which ultimately combines the best of storytelling, user experience design, and game design. This talk was the latest in my short history of conferences focused on immersive product design. Earlier in 2016 I gave an extended talk on immersive experience design at the Zurich FrontEnd conference and before that I gave a workshop at ISMAR 2014: Designing Location-Based Experiences.

Immersive Design Process

The primary design process for immersive experiences is similar to general UX or interaction design methods. I like to focus on concept mapping (UX journey, storyboard, etc.), then interaction design (based on how the user will interact with the virtual environment), then think about the actual technology platform (Vive, Hololens, etc.), followed up by building and testing. Naturally a project will move forward and backward, looping between the different phases as needed to deliver the final immersive product. Beyond the design flow, immersive design is generally a combination of themes and technologies from Film/Theater, Game Design, and User Experience Design. I approach VR and AR projects from this perspective, and search out the balance between story structure, technology, and user interaction needs and patterns.

VR is Not a Medium (yet)

Since about a year social media groups and tweets focused on VR and storytelling have touted the idea that VR is a new medium for storytelling, learning, etc. The problem is that VR is yet a defined medium. If you ask most people what a VR experience entails, they won’t be able to give a concrete answer. Part of the reason for this is that audiences don’t know what a VR (or AR or MR) experience is supposed to be. They haven’t learned what VR is supposed to be because it’s still in the early stages, and like with other entertainment mediums (such as film), it’s up to the industry to teach the audience. Compare the current state of VR to cinema, for example. The cinematic experience is quite clearly defined, it entails sitting a theater, generally with other people, and watching giant screen with a series of sound synchronized images displayed at 24 frames-per-second. It’s not complex, there aren’t that many rules, but when an audience enters a theater, they know what to expect (more or less). This medium however evolved literally over decades. Currently VR is basically at the early stage of cinema evolution, basically at the comparable stage of silent film era in the history of cinema. It’s up to the VR industry to provide great experiences and teach through doing what the audience should expect when they put on a VR headset. If this doesn’t happen, VR may go down the same pathway as 3D movies and television, in other words, ultimately be rejected by global audience.

Design Layers

When presented with a design challenge with multiple technologies and mediums to design from, I like to imagine a layers approach to product and experience design. This allows project participants and stakeholders to clearly see what needs to be done in order to deliver the final VR/AR/MR product. Then, through the specific tools and technologies of each design layer (for example user journey maps or game design documents), a designer and clearly negotiate the process designing and building an engaging immersive product using any of the growing number of immersive technologies available. More importantly, this approach allows for integration between story, code, and production. In a VR team, this is extremely important since it offers a way to unify the design process in a way that everyone can contribute to and understand.

Coverage of the Event

I would like to thank the organizers and sponsors of the 2017 #IIFVAR event, it was a wonderful combination of individuals and companies. I’m looking forward to seeing how the industry develops and how industries and societies will be influenced by access to great immersive experiences.

#IIFVAR 2017 in Zurich: a magic moment…
#IIFVAR 2017 Storify