Project Brief
In the fall of 2017, one year after shipping the first HoloLens, Windows 10 added support for Immersive headsets (VR). This represented an expansion of the holographic operating system into the fully digital end of the mixed reality spectrum, along with introducing mixed reality to consumers.
VR headsets sold by OEM partners required a tethered PC setup process, followed by an in-HMD experience. This in-device experience introduces mixed reality and how to get around in the virtual space; it needed to accommodate a variety of input, including Xbox game-pads, 6DoF controllers and voice.
Role: End-to-End Experience Designer
As one team developed the PC setup portion, I focused on developing the concept and interactions for the in-device experience, which introduces input and navigation. This experience needed to build on the excitement in the unboxing process and extend the Mixed Reality brand.
I developed the story, guided world design, and detailed interaction for a self-contained experience making learning easy and memorable.
Selected story frames from the PC setup and in-device experience; an interconnected story (above), and the v1 experience captured in its entirety (below).
Insights and Process
Brand cohesion
To extend the Mixed Reality brand into VR, I created a flow that felt familiar to HoloLens customers and reflected marketing messaging; I employed the same step-by-step, gated learning approach that I designed earlier for HoloLens.
Connection to ‘Place’
I focused my design research on understanding how environment affects us, especially in learning situations. How might we form connections to digital environments; are emotional and geographic connections possible when we can so easily switch worlds in VR, along with the intangible nature of digital things?
“‘Sense of Place’ refers to characteristics of a place that foster a sense of belonging and authentic human attachment with the place.” (Relf, 2007).
In the storyboard sketch, above, interactive learning moments are presented along the journey to the Mixed Reality ‘Home’.
The idea of forming a connection to the ‘virtual place’
drove my approach to the final design.
Using a controller to pull the Moon across the sky, causing multi-colored fireflies to emerge, before giving way to the sunrise and the arrival of seagulls.
Visual and Environment Design
Installations that lead us on paths - encountering moments of heightened engagement along the way - were particularly inspiring. I designed a path with visual and audio cues to guide us to the final destination.
Proxemics and Perceptual Clues
Understanding how to lead the participant along a path also required an understanding of proxemics and physical ergonomics and how they relate to the field-of-view of the VR headset. This helped inform spatial layout of elements and diegetic UI.
Comfort
“When I opened my eyes I saw nothing but the pool of nocturnal sky, for I was lying on my back with out-stretched arms, face to face with that hatchery of stars. Only half awake, still unaware that those depths were sky, having no roof between those depths and me, no branches to screen them, no root to cling to, I was seized with vertigo and felt myself as if flung forth and plunging downward like a diver.”
-Antoine de Saint-Exupéry: Wind, Sand, and Stars
It’s important to design for physical comfort by providing stable visual references and architectural elements. This is especially important for people new to virtual reality as many need time to become acclimated to the unique methods of locomotion.
The quote to the right, from one of my favorite books, provided inspiration while also foreshadowing the need to severely reduce the chance of vestibular discomfort.
From Sketch to Prototype
I made quick prototypes based on sketches and 3D visualizations to build story and refine spatial layout.
A style frame of an avatar using controllers to interact with glowing particles
I worked with a concept artist to explore potential environment designs
Time and Place in the Cliff House
‘Home’ in ‘Windows Mixed Reality Immersive’ (VR) is the ‘Cliff House’, a multi-room structure where you manage and open your apps and content; the fully digital equivalent of the HoloLens’ physical-world based approach. Developing a story where the human is at the center of “building” the structure aimed to create a sense of connection and attachment.
We begin back in a time when only the foundation of the house existed. Then, in the final stage of the journey, we transport to the present - finding ourselves inside of the completed structure, ready to explore content.
Above, storyboard images and actual captures from the before and after time-jumping experience.
Teleportation paths and the specifications for points of interest (POI) and interactive elements were added to the floorplan.
Final-stage prototypes and audio design
After making storyboards, I worked with 3D artists to build elements for more refined prototypes with greater interactivity and spatial audio, which I established in earlier prototypes using audio clips of ocean sounds, birds, wind rustling through trees. Additionally, I incorporated localized ‘sound bubbles’ containing Cortana’s voice guidance at each stop along the journey, marked by star-filled rings.
Design Specifications
Spatial layout specifications for visual and audio elements, along with storyboards
Above, the storyboard for the final design.
Guidance for the points of interests, gems, and spatial audio.
Audio clip specifications and line-of-sight to interactive gems.
The v2 experience
Based on customer feedback, the environment was brightened and redesigned to be more colorful, as seen below.
To try for yourself, open the “Learn Mixed Reality” app from your Windows Immersive headset.
