The Rookie was an industrial project in the second term of the Master of Digital Media Program. Our mission is to expand the boundaries of VR by creating a public narrative experience in a physically immersive environment. The project was also a chance for us to learn about and experiment with some new devices, such as HTC Vive, Kinect, Leap Motion, and Arduino, as well as how to integrate them into VR Unity.
The goal of the project was to create a public VR experience that combines motion simulation and normal walking. As it was a new topic in VR, I was in charge of Physical props design and art assets creating and managing, to combine the virtual and physical experience.
Our team followed the Agile Development process, in which we rapidly experimented, prototyped, tested and iterated.
Aside from being a 3D artist, my role in the art and design process was also managing the art assets and the production of the 3d virtual environment and physical setups, in order to ensure that the alignment with developer side and prop construction side could run smoothly and rapidly as we planned.
Different to most of the current VR games, we didn't use the Vive Controllers to control the movement. Instead, we embedded a racing wheel and a joystick into the experience to give the users an immersion of motion simulation.The user is controlling the spaceship by two levers.
Additionally, after some tests, we also found out that motion sickness was another big problem we had to solve in VR motion simulation. We minimized the motion sickness through gradual acceleration, limited rotations of movement, a view of the horizon, and limiting the user’s view with a vignette effect as well as a reduced view through the vehicle’s window of their moving environment.
In the walking part of The Rookie, we needed to have the players be able to interact with the physical objects. We researched and came up with some solutions using the Vive trackers to match the physical interactable objects with its virtual 3D model. However, due to the low budget of the prototyping phase, we decided to attach the existing Vive controllers with the objects to make them trackable by the HTC Vive Lighthouses.
Finally, the most difficult technical challenge we coped with was the calibration of the Vive controllers. Since the experience is the combination of the virtual and physical world, we needed to have the virtual objects match perfectly with the physical sets we built. However, the Vive controllers don't provide the accurate and stable tracking position, which means that everytime the Vive system is reset, the controllers appear in a different position than it was before. We tackled this problem by writing a script in Unity that fixes the initial position of the Vive controllers in the virtual world, and caches it everytime the system is reset.
Softwares: Unity5, Cinema4d, Rhinoceros, Adobe Photoshops, Adobe illustrator, After Effects
Project Manager: Parmida Zarrinkamar
Programmer: Hoa Nguyen, Robyn Choi, Frederik Nielsen
Prop constructor: Mateo Cano
Artist: Jimmz Zhang
Unity3d, Cinema4D, Substance Designer, Rhinoceros, 3D printers, Autodesk Recap 360.
Due to the NDA with the clients, the details and the source code of the project cannot be published.
The Haven is a VR Meditation installation that combined physical installation design, projection mapping and Virtual Reality application and collaborated with Marina Roselli. The core idea is to challenge the reversion of information from the physical and virtual world. And as artists, the purpose was to guide people to reflect their present state and question the distraction from mass information.
The concept is coming from the rethinking of distraction from social media and sense of meditation. The meditation brings back the self-identity and reflection of the relationship between the people and environment while excessive addiction to social media to gain information outside. We reverse the bound of the physical and virtual world, create the overwhelming of social media notification in the real world while audience could find their peaceful mind in VR headset. We gained the data and notification sounds from Facebook social media and projected the sound visualisation on the installation, which evokes a sense of annoyance as users instinctually check their phones. The colourful data stream projection reflects the vivid abundant mass information from Internet world which absorbs people's energy and attention in reality.
In the VR experience, we designed the simple minimal visuals with strong emotional impact. Once people inside the virtual world, the user will initially see a black or darken virtual environment simulating the transition of closing their eyes and entering into an imagined, meditative visualization. Comprised of a virtual environment, the user is presented with a naturalistic setting evoking a sense of serenity. Each beat of the story has a specific intent to build up the intended emotional journey. We also combined the audio like meditation bells and scents like petrichor and lavender to create a multi-sensory synaesthesia experience.
In the two videos below, the first video you can see a short version 360 video used for the 1st prototype (can be viewed with VR cardboard). In the second video, you can see a 360 automatic slideshow video of 5 different frames (can be viewed in VR cardboard). These frames are rendered at the higher quality.
3D artist and VR UX designer: Marina Roselli
3D artist and sound designer: Jimmz Zhang
Software: Cinema4D, Touchdesigner, After Effects, Ableton Live.
Cluster is a private VR experience installation in public space which attempt to introduce awe to the audience. By combining with a guided facilitator, pre-VR lounge, and non-controller locomotion design, this project guided immersiant to experience an unfamiliar, nostalgia and wonder narrative by going through the forest, deep sea and universe that shift user's cognition as ultimate purpose. This project was developed with Bernhard Reicke and School of Interactive Arts & Sciences (SIAT), Simon Fraser University.
In this project, I was responsible for deciding art direction, assets managing and sound designing.