For those who don’t know, I attended VISR’s Mixed Reality Accelerator earlier in the year. It finished this August, and I finally have the time to sit down and go through it. Thankfully I have been given permission to speak about our project without any restrictions from the Non-disclosure agreement. I do not have good quality promotional media yet though, so I will add those at a later date.
It was a great experience, no question. We created some really innovative solutions, and not only that, we formed a good community of like-minded people that will hopefully last well after the Accelerator. Below I’ve split the experience into two sections:
The First two weeks were dedicated to team-building, teaching and preparing the participants. We had lectures on C# programming, the VERTX platform, 3D asset creation, and Unity from the team at VISR. Around these lectures we were assigned into small groups and competed in two Game Jams in order to put our knowledge into practice, and foster teamwork and coordination. This was undoubtedly the best way to prepare 20+ students for a real-life project.
- The First GameJam was also our first time working with each other, and the concept was simply to make a Game and build it to the Xbox using UWP (Universal Windows Platform). This was tough but rewarding, as we had to figure out all aspects of collaboration, such as source control, task management, and planning, whilst some members of the group had just been introduced to the basics of Unity. Learning how to build to UWP was crucial, as this is the process we would later use to build for the Hololens.
- The Second GameJam was geared more towards preparing us for the Hololens, as each group had to create an app that allowed you to manipulate 3D objects in space, and had to include multiplayer networking logic. Thanks to the first Game Jam, we found it much easier to work as a team, and create the game itself. The real challenge was the networking logic, which was new to all of us, but it was a great learning experience. In order to complete this task we used VERTX plugins integrated into Unity.
After that, our groups were finally assigned the companies that we would be working with, and everybody met their Industry Representatives. Our group partnered with LearningMate, a large Education and Training development company. The POC (Proof Of Concept) they envisioned was an application to be used by engineers to aid in maintaining complex machinery. This was to be achieved by combining Hololens functionality with an IoT integrated electric box via the online VERTX platform.
We liaised with our Industry Reps on a daily basis, they were amazing help throughout the entire process, in all areas from Business, to Programming, Graphics and UX. Together we pinpointed 4 key functionalities that would serve as an MVP (Minimum Viable Product): Live Information of components, Interactive guide, Remote Assist, and a collaborative learning exercise. Additionally, all of these holographic functionalities needed to be positioned correctly respective to the physical electric box, and be easily accessible from a main menu. This was the storyboard I drew up for the User Journey:
Our group of 4 then divided up the tasks, I took the tasks of project management, 3D asset creation, and Unity/C# developer, another focused on the IoT aspect of the project as well as managing the teams source control (git) and the other two worked almost exclusively as Unity/C# developers.
We followed Agile development methods, having a scrum every morning to report progress and blockages, using an online Kan-ban board, and splitting the four aspects of our MVP into sprints: 2 weeks for the applications main menu, and the first functionality, 1 week each for the other three functionalities, and 1 week for final polish.
With a lot of hard work, and a few late nights near the end, we were able to deliver all of these, and the project was a resounding success. Our final application showcased:
- An intuitive method of scanning the environment and placing a visual anchor on the electric box for alignment.
- A simple an clear menu with subtle animations, perfect for all levels of Hololens users, leading professionals from Microsoft to praise our UI.
- A holographic exploded view of the Box’s components, with windows showing live information on the status of each of those components, as well as logs showing previous service history. All visible before even touching the physical box, and with the ability to leave a recorder message for future engineers.
- An interactive guide, directing the user to perform a maintenance bypass and replace a fuse safely. This was possible due to integrating IoT sensors on all components of the box, so that every physical action performed by the user was logged, and that information was sent directly to the Hololens for it to proceed to the next instruction. Having the holograms respond instantly to physical stimuli is an incredible feeling, and is the star of our application. It greatly increases immersion and usability.
- Integration of “Microsoft Remote Assist”. Allowing the user to place a call directly from within the app and receive audiovisual assistance from a remote expert.
- A collaborative feature showing the possibility of several Hololens users manipulating the same virtual components in order to construct a virtual electric box. Being able to see and interact with the same holograms as other users adds a lot to the perception of realism.
Images of each function are pending, and will be uploaded when possible, for now you can see short videos of all projects here: www.visr-vr.com/vertx-projects/#learningmate
Hopefully we will be able to get a video that shows off all functionalities soon.