I was approached to serve as the lead researcher for a gestural user interface that was displayed through a hologram through a government grant. This system was to be used as a situational awareness and planning tool for trained personel.
Create a gestural library, conduct concept testing, and conduct a literature review to present to the grant sponsor to be used in the future development of the system.
First, I conducted a literature review looking at ergonomic principles of gestural workspaces, virtual interfaces like CAD modeling software, as well as prior research in novel user interfaces and AR/VR devices. I knew that our system needed to control a hologram using your hands, but without touching it and yet still be valuable in communicating information to the user.
Next, I conducted a concept study to ascertain what mental models people had for controlling interfaces they could not touch. I also created a prototype of the system within virtual reality to make the experience more immersive and convincing as this project can seem a little far fetched. We pulled from computer aided design software conventions to help inform the participants as they were mostly engineering students. We could not test directly with the end user as they were a protected population, so we did the best with who we had available.
In the end, I worked with the grant sponsor to create a library of gestures that could be recorded with smart gloves that would then be identified by machine learning to control the holographic display. I created a training protocol for the smart gloves to be trained on, but did not oversee the actual training as that was scheduled for the next phase.
Due to the nature of this grant, I am not allowed to discuss the specifics of the project in depth. I can comment on my experience overall. I learned a great deal from this experience — from leading independent academic research to getting into the technical details of the systems we used. I hope that this project sees development and one day helps people. Who knows — maybe we don’t need to be in a galaxy far far away to have technology that feels like it is from outer space.