abstract image of a hand being used to represent hand tracking
August 30, 2022 · 11 min read

Hand-Tracking to Enhance Virtual Reality Surgical Training Simulations

Wraith’s Medical VR “hand-tracking” allows users more control without a controller.

Wraith VR surgery has been exploring how to improve user interfaces in surgical simulations by using hand-tracking. This fairly recent innovation allows users to control virtual objects with high precision with their hands while in mixed reality. This new feature, which is already being used in client projects, brings medical simulation to the next level for surgical training accuracy and learning.

Oculus has been experimenting and developing “Hand-Tracking” for the Quest since late 2019. Hand-tracking offers the user a controller-less experience on the Oculus Quest using your hands as controllers. As development has progressed so has the sophistication and usefulness. Hand Tracking 2.0 was released in the spring of 2022 with improved accuracy and the ability to detect touching.

In July, Wraith VR decided to explore the potential of hand-tracking for medical virtual reality applications. By building a demo app, with the motive to improve the experience for clients. Hand tracking modules allow Unity developers to tune how the user can grab and use interactive objects in a virtual environment. 

Wraith VR “Hand-tracking” makes it possible to handle virtual surgical tools with precision in a VR headset.
Wraith VR “Hand-tracking” makes it possible to handle virtual surgical tools with precision in a VR headset.

We learned that Oculus hand-tracking in Unity offered a lot of flexibility for the user to interact with medical devices and tooling. We can develop graspable objects using locked poses and hand positions that are generated based on where the tool is grabbed in augmented reality. VR with hand-tracking enhances the user experience and will be leveraged to improve comprehension in the simulations we develop for our valued clients.

Wraith VR hand-tracking technologies brings medical simulation to the next level for surgical training.

How to use hand-tracking for VR surgery?

If the Wraith VR surgical simulation you’ve chosen to load allows for hand-tracking, it will inform you as it is loading that you can use Touch Controllers or use hand-tracking. If you prefer hand tracking, simply put your upside down on their tracking rings on a level surface. In a few seconds you should see your hands digitally represented in front of you in the VR space. Then just grab whichever objects you want to grab and perform the surgery.