The HTC Vive & Oculus Touch controllers are a critical part of the VR experience, however the aesthetics highly resemble the likes of typical video game controllers.
Instead of the gamey wands, we wanted to create a haptic glove to provide a more realistic and intuitive experience for users, enabling an individual to touch and feel in virtual reality.
You are viewing: How Hard Would It Be To Program A Oculus Glove
The design of the haptic glove and paired demo was inspired by Iron Man, because what’s cooler than using a gesture controlled glove to shoot laser beams? When a user puts on the glove, a closed fist position would begin to charge the laser beam and after two seconds, the user would open their hand to release the charge and shoot the beam. During this process, a user would feel feedback from the haptic motors denoting when the glove is charging and when the charge was ready to be released. The purpose of this demo was to showcase the immersion of freely moving all five fingers, and the accuracy of response when doing so. The user would only be able to charge the laser beam if all five fingers were closed in the shape of a fist.
Conceptualizing
We originally thought we could incorporate just the internals of the HTC Vive controller and avoid it’s form factor, but in taking apart the controller, we found the positioning of IR receptors were specifically calculated (hence the doughnut shape).
Because the controllers were connected with delicate ribbon cables, we decided that it would be easier to mount the Vive controller as is for tracking orientation, rather than engineer a new idea around their internals.
Read more : How To Get Emperor’s New Gloves
Main Ingredients
EVA foam was the perfect material for our initial prototype. It gives a solid appearance while still being lightweight and flexible to fit a wide range of arm sizes. We sewed small pockets onto a cotton/spandex glove for the haptic motors and flex sensors.
We originally looked into leveraging the Vive controller’s wireless protocol but as it is 100% taken up by the sensor and button data. When you build an official device, the hardware is basically a black-box shape of your choosing; OpenVR handles everything for you and provides no room for customization.
This means anyone building a tracked object with a significantly different set of control features than the Vive controller needs to figure out a completely separate communication channel for their own data and let the Vive controller handle just the positioning of the device. Since we built the haptic glove around the Vive controller, this means the design will very easily adapt to the newly announced tracking pucks. We used Bluetooth for the communication protocol, as there are a number of hobby-grade micro-controller boards with Bluetooth available allowing for rapid prototyping.
The electrical system was built around the Adafruit Feather M0 Bluefruit LE. The Bluefruit LE is an Arduino-compatible micro-controller board with several nice features that fit well for this project. It features a Nordic nRF51822 chip for Bluetooth LE wireless communication and a 48 Mhz ARM Cortex M0+ for processing. The built-in battery charging circuitry had overcharge protection as well as charging-to-discharging switch-over. And there were a plethora of I/O pins, including several with analog to digital converters, giving us the ability to sample several sensors and drive several motors without having to overcomplicate our device with any multiplexing circuitry.
We used the same Spectra Symbol SEN-08606 flex sensors that were used by Thomas G. Zimmerman and Jaron Lanier in the original Nintendo Power Glove. And while a really quick-and-dirty build would have sufficed to sink the vibrating pager motors through the micro-controller, we decided to go the extra mile and use BJT transistors to switch power directly from the battery to the motors, thereby getting around the limited current-sourcing ability of the MCU. Once the haptic glove and paired Unity demo were built, the next step was syncing the two together over Bluetooth.
Bluetooth LE is a wireless personal area network protocol that was originally designed by Nokia under the name Wibree to be used to pair personal fitness trackers with smartphones. It’s seen limited adoption in the desktop computer space, but we were able to find a BLE USB dongle by a company named Plugable. This dongle works with the default Windows Bluetooth stack, so we didn’t have to perform any weird hacks with attempting to reflash the firmware on standard Bluetooth 4.0 USB dongles. However, the libraries available on Windows for accessing the Bluetooth stack aren’t compatible with Unity, so we built a driver server that used network sockets to perform interprocess communication with the game app to transmit the sensor readings and motor triggers back and forth. The driver server also worked as a handy testing tool, both for the physical hardware without the finished game as well as for the game without the physical hardware, as our team developing the project was spread across four different locations around the world.
Read more : How Do You Make Octopus Inside Glass Glove
Bluetooth devices communicate with PCs through two protocols, one called Generic Access Profile (GAP) and one called Generic Attribute Protocol (GATT). GAP controls how devices advertise themselves and how they connect to other devices, and GATT is the fine-grained detail of the features that the device exposes. GAP also supports broadcasting a small amount of data, which is how Bluetooth beacons work to be able to advertise a URL without needing to make a pairing relationship between devices. Theoretically, if we hadn’t needed to communicate back with the glove to trigger the haptic feedback motors, we could have performed all of the communication through GAP and eliminated the device pairing process.
But as it were, bidirectional communication requires a pairing relationship between the PC and the device. This has to be managed through the operating system. Desktop applications on Windows are not allowed to change the pairing relationship with devices directly. This is in contrast with Android, iOS, and Universal Windows Platform, but given that OpenVR is not compatible with UWP, we didn’t have a choice.
Once the device is paired with the PC, we can query the operating system for all attached devices. With GAP, we were able to set a custom name for our device, so it made identifying our own object in the list of hundreds of logical device connections quite easy. From there, the GATT interface carves the device up into “Services”, which are nothing more than named collections of values. Those values are called “Characteristics”, which are themselves nothing more than key-value pairs. There are large lists of standardized Services and Characteristics that the Bluetooth Consortium publishes, with the ability to also create your own custom Services and Characteristics.
However, along the way we discovered that the lack of significant work in this area means that most of the libraries available for both devices and PCs to work with Bluetooth do not work particularly well. We had to work around a number of difficult issues involving mismatched firmware versions and general instability in the data querying protocol. We finally settled on repurposing the standard Battery service, as we were originally planning on reporting the battery level of the device in real time. We used five Analog Characteristics to communicate the individual finger sensor values, and a single Analog Characteristic to bit-pack the state of the five motors in a single byte. This may not have been the most computationally efficient means of communicating the data, but it kept the protocol simple;, a necessity given the difficulties in getting the full system to work.
Results & Reflection
We debuted the Iron Man Gauntlet at this year’s MIT Tech Conference where hundreds of attendees were able to demo the device and didn’t encounter any problems with the functionality. We plan on making minor adjustments to the second iteration of the gauntlet such as making the glove tighter for a more form-fit experience, covering some of the wiring with a sleek clear encasing and making it more durable at the main stress points where the glove connects to the body of the gauntlet.
This post was originally posted via Medium: https://medium.com/@notiontheory/building-a-haptic-feedback-glove-for-virtual-reality-77232999a2d7#.sqmp57zev
Source: https://t-tees.com
Category: HOW
