Haptic System Creates Finger-Touch Sensations Hardware-Free​

One of the trippiest virtual-reality experiences that I’ve ever had was being able to touch an object that didn’t exist. Getting immersed in a virtual world through a headset that provides video and audio is one thing, but adding in touch is somehow transformative; physical interaction brings you into that virtual world in a much more direct way. But as with everything related to VR or AR, it’s all extremely hardware intensive. And especially in the case of augmented reality, all of that hardware can get in the way.

This hardware layer is a problem for virtual touch, because it seems like there’s just no way around it. Video and audio have ways of making themselves transparent, like with glasses or contact lenses or bone conduction, which let you experience the real world with an additional layer of reality added on top—augmented reality. With touch, though, there aren’t good options for transparency, because there has to be some physical thing touching, say, your fingers for you to actually feel sensations. Typically, this is some sort of glove, and whether or not the glove is transmitting a haptic signal, you’re still wearing it all the time, which is not transparent at all.

Researchers from the University of Chicago have figured out a clever way around this problem. In a paper that will be presented at the 2023 Conference on Human Factors in Computing Systems, or CHI 2023, (where it will also be honored with a best paper award), they demonstrate a wearable system that can generate tactile sensations across the bottom of your fingers and palm without any hardware getting in the way, by hacking into the nerves on the back of your hand instead.

This method depends on what’s called “referred sensation,” where stimulation of your body in one place gets felt in another place—it’s like when you accidentally bash your elbow against something but can feel tingling up through your fingers, because the signal has traveled along your nerves up through your hand. In some places, including in the fingers, referred sensation can be targeted with a reasonable amount of precision. With a signal electrode on the top of a finger and a ground electrode closer to the wrist, it’s possible to stimulate individual parts of each finger, creating 11 separately controllable tactile zones across five fingers and the palm.

However, just zapping fingers doesn’t provide a way of getting those signals to show up just on the bottom (palmar) side of the fingers, rather than the top (dorsal) side, which is where the signal is originating. Fortunately, the asymmetrical way that the nerves in our hands are set up makes this possible. The backhanded stimulation technique works because your palm and finger pads are way, way more sensitive than the backs of your hands, thanks to about 60 times more mechanoreceptors on the palmar side. So, if you use an electrode to stimulate the back of one of your fingers, the sensitivity on the front is so much higher that you’re going to feel it there much more strongly, even though the electrode is in direct contact with the back side. The researchers were able to find a stimulation intensity that was enough to trigger nerves on the finger pads, while staying below the detection threshold on the back side of the fingers, neatly solving the problem.

We asked Pedro Lopes, who runs the Human Computer Integration Lab at the University of Chicago, to describe what it feels like to use this system:

Electrotactile stimulation does not feel exactly like a real touch. But on contact, it does feel like a tap on my skin, which is pretty realistic. Touching the bouldering wall you see in the video, you feel your hand contacting the wall where you’d expect. Surprisingly, you feel it entirely in different points of your palm, not where the electrodes are—in VR, as I see my hand touching the wall, my brain expects the sensations in multiple points in the front, and that’s exactly what I feel. It feels good on contact, and the moment I first grab the hold, it feels right. There’s still some improvement to do to feel constant pressure, and if I keep grasping the hold for a minute, I start to notice this isn’t the same as a real hold.

In a pair of user studies, most participants reported feeling over 90 percent of tactile sensation on the palmar side of their hand, despite the electrodes being mounted on the dorsal side. And the location targeting was fairly robust, too. In mixed-reality manipulation tasks that included working with clay and virtual rock climbing, participants commented that the “the feeling itself was warm and tingly” and “I felt exactly what I expected to feel.” Arguably, the most important thing is that this system allows you to use your hands for other things at the same time, without impacting the VR/AR experience. This can be as simple as being able to use a keyboard or adjust your VR headset, or as complex as DJing, as Lopes explains:

While you see a lot of VR in the video, with this new level of hands-free haptics, we think we can unleash new use cases for haptics that go beyond VR/AR. For instance, we think haptics could be used in learning applications (while playing piano or drums, for example) or even while engaged in complex interactions. For instance, in the video you can see me DJing and being helped by a digital DJ system—while I am grabbing the faders and turntable I can feel tactile notifications that tell me when to release the record to achieve a better mix. No DJ would wear gloves just to enjoy this extra feedback, but this way, we think the trade-off is much more balanced.

There’s still a lot that can be done to improve this technology. Lopes says that in addition to being able to feel contacts, they already have the system set up to produce the feeling of textures, which is accomplished by sending continuous waveforms (rather than short pulses) to the electrodes.

Full-hand Electro-Tactile Feedback Without Obstructing Palmar Side of Hand, by Yudai Tanaka, Alan Shen, Andy Kong, and Pedro Lopes from the University of Chicago, will be presented at CHI 2023, in Hamburg.