Blind navigation with a wearable range camera and vibrotactile helment


We present a wayfinding system that uses a range camera and an array of vibrotactile elements we built into a helmet.

The range camera is a Kinect 3D sensor from Microsoft that is meant to be kept stationary, and used to watch the user (i.e. to detect the person's gestures). Rather than using the camera to look at the user, we reverse the situation, by putting the Kinect range camera on a helmet for being worn by the user. In our case, the Kinect is in motion rather than stationary.

Whereas stationary cameras have previously been used for gesture recognition, which the Kinect does very well, in our new modality, we take advantage of the Kinect's resilience against rapidly changing background scenery, where the background in our case is now in motion (i.e. a conventional wearable camera would be presented with a constantly changing background that is difficult to manage by mere background subtraction).

The goal of our project is collision avoidance for blind or visually impaired individuals, and for workers in harsh environments such as industrial environments with significant 3-dimensional obstacles, as well as use in low-light environments.

Here is the published paper, ACM Multimedia 2011