FingerSight: Fingertip Control and Haptic Sensing of the Visual Environment
Theme: Future History
|FingerSight is the underlying concept for a visual sensing device with haptic feedback that allows users to manipulate remote objects by gesture. The current model utilizes a small finger-mounted video camera to track graphical controls on a computer screen. Vibro-tactile feedback alerts users when they have locked onto a control.|
Because visual information is relayed through the sense of touch, FingerSight has potential application for the visually impaired. Haptic feedback allows users to receive additional information about their surroundings without interfering with auditory cues.
The basic goal is to create a novel application of machine vision and haptic feedback to enhance sensing and remote control of the environment by blind and sighted individuals. Technical goals include recognizing gestures over arbitrary visual fields, simultaneously maximizing information bandwidth and minimizing intrusiveness of haptic feedback, and incorporating multiple cameras.
FingerSight allows users to sense and manipulate completely virtual objects with a single finger. The system can be extended to multiple cameras, one per finger, each with different specifications, allowing synergistic merger of information. This supports capabilities such as 3D reconstruction, color coding, and simultaneous wide-range and focal views.
FingerSight could empower the low-vision population to interact with the distal environment in a new way. Sighted users can manipulate objects beyond their reach, so, for example, they can safely interact with remote targets in sterile or hazardous environments. Educators and gamers can use the technology in training simulations and virtual-reality systems.
Carnegie Mellon University
University of Pittsburgh