Sound gives ‘sight’ to blind

For decades, scientists have tried to find a way to let the blind see, such as implanting artificial retinas in the eyes, with poor results. Part of the problem with these techniques is that surgery on the eye is necessary.

Now an odd approach, seeing with sound but not by sonar, looks as if it may actually work. Sort of, anyway, which if you are blind is a lot better than nothing.

Called the vOICe (the middle vowels stand for “Oh, I see”), it was developed by Dr. Peter Meijer of Philips Research Laboratories in the Netherlands. The idea is that the user wears a camera on his head, which is connected to a computer, which is connected to stereo headphones. Given the small size of modern video cameras, and of computers, none of this need be particularly clumsy or conspicuous.

You can try it on the Web (www.seeingwithsound.com/voice.htm). After I played with sample “images” in sound for a few minutes on the Web site, it was beginning to work. For simple scenes, I began to correlate the sound of the scan with the picture being scanned.

At least for simple things like reading a graph, it clearly has potential.

How does it work? It’s a little tricky but makes sense.

To begin, you “look” at a scene, meaning that you point the camera at it. The camera generates a picture of the scene, which, being blind, you can’t see.

The software in the computer automatically scans the picture from left to right and turns it into sound that you hear in your headphones.

How do you turn a picture into sound? The system uses three rules.

First, the left-to-right scanning is mirrored in the headphones: When the scanning software is scanning the left side of the scene, the sound generated seems to come from the left and, as the scan sweeps across the scene, the apparent origin follows it.

Second, “bright” means “loud.” When the scan encounters a brightly lighted white door, for example, the sound becomes louder; when the scan passes the door and begins scanning the dark wall next to the door, the volume drops.

Third, pitch means elevation. That is, a bright object near the top of the scene will translate into a loud, high-pitched sound; the same object near the bottom of the scene would be a loud, low-pitched sound.

This means, for example, that a bright line running from the lower left of a dark wall to the upper right sounds like a steadily rising tone that moves from left to right in the stereo headphones.

It is more sophisticated than just this. You can ask it to tell you, in English, what the colors are in the scene. All sorts of things can be adjusted to taste — scan rate, etc.

Story Continues →