Smartphone app helps people with blindness navigate their surroundings

October 19, 2015 

PITTSBURGH ó Previously, when Chieko Asakawa navigated her way across the Carnegie Mellon University campus, she used her white cane to identify obstructions with her ears alert to recognizable sounds and intuition in full power to keep track of her location.

She is blind. So the IBM fellow from Japan and visiting faculty member at the universityís Robotics Institute would count steps and use known landmarks to document her progress. But if someone stopped to chat or bumped into her, she lost her step count and became disoriented.

"You have to always concentrate," she said. "For us, you are always thinking how many steps to the next turn and where you are. You canít walk while thinking about other things. Itís very easy to get lost if you are not careful."

But this week, Asakawa walked with her long, white cane through Newell-Simon Hall, including an elevator ride, to an ATM machine in Ween Hall, with assistance from the NavCog app downloaded onto her smartphone. Like a GPS system it provided a voice that described distances, what directions she was to turn and when she arrived at landmarks and the ATM machine.

NavCog, which she developed with help from Kris Kitani at Carnegie Mellonís Robotics Institute and IBM, now is available for free at the App Store downloads on iTunes. It helps people with visual impairment to find their way indoors and out. While still in pilot phase, the app and its open-source technology allows others to advance the technology and expand areas where it can be used.

NavCog uses Bluetooth sensors placed at turns, intersections and important destinations (elevators, entrances or ATM machines) to provide oneís position within 3 to 5 feet of actual location. Using a map editing tool and localization algorithms, it can identify your location almost in real time, which direction you are facing and additional information about your surroundings.

It allows Asakawa, who holds a Ph.D. in engineering, to walk unassisted through various buildings and sections of the campus based on signals from 200 battery-powered sensors. Thereís still the occasional bump, brush-by or problem determining whether the elevator is going up or down, but the campus system shows its potential.

"While visually impaired people like myself have become independent online, we are still challenged in the real world," she said, adding that the app can accelerate research and serve as the basis for more useful and expanded navigational system for people with blindness.

Already she and Kitani, a doctor of robotics, are testing a small camera hung over her ear and using emotion-recognition software to describe the emotional state of people approaching her. In time, computer-vision technology will compare previously downloaded images of an area with ones generated in real time by the smartphone to identify oneís location more accurately without sensors.

Elaine R. Welch, president and CEO of the Pennsylvania Association for the Blind, said people with visual impairment already use GPS technology to provide their general location outside. GPS cannot be used indoors. Such technology has potential to improve the quality of life of people with blindness, with self-driving cars representing the Holy Grail.

"Iíve been around since 1982, and I see what technology has done for people to achieve independence. Itís just been phenomenal and this is great for people with disabilities," she said. "I really applaud (the Carnegie Mellon-IBM team) for doing this. Anything that can help people to be independent and more sure of themselves, whether they are blind or sighted people in unfamiliar places, is great."



McClatchy-Tribune Information Services