New App Allows Vision-Impaired Guests to Navigate PIT

Airport served as testing ground for NavCog, developed by researchers at Carnegie Mellon University

By Matt Neistein

Published May 6, 2019

Read Time: 4 mins


Airports can be confusing places. Loud, sprawling and ever-changing, even the most confident people sometimes pause and look around to get their bearings.

Now imagine standing in that terminal and being blind.

A group of researchers from Carnegie Mellon University did just that while developing a smartphone app to help vision-impaired people navigate indoor spaces. In fact, one of them didn’t have to imagine at all.

Chieko Asakawa lost her vision as a child.

“She has been working in accessibility for over 30 years at IBM Research,” said fellow researcher Kris Kitani, an assistant research professor at CMU’s Robotics Institute. The two met when Asakawa brought her accessibility work to CMU, where Kitani was researching computer vision, using video and images to understand the world.

“We said it would be great if we could use computer vision and other machine-learning techniques to help with accessibility,” Kitani recalled.

The two joined researchers from IBM to help create the NavCog app, which relies on a network of Bluetooth beacons strategically placed inside a building to guide visually impaired travelers. As part of the project, the Allegheny County Airport Authority installed hundreds of the beacons throughout Pittsburgh International Airport.

A vision-impaired person tests out the BBeep “smart suitcase” at Pittsburgh International Airport. (Photo courtesy of Carnegie Mellon University)

“We are happy to serve as a living lab for CMU faculty and students doing meaningful research to improve the customer experience,” said Samantha Stedford, senior analyst for strategic initiatives at PIT. “We want Pittsburgh International to be industry-leading and our partnership with Carnegie Mellon allows us to pioneer new technologies and solutions.”

The app gives audio directions to the user, relying on a detailed map of the airport that includes restrooms, restaurants, gates, ticketing counters and more. Along with the airport, researchers have also tested the system at a shopping mall in Japan and a hospital in Pittsburgh.

“Each (building) has different elements to it. I think the airport really helped us to understand what it means to localize somebody in a very, very large scenario because the airport is so big,” Kitani said. “It’s also great as a testing ground because people who go to the airport – the majority of people – are there only once or twice, a few times, so it’s a place where people get lost often.”

Ten legally blind users tested the app with their iPhones. The results were encouraging, researchers said, as they were able to navigate escalators, moving walkways and the airport’s wide-open spaces without much difficulty.

The ultimate goal is not just to get blind people from ticketing counters to gates, Kitani said, but to allow them to explore the airport and find amenities just as easily as sighted people do.

“We’re hoping that places like the airport find value from a system like this to get people from A to B, but maybe the shops inside the airport can benefit from this by directing customers to their shops. There could be customer statistics, customer satisfaction technologies that could be built on top of it,” Kitani said. “We hope to move into many, many other sectors with a vision to just have NavCog everywhere so that a blind person can get around.”

One of the attributes that sets NavCog apart from similar technologies is the degree of accuracy, he said. The app estimates users’ locations to within a meter to a meter-and-a-half.

“We needed to hit that level of accuracy because, to a blind person, 1 to 1.5 meters of error is the difference between a wall and a door,” Kitani said.

NavCog is currently available in the App Store. What’s next? Likely a company spun off from CMU to develop the app commercially, he said.

“The airport played a really big role in pushing us in that direction,” Kitani said. To use the app, airport officials said it needed a company structure with support and maintenance.

“So it’s good that we worked together because it kind of pushed us into this spinout situation,” he said.

Stedford said NavCog not only helps improve the passenger experience but breaks new ground for the aviation industry.

“Making sure the airport is inclusive and accessible to all travelers is important to us, especially as we build a new terminal,” Stedford said. “This technology is indicative of the kinds of innovations that will result from our collaboration with CMU.”

Asakawa is presenting the results of the airport study at a conference in Scotland this week, along with a report on a related invention that was also tested at PIT: a “smart suitcase” called BBeep.

When blind people roll the suitcase ahead of themselves, it beeps to alert them of imminent impacts with objects or other pedestrians using a built-in camera and speaker. The noises can also alert sighted people in the area to be aware of a blind person’s path.

Go to Top