Seeing AI

Country
United States
Sustainable development goals
  • Good health and well-being
  • Quality education
Project link
https://itunes.apple.com/us/app/seeing-ai/id999062298?mt=8

About the project

Seeing AI is a free app which uses artificial intelligence to narrates any environment through smartphone cameras and microphones. The app has been designed for the visually impaired community by a Microsoft Deep Vision hackathon team.

More project information

Seeing AI is a free app which uses artificial intelligence to narrates any environment through smartphone cameras and microphones. The app has been designed for the visually impaired community by a Microsoft Deep Vision hackathon team. The research team which included several visually impaired members, such as Saqib Shaikh who was a key developer of the app, wanted to tackle the issue of lack of available technology to the 253 million people around the world which live with either partial or total blindness. The team’s executive summary of the app indicate that the software integrates use of the built-in camera to recognize people, specific items, or a general environment around the user. Thanks to image recognition and artificial intelligence, users can aim their phones at almost anything and receive a vocal announcement of what it is. Facial-recognition technology enables the app to

remember and recognize friends’ faces and even perform some emotion recognition to determine the person’s mood. It can also read printed text in books or newspapers aloud. Optimized for use with VoiceOver, the app enables its user to recognize: • Short Text - Speaks text as soon as it appears in front of the camera. • Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting. • Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available. (iPhone 6 and later). • People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and emotions. • Scenes (early preview) - Hear an overall description of the scene captured. • Currency - Recognizes currency notes. (iOS 11). • Color - Identifies color. • Handwriting - Reads handwritten text like in greeting cards. • Light - Generates an audible tone corresponding to the brightness in the surroundings. • Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more. • Photo browsing experience - Describes photos on your phone. To date, the Seeing AI app has given 150,000 visually impaired people around the world the ability to gain new insights into their surroundings. The project also recently received several awards, including the prestigious Helen Keller Achievement Award from the American Foundation for the Blind. With its groundbreaking technology, the project contributes to the SDG 3 (Good Health and Well-Being) as well as the SDG 4 (Quality Education) as it allows the visually impaired community to gain unprecedented independence in pursuing an education.