Microsoft has announced that its popular app Seeing AI, which narrates the world for the visually impaired, is now available on Android devices. The app, which was previously exclusive to iOS, uses artificial intelligence to recognize and describe various objects, people, and scenes using the device’s camera and voice feedback. Seeing AI is a free app that aims to provide a more accessible and empowering experience for low-vision and blind users.
What is Seeing AI and What Can It Do?
Seeing AI is an app that uses the power of the cloud and AI to deliver an intelligent app designed to help low-vision and blind users navigate their day. The app allows users to point their phone’s camera at anything they want to know more about and hear a description of what the AI has recognized around them.
Seeing AI has multiple functions in one app, which can be switched between channels. Some of the channels include:
- Short Text: Speaks text as soon as it appears in front of the camera, such as signs, labels, or menus.
- Documents: Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting, such as headings, columns, or tables.
- Products: Gives audio beeps to help locate barcodes and then scans them to identify products, and hear the name and package information when available.
- Person: Recognizes friends and describes people around you, including their facial characteristics, approximate age, gender, and expression.
- Scene: An experimental feature that describes the scene around you, such as a park, a kitchen, or a street. You can also explore the photo by moving your finger over the screen to hear the location of different objects.
- World: An Audio Augmented Reality experience to explore an unfamiliar environment, such as hearing objects announced around you with Spatial Audio (requires a device with LiDAR, and Android 11+).
- Indoor Navigation: Available on the World Channel, allows you to define routes through a facility, like “entrance to the classroom”, and navigate by sound (A9 or later processor, Android 11+).
- Colors: Identifies colors, such as clothing or paint.
- Handwriting: Reads handwritten text, such as in greeting cards or notes (available in a subset of languages).
- Light: Generates an audible tone corresponding to the brightness in the surroundings, such as to check if the lights are on.
- Images in Other Apps: Allows you to describe images from other apps, such as Mail, Photos, Twitter, WhatsApp, and more, by tapping “Share” and “Recognize with Seeing AI”.
Benefits of Seeing AI for Low-vision and Blind Users
Seeing AI is a powerful app that provides a more accessible and empowering experience for low-vision and blind users. Some of the benefits of Seeing AI include:
- Recognition of friends and their facial expressions: Seeing AI can save people’s faces and names, so you can recognize them when you meet them again. You can also hear their approximate age, gender, and emotions, such as happy, sad, or surprised.
- Ability to read text quickly: Seeing AI can read any text that appears in front of the camera, such as signs, labels, menus, books, or newspapers. You can also scan documents and hear the text with its original formatting, such as headings, columns, or tables. You can also ask questions about the document’s contents, such as finding out the vegetarian options on a menu, or the cost of a particular item on a receipt.
- Independence and confidence: Seeing AI can help you with various daily tasks, such as identifying products, colors, currency, or light. You can also explore your surroundings with the Scene and World channels, and hear what’s around you, such as a park, a kitchen, or a street. You can also create and follow routes through a building with an Indoor Navigation feature, such as from the entrance to the classroom.
How Seeing AI is Expanding to Android?
Seeing AI was first launched in 2017 as a Microsoft research project for iOS devices. Since then, the app has been downloaded by over one million users and has received positive feedback and suggestions from the low-vision and blind community. Seeing AI has also won several awards, such as the AppleVis Golden Apple Award for Best Assistive App, and the Helen Keller Achievement Award from the American Foundation for the Blind.
Now, Microsoft is expanding Seeing AI to Android devices, making it available for anyone with an Android device running Android 6.0 or later. The app can be downloaded for free from the Google Play Store. The features and functions of Seeing AI remain the same on both platforms, and the app supports multiple languages, such as English, Spanish, French, German, Japanese, Italian, Turkish, and Dutch.
Seeing AI is a remarkable app that demonstrates how AI can be used to improve the lives of people with disabilities. By bringing Seeing AI to Android, Microsoft is making the app more accessible and inclusive for low-vision and blind users around the world. Seeing AI is an app that narrates the world for the visually impaired, and empowers them to achieve more.
