Mobile Applications Aiding the Blind

Main researcher: Piotr Skulimowski, Ph.D.

The "Assistant" application offers an approach that is especially dedicated to the blind and visually impaired. The target platforms of the suite are the Symbian Series 60 phones. Menu structure and applications were designed from the very beginning specifically for the blind. Interaction with the user is performed with the help of a speech synthesizer, substituted in the case of common messages by speaker voice recordings. Graphical and textual interface, which use only large, distinguishable, high contrast characters, are supplementary, aiding users who retained partial sight. Main application functions: standard phone functions, with a phonebook, text and media messaging, alarm clock and scheduler, tools: voice recorder, notepad, calculator, stopper, color and light source recognition, GPS localization services

For the Android platform a method for recognition of objects in scenes recorded by a built-in mobile phone camera is proposed. This application was written using the Scale-Invariant Feature Transform, which was specially modified for processing low resolution images. Other developed indoor navigation system consists of a set of embedded display devices, each showing a sequence of information messages encoded as two-dimensional barcodes (using a modified Data Matrix standard) and an Android smart-phone with an integrated video camera that detects, decodes and reads relayed messages using voice synthesis.

Main menu of the "Assistant" software

Color detection module

Entering SMS message using the Assistant software

Object recognition application runs on a HTC Smartphone. The application software successfully recognizes the object (apricot yogurt) and displays the reference image.

An element of the local navigation system mounted near the door of the room