A team of Stony Brook University Department of Computer Science faculty have developed a new technology to help blind people complete printed forms independently. The Write-it-Yourself guide (WiYG) system uses a custom, compact, 3D-printed smartphone attachment, smartphone app and a signature guide to direct the user to different fields on a form.

For the blind, assistive technology (AT) allows independence for a variety of activities. Most of these technologies focus on reading assistance. But for a non-digital, paper-based task such as writing information on forms, bank checks or contracts, blind people still need the assistance of a sighted person. WiYG automatically guides the user to the appropriate form fields with voice instructions. Watch the video below for a demonstration.

“WiYG is a breakthrough assistive technology,” said Fotis Sotiropoulos, Dean, College of Engineering and Applied Sciences. “The team’s human-centered computing focus provides an independent, portable solution for a common challenge faced by the blind community.”

The WiYG team performed a study involving 13 subjects ranging in age from 26 to 65 years old. All were able to write on paper and were experienced with smartphones. After an initial practice time, they were given 10 minutes to use WiYG to complete four unique documents including a letter-size consent form, two different bank checks and a restaurant receipt. Accuracy and overlap were measured human assessors, with one form achieving 89.5% accuracy. At the end of the study, all 13 subjects stated that they would use WiYG in their everyday life, and the majority affirming it would help them complete related tasks quickly.

The illustration above demonstrates how a blind user will fill out a paper form with WiYG: The user (a) attaches the 3D-printed apparatus to the phone and places it in front of the paper; (b) aligns a signature guide to the top-left corner of the paper for calibration; (c) taps the screen to start receiving voice instructions for moving the signature guide to the first form-field; (d) follows the instructions and moves the signature guide; (e) writes the information requested by the application, and then taps the screen to start receiving instructions for the next form-field. This process continues until the paper form is filled out completely.

“A guided interface like WiYG is potentially life-changing for blind users,” said Samir Das, Stony Brook’s Chair of the Department of Computer Science. “This technology will empower them to overcome barriers and open new educational and employment opportunities.”

Upcoming enhancements to the WiYG system will focus on making the system applicable in a number of scenarios such as filling out forms that are clipped to a clipboard and integrating reading capability in WiYG.

Research for the WiYG project was supported in part by NSF: 1806076, NEI/NIH: R01EY02662, and NIDILRR: 90IF0117-01-00.

The research team included IV Ramakrishnan, Professor in the Department of Computer Science and Associate Dean of Research in the College of Engineering and Applied Sciences, with support from a team of graduate students and faculty, including: Vikas Ashok, PhD, Syed Masum Billah, PhD, Shirin Feiz, MS, and Roy Shilkrot, PhD and Assistant Professor of Computer Science.