Stony Brook Department of Computer Science Professor Xiaojun Bi and his collaborators won Best Paper at ACM Symposium on User Interface Software and Technology (UIST 2019) a premier conference in the technical Human Computer Interaction community. The award-winning paper, entitled TipText: Eyes-Free Text Entry on a Fingertip Keyboard, covers their invention of this new text entry system. Only one percent of submissions to UIST 2019 were selected to receive Best Paper awards.

The team conducted the study in collaboration with the Human and Computer Interaction Lab at Dartmouth University. The new text entry technology they developed uses an invisible 2 x 3 grid QWERTY keyboard placed on the tip of the index finger, which allows for one-handed text input. Figure 1 depicts a sample keyboard.

The user types with the keyboard by tapping their thumb along the index finger as if it were a full-sized keyboard. The program recognizes the order the letters are selected and offers possible words. The user then selects the correct word by swiping right against their finger. Swiping left against the fingertip deletes the last letter typed. A space is automatically included after the word is selected.

Professor Bi and his colleagues developed three different prototypes for TipText: The first uses PET film, the second a printed flexible circuit (PFC) and the last was printed on temporary tattoo paper. They used the PFC prototype in their experiments, however, they believe the temporary tattoo paper would be better suited for future experiments or production. Figure 2 shows all three prototypes applied to a finger.

The team believes that being able to type inconspicuously without having to look at a keyboard will change the way we function. “Humans have a strong capability to type on a small area, paired with a statistical coding algorithm to reduce errors, it can provide high input performance,” said Professor Bi. TipText will sync to smartphones, smartwatches and other smart accessories, including those without screens. It will also include an audio component so that it can be used without a screen.

TipText is the first fingertip-keyboard in development. While it is currently only for research purposes, the potential for commercial use is undeniable. With further research and development, users may soon have a new world of communicating at their fingertips.

Professor Xiaojun Bi is an expert in Human Computer Interaction, Mobile Computing and theoretical issues of user interface design. He won Google Research Awards in 2017 and 2018 and was recognized with the ACM CHI Honorable Mention Awards in 2018, 2017, 2015 and 2011. Prior to joining Stony Brook, he was a Research Scientist at Google Inc. in Mountain View, California.

The ACM Symposium on User Interface Software and Technology is the premier forum for innovations in human-computer interfaces. The symposium is sponsored by various companies including the ACM, Google, Microsoft, Apple and Adobe. The symposium allows Individuals from various fields to discuss their research and ideas.

-- Aamna Atif