Abstract: Finger touch has become one of the main input modalities, as touchscreen devices such as smartphones and tablets are widely adopted. Touch interaction is often intuitive and well-suited in mobile scenarios as it does not require carrying extra input tools. Among various touch input methods, word-gesture input, also known as gesture typing or ShapeWriting, is a touch gestural text entry method that has been widely adopted on touchscreen mobile devices. It allows users to enter a word by tracing letters, instead of tapping them. As users gain expertise, they may gradually transit to directly recalling the gesture without relying on the visual guidance provided by the keyboard.
Our research expands the word-gesture input beyond regular text entry. We extend it to support command input and text editing. In the first part of our research, we explored the possibility of combining gesture input on the trackpads of laptops with hotkeys. We created HotStrokes which support efficient command input with finger gestures. A user enters a HotStroke by holding a modifier key, then gesture typing a command name on a laptop trackpad as if on an imaginary virtual keyboard. A controlled experiment shows HotStrokes effectively augments the existing menu and hotkey command activation paradigm. Results show HotStrokes is efficient by reducing command activation time by 43% compared to linear menus. The method is also easy to learn with a high adoption rate, replacing 91% of linear menu usage. In the second part, we explored how to use gesture input to correct errors in text entry. We present JustCorrect, an intelligent post hoc text correction technique for smartphones. To make a correction, the user simply types the correct text at the end of their current input, and JustCorrect will automatically detect the error and apply the correction in the form of an insertion or a substitution. In this way, manual navigation steps are bypassed, and the correction can be committed with a single tap. In the third part, we explored the possibility of implementing gesture input in back-of-device interaction. We create a back-of-device command and text input technique called BackSwipe, which allows a user to hold a smartphone with one hand, and use the index finger of the same hand to draw a word-gesture anywhere at the back of the smartphone to enter commands and text. Our user study shows BackSwipe is feasible and a promising input method, especially for command input in the one-hand holding posture. It complements the extant smartphone interaction by leveraging the back of the device as a gestural input surface. Overall, HotStrokes, JustCorrect and BackSwipe extend word-gesture input to support a wide range of applications such as command input and text editing.


Contact events [at] cs.stonybrook.edu for Zoom info.