CHI’22: ACM Conference on Human Factors in Computing Systems
Session: Brains, Senses and Virtual Reality
This paper discusses the Keyboard Augmentation Toolkit (KAT) which supports the creation of virtual keyboards that can be used both for standalone input (e.g. for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality.
In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display – 2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity – supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements e.g. enabling application-dependent interactions; and flexible layouts – allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g. mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.
Pre-recorded presentations of CHI 2022