Perceptual Interfaces

Report ID: 
Matthew Turk and Mathias Kolsch
2003-10-01 05:00:00


In recent years, perceptual interfaces have emerged as anincreasingly important research direction. The general focus of thisarea is to integrate multiple perceptual modalities (such as computervision, speech and sound processing, and haptic I/O) into the userinterface. Broadly defined, perceptual interfaces are highlyinteractive, multimodal interfaces that enable rich, natural, andefficient interaction with computers. More specifically, perceptualinterfaces seek to leverage sensing (input) and rendering (output)technologies in order to provide interactions not feasible withstandard interfaces and the common triumvirate of I/O devices: thekeyboard, mouse and monitor.In this report, we seek to communicate the motivations and goals ofperceptual interfaces, to enumerate the relevant technologies, todiscuss the integration of multiple modalities, and to describe inmore detail the role of computer vision in human-computerinteraction. We cover vision problems, constraints, and approachesthat are apropos to the area, survey the state of the art in computervision research and multi-modal interfaces, and take a look at otherperceptual technologies such as brain-computer interfaces. We focuson their application to perceptual interfaces, describe severalnear-term applications, and suggest promising research directions.