MultiPro: Prototyping Multimodal UI with Anthropomorphic Agents

Abstract

Modern user interfaces (UI) often provide natural and multimodal interaction, sometimes modelled in the form of a conversation with an anthropomorphic agent embedded in the system. Designing such interfac-es is a challenging task. Furthermore, it is often not clear which user scenarios will profit from a social, conversational interaction paradigm and how it could be integrated with classic paradigms like direct manipulation interfaces. Our multimodal prototyping framework MultiPro helps designers in rapidly designing UIs to explore these questions. It allows the easy prototyping and evaluation of UI using a mix of anthropomorphic (e.g., human-like appearance, speech) and classic graphical elements. We illustrate how designers can specify the look and interaction flow with MultiPro, how MultiPro supports user-based evaluation of the designs, and how MultiPro is used in HCI research applications.