Publications

Advanced search

Abstract

Vicent Alabau, Daniel Ortiz-Martínez, Alberto Sanchis, Francisco Casacuberta. Multimodal Interactive Machine Translation. ICMI-MLMI '10: Proceedings of the 2010 International Conference on Multimodal Interfaces, 2010. DOI: 10.1145/1891903.1891960

Interactive machine translation (IMT) [1] is an alternative approach to machine translation, integrating human exper- tise into the automatic translation process. In this frame- work, a human iteratively interacts with a system until the output desired by the human is completely generated. Tra- ditionally, interaction has been performed using a keyboard and a mouse. However, the use of touchscreens has been popularised recently. Many touchscreen devices already ex- ist in the market, namely mobile phones, laptops and tablet computers like the iPad. In this work, we propose a new interaction modality to take advantage of such devices, for which online handwritten text seems a very natural way of input. Multimodality is formulated as an extension to the traditional IMT protocol where the user can amend errors by writing text with an electronic pen or a stylus on a touch- screen. Different approaches to modality fusion have been studied. In addition, these approaches have been assessed on the Xerox task. Finally, a thorough study of the errors committed by the online handwritten system will show fu- ture work directions.