Advanced search


Álvaro Peris, Francisco Casacuberta. A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, 2019. pp. 81-86. ACL.

We present a demonstration of a neuralinteractive-predictive system for tackling mul-timodal sequence to sequence tasks. The sys-tem generates text predictions to different se-quence to sequence tasks: machine translation,image and video captioning. These predictionsare revised by a human agent, who introducescorrections in the form of characters. The sys-tem reacts to each correction, providing alter-native hypotheses, compelling with the feed-back provided by the user. The final objectiveis to reduce the human effort required duringthis correction process.This system is implemented following aclient–server architecture. For accessing thesystem, we developed a website, which com-municates with the neural model, hosted ina local server. From this website, the dif-ferent tasks can be tackled following theinteractive-predictive framework. We open-source all the code developed for buildingthis system. The demonstration in hostedin