Publications

Advanced search

Abstract

Carlos D. Martínez-Hinarejos, Josep Lladós, Alicia Fornés, Francisco Casacuberta, Lluis Heras, Joan Mas, Moisés Pastor, Oriol Ramos-Terrades, Joan-Andreu Sánchez, Enrique Vidal, Fernando Vilariño. Context, multimodality, and user collaboration in handwritten text processing: the CoMUN-HaT project. Proceedings of IberSpeech 2016, 2016. pp. 375-383.

Processing of handwritten documents is a task that is of wide interest for many purposes, such as those related to preserve cultural heritage. Handwritten text recognition techniques have been successfully applied during the last decade to obtain transcriptions of handwritten documents, and keyword spotting techniques have been applied for searching specific terms in image collections of handwritten documents. However, results on tran- scription and indexing are far from perfect. In this framework, the use of new data sources arises as a new paradigm that will allow for a better transcription and indexing of handwritten documents. Three main different data sources could be considered: context of the document (style, writer, historical time, topics,. . . ), multimodal data (representations of the document in a different modality, such as the speech signal of the dictation of the text), and user feedback (corrections, amendments,. . . ). The CoMUN-HaT project aims at the integration of these different data sources into the transcription and indexing task for handwritten documents: the use of context derived from the analysis of the documents, how multimodality can aid the recognition process to obtain more accurate transcriptions (including transcription in a modern version of the language), and integration into a user- in-the-loop assisted text transcription framework. This will be reflected in the construction of a transcription and indexing platform that can be used by both professional and non-professional users, contributing to crowd-sourcing activities to preserve cultural heritage and to obtain an accessible version of the involved corpus.