Advanced search


Daniel Ortiz-Martínez, Ismael García-Varea, Francisco Casacuberta. A general framework to deal with the scaling problem in phrase-based statistical machine translation. Proceedings of the 3rd Iberian Conference on Pattern Recognition and Image Analysis, Volume 4477 of LNCS, 2007. pp. 314-322.

In this paper, we address the topic of how to estimate phrase-based models from very large corpora and apply them in statistical machine translation. The great number of sentence pairs contained in recent corpora like the well-known Europarl corpus have enormously increased the memory requirements to train phrase-based models and to apply them within a decoding process. We propose a general framework that deals with this problem without introducing significant time overhead by means of the combination of different scaling techniques. This new framework is based on the use of counts instead of probabilities, and on the concept of cache memory.