Handling Unknown Words in English to Nepali Statistical Machine Translation Using Analogical Learning Approach

Abstract

Unknown words are one of the great difficulties in the field of machine translation. In the process of translation, a system is most likely to encounter words that were not present in the available training data. While this is in part due to the segmentation issues, it is also often simply due to the lack of training data. In statistical machine translation to translate a sentence from one language to another we make use of a parallel corpus but it is not possible to a corpus to contain all the words from a whole language domain, hence the unknown word problem is obvious. In this thesis work an effective approach is used to translate those unknown words in English to Nepali statistical machine translation using word analogy. In this method the meaning of the unknown word is identified on the basis of other words presented in the corpus and the analogy between the prefixes and suffixes of those words with the unknown word.