Published January 1, 2018 | Version v1
Conference paper Open

Turkish PoS Tagging by Reducing Sparsity with Morpheme Tags in Small Datasets

  • 1. Hacettepe Univ, Dept Comp Engn, TR-06800 Ankara, Turkey
  • 2. Middle East Tech Univ ODTU, Cognit Sci Dept, Informat Inst, TR-06800 Ankara, Turkey

Description

Sparsity is one of the major problems in natural language processing. The problem becomes even more severe in agglutinating languages that are highly prone to be inflected. We deal with sparsity in Turkish by adopting morphological features for part-of-speech tagging. We learn inflectional and derivational morpheme tags in Turkish by using conditional random fields (CRF) and we employ the morpheme tags in part-of-speech (PoS) tagging by using hidden Markov models (HMMs) to mitigate sparsity. Results show that using morpheme tags in PoS tagging helps alleviate the sparsity in emission probabilities. Our model outperforms other hidden Markov model based PoS tagging models for small training datasets in Turkish. We obtain an accuracy of 94.1% in morpheme tagging and 89.2% in PoS tagging on a 5K training dataset.

Files

bib-6661ff54-5fc6-44f6-8216-729fce8d1893.txt

Files (199 Bytes)

Name Size Download all
md5:feb9e8cafbee71be1d4ffb6c78f750c3
199 Bytes Preview Download