Yayınlanmış 1 Ocak 2018 | Sürüm v1
Konferans bildirisi Açık

Turkish PoS Tagging by Reducing Sparsity with Morpheme Tags in Small Datasets

  • 1. Hacettepe Univ, Dept Comp Engn, TR-06800 Ankara, Turkey
  • 2. Middle East Tech Univ ODTU, Cognit Sci Dept, Informat Inst, TR-06800 Ankara, Turkey

Açıklama

Sparsity is one of the major problems in natural language processing. The problem becomes even more severe in agglutinating languages that are highly prone to be inflected. We deal with sparsity in Turkish by adopting morphological features for part-of-speech tagging. We learn inflectional and derivational morpheme tags in Turkish by using conditional random fields (CRF) and we employ the morpheme tags in part-of-speech (PoS) tagging by using hidden Markov models (HMMs) to mitigate sparsity. Results show that using morpheme tags in PoS tagging helps alleviate the sparsity in emission probabilities. Our model outperforms other hidden Markov model based PoS tagging models for small training datasets in Turkish. We obtain an accuracy of 94.1% in morpheme tagging and 89.2% in PoS tagging on a 5K training dataset.

Dosyalar

bib-6661ff54-5fc6-44f6-8216-729fce8d1893.txt

Dosyalar (199 Bytes)

Ad Boyut Hepisini indir
md5:feb9e8cafbee71be1d4ffb6c78f750c3
199 Bytes Ön İzleme İndir