Please use this identifier to cite or link to this item:
https://hdl.handle.net/2440/81013
Citations | ||
Scopus | Web of Science® | Altmetric |
---|---|---|
?
|
?
|
Type: | Journal article |
Title: | Improved part-of-speech prediction in suffix analysis |
Author: | Fruzangohar, M. Kroeger, T. Adelson, D. |
Citation: | PLoS One, 2013; 8(10):1-6 |
Publisher: | Public Library of Science |
Issue Date: | 2013 |
ISSN: | 1932-6203 1932-6203 |
Editor: | Bourdon, J. |
Statement of Responsibility: | Mario Fruzangohar, Trent A. Kroeger, David L. Adelson |
Abstract: | <h4>Motivation</h4>Predicting the part of speech (POS) tag of an unknown word in a sentence is a significant challenge. This is particularly difficult in biomedicine, where POS tags serve as an input to training sophisticated literature summarization techniques, such as those based on Hidden Markov Models (HMM). Different approaches have been taken to deal with the POS tagger challenge, but with one exception--the TnT POS tagger--previous publications on POS tagging have omitted details of the suffix analysis used for handling unknown words. The suffix of an English word is a strong predictor of a POS tag for that word. As a pre-requisite for an accurate HMM POS tagger for biomedical publications, we present an efficient suffix prediction method for integration into a POS tagger.<h4>Results</h4>We have implemented a fully functional HMM POS tagger using experimentally optimised suffix based prediction. Our simple suffix analysis method, significantly outperformed the probability interpolation based TnT method. We have also shown how important suffix analysis can be for probability estimation of a known word (in the training corpus) with an unseen POS tag; a common scenario with a small training corpus. We then integrated this simple method in our POS tagger and determined an optimised parameter set for both methods, which can help developers to optimise their current algorithm, based on our results. We also introduce the concept of counting methods in maximum likelihood estimation for the first time and show how counting methods can affect the prediction result. Finally, we describe how machine-learning techniques were applied to identify words, for which prediction of POS tags were always incorrect and propose a method to handle words of this type.<h4>Availability and implementation</h4>Java source code, binaries and setup instructions are freely available at http://genomes.sapac.edu.au/text_mining/pos_tagger.zip. |
Rights: | © 2013 Fruzangohar et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
DOI: | 10.1371/journal.pone.0076042 |
Published version: | http://dx.doi.org/10.1371/journal.pone.0076042 |
Appears in Collections: | Aurora harvest 4 Molecular and Biomedical Science publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
hdl_81013.pdf | Published version | 116.62 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.