Approaches for temporal expression recognition in hindi

Abstract

Temporal annotation of plain text is considered as a useful component of modern information retrieval tasks. In this work, different approaches for identification and classification of temporal expressions in Hindi are developed and analyzed. Firstly, a rule based approach is developed, which takes plain text as input and based on a set of hand-crafted rules, produces a tagged output with identified temporal expressions. This approach is shown to have a strict F1-measure of 0.83. In another approach, a CRF based classifier is trained with human tagged data and is then tested on a test dataset. The trained classifier identifies the time expressions from plain text and further classifies them to various classes. This approach is shown to have a strict F1-measure of 0.78. Next, the CRF is replaced by a SVM based classifier and the same experiment is performed with the same features. This approach is shown to be comparable to the CRF and shows a strict F1-measure of 0.77. With three different comparable systems performing the extraction task, merging them to take advantage of their positives is the next step. As the first merge experiment, rule based tagged data is fed to the CRF and SVM classifiers as training data. Evaluation results report an increase in F1-measure of the CRF from 0.78 to 0.8. Secondly, a voting based approach is implemented, which chooses the best class for each token from the outputs of the above three approaches. This approach results in the best performance for this task with a strict F1-measure of 0.88. In this process a reusable gold standard dataset for temporal tagging in Hindi is also developed. Named the ILTIMEX2012 corpus, it consists of 300 manually tagged Hindi news documents.