M Tech Dissertations
Permanent URI for this collectionhttp://drsr.daiict.ac.in/handle/123456789/3
Browse
2 results
Search Results
Item Open Access Part of speech tagging for Gujarati text(Dhirubhai Ambani Institute of Information and Communication Technology, 2011) Dave, Mainak; Pandya, AbhinayPart-of-speech (POS) tagging is a process of assigning the lexicon category to each lexicons in a given natural language sentence, that best suits the definition of the lexicon as well as the context of the sentence in which it is used. Part-of-speech tagging is an important part of Natural Language Processing (NLP) and is useful for most NLP applications. Part-of-speech tagging is often a primary step in most of the NLP tasks such as chunking, parsing, etc. Gujarati is the state language of Gujarat, a western state of India, and is spoken by 70 percent of the state's population. More than 46 million people worldwide consider Gujarati as their first language. Apart from Gujarat, it is widely spoken in the states of Maharashtra, Rajasthan, Karnataka and Madhya Pradesh and also around the world. Natural language processing of Gujarati is in its early stage of existence. Gujarati POS tagger is a core component for most NLP applications. Information retrieval, machine translation, shallow parsing and word sense disambiguation tasks can be work more effectively and efficiently with the existence of a POS tagger. Our focus in this work is to develop an effective Gujarati text POS tagger. Our main task of thesis is to built a system which can annotate part-of-speech for Gujarati texts automatically, with the help of various machine learning algorithms. We have used tag sets defined by IIIT Hyderabad. We have used two machine learning techniques one is Hidden Markov Model (HMM) and second is Conditional random Field (CRF). Since Gujarati is a morphologically reach language, we can use Morphological Analyzer (MA) to restrict the set of possible tags for a given words. Gujarati language is based on Paninian framework, rules of morphology are well-defined. Hence we have defined morphological rules for Gujarati. While MA helps us to restrict the possible choice of tags for a given word, one can also use prefix/suffix information (i.e., the sequence of first/last few characters of a word) to further improve the models. HMM model uses suffix information during smoothing process while CRF uses suffixes as a feature. (http://www.oclc.org/languagesets/educational/languages/india.htm)Item Open Access Shallow parsing of Gujarati text(Dhirubhai Ambani Institute of Information and Communication Technology, 2011) Dave, Vidhi; Pandya, AbhinayShallow parsing is the process of assigning tag to minimal, non recursive phrase of the sentence. It is useful for many applications like question answering system, information retrieval where there is no need of full parsing. Gujarati is one of the main languages of India and 26th most spoken native language in the world. There are more than 50 million speakers of Gujarati language worldwide. Natural language processing of Gujarati is in its infancy. Now days there are many data available in Gujarati on websites but due to lack of resources it is hard for users to retrieve it efficiently. So, shallow parsing of Gujarati can make task easier for another tasks like machine translation, information extraction and retrieval. In this thesis, we have worked on the automatic annotation of Shallow Parsing of Gujarati. 400 sentences have been manually tagged. Different Machine Learning techniques namely Hidden Markov Model and Conditional Random Field have been used. We achieved good accuracy and it is similar to Hindi chunker even though resources available for Gujarati are very less. The best performance is achieved using CRF with contextual information and Part-of-speech tags.