Automatic Learning Assistant in T elugu (2021)

Abstract: This paper presents a learning assistant that tests one’s knowledge and gives feedback that helps a person learn at a faster pace. A learning assistant (based on automated question generation) has extensive uses in education, information websites, self-assessment, FAQs, testing ML agents, research, etc. Multiple researchers, and companies have worked on Virtual Assistance, but majorly in English. We built our learning assistant for Telugu language to help with teaching in the mother tongue, which is the most efficient way of learning. Our system is built primarily based on Question Generation in Telugu. Many experiments were conducted on Question Generation in English in multiple ways. We have built the first hybrid machine learning and rule-based solution in Telugu, which proves efficient for short stories or short passages in children’s books. Our work covers the fundamental question forms with question types: adjective, yes/no, adverb, verb, when, where, whose, quotative, and quantitative (how many/how much). We constructed rules for question generation using Part of Speech (POS) tags and Universal Dependency (UD) tags along with linguistic information of the surrounding relevant context of the word. We used keyword matching, multilingual sentence embedding to evaluate the answer. Our system is primarily built on question generation in Telugu, and is also capable of evaluating the user’s answers to the generated questions.

## introduction research on virtual assistants is renowned since they being widely used in recent times for numerous tasks. these assistants are generated using large datasets and high-end natural language understanding (nlu) and natural language generation (nlg) tools. nlu and nlg are used in 1 (roshni, 2020) (nishanthi, 2020) interactive nlp applications such as ai-based dialogue systems/voice assistants like siri, google assistant, alexa, and similar personal assistants. research is still going on to make these assistants work in major indian languages as well. an automated learning assistant like our system is not only useful for the learning process for humans but also for machines in the process of testing ml systems foot_0 . research has been done for question answer generating system in english foot_1 , concentrating on basic wh-questions with a rule-based approach foot_2 , question template based approaches foot_3 etc. for a low-resourced language like telugu, a complete ai-based solution can be non-viable. there are hardly any datasets available for the system to produce significant accuracy. a completely rule-based system might leave out principle parts of the abstract. there is a chance that all the questions cannot be captured inclusively by completely handwritten rules. hence, we want to introduce a mixed rule-based and ai-based solution to this problem. our system works on the following three crucial steps: we attempted to produce questions, concentrating on the critical points of a text that are generally asked in assessment tests. questions posed to an individual challenge their knowledge and understanding of specific topics, so we formed questions in each sentence in as many ways as possible. we based this model on children's stories, so the questions we wanted to produce aim to be simpler and more objective. based on the observation of the data chosen and analysis of all the possible causes, we developed a set of rules for each part of speech that can be formed into a question word in telugu. we maximized the possible number of questions in each sentence with all the keywords. we built rules for question generation based on pos tags, ud tags and information surrounding the word, which is comparable with vibhaktis (case markers) in telugu grammar. the question generation in manually evaluated and the detailed error analysis is given in section 8.1. our learning assistant evaluates using string matching, keyword matching for telugu answers, and a pre-trained sentence transformer model using xlm-r.( nils reimers, 2019 ) 2 related work previously, holy lovenia, felix limanta et al.[2018] (holy lovenia, 2018) experimented on q&a pair generation in english where they succeeded in forming what, who, and where questions. rami reddy et al.[2006] (rami reddy nandi reddy, 2006) worked on dialogue based question answering system in telugu for railway inquiries, which majorly concentrated on answer generation for a given query. similar work has done by (hoojung chung) on dealing with practical question answering system in restricted domain. shudipta sharma et al.[2018](shudipta sharma) worked on automatic q&a pair generation for english and bengali texts using nlp tasks like verb decomposition, subject auxiliary inversion for a question tag. ## dataset we have used a telugu stories dataset taken from a website called kathalu wordpress". @xcite this dataset was chosen because of a variety in the themes of the stories, wide vocabulary and sentences of varying lengths. ## summarization since telugu is a low resource language, we used statistical and unsupervised methods for this task. summarization also ensures the portability of our system to other similar low resource languages. for summarization, we did a basic data preprocessing (spaces, special characters, etc.) in addition to root-word extraction using shiva reddy's pos tagger @xcite . we used two types of existing summarization techniques: ## answer phrase selection candidate answers are words/phrases that depict some vital information in a sentence. adjectives, adverbs, and the subject of a sentence are some examples of such candidates. the answer selection module utilizes two main nlp components -pos tagging (part of speech tagging) and ud parsing (universal dependency parsing), along with language-specific rules to determine the answer words in an input sentence. ## question formation questions are formed according to the chosen phrases chosen previously, and the question words are replaced using further conditions if required. 1. quantifiers, adjectives, adverbs: the words that are marked jj pos are replaced with "ఎటువంటి" (etuvanti"-what kind of) rb pos tagged that are followed by verbs with "గా" (ga") suffix are replaced by "ఎలా" (ela"-how) and the qc tagged words that are not articles ("ఒక" (oka"-one/once) were chosen and changed based on the following word. if the quantifier is followed by "శాతం", "మంది" ,"వరకు" (shatam",mamdi",varaku") then the word is replaced with "ఎంత" (emta"how much), if the quantifier has a suffix it is added to the question word. for example: "1700కు" -"ఎంతకు" (emtaku) and the rest of the quantifiers like ఐదు పిచుచ్కలు (meaning five sparrows) are replaced with "ఎనిన్" (enni"-how many) ("ఎనిన్ పిచుచ్కలు" (how many sparrows) in this case). ## answer evaluation user's answer for the question generated is evaluated in two ways depending on the form of input. ## results we obtained results that resemble commonly used questions covering nine pos and ud tags. the questions generated by this system are successful and are most similar to academic questions we see in textbooks. we did manual error analysis for the question and answer pair generated. in most cases, it has produced legible results that resemble human-made questions, but there were errors in a few complex sentences. out of the 916 questions formed, only 34 were either completely erroneous or illegible. the rest were both grammatically correct and significant for the context of the story. the system successfully obtained all possible questions for each simple sentence, not requiring further linguistic analysis. ## conclusions we have built a mixed rule-based and aibased question and answer generating system with 96.28% accuracy. we used two methods for summarization and two similarity measures. we constructed observation-based rules for the dataset in a particular domain. there is a chance of varying results if we test this system for data in a different domain, but it gives accuracies above 95% for any data in the domain chosen. we tested question generation in the news article domain, which gave grammatically correct questions. the error rate may increase if we use complex words and phrases that need tags beyond the proposed set of rules. we plan to extend our work to be able to include: