pwc-archive/papers-with-abstracts huggingface dataset:
An Investigation of Hybrid architectures for Low Resource Multilingual Speech Recognition system in Indian context
Authors: Ganesh Mirishkar, Aditya Yadavalli, Anil Kumar Vuppala
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Diversity, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

India is a land of language diversity. There are approximately 2000 languages spoken around, and among which officially registered are 23. In those, there are very few with Automatic Speech Recognition (ASR) capability. The reason for this is the fact that building an ASR system requires thousands of hours of annotated speech data, a vast amount of text, and a lexicon that can span all the words in the language. At the same time, it is observed that Indian languages share a common phonetic base. In this work, we build a multilingual speech recognition system for low-resource languages by leveraging the shared phonetic space. Deep Neural architectures play a vital role in improving the performance of low-resource ASR systems. The typical strategy used to train the multilingual acoustic model is merging various languages as a unified group. In this paper, the speech recognition system is built using six Indian languages, namely Gujarati, Hindi, Marathi, Odia, Tamil, and Telugu. Various state-of-the-art experiments were performed using different acoustic modeling and language modeling techniques.
Improve Sinhala Speech Recognition Through e2e LF-MMI Model
Authors: Buddhi Gamage, Randil Pushpananda, Thilini Nadungodage, Ruwan Weerasinghe
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, Speech-to-Text, Speech-to-Text Translation

Automatic speech recognition (ASR) has experienced several paradigm shifts over the years from template-based approaches and statistical modeling to the popular GMM-HMM approach and then to deep learning hybrid model DNN-HMM. The latest shift is to end-to-end (e2e) DNN architecture. We present a study to build an e2e ASR system using state-of-the-art deep learning models to verify the applicability of e2e ASR models for the highly inflected and yet low-resource Sinhala language. We experimented on e2e Lattice-Free Maximum Mutual Information (e2e LF-MMI) model with the baseline statistical models with 40 hours of training data to evaluate. We used the same corpus for creating language models and lexicon in our previous study, which resulted in the best accuracy for the Sinhala language. We were able to achieve a Word-error-rate (WER) of 28.55% for Sinhala, only slightly worse than the existing best hybrid model. Our model, however, is more context-independent and faster for Sinhala speech recognition and so more suitable for general purpose speech-to-text translation.
Image Caption Generation Framework for Assamese News using Attention Mechanism
Authors: Ringki Das, Thoudam Doren Singh
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Caption Generation, Decoder, Image Captioning

Automatic caption generation is an artificial intelligence problem that falls at the intersection of computer vision and natural language processing. Although significant works have been reported in image captioning, the contribution is limited to English and few major languages with sufficient resources. But, no work on image captioning has been reported in a resource-constrained language like Assamese. With this inspiration, we propose an encoder-decoder based framework for image caption generation in the Assamese news domain. The VGG-16 pre-trained model at the encoder side and LSTM with an attention mechanism are employed at the decoder side to generate the Assamese caption. We train the proposed model on the dataset built in-house consisting of 10,000 images with a single caption for each image. We describe our experimental methodology, quantitative and qualitative results which validate the effectiveness of our model for caption generation. The proposed model shows a BLEU score of 12.1 outperforming the baseline model.
An Efficient Keyframes Selection Based Framework for Video Captioning
Authors: Alok Singh, Loitongbam Sanayai Meetei, Salam Michael Singh, Thoudam Doren Singh, Sivaji Bandyopadhyay
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Text Generation, Video Captioning, Video Description

Describing a video is a challenging yet attractive task since it falls into the intersection of computer vision and natural language generation. The attention-based models have reported the best performance. However, all these models follow similar procedures, such as segmenting videos into chunks of frames or sampling frames at equal intervals for visual encoding. The process of segmenting video into chunks or sampling frames at equal intervals causes encoding of redundant visual information and requires additional computational cost since a video consists of a sequence of similar frames and suffers from inescapable noise such as uneven illumination, occlusion and motion effects. In this paper, a boundary-based keyframes selection approach for video description is proposed that allow the system to select a compact subset of keyframes to encode the visual information and generate a description for a video without much degradation. The proposed approach uses 3 4 frames per video and yields competitive performance over two benchmark datasets MSVD and MSR-VTT (in both English and Hindi).
A Scaled Encoder Decoder Network for Image Captioning in Hindi
Authors: Santosh Kumar Mishra, Sriparna Saha, Pushpak Bhattacharyya
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Decoder, Deep Learning, Image Captioning

Image captioning is a prominent research area in computer vision and natural language processing, which automatically generates natural language descriptions for images. Most of the existing works have focused on developing models for image captioning in the English language. The current paper introduces a novel deep learning architecture based on encoder-decoder with an attention mechanism for image captioning in the Hindi language. For encoder, decoder, and attention, several deep learning-based architectures have been explored. Hindi, the fourth-most spoken language globally, is widely spoken in India and South Asia and is one of India’s official languages. The proposed encoder-decoder architecture utilizes scaling in convolution neural networks to achieve better accuracy than state-of-the-art image captioning methods in Hindi. The proposed method’s performance is compared with state-of-the-art methods in terms of BLEU scores and manual evaluation (in terms of adequacy and fluency). The obtained results demonstrate the efficacy of the proposed method.
Co-attention based Multimodal Factorized Bilinear Pooling for Internet Memes Analysis
Authors: Gitanjali Kumari, Amitava Das, Asif Ekbal
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12

Social media platforms like Facebook, Twitter, and Instagram have a significant impact on several aspects of society. Memes are a new type of social media communication found on social platforms. Even though memes are primarily used to distribute humorous content, certain memes propagate hate speech through dark humor. It is critical to properly analyze and filter out these toxic memes from social media. But the presence of sarcasm and humor in an implicit way analyzes memes more challenging. This paper proposes an end-to-end neural network architecture that learns the complex association between text and image of a meme. For this purpose, we use a recent SemEval-2020 Task-8 multimodal dataset. We proposed an end-to-end CNN-based deep neural network architecture with two sub-modules viz. (i)Co-attention based sub-module and (ii) Multimodal Factorized Bilinear Pooling(MFB) sub-module to represent the textual and visual features of a meme in a more fine-grained way. We demonstrated the effectiveness of our proposed work through extensive experiments. The experimental results show that our proposed model achieves a 36.81% macro F1-score, outperforming all the baseline models.
How effective is incongruity? Implications for code-mixed sarcasm detection
Authors: Aditya Shah, Chandresh Maurya
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Sarcasm Detection

The presence of sarcasm in conversational systems and social media like chatbots, Facebook, Twitter, etc. poses several challenges for downstream NLP tasks. This is attributed to the fact that the intended meaning of a sarcastic text is contrary to what is expressed. Further, the use of code-mix language to express sarcasm is increasing day by day. Current NLP techniques for code-mix data have limited success due to the use of different lexicon, syntax, and scarcity of labeled corpora. To solve the joint problem of code-mixing and sarcasm detection, we propose the idea of capturing incongruity through sub-word level embeddings learned via fastText. Empirical results show that our proposed model achieves an F1-score on code-mix Hinglish dataset comparable to pretrained multilingual models while training 10x faster and using a lower memory footprint.
Contrastive Learning of Sentence Representations
Authors: Hefei Qiu, Wei Ding, Ping Chen
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Contrastive Learning, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings

Learning sentence representations which capture rich semantic meanings has been crucial for many NLP tasks. Pre-trained language models such as BERT have achieved great success in NLP, but sentence embeddings extracted directly from these models do not perform well without fine-tuning. We propose Contrastive Learning of Sentence Representations (CLSR), a novel approach which applies contrastive learning to learn universal sentence representations on top of pre-trained language models. CLSR utilizes semantic similarity of two sentences to construct positive instance for contrastive learning. Semantic information that has been captured by the pre-trained models is kept by getting sentence embeddings from these models with proper pooling strategy. An encoder followed by a linear projection takes these embeddings as inputs and is trained under a contrastive objective. To evaluate the performance of CLSR, we run experiments on a range of pre-trained language models and their variants on a series of Semantic Contextual Similarity tasks. Results show that CLSR gains significant performance improvements over existing SOTA language models.
Classifying Verses of the Quran using Doc2vec
Authors: Menwa Alshammeri, Eric Atwell, Mohammad Alsalka
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: regression, Semantic Similarity, Semantic Textual Similarity

The Quran, as a significant religious text, bears important spiritual and linguistic values. Understanding the text and inferring the underlying meanings entails semantic similarity analysis. We classified the verses of the Quran into 15 pre-defined categories or concepts, based on the Qurany corpus, using Doc2Vec and Logistic Regression. Our classifier scored 70% accuracy, and 60% F1-score using the distributed bag-of-words architecture. We then measured how similar the documents within the same category are to each other semantically and use this information to evaluate our model. We calculated the mean difference and average similarity values for each category to indicate how well our model describes that category.
Training data reduction for multilingual Spoken Language Understanding systems
Authors: Anmol Bansal, Anjali Shenoy, Krishna Chaitanya Pappu, Kay Rottmann, Anurag Dwarakanath
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: intent-classification, Intent Classification, Spoken Language Understanding

Fine-tuning self-supervised pre-trained language models such as BERT has significantly improved state-of-the-art performance on natural language processing tasks. Similar finetuning setups can also be used in commercial large scale Spoken Language Understanding (SLU) systems to perform intent classification and slot tagging on user queries. Finetuning such powerful models for use in commercial systems requires large amounts of training data and compute resources to achieve high performance. This paper is a study on the different empirical methods of identifying training data redundancies for the fine tuning paradigm. Particularly, we explore rule based and semantic techniques to reduce data in a multilingual fine tuning setting and report our results on key SLU metrics. Through our experiments, we show that we can achieve on par/better performance on fine-tuning using a reduced data set as compared to a model finetuned on the entire data set.
Leveraging Expectation Maximization for Identifying Claims in Low Resource Indian Languages
Authors: Rudra Dhar, Dipankar Das
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12

Identification of the checkable claims is one of the important prior tasks while dealing with infinite amount of data streaming from social web and the task becomes a compulsory one when we analyze them on behalf of a multilingual country like India that contains more than 1 billion people. In the present work, we describe our system which is made for detecting check-worthy claim sentences in resource scarce Indian languages (e.g., Bengali and Hindi). Firstly, we collected sentences from various sources in Bengali and Hindi and vectorized them with several NLP features. We labeled a small portion of them for check-worthy claims manually. However, in order to label rest amount of data in a semi-supervised fashion, we employed the Expectation Maximization (EM) algorithm tuned with the Multivariate Gaussian Mixture Model (GMM) to assign weakly labels. The optimal number of Gaussians in this algorithm is traced by using Logistic Regression. Furthermore, we used different ratios of manually labeled data and weakly labeled data to train our various machine learning models. We tabulated and plotted the performances of the models along with the stepwise decrement in proportion of manually labeled data. The experimental results were at par with our theoretical understanding, and we conclude that the weakly labeling of check-worthy claim sentences in low resource languages with EM algorithm has true potential.
Performance of BERT on Persuasion for Good
Authors: Saumajit Saha, Kanika Kalra, Manasi Patwardhan, Shirish Karande
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12

We consider the task of automatically classifying the persuasion strategy employed by an utterance in a dialog. We base our work on the PERSUASION-FOR-GOOD dataset, which is composed of conversations between crowdworkers trying to convince each other to make donations to a charity. Currently, the best known performance on this dataset, for classification of persuader’s strategy, is not derived by employing pretrained language models like BERT. We observe that a straightforward fine-tuning of BERT does not provide significant performance gain. Nevertheless, nonuniformly sampling to account for the class imbalance and a cost function enforcing a hierarchical probabilistic structure on the classes provides an absolute improvement of 10.79% F1 over the previously reported results. On the same dataset, we replicate the framework for classifying the persuadee’s response.
Multi-Turn Target-Guided Topic Prediction with Monte Carlo Tree Search
Authors: Jingxuan Yang, Si Li, Jun Guo
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Prediction, Retrieval

This paper concerns the problem of topic prediction in target-guided conversation, which requires the system to proactively and naturally guide the topic thread of the conversation, ending up with achieving a designated target subject. Existing studies usually resolve the task with a sequence of single-turn topic prediction. Greedy decision is made at each turn since it is impossible to explore the topics in future turns under the single-turn topic prediction mechanism. As a result, these methods often suffer from generating sub-optimal topic threads. In this paper, we formulate the target-guided conversation as a problem of multi-turn topic prediction and model it under the framework of Markov decision process (MDP). To alleviate the problem of generating sub-optimal topic thread, Monte Carlo tree search (MCTS) is employed to improve the topic prediction by conducting long-term planning. At online topic prediction, given a target and a start utterance, our proposed MM-TP (MCTS-enhanced MDP for Topic Prediction) firstly performs MCTS to enhance the policy for predicting the topic for each turn. Then, two retrieval models are respectively used to generate the responses of the agent and the user. Quantitative evaluation and qualitative study showed that MM-TP significantly improved the state-of-the-art baselines.
Resolving Prepositional Phrase Attachment Ambiguities with Contextualized Word Embeddings
Authors: Adwait Ratnaparkhi, Atul Kumar
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Prepositional Phrase Attachment, Sentence, Word Embeddings

This paper applies contextualized word embedding models to a long-standing problem in the natural language parsing community, namely prepositional phrase attachment. Following past formulations of this problem, we use data sets in which the attachment decision is both a binary-valued choice as well as a multi-valued choice. We present a deep learning architecture that fine-tunes the output of a contextualized word embedding model for the purpose of predicting attachment decisions. We present experiments on two commonly used datasets that outperform the previous best results, using only the original training data and the unannotated full sentence context.
Multi-Source Cross-Lingual Constituency Parsing
Authors: Hour Kaing, Chenchen Ding, Katsuhito Sudoh, Masao Utiyama, Eiichiro Sumita, Satoshi Nakamura
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Constituency Parsing, Cross-Lingual Transfer, Diversity, Pretrained Multilingual Language Models

Pretrained multilingual language models have become a key part of cross-lingual transfer for many natural language processing tasks, even those without bilingual information. This work further investigates the cross-lingual transfer ability of these models for constituency parsing and focuses on multi-source transfer. Addressing structure and label set diversity problems, we propose the integration of typological features into the parsing model and treebank normalization. We trained the model on eight languages with diverse structures and use transfer parsing for an additional six low-resource languages. The experimental results show that the treebank normalization is essential for cross-lingual transfer performance and the typological features introduce further improvement. As a result, our approach improves the baseline F1 of multi-source transfer by 5 on average.
Kannada Sandhi Generator for Lopa and Adesha Sandhi
Authors: Musica Supriya, Dinesh U. Acharya, Ashalatha Nayak, Arjuna S. R
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12

Kannada is one of the major spoken classical languages in India. It is morphologically rich and highly agglutinative in nature. One of the important grammatical aspects is the concept of sandhi(euphonic change). There has not been a sandhi generator for Kannada and this work aims at basic sandhi generation. In this paper, we present algorithms for lopa and Adesha sandhi using a rule-based approach. The proposed method generates the sandhied word and corresponding sandhi without any help of dictionary. This work is significant for agglutinative languages especially to Dravidian languages and can be used to enhance the vocabulary for language related tasks.
Semantics of Spatio-Directional Geometric Terms of Indian Languages
Authors: Sukhada Sukhada, Paul Soma, Rahul Kumar, Karthik Puranik
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Machine Translation

This paper examines widely prevalent yet little-studied expressions in Indian languages which are known as geometrical terms be-cause “they engage locations along the axes of the reference object”. These terms are andara (inside), b ̄ahara (outside), ̄age (in front of), s ̄amane (in front of), p ̄ıche (back), ̄upara (above/over), n ̄ıce (under/below), d ̄ayem. (right), b ̄ayem. (left), p ̄asa (near), d ̄ura (away/far) in Hindi. The way these terms have been interpreted by the scholars of the Hindi language and handled in the Hindi Dependency treebank is misleading. This paper proposes an alternative analysis of these terms focusing on their triple – nominal, modifier and relational - functions and presents abstract semantic representations of these terms following the proposed analysis. The semantic representation will be explicit, unambiguous abstract and therefore universal in nature. The correspondence of these terms in Bangla and Kannada are also identified. Disambiguation of geometric terms will facilitate parsing and machine translation especially from Indian Language to English because these geometric terms of Indian languages are variedly translated in English de-pending on context.
Auditing Keyword Queries Over Text Documents
Authors: Bharath Kumar Reddy Apparreddy, Sailaja Rajanala, Manish Singh
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Anomaly Detection

Data security and privacy is an issue of growing importance in the healthcare domain. In this paper, we present an auditing system to detect privacy violations for unstructured text documents such as healthcare records. Given a sensitive document, we present an anomaly detection algorithm that can find the top-k suspicious keyword queries that may have accessed the sensitive document. Since unstructured healthcare data, such as medical reports and query logs, are not easily available for public research, in this paper, we show how one can use the publicly available DBLP data to create an equivalent healthcare data and query log, which can then be used for experimental evaluation.
A Method to Disambiguate a Word by Using Restricted Boltzmann Machine
Authors: Nazreena Rahman, Bhogeswar Borah
Date: 2021-12-01 00:00:00
Proceeding: ICON 2021 12
Tasks: Articles, Information Retrieval, Retrieval, Word Sense Disambiguation

Finding the correct sense of a word is of great importance in many textual data related applications such as information retrieval, text mining and natural language processing. We have proposed one novel Word Sense Disambiguation (WSD) method according to its context. Based on collocation extraction score, the proposed method extracts three different features for each sense definition of a target word. These features create a feature vector and all the feature vectors create a sense matrix. Here, Restricted Boltzmann Machine (RBM) is used to enhance the sense matrix. Comparison of the proposed WSD method is made with current state-of-the-art systems using SENSEVAL and Sem Eval datasets. The proposed WSD method shows the practical implementation by applying on query-based text summary. For evaluation on query-based text summary, the proposed WSD method uses DUC datasets containing news-wire articles. Finally, the experimental analysis shows that our proposed WSD method performs better as compared to the current systems.