pwc-archive/papers-with-abstracts huggingface dataset:
ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution
Authors: Milan Straka, Jana Straková
Date: 2022-09-15 00:00:00
Proceeding: CRAC (ACL) 2022 10
Tasks: coreference-resolution, Coreference Resolution

We describe the winning submission to the CRAC 2022 Shared Task on Multilingual Coreference Resolution. Our system first solves mention detection and then coreference linking on the retrieved spans with an antecedent-maximization approach, and both tasks are fine-tuned jointly with shared Transformer weights. We report results of fine-tuning a wide range of pretrained models. The center of this contribution are fine-tuned multilingual models. We found one large multilingual model with sufficiently large encoder to increase performance on all datasets across the board, with the benefit not limited only to the underrepresented languages or groups of typologically relative languages. The source code is available at https://github.com/ufal/crac2022-corpipe.
Graph-to-Text Generation with Dynamic Structure Pruning
Authors: Liang Li, Ruiying Geng, Bowen Li, Can Ma, Yinliang Yue, Binhua Li, Yongbin Li
Date: 2022-09-15 00:00:00
Proceeding: COLING 2022 10
Tasks: Decoder, Text Generation

Most graph-to-text works are built on the encoder-decoder framework with cross-attention mechanism. Recent studies have shown that explicitly modeling the input graph structure can significantly improve the performance. However, the vanilla structural encoder cannot capture all specialized information in a single forward pass for all decoding steps, resulting in inaccurate semantic representations. Meanwhile, the input graph is flatted as an unordered sequence in the cross attention, ignoring the original graph structure. As a result, the obtained input graph context vector in the decoder may be flawed. To address these issues, we propose a Structure-Aware Cross-Attention (SACA) mechanism to re-encode the input graph representation conditioning on the newly generated context at each decoding step in a structure aware manner. We further adapt SACA and introduce its variant Dynamic Graph Pruning (DGP) mechanism to dynamically drop irrelevant nodes in the decoding process. We achieve new state-of-the-art results on two graph-to-text datasets, LDC2020T02 and ENT-DESC, with only minor increase on computational cost.
Linear Transformations for Cross-lingual Sentiment Analysis
Authors: Pavel Přibáň, Jakub Šmíd, Adam Mištera, Pavel Král
Date: 2022-09-15 00:00:00
Tasks: Classification, Sentiment Analysis

This paper deals with cross-lingual sentiment analysis in Czech, English and French languages. We perform zero-shot cross-lingual classification using five linear transformations combined with LSTM and CNN based classifiers. We compare the performance of the individual transformations, and in addition, we confront the transformation-based approach with existing state-of-the-art BERT-like models. We show that the pre-trained embeddings from the target domain are crucial to improving the cross-lingual classification results, unlike in the monolingual classification, where the effect is not so distinctive.
UBARv2: Towards Mitigating Exposure Bias in Task-Oriented Dialogs
Authors: Yunyi Yang, Hong Ding, Qingyi Liu, Xiaojun Quan
Date: 2022-09-15 00:00:00

This paper studies the exposure bias problem in task-oriented dialog systems, where the model's generated content over multiple turns drives the dialog context away from the ground-truth distribution at training time, introducing error propagation and damaging the robustness of the TOD system. To bridge the gap between training and inference for multi-turn task-oriented dialogs, we propose session-level sampling which explicitly exposes the model to sampled generated content of dialog context during training. Additionally, we employ a dropout-based consistency regularization with the masking strategy R-Mask to further improve the robustness and performance of the model. The proposed UBARv2 achieves state-of-the-art performance on the standardized evaluation benchmark MultiWOZ and extensive experiments show the effectiveness of the proposed methods.
TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Media
Authors: Daniel Loureiro, Aminette D'Souza, Areej Nasser Muhajab, Isabella A. White, Gabriel Wong, Luis Espinosa Anke, Leonardo Neves, Francesco Barbieri, Jose Camacho-Collados
Date: 2022-09-15 00:00:00
Proceeding: COLING 2022 10

Language evolves over time, and word meaning changes accordingly. This is especially true in social media, since its dynamic nature leads to faster semantic shifts, making it challenging for NLP models to deal with new content and trends. However, the number of datasets and models that specifically address the dynamic nature of these social platforms is scarce. To bridge this gap, we present TempoWiC, a new benchmark especially aimed at accelerating research in social media-based meaning shift. Our results show that TempoWiC is a challenging benchmark, even for recently-released language models specialized in social media.
The Impact of Edge Displacement Vaserstein Distance on UD Parsing Performance
Authors: Mark Anderson, Carlos Gómez-Rodríguez
Date: 2022-09-15 00:00:00
Proceeding: CL (ACL) 2022 9

We contribute to the discussion on parsing performance in NLP by introducing a measurement that evaluates the differences between the distributions of edge displacement (the directed distance of edges) seen in training and test data. We hypothesize that this measurement will be related to differences observed in parsing performance across treebanks. We motivate this by building upon previous work and then attempt to falsify this hypothesis by using a number of statistical methods. We establish that there is a statistical correlation between this measurement and parsing performance even when controlling for potential covariants. We then use this to establish a sampling technique that gives us an adversarial and complementary split. This gives an idea of the lower and upper bounds of parsing systems for a given treebank in lieu of freshly sampled data. In a broader sense, the methodology presented here can act as a reference for future correlation-based exploratory work in NLP.
uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers
Authors: Piji Li
Date: 2022-09-15 00:00:00
Proceeding: COLING 2022 10
Tasks: Language Modeling, Language Modelling, Sentence

The task of Chinese Spelling Check (CSC) is aiming to detect and correct spelling errors that can be found in the text. While manually annotating a high-quality dataset is expensive and time-consuming, thus the scale of the training dataset is usually very small (e.g., SIGHAN15 only contains 2339 samples for training), therefore supervised-learning based models usually suffer the data sparsity limitation and over-fitting issue, especially in the era of big language models. In this paper, we are dedicated to investigating the \textbf{unsupervised} paradigm to address the CSC problem and we propose a framework named \textbf{uChecker} to conduct unsupervised spelling error detection and correction. Masked pretrained language models such as BERT are introduced as the backbone model considering their powerful language diagnosis capability. Benefiting from the various and flexible MASKing operations, we propose a Confusionset-guided masking strategy to fine-train the masked language model to further improve the performance of unsupervised detection and correction. Experimental results on standard datasets demonstrate the effectiveness of our proposed model uChecker in terms of character-level and sentence-level Accuracy, Precision, Recall, and F1-Measure on tasks of spelling error detection and correction respectively.
Accuracy of the Uzbek stop words detection: a case study on "School corpus"
Authors: Khabibulla Madatov, Shukurla Bekchanov, Jernej Vičič
Date: 2022-09-15 00:00:00
Tasks: Information Retrieval, Retrieval, Sentence

Stop words are very important for information retrieval and text analysis investigation tasks of natural language processing. Current work presents a method to evaluate the quality of a list of stop words aimed at automatically creating techniques. Although the method proposed in this paper was tested on an automatically-generated list of stop words for the Uzbek language, it can be, with some modifications, applied to similar languages either from the same family or the ones that have an agglutinative nature. Since the Uzbek language belongs to the family of agglutinative languages, it can be explained that the automatic detection of stop words in the language is a more complex process than in inflected languages. Moreover, we integrated our previous work on stop words detection in the example of the "School corpus" by investigating how to automatically analyse the detection of stop words in Uzbek texts. This work is devoted to answering whether there is a good way of evaluating available stop words for Uzbek texts, or whether it is possible to determine what part of the Uzbek sentence contains the majority of the stop words by studying the numerical characteristics of the probability of unique words. The results show acceptable accuracy of the stop words lists.
A semantic hierarchical graph neural network for text classification
Authors: Shuai Hua, Xinxin Li, Yunpeng Jing, Qunfeng Liu
Date: 2022-09-15 00:00:00
Tasks: Classification, Graph Neural Network, Sentence, text-classification, Text Classification

The key to the text classification task is language representation and important information extraction, and there are many related studies. In recent years, the research on graph neural network (GNN) in text classification has gradually emerged and shown its advantages, but the existing models mainly focus on directly inputting words as graph nodes into the GNN models ignoring the different levels of semantic structure information in the samples. To address the issue, we propose a new hierarchical graph neural network (HieGNN) which extracts corresponding information from word-level, sentence-level and document-level respectively. Experimental results on several benchmark datasets achieve better or similar results compared to several baseline methods, which demonstrate that our model is able to obtain more useful information for classification from samples.
VIPHY: Probing "Visible" Physical Commonsense Knowledge
Authors: Shikhar Singh, Ehsan Qasemi, Muhao Chen
Date: 2022-09-15 00:00:00
Tasks: Visual Reasoning

In recent years, vision-language models (VLMs) have shown remarkable performance on visual reasoning tasks (e.g. attributes, location). While such tasks measure the requisite knowledge to ground and reason over a given visual instance, they do not, however, measure the ability of VLMs to retain and generalize such knowledge. In this work, we evaluate their ability to acquire "visible" physical knowledge -- the information that is easily accessible from images of static scenes, particularly across the dimensions of object color, size and space. We build an automatic pipeline to derive a comprehensive knowledge resource for calibrating and probing these models. Our results indicate a severe gap between model and human performance across all three tasks. Furthermore, our caption pretrained baseline (CapBERT) significantly outperforms VLMs on both size and spatial tasks -- highlighting that despite sufficient access to ground language with visual modality, they struggle to retain such knowledge. The dataset and code are available at https://github.com/Axe--/ViPhy .
Robust Product Classification with Instance-Dependent Noise
Authors: Huy Nguyen, Devashish Khatwani
Date: 2022-09-14 00:00:00
Proceeding: ECNLP (ACL) 2022 5
Tasks: Classification, Denoising, Product Categorization

Noisy labels in large E-commerce product data (i.e., product items are placed into incorrect categories) are a critical issue for product categorization task because they are unavoidable, non-trivial to remove and degrade prediction performance significantly. Training a product title classification model which is robust to noisy labels in the data is very important to make product classification applications more practical. In this paper, we study the impact of instance-dependent noise to performance of product title classification by comparing our data denoising algorithm and different noise-resistance training algorithms which were designed to prevent a classifier model from over-fitting to noise. We develop a simple yet effective Deep Neural Network for product title classification to use as a base classifier. Along with recent methods of stimulating instance-dependent noise, we propose a novel noise stimulation algorithm based on product title similarity. Our experiments cover multiple datasets, various noise methods and different training solutions. Results uncover the limit of classification task when noise rate is not negligible and data distribution is highly skewed.
Prompt-based Conservation Learning for Multi-hop Question Answering
Authors: Zhenyun Deng, Yonghua Zhu, Yang Chen, Qianqian Qi, Michael Witbrock, Patricia Riddle
Date: 2022-09-14 00:00:00
Proceeding: COLING 2022 10
Tasks: Multi-hop Question Answering, Question Answering

Multi-hop question answering (QA) requires reasoning over multiple documents to answer a complex question and provide interpretable supporting evidence. However, providing supporting evidence is not enough to demonstrate that a model has performed the desired reasoning to reach the correct answer. Most existing multi-hop QA methods fail to answer a large fraction of sub-questions, even if their parent questions are answered correctly. In this paper, we propose the Prompt-based Conservation Learning (PCL) framework for multi-hop QA, which acquires new knowledge from multi-hop QA tasks while conserving old knowledge learned on single-hop QA tasks, mitigating forgetting. Specifically, we first train a model on existing single-hop QA tasks, and then freeze this model and expand it by allocating additional sub-networks for the multi-hop QA task. Moreover, to condition pre-trained language models to stimulate the kind of reasoning required for specific multi-hop questions, we learn soft prompts for the novel sub-networks to perform type-specific reasoning. Experimental results on the HotpotQA benchmark show that PCL is competitive for multi-hop QA and retains good performance on the corresponding single-hop sub-questions, demonstrating the efficacy of PCL in mitigating knowledge loss by forgetting.
ESSumm: Extractive Speech Summarization from Untranscribed Meeting
Authors: Jun Wang
Date: 2022-09-14 00:00:00
Tasks: speech-recognition, Speech Recognition

In this paper, we propose a novel architecture for direct extractive speech-to-speech summarization, ESSumm, which is an unsupervised model without dependence on intermediate transcribed text. Different from previous methods with text presentation, we are aimed at generating a summary directly from speech without transcription. First, a set of smaller speech segments are extracted based on speech signal's acoustic features. For each candidate speech segment, a distance-based summarization confidence score is designed for latent speech representation measure. Specifically, we leverage the off-the-shelf self-supervised convolutional neural network to extract the deep speech features from raw audio. Our approach automatically predicts the optimal sequence of speech segments that capture the key information with a target summary length. Extensive results on two well-known meeting datasets (AMI and ICSI corpora) show the effectiveness of our direct speech-based method to improve the summarization quality with untranscribed data. We also observe that our unsupervised speech-based method even performs on par with recent transcript-based summarization approaches, where extra speech recognition is required.
Medical Image Segmentation using LeViT-UNet++: A Case Study on GI Tract Data
Authors: Praneeth Nemani, Satyanarayana Vollala
Date: 2022-09-15 00:00:00
Tasks: image-classification, Image Classification, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Gastro-Intestinal Tract cancer is considered a fatal malignant condition of the organs in the GI tract. Due to its fatality, there is an urgent need for medical image segmentation techniques to segment organs to reduce the treatment time and enhance the treatment. Traditional segmentation techniques rely upon handcrafted features and are computationally expensive and inefficient. Vision Transformers have gained immense popularity in many image classification and segmentation tasks. To address this problem from a transformers' perspective, we introduced a hybrid CNN-transformer architecture to segment the different organs from an image. The proposed solution is robust, scalable, and computationally efficient, with a Dice and Jaccard coefficient of 0.79 and 0.72, respectively. The proposed solution also depicts the essence of deep learning-based automation to improve the effectiveness of the treatment
A Genetic Quantum Annealing Algorithm
Authors: Steven Abel, Luca A. Nutricati, Michael Spannowsky
Date: 2022-09-15 00:00:00

A genetic algorithm (GA) is a search-based optimization technique based on the principles of Genetics and Natural Selection. We present an algorithm which enhances the classical GA with input from quantum annealers. As in a classical GA, the algorithm works by breeding a population of possible solutions based on their fitness. However, the population of individuals is defined by the continuous couplings on the quantum annealer, which then give rise via quantum annealing to the set of corresponding phenotypes that represent attempted solutions. This introduces a form of directed mutation into the algorithm that can enhance its performance in various ways. Two crucial enhancements come from the continuous couplings having strengths that are inherited from the fitness of the parents (so-called nepotism) and from the annealer couplings allowing the entire population to be influenced by the fittest individuals (so-called quantum-polyandry). We find our algorithm to be significantly more powerful on several simple problems than a classical GA.
Astromorphic Self-Repair of Neuromorphic Hardware Systems
Authors: Zhuangyu Han, A N M Nafiul Islam, Abhronil Sengupta
Date: 2022-09-15 00:00:00

While neuromorphic computing architectures based on Spiking Neural Networks (SNNs) are increasingly gaining interest as a pathway toward bio-plausible machine learning, attention is still focused on computational units like the neuron and synapse. Shifting from this neuro-synaptic perspective, this paper attempts to explore the self-repair role of glial cells, in particular, astrocytes. The work investigates stronger correlations with astrocyte computational neuroscience models to develop macro-models with a higher degree of bio-fidelity that accurately captures the dynamic behavior of the self-repair process. Hardware-software co-design analysis reveals that bio-morphic astrocytic regulation has the potential to self-repair hardware realistic faults in neuromorphic hardware systems with significantly better accuracy and repair convergence for unsupervised learning tasks on the MNIST and F-MNIST datasets. Our implementation source code and trained models are available at https://github.com/NeuroCompLab-psu/Astromorphic_Self_Repair.
Optimization of Rocker-Bogie Mechanism using Heuristic Approaches
Authors: Harsh Senjaliya, Pranshav Gajjar, Brijan Vaghasiya, Pooja Shah, Paresh Gujarati
Date: 2022-09-14 00:00:00
Tasks: Computational Efficiency

Optimal locomotion and efficient traversal of extraterrestrial rovers in dynamic terrains and environments is an important problem statement in the field of planetary science and geophysical systems. Designing a superlative and efficient architecture for the suspension mechanism of planetary rovers is a crucial step towards robust rovers. This paper focuses on the Rocker Bogie mechanism, a standard suspension methodology associated with foreign terrains. After scrutinizing the available previous literature and by leveraging various optimization and global minimization algorithms, this paper offers a novel study on mechanical design optimization of a rovers suspension mechanism. This paper presents extensive tests on Simulated Annealing, Genetic Algorithms, Swarm Intelligence techniques, Basin Hoping and Differential Evolution, while thoroughly assessing every related hyper parameter, to find utility driven solutions. We also assess Dual Annealing and subsidiary algorithms for the aforementioned task while maintaining an unbiased testing standpoint for ethical research. Computational efficiency and overall fitness are considered key valedictory parameters for assessing the related algorithms, emphasis is also given to variable input seeds to find the most suitable utility driven strategy. Simulated Annealing was obtained empirically to be the top performing heuristic strategy, with a fitness of 760, which was considerably superior to other algorithms and provided consistent performance across various input seeds and individual performance indicators.
Heart Attack Classification System using Neural Network Trained with Particle Swarm Optimization
Authors: Askandar H. Amin, Botan K. Ahmed, Bestan B. Maaroof, Tarik A. Rashid
Date: 2022-08-21 00:00:00

The prior detection of a heart attack could lead to the saving of one's life. Putting specific criteria into a system that provides an early warning of an imminent at-tack will be advantageous to a better prevention plan for an upcoming heart attack. Some studies have been conducted for this purpose, but yet the goal has not been reached to prevent a patient from getting such a disease. In this paper, Neural Network trained with Particle Swarm Optimization (PSONN) is used to analyze the input criteria and enhance heart attack anticipation. A real and novel dataset that has been recorded on the disease is used. After preprocessing the data, the features are fed into the system. As a result, the outcomes from PSONN have been evaluated against those from other algorithms. Decision Tree, Random Forest, Neural network trained with Backpropagation (BPNN), and Naive Bayes were among those employed. Then the results of 100%, 99.2424%, 99.2323%, 81.3131%, and 66.4141% are produced concerning the mentioned algorithms, which show that PSONN has recorded the highest accuracy rate among all other tested algorithms.
Forecasting Evolution of Clusters in Game Agents with Hebbian Learning
Authors: Beomseok Kang, Saibal Mukhopadhyay
Date: 2022-08-19 00:00:00
Tasks: Clustering, Multi-agent Reinforcement Learning, Real-Time Strategy Games, Self-Supervised Learning, Starcraft, Starcraft II

Large multi-agent systems such as real-time strategy games are often driven by collective behavior of agents. For example, in StarCraft II, human players group spatially near agents into a team and control the team to defeat opponents. In this light, clustering the agents in the game has been used for various purposes such as the efficient control of the agents in multi-agent reinforcement learning and game analytic tools for the game users. However, despite the useful information provided by clustering, learning the dynamics of multi-agent systems at a cluster level has been rarely studied yet. In this paper, we present a hybrid AI model that couples unsupervised and self-supervised learning to forecast evolution of the clusters in StarCraft II. We develop an unsupervised Hebbian learning method in a set-to-cluster module to efficiently create a variable number of the clusters with lower inference time complexity than K-means clustering. Also, a long short-term memory based prediction module is designed to recursively forecast state vectors generated by the set-to-cluster module to define cluster configuration. We experimentally demonstrate the proposed model successfully predicts complex movement of the clusters in the game.