pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Representation Learning and Anomaly Detection in ECG Sequences
Authors: João Pereira, Margarida Silveira
Date: 2019-08-01 00:00:00
Tasks: Anomaly Detection, Clustering, Representation Learning, Time Series, Time Series Analysis, Time Series Classification

While the big data revolution takes place, large amounts of electronic health records, such as electrocardiograms (ECGs) and vital signs data, have become available. These signals are often recorded as time series of observations and are now easier to obtain. In particular, with the arise of smart devices that can perform ECG, there is the quest for developing novel approaches that allow to monitor these signals efficiently, and quickly detect anomalies. However, since most data generated remains unlabelled, the task of anomaly detection is still very challenging. Unsupervised representation learning using deep generative models (e.g., variational autoencoders) has been used to learn expressive feature representations of sequences that can make downstream tasks, such as anomaly detection, easier to execute and more accurate. We propose an approach for unsupervised representation learning of ECG sequences using a variational autoencoder parameterised by recurrent neural networks, and use the learned representations for anomaly detection using multiple detection strategies. We tested our approach on the ECG5000 electrocardiogram dataset of the UCR time series classification archive. Our results show that the proposed approach is able to learn expressive representations of ECG sequences, and to detect anomalies with scores that outperform other both supervised and unsupervised methods.
Knowledge Enhanced Masked Language Model for Stance Detection
Authors: Kornraphop Kawintiranon, Lisa Singh
Date: 2021-05-26 00:00:00
Proceeding: NAACL 2021 4
Tasks: Language Modeling, Language Modelling, Sentence, Stance Detection, Stance Detection (US Election 2020 - Biden), Stance Detection (US Election 2020 - Trump)

Detecting stance on Twitter is especially challenging because of the short length of each tweet, the continuous coinage of new terminology and hashtags, and the deviation of sentence structure from standard prose. Fine-tuned language models using large-scale in-domain data have been shown to be the new state-of-the-art for many NLP tasks, including stance detection. In this paper, we propose a novel BERT-based fine-tuning method that enhances the masked language model for stance detection. Instead of random token masking, we propose using a weighted log-odds-ratio to identify words with high stance distinguishability and then model an attention mechanism that focuses on these words. We show that our proposed approach outperforms the state of the art for stance detection on Twitter data about the 2020 US Presidential election.
Prediction of Clinical Risk Factors of Diabetes Using Multiple Machine Learning Techniques Resolving Class Imbalance
Authors: Kazi Amit Hasan, Md. Al Mehedi Hasan
Date: 2020-12-19 00:00:00
Tasks: BIG-bench Machine Learning, Medical Diagnosis

Prediction of Clinical Risk Factors of Diabetes Using Multiple Machine Learning TechBeing the most common and rapidly growing disease, Diabetes affecting a huge number of people from all span of ages each year that reduces the lifespan. Having a high affecting rate, it increases the significance of initial diagnosis. Diabetes brings other complicated complications like cardiovascular disease, kidney failure, stroke, damaging the vital organs etc. Early diagnosis of diabetes reduces the likelihood of transiting it into a chronic and severe state. The identification and analysis of risk factors of different spinal attributes help to identify the prevalence of diabetes in medical diagnosis. The prevalence measure and identification of diabetes in the early stages reduce the chances of future complications. In this research, the collective NHANES dataset of 1999-2000 to 2015-2016 was used and the purposes of this research were to analyze and ascertain the potential risk factors correlated with diabetes by using Logistic Regression, ANOVA and also to identify the abnormalities by using multiple supervised machine learning algorithms. Class imbalance, outlier problems were handled and experimental results show that age, blood-related diabetes, cholesterol and BMI are the most significant risk factors that associated with diabetes. Along with this, the highest accuracy score .90 was achieved with the random forest classification method.
Focus on Local: Detecting Lane Marker from Bottom Up via Key Point
Authors: Zhan Qu, Huan Jin, Yang Zhou, Zhen Yang, Wei zhang
Date: 2021-05-28 00:00:00
Proceeding: CVPR 2021 1
Tasks: Lane Detection

Mainstream lane marker detection methods are implemented by predicting the overall structure and deriving parametric curves through post-processing. Complex lane line shapes require high-dimensional output of CNNs to model global structures, which further increases the demand for model capacity and training data. In contrast, the locality of a lane marker has finite geometric variations and spatial coverage. We propose a novel lane marker detection solution, FOLOLane, that focuses on modeling local patterns and achieving prediction of global structures in a bottom-up manner. Specifically, the CNN models lowcomplexity local patterns with two separate heads, the first one predicts the existence of key points, and the second refines the location of key points in the local range and correlates key points of the same lane line. The locality of the task is consistent with the limited FOV of the feature in CNN, which in turn leads to more stable training and better generalization. In addition, an efficiency-oriented decoding algorithm was proposed as well as a greedy one, which achieving 36% runtime gains at the cost of negligible performance degradation. Both of the two decoders integrated local information into the global geometry of lane markers. In the absence of a complex network architecture design, the proposed method greatly outperforms all existing methods on public datasets while achieving the best state-of-the-art results and real-time processing simultaneously.
Improving Generalization in Mountain Car Through the Partitioned Parameterized Policy Approach via Quasi-Stochastic Gradient Descent
Authors: Caleb M. Bowyer
Date: 2021-05-28 00:00:00

The reinforcement learning problem of finding a control policy that minimizes the minimum time objective for the Mountain Car environment is considered. Particularly, a class of parameterized nonlinear feedback policies is optimized over to reach the top of the highest mountain peak in minimum time. The optimization is carried out using quasi-Stochastic Gradient Descent (qSGD) methods. In attempting to find the optimal minimum time policy, a new parameterized policy approach is considered that seeks to learn an optimal policy parameter for different regions of the state space, rather than rely on a single macroscopic policy parameter for the entire state space. This partitioned parameterized policy approach is shown to outperform the uniform parameterized policy approach and lead to greater generalization than prior methods, where the Mountain Car became trapped in circular trajectories in the state space.
Confident in the Crowd: Bayesian Inference to Improve Data Labelling in Crowdsourcing
Authors: Pierce Burke, Richard Klein
Date: 2021-05-28 00:00:00
Tasks: Bayesian Inference, Binary Classification

With the increased interest in machine learning and big data problems, the need for large amounts of labelled data has also grown. However, it is often infeasible to get experts to label all of this data, which leads many practitioners to crowdsourcing solutions. In this paper, we present new techniques to improve the quality of the labels while attempting to reduce the cost. The naive approach to assigning labels is to adopt a majority vote method, however, in the context of data labelling, this is not always ideal as data labellers are not equally reliable. One might, instead, give higher priority to certain labellers through some kind of weighted vote based on past performance. This paper investigates the use of more sophisticated methods, such as Bayesian inference, to measure the performance of the labellers as well as the confidence of each label. The methods we propose follow an iterative improvement algorithm which attempts to use the least amount of workers necessary to achieve the desired confidence in the inferred label. This paper explores simulated binary classification problems with simulated workers and questions to test the proposed methods. Our methods outperform the standard voting methods in both cost and accuracy while maintaining higher reliability when there is disagreement within the crowd.
Relation Matters in Sampling: A Scalable Multi-Relational Graph Neural Network for Drug-Drug Interaction Prediction
Authors: Arthur Feeney, Rishabh Gupta, Veronika Thost, Rico Angell, Gayathri Chandu, Yash Adhikari, Tengfei Ma
Date: 2021-05-28 00:00:00
Tasks: Graph Neural Network, Relation

Sampling is an established technique to scale graph neural networks to large graphs. Current approaches however assume the graphs to be homogeneous in terms of relations and ignore relation types, critically important in biomedical graphs. Multi-relational graphs contain various types of relations that usually come with variable frequency and have different importance for the problem at hand. We propose an approach to modeling the importance of relation types for neighborhood sampling in graph neural networks and show that we can learn the right balance: relation-type probabilities that reflect both frequency and importance. Our experiments on drug-drug interaction prediction show that state-of-the-art graph neural networks profit from relation-dependent sampling in terms of both accuracy and efficiency.
Cisco at SemEval-2021 Task 5: What's Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments
Authors: Sreyan Ghosh, Sonal Kumar
Date: 2021-05-28 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Attribute, Binary Classification, Dependency Parsing, Named Entity Recognition (NER), Sentence, TAG, Toxic Spans Detection

Social network platforms are generally used to share positive, constructive, and insightful content. However, in recent times, people often get exposed to objectionable content like threat, identity attacks, hate speech, insults, obscene texts, offensive remarks or bullying. Existing work on toxic speech detection focuses on binary classification or on differentiating toxic speech among a small set of categories. This paper describes the system proposed by team Cisco for SemEval-2021 Task 5: Toxic Spans Detection, the first shared task focusing on detecting the spans in the text that attribute to its toxicity, in English language. We approach this problem primarily in two ways: a sequence tagging approach and a dependency parsing approach. In our sequence tagging approach we tag each token in a sentence under a particular tagging scheme. Our best performing architecture in this approach also proved to be our best performing architecture overall with an F1 score of 0.6922, thereby placing us 7th on the final evaluation phase leaderboard. We also explore a dependency parsing approach where we extract spans from the input sentence under the supervision of target span boundaries and rank our spans using a biaffine model. Finally, we also provide a detailed analysis of our results and model performance in our paper.
Changing the World by Changing the Data
Authors: Anna Rogers
Date: 2021-05-28 00:00:00
Proceeding: ACL 2021 5
Tasks: Position

NLP community is currently investing a lot more research and resources into development of deep learning models than training data. While we have made a lot of progress, it is now clear that our models learn all kinds of spurious patterns, social biases, and annotation artifacts. Algorithmic solutions have so far had limited success. An alternative that is being actively discussed is more careful design of datasets so as to deliver specific signals. This position paper maps out the arguments for and against data curation, and argues that fundamentally the point is moot: curation already is and will be happening, and it is changing the world. The question is only how much thought we want to invest into that process.
Training of SSD(Single Shot Detector) for Facial Detection using Nvidia Jetson Nano
Authors: Saif Ur Rehman, Muhammad Rashid Razzaq, Muhammad Hadi Hussian
Date: 2021-05-28 00:00:00

In this project, we have used the computer vision algorithm SSD (Single Shot detector) computer vision algorithm and trained this algorithm from the dataset which consists of 139 Pictures. Images were labeled using Intel CVAT (Computer Vision Annotation Tool) We trained this model for facial detection. We have deployed our trained model and software in the Nvidia Jetson Nano Developer kit. Model code is written in Pytorch's deep learning framework. The programming language used is Python.
Knowledge Inheritance for Pre-trained Language Models
Authors: Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie zhou
Date: 2021-05-28 00:00:00
Proceeding: NAACL 2022 7
Tasks: Domain Adaptation, Knowledge Distillation, Self-Learning, Transfer Learning

Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PLMs. However, it requires tremendous computational resources to train a large-scale PLM, which may be practically unaffordable. In addition, existing large-scale PLMs are mainly trained from scratch individually, ignoring that many well-trained PLMs are available. To this end, we explore the question how could existing PLMs benefit training large-scale PLMs in future. Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs. Experimental results demonstrate the superiority of KI in training efficiency. We also conduct empirical analyses to explore the effects of teacher PLMs' pre-training settings, including model architecture, pre-training data, etc. Finally, we show that KI could be applied to domain adaptation and knowledge transfer.
Accelerating BERT Inference for Sequence Labeling via Early-Exit
Authors: Xiaonan Li, Yunfan Shao, Tianxiang Sun, Hang Yan, Xipeng Qiu, Xuanjing Huang
Date: 2021-05-28 00:00:00
Proceeding: ACL 2021 5
Tasks: Sentence

Both performance and efficiency are crucial factors for sequence labeling tasks in many real-world scenarios. Although the pre-trained models (PTMs) have significantly improved the performance of various sequence labeling tasks, their computational cost is expensive. To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks. However, existing early-exit mechanisms are specifically designed for sequence-level tasks, rather than sequence labeling. In this paper, we first propose a simple extension of sentence-level early-exit for sequence labeling tasks. To further reduce the computational cost, we also propose a token-level early-exit mechanism that allows partial tokens to exit early at different layers. Considering the local dependency inherent in sequence labeling, we employed a window-based criterion to decide for a token whether or not to exit. The token-level early-exit brings the gap between training and inference, so we introduce an extra self-sampling fine-tuning stage to alleviate it. The extensive experiments on three popular sequence labeling tasks show that our approach can save up to 66%-75% inference cost with minimal performance degradation. Compared with competitive compressed models such as DistilBERT, our approach can achieve better performance under the same speed-up ratios of 2X, 3X, and 4X.
Learning Approximate and Exact Numeral Systems via Reinforcement Learning
Authors: Emil Carlsson, Devdatt Dubhashi, Fredrik D. Johansson
Date: 2021-05-28 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent work (Xu et al., 2020) has suggested that numeral systems in different languages are shaped by a functional need for efficient communication in an information-theoretic sense. Here we take a learning-theoretic approach and show how efficient communication emerges via reinforcement learning. In our framework, two artificial agents play a Lewis signaling game where the goal is to convey a numeral concept. The agents gradually learn to communicate using reinforcement learning and the resulting numeral systems are shown to be efficient in the information-theoretic framework of Regier et al. (2015); Gibson et al. (2017). They are also shown to be similar to human numeral systems of same type. Our results thus provide a mechanistic explanation via reinforcement learning of the recent results in Xu et al. (2020) and can potentially be generalized to other semantic domains.
Lightweight Cross-Lingual Sentence Representation Learning
Authors: Zhuoyuan Mao, Prakhar Gupta, Pei Wang, Chenhui Chu, Martin Jaggi, Sadao Kurohashi
Date: 2021-05-28 00:00:00
Proceeding: ACL 2021 5
Tasks: Contrastive Learning, Document Classification, Language Modeling, Language Modelling, Representation Learning, Retrieval, Sentence, Sentence Retrieval

Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on downstream tasks. However, further increases and modifications based on such large-scale models are usually impractical due to memory limitations. In this work, we introduce a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations. We explore different training tasks and observe that current cross-lingual training tasks leave a lot to be desired for this shallow architecture. To ameliorate this, we propose a novel cross-lingual language model, which combines the existing single-word masked language model with the newly proposed cross-lingual token-level reconstruction task. We further augment the training task by the introduction of two computationally-lite sentence-level contrastive learning tasks to enhance the alignment of cross-lingual sentence representation space, which compensates for the learning bottleneck of the lightweight transformer for generative tasks. Our comparisons with competing models on cross-lingual sentence retrieval and multilingual document classification confirm the effectiveness of the newly proposed training tasks for a shallow model.
Neonatal seizure detection from raw multi-channel EEG using a fully convolutional architecture
Authors: Alison O'Shea, Gordon Lightbody, Geraldine Boylan, Andriy Temko
Date: 2021-05-28 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Seizure Detection

A deep learning classifier for detecting seizures in neonates is proposed. This architecture is designed to detect seizure events from raw electroencephalogram (EEG) signals as opposed to the state-of-the-art hand engineered feature-based representation employed in traditional machine learning based solutions. The seizure detection system utilises only convolutional layers in order to process the multichannel time domain signal and is designed to exploit the large amount of weakly labelled data in the training stage. The system performance is assessed on a large database of continuous EEG recordings of 834h in duration; this is further validated on a held-out publicly available dataset and compared with two baseline SVM based systems. The developed system achieves a 56% relative improvement with respect to a feature-based state-of-the art baseline, reaching an AUC of 98.5%; this also compares favourably both in terms of performance and run-time. The effect of varying architectural parameters is thoroughly studied. The performance improvement is achieved through novel architecture design which allows more efficient usage of available training data and end-to-end optimisation from the front-end feature extraction to the back-end classification. The proposed architecture opens new avenues for the application of deep learning to neonatal EEG, where the performance becomes a function of the amount of training data with less dependency on the availability of precise clinical labels.
A General Taylor Framework for Unifying and Revisiting Attribution Methods
Authors: Huiqi Deng, Na Zou, Mengnan Du, Weifu Chen, Guocan Feng, Xia Hu
Date: 2021-05-28 00:00:00
Tasks: Benchmarking, Decision Making

Attribution methods provide an insight into the decision-making process of machine learning models, especially deep neural networks, by assigning contribution scores to each individual feature. However, the attribution problem has not been well-defined, which lacks a unified guideline to the contribution assignment process. Furthermore, existing attribution methods often built upon various empirical intuitions and heuristics. There still lacks a general theoretical framework that not only can offer a good description of the attribution problem, but also can be applied to unifying and revisiting existing attribution methods. To bridge the gap, in this paper, we propose a Taylor attribution framework, which models the attribution problem as how to decide individual payoffs in a coalition. Then, we reformulate fourteen mainstream attribution methods into the Taylor framework and analyze these attribution methods in terms of rationale, fidelity, and limitation in the framework. Moreover, we establish three principles for a good attribution in the Taylor attribution framework, i.e., low approximation error, correct Taylor contribution assignment, and unbiased baseline selection. Finally, we empirically validate the Taylor reformulations and reveal a positive correlation between the attribution performance and the number of principles followed by the attribution method via benchmarking on real-world datasets.
A proxemics game between festival visitors and an industrial robot
Authors: Brigitte Krenn, Stephanie Gross, Bernhard Dieber, Horst Pichler, Kathrin Meyer
Date: 2021-05-28 00:00:00

With increased applications of collaborative robots (cobots) in industrial workplaces, behavioural effects of human-cobot interactions need to be further investigated. This is of particular importance as nonverbal behaviours of collaboration partners in human-robot teams significantly influence the experience of the human interaction partners and the success of the collaborative task. During the Ars Electronica 2020 Festival for Art, Technology and Society (Linz, Austria), we invited visitors to exploratively interact with an industrial robot, exhibiting restricted interaction capabilities: extending and retracting its arm, depending on the movements of the volunteer. The movements of the arm were pre-programmed and telecontrolled for safety reasons (which was not obvious to the participants). We recorded video data of these interactions and investigated general nonverbal behaviours of the humans interacting with the robot, as well as nonverbal behaviours of people in the audience. Our results showed that people were more interested in exploring the robot's action and perception capabilities than just reproducing the interaction game as introduced by the instructors. We also found that the majority of participants interacting with the robot approached it up to a distance which would be perceived as threatening or intimidating, if it were a human interaction partner. Regarding bystanders, we found examples where people made movements as if trying out variants of the current participant's behaviour.