pwc-archive/papers-with-abstracts huggingface dataset:
Detecting Requirements Smells With Deep Learning: Experiences, Challenges and Future Work
Authors: Mohammad Kasra Habib, Stefan Wagner, Daniel Graziotin
Date: 2021-08-06 00:00:00
Tasks: Ensemble Learning, Transfer Learning, Word Embeddings

Requirements Engineering (RE) is the initial step towards building a software system. The success or failure of a software project is firmly tied to this phase, based on communication among stakeholders using natural language. The problem with natural language is that it can easily lead to different understandings if it is not expressed precisely by the stakeholders involved, which results in building a product different from the expected one. Previous work proposed to enhance the quality of the software requirements detecting language errors based on ISO 29148 requirements language criteria. The existing solutions apply classical Natural Language Processing (NLP) to detect them. NLP has some limitations, such as domain dependability which results in poor generalization capability. Therefore, this work aims to improve the previous work by creating a manually labeled dataset and using ensemble learning, Deep Learning (DL), and techniques such as word embeddings and transfer learning to overcome the generalization problem that is tied with classical NLP and improve precision and recall metrics using a manually labeled dataset. The current findings show that the dataset is unbalanced and which class examples should be added more. It is tempting to train algorithms even if the dataset is not considerably representative. Whence, the results show that models are overfitting; in Machine Learning this issue is solved by adding more instances to the dataset, improving label quality, removing noise, and reducing the learning algorithms complexity, which is planned for this research.
Transferring Knowledge Distillation for Multilingual Social Event Detection
Authors: Jiaqian Ren, Hao Peng, Lei Jiang, Jia Wu, Yongxin Tong, Lihong Wang, Xu Bai, Bo wang, Qiang Yang
Date: 2021-08-06 00:00:00
Tasks: Cross-Lingual Word Embeddings, Event Detection, Knowledge Distillation, Word Embeddings

Recently published graph neural networks (GNNs) show promising performance at social event detection tasks. However, most studies are oriented toward monolingual data in languages with abundant training samples. This has left the more common multilingual settings and lesser-spoken languages relatively unexplored. Thus, we present a GNN that incorporates cross-lingual word embeddings for detecting events in multilingual data streams. The first exploit is to make the GNN work with multilingual data. For this, we outline a construction strategy that aligns messages in different languages at both the node and semantic levels. Relationships between messages are established by merging entities that are the same but are referred to in different languages. Non-English message representations are converted into English semantic space via the cross-lingual word embeddings. The resulting message graph is then uniformly encoded by a GNN model. In special cases where a lesser-spoken language needs to be detected, a novel cross-lingual knowledge distillation framework, called CLKD, exploits prior knowledge learned from similar threads in English to make up for the paucity of annotated data. Experiments on both synthetic and real-world datasets show the framework to be highly effective at detection in both multilingual data and in languages where training samples are scarce.
Rectified Euler k-means and Beyond
Authors: Yunxia Lin, Songcan Chen
Date: 2021-08-06 00:00:00

Euler k-means (EulerK) first maps data onto the unit hyper-sphere surface of equi-dimensional space via a complex mapping which induces the robust Euler kernel and next employs the popular $k$-means. Consequently, besides enjoying the virtues of k-means such as simplicity and scalability to large data sets, EulerK is also robust to noises and outliers. Although so, the centroids captured by EulerK deviate from the unit hyper-sphere surface and thus in strict distributional sense, actually are outliers. This weird phenomenon also occurs in some generic kernel clustering methods. Intuitively, using such outlier-like centroids should not be quite reasonable but it is still seldom attended. To eliminate the deviation, we propose two Rectified Euler k-means methods, i.e., REK1 and REK2, which retain the merits of EulerK while acquire real centroids residing on the mapped space to better characterize the data structures. Specifically, REK1 rectifies EulerK by imposing the constraint on the centroids while REK2 views each centroid as the mapped image from a pre-image in the original space and optimizes these pre-images in Euler kernel induced space. Undoubtedly, our proposed REKs can methodologically be extended to solve problems of such a category. Finally, the experiments validate the effectiveness of REK1 and REK2.
Machine learning for surface prediction in ACTS
Authors: Benjamin Huth, Andreas Salzburger, Tilo Wettig
Date: 2021-08-06 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

We present an ongoing R&D activity for machine-learning-assisted navigation through detectors to be used for track reconstruction. We investigate different approaches of training neural networks for surface prediction and compare their results. This work is carried out in the context of the ACTS tracking toolkit.
Deriving Disinformation Insights from Geolocalized Twitter Callouts
Authors: David Tuxworth, Dimosthenis Antypas, Luis Espinosa-Anke, Jose Camacho-Collados, Alun Preece, David Rogers
Date: 2021-08-06 00:00:00
Tasks: Language Modelling, Specificity, Word Embeddings

This paper demonstrates a two-stage method for deriving insights from social media data relating to disinformation by applying a combination of geospatial classification and embedding-based language modelling across multiple languages. In particular, the analysis in centered on Twitter and disinformation for three European languages: English, French and Spanish. Firstly, Twitter data is classified into European and non-European sets using BERT. Secondly, Word2vec is applied to the classified texts resulting in Eurocentric, non-Eurocentric and global representations of the data for the three target languages. This comparative analysis demonstrates not only the efficacy of the classification method but also highlights geographic, temporal and linguistic differences in the disinformation-related media. Thus, the contributions of the work are threefold: (i) a novel language-independent transformer-based geolocation method; (ii) an analytical approach that exploits lexical specificity and word embeddings to interrogate user-generated content; and (iii) a dataset of 36 million disinformation related tweets in English, French and Spanish.
Identifiable Energy-based Representations: An Application to Estimating Heterogeneous Causal Effects
Authors: Yao Zhang, Jeroen Berrevoets, Mihaela van der Schaar
Date: 2021-08-06 00:00:00
Tasks: Dimensionality Reduction

Conditional average treatment effects (CATEs) allow us to understand the effect heterogeneity across a large population of individuals. However, typical CATE learners assume all confounding variables are measured in order for the CATE to be identifiable. This requirement can be satisfied by collecting many variables, at the expense of increased sample complexity for estimating CATEs. To combat this, we propose an energy-based model (EBM) that learns a low-dimensional representation of the variables by employing a noise contrastive loss function. With our EBM we introduce a preprocessing step that alleviates the dimensionality curse for any existing learner developed for estimating CATEs. We prove that our EBM keeps the representations partially identifiable up to some universal constant, as well as having universal approximation capability. These properties enable the representations to converge and keep the CATE estimates consistent. Experiments demonstrate the convergence of the representations, as well as show that estimating CATEs on our representations performs better than on the variables or the representations obtained through other dimensionality reduction methods.
Inspecting the Process of Bank Credit Rating via Visual Analytics
Authors: Qiangqiang Liu, Quan Li, Zhihua Zhu, Tangzhi Ye, Xiaojuan Ma
Date: 2021-08-06 00:00:00
Tasks: Management

Bank credit rating classifies banks into different levels based on publicly disclosed and internal information, serving as an important input in financial risk management. However, domain experts have a vague idea of exploring and comparing different bank credit rating schemes. A loose connection between subjective and quantitative analysis and difficulties in determining appropriate indicator weights obscure understanding of bank credit ratings. Furthermore, existing models fail to consider bank types by just applying a unified indicator weight set to all banks. We propose RatingVis to assist experts in exploring and comparing different bank credit rating schemes. It supports interactively inferring indicator weights for banks by involving domain knowledge and considers bank types in the analysis loop. We conduct a case study with real-world bank data to verify the efficacy of RatingVis. Expert feedback suggests that our approach helps them better understand different rating schemes.
Auxiliary Class Based Multiple Choice Learning
Authors: Sihwan Kim, Dae Yon Jung, Taejang Park
Date: 2021-08-06 00:00:00
Tasks: Diversity, Ensemble Learning, image-classification, Image Classification, Multiple-choice

The merit of ensemble learning lies in having different outputs from many individual models on a single input, i.e., the diversity of the base models. The high quality of diversity can be achieved when each model is specialized to different subsets of the whole dataset. Moreover, when each model explicitly knows to which subsets it is specialized, more opportunities arise to improve diversity. In this paper, we propose an advanced ensemble method, called Auxiliary class based Multiple Choice Learning (AMCL), to ultimately specialize each model under the framework of multiple choice learning (MCL). The advancement of AMCL is originated from three novel techniques which control the framework from different directions: 1) the concept of auxiliary class to provide more distinct information through the labels, 2) the strategy, named memory-based assignment, to determine the association between the inputs and the models, and 3) the feature fusion module to achieve generalized features. To demonstrate the performance of our method compared to all variants of MCL methods, we conduct extensive experiments on the image classification and segmentation tasks. Overall, the performance of AMCL exceeds all others in most of the public datasets trained with various networks as members of the ensembles.
Unsupervised Learning of Debiased Representations with Pseudo-Attributes
Authors: Seonguk Seo, Joon-Young Lee, Bohyung Han
Date: 2021-08-06 00:00:00
Proceeding: CVPR 2022 1
Tasks: Attribute

Dataset bias is a critical challenge in machine learning since it often leads to a negative impact on a model due to the unintended decision rules captured by spurious correlations. Although existing works often handle this issue based on human supervision, the availability of the proper annotations is impractical and even unrealistic. To better tackle the limitation, we propose a simple but effective unsupervised debiasing technique. Specifically, we first identify pseudo-attributes based on the results from clustering performed in the feature embedding space even without an explicit bias attribute supervision. Then, we employ a novel cluster-wise reweighting scheme to learn debiased representation; the proposed method prevents minority groups from being discounted for minimizing the overall loss, which is desirable for worst-case generalization. The extensive experiments demonstrate the outstanding performance of our approach on multiple standard benchmarks, even achieving the competitive accuracy to the supervised counterpart.
Is it Fake? News Disinformation Detection on South African News Websites
Authors: Harm de Wet, Vukosi Marivate
Date: 2021-08-06 00:00:00
Tasks: Fake News Detection, Interpretable Machine Learning

Disinformation through fake news is an ongoing problem in our society and has become easily spread through social media. The most cost and time effective way to filter these large amounts of data is to use a combination of human and technical interventions to identify it. From a technical perspective, Natural Language Processing (NLP) is widely used in detecting fake news. Social media companies use NLP techniques to identify the fake news and warn their users, but fake news may still slip through undetected. It is especially a problem in more localised contexts (outside the United States of America). How do we adjust fake news detection systems to work better for local contexts such as in South Africa. In this work we investigate fake news detection on South African websites. We curate a dataset of South African fake news and then train detection models. We contrast this with using widely available fake news datasets (from mostly USA website). We also explore making the datasets more diverse by combining them and observe the differences in behaviour in writing between nations' fake news using interpretable machine learning.
Incremental Feature Learning For Infinite Data
Authors: Armin Sadreddin, Samira Sadaoui
Date: 2021-08-06 00:00:00
Tasks: Fraud Detection, Transfer Learning

This study addresses the actual behavior of the credit-card fraud detection environment where financial transactions containing sensitive data must not be amassed in an enormous amount to conduct learning. We introduce a new adaptive learning approach that adjusts frequently and efficiently to new transaction chunks; each chunk is discarded after each incremental training step. Our approach combines transfer learning and incremental feature learning. The former improves the feature relevancy for subsequent chunks, and the latter, a new paradigm, increases accuracy during training by determining the optimal network architecture dynamically for each new chunk. The architectures of past incremental approaches are fixed; thus, the accuracy may not improve with new chunks. We show the effectiveness and superiority of our approach experimentally on an actual fraud dataset.
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Authors: Kenny Peng, Arunesh Mathur, Arvind Narayanan
Date: 2021-08-06 00:00:00
Tasks: BIG-bench Machine Learning, Management, Person Recognition

Machine learning datasets have elicited concerns about privacy, bias, and unethical applications, leading to the retraction of prominent datasets such as DukeMTMC, MS-Celeb-1M, and Tiny Images. In response, the machine learning community has called for higher ethical standards in dataset creation. To help inform these efforts, we studied three influential but ethically problematic face and person recognition datasets -- Labeled Faces in the Wild (LFW), MS-Celeb-1M, and DukeMTM -- by analyzing nearly 1000 papers that cite them. We found that the creation of derivative datasets and models, broader technological and social change, the lack of clarity of licenses, and dataset management practices can introduce a wide range of ethical concerns. We conclude by suggesting a distributed approach to harm mitigation that considers the entire life cycle of a dataset.
Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents
Authors: Amit Gupte, Alexey Romanov, Sahitya Mantravadi, Dalitso Banda, Jianjie Liu, Raza Khan, Lakshmanan Ramu Meenal, Benjamin Han, Soundar Srinivasan
Date: 2021-08-06 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Optical Character Recognition, Optical Character Recognition (OCR)

Document digitization is essential for the digital transformation of our societies, yet a crucial step in the process, Optical Character Recognition (OCR), is still not perfect. Even commercial OCR systems can produce questionable output depending on the fidelity of the scanned documents. In this paper, we demonstrate an effective framework for mitigating OCR errors for any downstream NLP task, using Named Entity Recognition (NER) as an example. We first address the data scarcity problem for model training by constructing a document synthesis pipeline, generating realistic but degraded data with NER labels. We measure the NER accuracy drop at various degradation levels and show that a text restoration model, trained on the degraded data, significantly closes the NER accuracy gaps caused by OCR errors, including on an out-of-domain dataset. For the benefit of the community, we have made the document synthesis pipeline available as an open-source project.
RIS-assisted UAV Communications for IoT with Wireless Power Transfer Using Deep Reinforcement Learning
Authors: Khoi Khac Nguyen, Antonino Masaracchia, Tan Do-Duy, H. Vincent Poor, Trung Q. Duong
Date: 2021-08-05 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Many of the devices used in Internet-of-Things (IoT) applications are energy-limited, and thus supplying energy while maintaining seamless connectivity for IoT devices is of considerable importance. In this context, we propose a simultaneous wireless power transfer and information transmission scheme for IoT devices with support from reconfigurable intelligent surface (RIS)-aided unmanned aerial vehicle (UAV) communications. In particular, in a first phase, IoT devices harvest energy from the UAV through wireless power transfer; and then in a second phase, the UAV collects data from the IoT devices through information transmission. To characterise the agility of the UAV, we consider two scenarios: a hovering UAV and a mobile UAV. Aiming at maximizing the total network sum-rate, we jointly optimize the trajectory of the UAV, the energy harvesting scheduling of IoT devices, and the phaseshift matrix of the RIS. We formulate a Markov decision process and propose two deep reinforcement learning algorithms to solve the optimization problem of maximizing the total network sum-rate. Numerical results illustrate the effectiveness of the UAV's flying path optimization and the network's throughput of our proposed techniques compared with other benchmark schemes. Given the strict requirements of the RIS and UAV, the significant improvement in processing time and throughput performance demonstrates that our proposed scheme is well applicable for practical IoT applications.
Interpolation can hurt robust generalization even when there is no noise
Authors: Konstantin Donhauser, Alexandru Ţifrea, Michael Aerni, Reinhard Heckel, Fanny Yang
Date: 2021-08-05 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: regression

Numerous recent works show that overparameterization implicitly reduces variance for min-norm interpolators and max-margin classifiers. These findings suggest that ridge regularization has vanishing benefits in high dimensions. We challenge this narrative by showing that, even in the absence of noise, avoiding interpolation through ridge regularization can significantly improve generalization. We prove this phenomenon for the robust risk of both linear regression and classification and hence provide the first theoretical result on robust overfitting.
Understanding Human Innate Immune System Dependencies using Graph Neural Networks
Authors: Shagufta Henna
Date: 2021-08-05 00:00:00
Tasks: Graph Neural Network

Since the rapid outbreak of Covid-19 and with no approved vaccines to date, profound research interest has emerged to understand the innate immune response to viruses. This understanding can help to inhibit virus replication, prolong adaptive immune response, accelerated virus clearance, and tissue recovery, a key milestone to propose a vaccine to combat coronaviruses (CoVs), e.g., Covid-19. Although an innate immune system triggers inflammatory responses against CoVs upon recognition of viruses, however, a vaccine is the ultimate protection against CoV spread. The development of this vaccine is time-consuming and requires a deep understanding of the innate immune response system. In this work, we propose a graph neural network-based model that exploits the interactions between pattern recognition receptors (PRRs), i.e., the human immune response system. These interactions can help to recognize pathogen-associated molecular patterns (PAMPs) to predict the activation requirements of each PRR. The immune response information of each PRR is derived from combining its historical PAMPs activation coupled with the modeled effect on the same from PRRs in its neighborhood. On one hand, this work can help to understand how long Covid-19 can confer immunity where a strong immune response means people already been infected can safely return to work. On the other hand, this GNN-based understanding can also abode well for vaccine development efforts. Our proposal has been evaluated using CoVs immune response dataset, with results showing an average IFNs activation prediction accuracy of 90%, compared to 85% using feed-forward neural networks.
Enterprise Analytics using Graph Database and Graph-based Deep Learning
Authors: Shagufta Henna, Shyam Krishnan Kalliadan
Date: 2021-08-05 00:00:00
Tasks: Attribute, Binary Classification, Deep Learning, Management

In a business-to-business (B2B) customer relationship management (CRM) use case, each client is a potential business organization/company with a solid business strategy and focused and rational decisions. This paper introduces a graph-based analytics approach to improve CRM within a B2B environment. In our approach, in the first instance, we have designed a graph database using the Neo4j platform. Secondly, the graph database has been investigated by using data mining and exploratory analysis coupled with cypher graph query language. Specifically, we have applied the graph convolution network (GCN) to enable CRM analytics to forecast sales. This is the first step towards a GCN-based binary classification based on graph databases in the domain of B2B CRM. We evaluate the performance of the proposed GCN model on graph databases and compare it with Random Forest (RF), Convolutional Neural Network (CNN), and Artificial Neural Network (ANN). The proposed GCN approach is further augmented with the shortest path and eigenvector centrality attribute to significantly improve the accuracy of sales prediction. Experimental results reveal that the proposed graph-based deep learning approach outperforms the Random Forests (RsF) and two deep learning models, i.e., CNN and ANN under different combinations of graph features.