pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification
Authors: Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Tom Vu, Likun Lei
Date: 2022-04-02 00:00:00
Tasks: Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories. Recently, large pre-trained Transformer models have made significant performance improvements in XMTC, which typically use the embedding of the special CLS token to represent the entire document semantics as a global feature vector, and match it against candidate labels. However, we argue that such a global feature vector may not be sufficient to represent different granularity levels of semantics in the document, and that complementing it with the local word-level features could bring additional gains. Based on this insight, we propose an approach that combines both the local and global features produced by Transformer models to improve the prediction power of the classifier. Our experiments show that the proposed model either outperforms or is comparable to the state-of-the-art methods on benchmark datasets.
Risk-Aware Control and Optimization for High-Renewable Power Grids
Authors: Neil Barry, Minas Chatzos, Wenbo Chen, Dahye Han, Chaofan Huang, Roshan Joseph, Michael Klamkin, Seonho Park, Mathieu Tanneau, Pascal Van Hentenryck, Shangkun Wang, Hanyu Zhang, Haoruo Zhao
Date: 2022-04-02 00:00:00
Tasks: Uncertainty Quantification, Vocal Bursts Intensity Prediction

The transition of the electrical power grid from fossil fuels to renewable sources of energy raises fundamental challenges to the market-clearing algorithms that drive its operations. Indeed, the increased stochasticity in load and the volatility of renewable energy sources have led to significant increases in prediction errors, affecting the reliability and efficiency of existing deterministic optimization models. The RAMC project was initiated to investigate how to move from this deterministic setting into a risk-aware framework where uncertainty is quantified explicitly and incorporated in the market-clearing optimizations. Risk-aware market-clearing raises challenges on its own, primarily from a computational standpoint. This paper reviews how RAMC approaches risk-aware market clearing and presents some of its innovations in uncertainty quantification, optimization, and machine learning. Experimental results on real networks are presented.
A Dynamic Meta-Learning Model for Time-Sensitive Cold-Start Recommendations
Authors: Krishna Prasad Neupane, Ervine Zheng, Yu Kong, Qi Yu
Date: 2022-04-03 00:00:00
Tasks: Meta-Learning, Recommendation Systems

We present a novel dynamic recommendation model that focuses on users who have interactions in the past but turn relatively inactive recently. Making effective recommendations to these time-sensitive cold-start users is critical to maintain the user base of a recommender system. Due to the sparse recent interactions, it is challenging to capture these users' current preferences precisely. Solely relying on their historical interactions may also lead to outdated recommendations misaligned with their recent interests. The proposed model leverages historical and current user-item interactions and dynamically factorizes a user's (latent) preference into time-specific and time-evolving representations that jointly affect user behaviors. These latent factors further interact with an optimized item embedding to achieve accurate and timely recommendations. Experiments over real-world data help demonstrate the effectiveness of the proposed time-sensitive cold-start recommendation model.
Towards Web Phishing Detection Limitations and Mitigation
Authors: Alsharif Abuadbba, Shuo Wang, Mahathir Almashor, Muhammed Ejaz Ahmed, Raj Gaire, Seyit Camtepe, Surya Nepal
Date: 2022-04-03 00:00:00
Tasks: Attribute

Web phishing remains a serious cyber threat responsible for most data breaches. Machine Learning (ML)-based anti-phishing detectors are seen as an effective countermeasure, and are increasingly adopted by web-browsers and software products. However, with an average of 10K phishing links reported per hour to platforms such as PhishTank and VirusTotal (VT), the deficiencies of such ML-based solutions are laid bare. We first explore how phishing sites bypass ML-based detection with a deep dive into 13K phishing pages targeting major brands such as Facebook. Results show successful evasion is caused by: (1) use of benign services to obscure phishing URLs; (2) high similarity between the HTML structures of phishing and benign pages; (3) hiding the ultimate phishing content within Javascript and running such scripts only on the client; (4) looking beyond typical credentials and credit cards for new content such as IDs and documents; (5) hiding phishing content until after human interaction. We attribute the root cause to the dependency of ML-based models on the vertical feature space (webpage content). These solutions rely only on what phishers present within the page itself. Thus, we propose Anti-SubtlePhish, a more resilient model based on logistic regression. The key augmentation is the inclusion of a horizontal feature space, which examines correlation variables between the final render of suspicious pages against what trusted services have recorded (e.g., PageRank). To defeat (1) and (2), we correlate information between WHOIS, PageRank, and page analytics. To combat (3), (4) and (5), we correlate features after rendering the page. Experiments with 100K phishing/benign sites show promising accuracy (98.8%). We also obtained 100% accuracy against 0-day phishing pages that were manually crafted, comparing well to the 0% recorded by VT vendors over the first four days.
A Computational Analysis of Pitch Drift in Unaccompanied Solo Singing using DBSCAN Clustering
Authors: Sepideh Shafiei, S. Hakam
Date: 2022-04-03 00:00:00
Tasks: Clustering

Unaccompanied vocalists usually change the tuning unintentionally and end up with a higher or lower pitch than the starting point during a long performance. This phenomenon is called pitch drift, which is dependent on various elements, such as the skill of the performer, and the length and difficulty of the performance. In this paper, we propose a computational method for measuring pitch drift in the course of an unaccompanied vocal performance, using pitch histogram and DBSCAN clustering.
On Efficiently Acquiring Annotations for Multilingual Models
Authors: Joel Ruben Antony Moniz, Barun Patra, Matthew R. Gormley
Date: 2022-04-03 00:00:00
Proceeding: ACL 2022 5
Tasks: Active Learning, Dependency Parsing

When tasked with supporting multiple languages for a given problem, two approaches have arisen: training a model for each language with the annotation budget divided equally among them, and training on a high-resource language followed by zero-shot transfer to the remaining languages. In this work, we show that the strategy of joint learning across multiple languages using a single model performs substantially better than the aforementioned alternatives. We also demonstrate that active learning provides additional, complementary benefits. We show that this simple approach enables the model to be data efficient by allowing it to arbitrate its annotation budget to query languages it is less certain on. We illustrate the effectiveness of our proposed method on a diverse set of tasks: a classification task with 4 languages, a sequence tagging task with 4 languages and a dependency parsing task with 5 languages. Our proposed method, whilst simple, substantially outperforms the other viable alternatives for building a model in a multilingual setting under constrained budgets.
A Differentially Private Framework for Deep Learning with Convexified Loss Functions
Authors: Zhigang Lu, Hassan Jameel Asghar, Mohamed Ali Kaafar, Darren Webb, Peter Dickinson
Date: 2022-04-03 00:00:00
Tasks: Sensitivity

Differential privacy (DP) has been applied in deep learning for preserving privacy of the underlying training sets. Existing DP practice falls into three categories - objective perturbation, gradient perturbation and output perturbation. They suffer from three main problems. First, conditions on objective functions limit objective perturbation in general deep learning tasks. Second, gradient perturbation does not achieve a satisfactory privacy-utility trade-off due to over-injected noise in each epoch. Third, high utility of the output perturbation method is not guaranteed because of the loose upper bound on the global sensitivity of the trained model parameters as the noise scale parameter. To address these problems, we analyse a tighter upper bound on the global sensitivity of the model parameters. Under a black-box setting, based on this global sensitivity, to control the overall noise injection, we propose a novel output perturbation framework by injecting DP noise into a randomly sampled neuron (via the exponential mechanism) at the output layer of a baseline non-private neural network trained with a convexified loss function. We empirically compare the privacy-utility trade-off, measured by accuracy loss to baseline non-private models and the privacy leakage against black-box membership inference (MI) attacks, between our framework and the open-source differentially private stochastic gradient descent (DP-SGD) approaches on six commonly used real-world datasets. The experimental evaluations show that, when the baseline models have observable privacy leakage under MI attacks, our framework achieves a better privacy-utility trade-off than existing DP-SGD implementations, given an overall privacy budget $\epsilon \leq 1$ for a large number of queries.
Understanding the unstable convergence of gradient descent
Authors: Kwangjun Ahn, Jingzhao Zhang, Suvrit Sra
Date: 2022-04-03 00:00:00

Most existing analyses of (stochastic) gradient descent rely on the condition that for $L$-smooth costs, the step size is less than $2/L$. However, many works have observed that in machine learning applications step sizes often do not fulfill this condition, yet (stochastic) gradient descent still converges, albeit in an unstable manner. We investigate this unstable convergence phenomenon from first principles, and discuss key causes behind it. We also identify its main characteristics, and how they interrelate based on both theory and experiments, offering a principled view toward understanding the phenomenon.
A Survey on Machine Learning Solutions for Graph Pattern Extraction
Authors: Kai Siong Yow, Ningyi Liao, Siqiang Luo, Reynold Cheng, Chenhao Ma, Xiaolin Han
Date: 2022-04-03 00:00:00
Tasks: Community Detection, Community Search, Survey

A subgraph is constructed by using a subset of vertices and edges of a given graph. There exist many graph properties that are hereditary for subgraphs. Hence, researchers from different communities have paid a great deal of attention in studying numerous subgraph problems, on top of the ordinary graph problems. Many algorithms are proposed in studying subgraph problems, where one common approach is by extracting the patterns and structures of a given graph. Due to the complex structures of certain types of graphs and to improve overall performances of the existing frameworks, machine learning techniques have recently been employed in dealing with various subgraph problems. In this article, we present a comprehensive review on five well known subgraph problems that have been tackled by using machine learning methods. They are subgraph isomorphism (both counting and matching), maximum common subgraph, community detection and community search problems. We provide an outline of each proposed method, and examine its designs and performances. We also explore non-learning-based algorithms for each problem and a brief discussion is given. We then suggest some promising research directions in this area, hoping that relevant subgraph problems can be tackled by using a similar strategy. Since there is a huge growth in employing machine learning techniques in recent years, we believe that this survey will serve as a good reference point to relevant research communities.
Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
Authors: Mahima Pushkarna, Andrew Zaldivar, Oddur Kjartansson
Date: 2022-04-03 00:00:00

As research and industry moves towards large-scale models capable of numerous downstream tasks, the complexity of understanding multi-modal datasets that give nuance to models rapidly increases. A clear and thorough understanding of a dataset's origins, development, intent, ethical considerations and evolution becomes a necessary step for the responsible and informed deployment of models, especially those in people-facing contexts and high-risk domains. However, the burden of this understanding often falls on the intelligibility, conciseness, and comprehensiveness of the documentation. It requires consistency and comparability across the documentation of all datasets involved, and as such documentation must be treated as a user-centric product in and of itself. In this paper, we propose Data Cards for fostering transparent, purposeful and human-centered documentation of datasets within the practical contexts of industry and research. Data Cards are structured summaries of essential facts about various aspects of ML datasets needed by stakeholders across a dataset's lifecycle for responsible AI development. These summaries provide explanations of processes and rationales that shape the data and consequently the models, such as upstream sources, data collection and annotation methods; training and evaluation methods, intended use; or decisions affecting model performance. We also present frameworks that ground Data Cards in real-world utility and human-centricity. Using two case studies, we report on desirable characteristics that support adoption across domains, organizational structures, and audience groups. Finally, we present lessons learned from deploying over 20 Data Cards.
A System for Interactive Examination of Learned Security Policies
Authors: Kim Hammar, Rolf Stadler
Date: 2022-04-03 00:00:00

We present a system for interactive examination of learned security policies. It allows a user to traverse episodes of Markov decision processes in a controlled manner and to track the actions triggered by security policies. Similar to a software debugger, a user can continue or or halt an episode at any time step and inspect parameters and probability distributions of interest. The system enables insight into the structure of a given policy and in the behavior of a policy in edge cases. We demonstrate the system with a network intrusion use case. We examine the evolution of an IT infrastructure's state and the actions prescribed by security policies while an attack occurs. The policies for the demonstration have been obtained through a reinforcement learning approach that includes a simulation system where policies are incrementally learned and an emulation system that produces statistics that drive the simulation runs.
Proceedings of TDA: Applications of Topological Data Analysis to Data Science, Artificial Intelligence, and Machine Learning Workshop at SDM 2022
Authors: R. W. R. Darling, John A. Emanuello, Emilie Purvine, Ahmad Ridley
Date: 2022-04-03 00:00:00
Tasks: Topological Data Analysis

Topological Data Analysis (TDA) is a rigorous framework that borrows techniques from geometric and algebraic topology, category theory, and combinatorics in order to study the "shape" of such complex high-dimensional data. Research in this area has grown significantly over the last several years bringing a deeply rooted theory to bear on practical applications in areas such as genomics, natural language processing, medicine, cybersecurity, energy, and climate change. Within some of these areas, TDA has also been used to augment AI and ML techniques. We believe there is further utility to be gained in this space that can be facilitated by a workshop bringing together experts (both theorists and practitioners) and non-experts. Currently there is an active community of pure mathematicians with research interests in developing and exploring the theoretical and computational aspects of TDA. Applied mathematicians and other practitioners are also present in community but do not represent a majority. This speaks to the primary aim of this workshop which is to grow a wider community of interest in TDA. By fostering meaningful exchanges between these groups, from across the government, academia, and industry, we hope to create new synergies that can only come through building a mutual comprehensive awareness of the problem and solution spaces.
Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusion
Authors: Tianchen Ji, Arun Narenthiran Sivakumar, Girish Chowdhary, Katherine Driggs-Campbell
Date: 2022-04-03 00:00:00
Tasks: Anomaly Detection, Robot Navigation, Sensor Fusion

Despite the rapid advancement of navigation algorithms, mobile robots often produce anomalous behaviors that can lead to navigation failures. The ability to detect such anomalous behaviors is a key component in modern robots to achieve high-levels of autonomy. Reactive anomaly detection methods identify anomalous task executions based on the current robot state and thus lack the ability to alert the robot before an actual failure occurs. Such an alert delay is undesirable due to the potential damage to both the robot and the surrounding objects. We propose a proactive anomaly detection network (PAAD) for robot navigation in unstructured and uncertain environments. PAAD predicts the probability of future failure based on the planned motions from the predictive controller and the current observation from the perception module. Multi-sensor signals are fused effectively to provide robust anomaly detection in the presence of sensor occlusion as seen in field environments. Our experiments on field robot data demonstrates superior failure identification performance than previous methods, and that our model can capture anomalous behaviors in real-time while maintaining a low false detection rate in cluttered fields. Code, dataset, and video are available at https://github.com/tianchenji/PAAD
Best-Response Bayesian Reinforcement Learning with Bayes-adaptive POMDPs for Centaurs
Authors: Mustafa Mert Çelikok, Frans A. Oliehoek, Samuel Kaski
Date: 2022-04-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

Centaurs are half-human, half-AI decision-makers where the AI's goal is to complement the human. To do so, the AI must be able to recognize the goals and constraints of the human and have the means to help them. We present a novel formulation of the interaction between the human and the AI as a sequential game where the agents are modelled using Bayesian best-response models. We show that in this case the AI's problem of helping bounded-rational humans make better decisions reduces to a Bayes-adaptive POMDP. In our simulated experiments, we consider an instantiation of our framework for humans who are subjectively optimistic about the AI's future behaviour. Our results show that when equipped with a model of the human, the AI can infer the human's bounds and nudge them towards better decisions. We discuss ways in which the machine can learn to improve upon its own limitations as well with the help of the human. We identify a novel trade-off for centaurs in partially observable tasks: for the AI's actions to be acceptable to the human, the machine must make sure their beliefs are sufficiently aligned, but aligning beliefs might be costly. We present a preliminary theoretical analysis of this trade-off and its dependence on task structure.
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Authors: Kushal Arora, Layla El Asri, Hareesh Bahuleyan, Jackie Chi Kit Cheung
Date: 2022-04-03 00:00:00
Proceeding: Findings (ACL) 2022 5
Tasks: Imitation Learning, Text Generation

Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis is that this brittleness of generation models is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this paper, we verify this hypothesis by analyzing exposure bias from an imitation learning perspective. We show that exposure bias leads to an accumulation of errors, analyze why perplexity fails to capture this accumulation, and empirically show that this accumulation results in poor generation quality. Source code to reproduce these experiments is available at https://github.com/kushalarora/quantifying_exposure_bias
Continuous Variable Quantum MNIST Classifiers
Authors: Sophie Choe
Date: 2022-04-04 00:00:00

In this paper, classical and continuous variable (CV) quantum neural network hybrid multiclassifiers are presented using the MNIST dataset. The combination of cutoff dimension and probability measurement method in the CV model allows a quantum circuit to produce output vectors of size equal to n raised to the power of n where n represents cutoff dimension and m, the number of qumodes. They are then translated as one-hot encoded labels, padded with an appropriate number of zeros. The total of eight different classifiers are built using 2,3,...,8 qumodes, based on the binary classifier architecture proposed in Continuous variable quantum neural networks. The displacement gate and the Kerr gate in the CV model allow for the bias addition and nonlinear activation components of classical neural networks to quantum. The classifiers are composed of a classical feedforward neural network, a quantum data encoding circuit, and a CV quantum neural network circuit. On a truncated MNIST dataset of 600 samples, a 4 qumode hybrid classifier achieves 100% training accuracy.
Capturing positive network attributes during the estimation of recursive logit models: A prism-based approach
Authors: Yuki Oyama
Date: 2022-04-04 00:00:00
Tasks: parameter estimation

Although the recursive logit (RL) model has been recently popular and has led to many applications and extensions, an important numerical issue with respect to the computation of value functions remains unsolved. This issue is particularly significant for model estimation, during which the parameters are updated every iteration and may violate the feasibility condition of the value function. To solve this numerical issue of the value function in the model estimation, this study performs an extensive analysis of a prism-constrained RL (Prism-RL) model proposed by Oyama and Hato (2019), which has a path set constrained by the prism defined based upon a state-extended network representation. The numerical experiments have shown two important properties of the Prism-RL model for parameter estimation. First, the prism-based approach enables estimation regardless of the initial and true parameter values, even in cases where the original RL model cannot be estimated due to the numerical problem. We also successfully captured a positive effect of the presence of street green on pedestrian route choice in a real application. Second, the Prism-RL model achieved better fit and prediction performance than the RL model, by implicitly restricting paths with large detour or many loops. Defining the prism-based path set in a data-oriented manner, we demonstrated the possibility of the Prism-RL model describing more realistic route choice behavior. The capture of positive network attributes while retaining the diversity of path alternatives is important in many applications such as pedestrian route choice and sequential destination choice behavior, and thus the prism-based approach significantly extends the practical applicability of the RL model.