pwc-archive/papers-with-abstracts huggingface dataset:
A Finer Calibration Analysis for Adversarial Robustness
Authors: Pranjal Awasthi, Anqi Mao, Mehryar Mohri, Yutao Zhong
Date: 2021-05-04 00:00:00
Tasks: Adversarial Robustness, BIG-bench Machine Learning, Robust classification

We present a more general analysis of $H$-calibration for adversarially robust classification. By adopting a finer definition of calibration, we can cover settings beyond the restricted hypothesis sets studied in previous work. In particular, our results hold for most common hypothesis sets used in machine learning. We both fix some previous calibration results (Bao et al., 2020) and generalize others (Awasthi et al., 2021). Moreover, our calibration results, combined with the previous study of consistency by Awasthi et al. (2021), also lead to more general $H$-consistency results covering common hypothesis sets.
Apparel Recommender System based on Bilateral image shape features
Authors: Yichi Lu, Mingtian Gao, Ryosuke Saga
Date: 2021-05-04 00:00:00
Tasks: Recommendation Systems

Probabilistic matrix factorization (PMF) is a well-known model of recommender systems. With the development of image recognition technology, some PMF recommender systems that combine images have emerged. Some of these systems use the image shape features of the recommended products to achieve better results compared to those of the traditional PMF. However, in the existing methods, no PMF recommender system can combine the image features of products previously purchased by customers and of recommended products. Thus, this study proposes a novel probabilistic model that integrates double convolutional neural networks (CNNs) into PMF. For apparel goods, two trained CNNs from the image shape features of users and items are combined, and the latent variables of users and items are optimized based on the vectorized features of CNNs and ratings. Extensive experiments show that our model predicts outcome more accurately than do other recommender models.
Neural Weighted A*: Learning Graph Costs and Heuristics with Differentiable Anytime A*
Authors: Alberto Archetti, Marco Cannici, Matteo Matteucci
Date: 2021-05-04 00:00:00

Recently, the trend of incorporating differentiable algorithms into deep learning architectures arose in machine learning research, as the fusion of neural layers and algorithmic layers has been beneficial for handling combinatorial data, such as shortest paths on graphs. Recent works related to data-driven planning aim at learning either cost functions or heuristic functions, but not both. We propose Neural Weighted A*, a differentiable anytime planner able to produce improved representations of planar maps as graph costs and heuristics. Training occurs end-to-end on raw images with direct supervision on planning examples, thanks to a differentiable A* solver integrated into the architecture. More importantly, the user can trade off planning accuracy for efficiency at run-time, using a single, real-valued parameter. The solution suboptimality is constrained within a linear bound equal to the optimal path cost multiplied by the tradeoff parameter. We experimentally show the validity of our claims by testing Neural Weighted A* against several baselines, introducing a novel, tile-based navigation dataset. We outperform similar architectures in planning accuracy and efficiency.
Intelligent Zero Trust Architecture for 5G/6G Networks: Principles, Challenges, and the Role of Machine Learning in the context of O-RAN
Authors: Keyvan Ramezanpour, Jithin Jagannath
Date: 2021-05-04 00:00:00
Tasks: Edge-computing, Position

In this position paper, we discuss the critical need for integrating zero trust (ZT) principles into next-generation communication networks (5G/6G). We highlight the challenges and introduce the concept of an intelligent zero trust architecture (i-ZTA) as a security framework in 5G/6G networks with untrusted components. While network virtualization, software-defined networking (SDN), and service-based architectures (SBA) are key enablers of 5G networks, operating in an untrusted environment has also become a key feature of the networks. Further, seamless connectivity to a high volume of devices has broadened the attack surface on information infrastructure. Network assurance in a dynamic untrusted environment calls for revolutionary architectures beyond existing static security frameworks. To the best of our knowledge, this is the first position paper that presents the architectural concept design of an i-ZTA upon which modern artificial intelligence (AI) algorithms can be developed to provide information security in untrusted networks. We introduce key ZT principles as real-time Monitoring of the security state of network assets, Evaluating the risk of individual access requests, and Deciding on access authorization using a dynamic trust algorithm, called MED components. To ensure ease of integration, the envisioned architecture adopts an SBA-based design, similar to the 3GPP specification of 5G networks, by leveraging the open radio access network (O-RAN) architecture with appropriate real-time engines and network interfaces for collecting necessary machine learning data. Therefore, this work provides novel research directions to design machine learning based components that contribute towards i-ZTA for the future 5G/6G networks.
Sentiment and Emotion Classification of Epidemic Related Bilingual data from Social Media
Authors: Muhammad Zain Ali, Kashif Javed, Ehsan ul Haq, Anoshka Tariq
Date: 2021-05-04 00:00:00
Tasks: Emotion Classification, General Classification, Sentiment Analysis

In recent years, sentiment analysis and emotion classification are two of the most abundantly used techniques in the field of Natural Language Processing (NLP). Although sentiment analysis and emotion classification are used commonly in applications such as analyzing customer reviews, the popularity of candidates contesting in elections, and comments about various sporting events; however, in this study, we have examined their application for epidemic outbreak detection. Early outbreak detection is the key to deal with epidemics effectively, however, the traditional ways of outbreak detection are time-consuming which inhibits prompt response from the respective departments. Social media platforms such as Twitter, Facebook, Instagram, etc. allow the users to express their thoughts related to different aspects of life, and therefore, serve as a substantial source of information in such situations. The proposed study exploits the bilingual (Urdu and English) data from Twitter and NEWS websites related to the dengue epidemic in Pakistan, and sentiment analysis and emotion classification are performed to acquire deep insights from the data set for gaining a fair idea related to an epidemic outbreak. Machine learning and deep learning algorithms have been used to train and implement the models for the execution of both tasks. The comparative performance of each model has been evaluated using accuracy, precision, recall, and f1-measure.
On the Sample Complexity of Rank Regression from Pairwise Comparisons
Authors: Berkan Kadioglu, Peng Tian, Jennifer Dy, Deniz Erdogmus, Stratis Ioannidis
Date: 2021-05-04 00:00:00
Tasks: regression

We consider a rank regression setting, in which a dataset of $N$ samples with features in $\mathbb{R}^d$ is ranked by an oracle via $M$ pairwise comparisons. Specifically, there exists a latent total ordering of the samples; when presented with a pair of samples, a noisy oracle identifies the one ranked higher with respect to the underlying total ordering. A learner observes a dataset of such comparisons and wishes to regress sample ranks from their features. We show that to learn the model parameters with $\epsilon > 0$ accuracy, it suffices to conduct $M \in \Omega(dN\log^3 N/\epsilon^2)$ comparisons uniformly at random when $N$ is $\Omega(d/\epsilon^2)$.
Online Transfer Learning: Negative Transfer and Effect of Prior Knowledge
Authors: Xuetong Wu, Jonathan H. Manton, Uwe Aickelin, Jingge Zhu
Date: 2021-05-04 00:00:00
Tasks: Transfer Learning

Transfer learning is a machine learning paradigm where the knowledge from one task is utilized to resolve the problem in a related task. On the one hand, it is conceivable that knowledge from one task could be useful for solving a related problem. On the other hand, it is also recognized that if not executed properly, transfer learning algorithms could in fact impair the learning performance instead of improving it - commonly known as "negative transfer". In this paper, we study the online transfer learning problems where the source samples are given in an offline way while the target samples arrive sequentially. We define the expected regret of the online transfer learning problem and provide upper bounds on the regret using information-theoretic quantities. We also obtain exact expressions for the bounds when the sample size becomes large. Examples show that the derived bounds are accurate even for small sample sizes. Furthermore, the obtained bounds give valuable insight on the effect of prior knowledge for transfer learning in our formulation. In particular, we formally characterize the conditions under which negative transfer occurs.
mil-benchmarks: Standardized Evaluation of Deep Multiple-Instance Learning Techniques
Authors: Daniel Grahn
Date: 2021-05-04 00:00:00
Tasks: Multiple Instance Learning, Weakly-supervised Learning

Multiple-instance learning is a subset of weakly supervised learning where labels are applied to sets of instances rather than the instances themselves. Under the standard assumption, a set is positive only there is if at least one instance in the set which is positive. This paper introduces a series of multiple-instance learning benchmarks generated from MNIST, Fashion-MNIST, and CIFAR10. These benchmarks test the standard, presence, absence, and complex assumptions and provide a framework for future benchmarks to be distributed. I implement and evaluate several multiple-instance learning techniques against the benchmarks. Further, I evaluate the Noisy-And method with label noise and find mixed results with different datasets. The models are implemented in TensorFlow 2.4.1 and are available on GitHub. The benchmarks are available from PyPi as mil-benchmarks and on GitHub.
NeuralLog: a Neural Logic Language
Authors: Victor Guimarães, Vítor Santos Costa
Date: 2021-05-04 00:00:00
Tasks: Link Prediction, Prediction

Application domains that require considering relationships among objects which have real-valued attributes are becoming even more important. In this paper we propose NeuralLog, a first-order logic language that is compiled to a neural network. The main goal of NeuralLog is to bridge logic programming and deep learning, allowing advances in both fields to be combined in order to obtain better machine learning models. The main advantages of NeuralLog are: to allow neural networks to be defined as logic programs; and to be able to handle numeric attributes and functions. We compared NeuralLog with two distinct systems that use first-order logic to build neural networks. We have also shown that NeuralLog can learn link prediction and classification tasks, using the same theory as the compared systems, achieving better results for the area under the ROC curve in four datasets: Cora and UWCSE for link prediction; and Yelp and PAKDD15 for classification; and comparable results for link prediction in the WordNet dataset.
Distributive Justice and Fairness Metrics in Automated Decision-making: How Much Overlap Is There?
Authors: Matthias Kuppler, Christoph Kern, Ruben L. Bach, Frauke Kreuter
Date: 2021-05-04 00:00:00
Tasks: Decision Making, Fairness

The advent of powerful prediction algorithms led to increased automation of high-stake decisions regarding the allocation of scarce resources such as government spending and welfare support. This automation bears the risk of perpetuating unwanted discrimination against vulnerable and historically disadvantaged groups. Research on algorithmic discrimination in computer science and other disciplines developed a plethora of fairness metrics to detect and correct discriminatory algorithms. Drawing on robust sociological and philosophical discourse on distributive justice, we identify the limitations and problematic implications of prominent fairness metrics. We show that metrics implementing equality of opportunity only apply when resource allocations are based on deservingness, but fail when allocations should reflect concerns about egalitarianism, sufficiency, and priority. We argue that by cleanly distinguishing between prediction tasks and decision tasks, research on fair machine learning could take better advantage of the rich literature on distributive justice.
Enhancing Generalizability of Predictive Models with Synergy of Data and Physics
Authors: Yingjun Shen, Zhe Song, Andrew Kusiak
Date: 2021-05-04 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Philosophy, Prediction

Wind farm needs prediction models for predictive maintenance. There is a need to predict values of non-observable parameters beyond ranges reflected in available data. A prediction model developed for one machine many not perform well in another similar machine. This is usually due to lack of generalizability of data-driven models. To increase generalizability of predictive models, this research integrates the data mining with first-principle knowledge. Physics-based principles are combined with machine learning algorithms through feature engineering, strong rules and divide-and-conquer. The proposed synergy concept is illustrated with the wind turbine blade icing prediction and achieves significant prediction accuracy across different turbines. The proposed process is widely accepted by wind energy predictive maintenance practitioners because of its simplicity and efficiency. Furthermore, this paper demonstrates the importance of embedding physical principles within the machine learning process, and also highlight an important point that the need for more complex machine learning algorithms in industrial big data mining is often much less than it is in other applications, making it essential to incorporate physics and follow Less is More philosophy.
On the Linear convergence of Natural Policy Gradient Algorithm
Authors: Sajad Khodadadian, Prakirt Raj Jhunjhunwala, Sushil Mahavir Varma, Siva Theja Maguluri
Date: 2021-05-04 00:00:00
Tasks: Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Markov Decision Processes are classically solved using Value Iteration and Policy Iteration algorithms. Recent interest in Reinforcement Learning has motivated the study of methods inspired by optimization, such as gradient ascent. Among these, a popular algorithm is the Natural Policy Gradient, which is a mirror descent variant for MDPs. This algorithm forms the basis of several popular Reinforcement Learning algorithms such as Natural actor-critic, TRPO, PPO, etc, and so is being studied with growing interest. It has been shown that Natural Policy Gradient with constant step size converges with a sublinear rate of O(1/k) to the global optimal. In this paper, we present improved finite time convergence bounds, and show that this algorithm has geometric (also known as linear) asymptotic convergence rate. We further improve this convergence result by introducing a variant of Natural Policy Gradient with adaptive step sizes. Finally, we compare different variants of policy gradient methods experimentally.
Learning Traffic Speed Dynamics from Visualizations
Authors: Bilal Thonnam Thodi, Zaid Saeed Khan, Saif Eddin Jabari, Monica Menendez
Date: 2021-05-04 00:00:00
Tasks: State Estimation

Space-time visualizations of macroscopic or microscopic traffic variables is a qualitative tool used by traffic engineers to understand and analyze different aspects of road traffic dynamics. We present a deep learning method to learn the macroscopic traffic speed dynamics from these space-time visualizations, and demonstrate its application in the framework of traffic state estimation. Compared to existing estimation approaches, our approach allows a finer estimation resolution, eliminates the dependence on the initial conditions, and is agnostic to external factors such as traffic demand, road inhomogeneities and driving behaviors. Our model respects causality in traffic dynamics, which improves the robustness of estimation. We present the high-resolution traffic speed fields estimated for several freeway sections using the data obtained from the Next Generation Simulation Program (NGSIM) and German Highway (HighD) datasets. We further demonstrate the quality and utility of the estimation by inferring vehicle trajectories from the estimated speed fields, and discuss the benefits of deep neural network models in approximating the traffic dynamics.
Training Quantized Neural Networks to Global Optimality via Semidefinite Programming
Authors: Burak Bartan, Mert Pilanci
Date: 2021-05-04 00:00:00
Tasks: Quantization

Neural networks (NNs) have been extremely successful across many tasks in machine learning. Quantization of NN weights has become an important topic due to its impact on their energy efficiency, inference time and deployment on hardware. Although post-training quantization is well-studied, training optimal quantized NNs involves combinatorial non-convex optimization problems which appear intractable. In this work, we introduce a convex optimization strategy to train quantized NNs with polynomial activations. Our method leverages hidden convexity in two-layer neural networks from the recent literature, semidefinite lifting, and Grothendieck's identity. Surprisingly, we show that certain quantized NN problems can be solved to global optimality in polynomial-time in all relevant parameters via semidefinite relaxations. We present numerical examples to illustrate the effectiveness of our method.
A Review on Oracle Issues in Machine Learning
Authors: Diogo Seca
Date: 2021-05-04 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning contrasts with traditional software development in that the oracle is the data, and the data is not always a correct representation of the problem that machine learning tries to model. We present a survey of the oracle issues found in machine learning and state-of-the-art solutions for dealing with these issues. These include lines of research for differential testing, metamorphic testing, and test coverage. We also review some recent improvements to robustness during modeling that reduce the impact of oracle issues, as well as tools and frameworks for assisting in testing and discovering issues specific to the dataset.
TimeGym: Debugging for Time Series Modeling in Python
Authors: Diogo Seca
Date: 2021-05-04 00:00:00
Tasks: test driven development, Time Series, Time Series Analysis, Time Series Forecasting

We introduce the TimeGym Forecasting Debugging Toolkit, a Python library for testing and debugging time series forecasting pipelines. TimeGym simplifies the testing forecasting pipeline by providing generic tests for forecasting pipelines fresh out of the box. These tests are based on common modeling challenges of time series. Our library enables forecasters to apply a Test-Driven Development approach to forecast modeling, using specified oracles to generate artificial data with noise.
Using Twitter Attribute Information to Predict Stock Prices
Authors: Roderick Karlemstrand, Ebba Leckström
Date: 2021-05-04 00:00:00
Tasks: Attribute, BIG-bench Machine Learning, Sentiment Analysis, Time Series Analysis, Time Series Forecasting

Being able to predict stock prices might be the unspoken wish of stock investors. Although stock prices are complicated to predict, there are many theories about what affects their movements, including interest rates, news and social media. With the help of Machine Learning, complex patterns in data can be identified beyond the human intellect. In this thesis, a Machine Learning model for time series forecasting is created and tested to predict stock prices. The model is based on a neural network with several layers of LSTM and fully connected layers. It is trained with historical stock values, technical indicators and Twitter attribute information retrieved, extracted and calculated from posts on the social media platform Twitter. These attributes are sentiment score, favourites, followers, retweets and if an account is verified. To collect data from Twitter, Twitter's API is used. Sentiment analysis is conducted with VADER. The results show that by adding more Twitter attributes, the MSE between the predicted prices and the actual prices improved by 3%. With technical analysis taken into account, MSE decreases from 0.1617 to 0.1437, which is an improvement of around 11%. The restrictions of this study include that the selected stock has to be publicly listed on the stock market and popular on Twitter and among individual investors. Besides, the stock markets' opening hours differ from Twitter, which constantly available. It may therefore introduce noises in the model.
Optimal Algorithms for Range Searching over Multi-Armed Bandits
Authors: Siddharth Barman, Ramakrishnan Krishnamurthy, Saladi Rahul
Date: 2021-05-04 00:00:00
Tasks: Multi-Armed Bandits

This paper studies a multi-armed bandit (MAB) version of the range-searching problem. In its basic form, range searching considers as input a set of points (on the real line) and a collection of (real) intervals. Here, with each specified point, we have an associated weight, and the problem objective is to find a maximum-weight point within every given interval. The current work addresses range searching with stochastic weights: each point corresponds to an arm (that admits sample access) and the point's weight is the (unknown) mean of the underlying distribution. In this MAB setup, we develop sample-efficient algorithms that find, with high probability, near-optimal arms within the given intervals, i.e., we obtain PAC (probably approximately correct) guarantees. We also provide an algorithm for a generalization wherein the weight of each point is a multi-dimensional vector. The sample complexities of our algorithms depend, in particular, on the size of the optimal hitting set of the given intervals. Finally, we establish lower bounds proving that the obtained sample complexities are essentially tight. Our results highlight the significance of geometric constructs -- specifically, hitting sets -- in our MAB setting.
Deep Extended Feedback Codes
Authors: Anahid Robert Safavi, Alberto G. Perotti, Branislav M. Popovic, Mahdi Boloursaz Mashhadi, Deniz Gunduz
Date: 2021-05-04 00:00:00

A new deep-neural-network (DNN) based error correction encoder architecture for channels with feedback, called Deep Extended Feedback (DEF), is presented in this paper. The encoder in the DEF architecture transmits an information message followed by a sequence of parity symbols which are generated based on the message as well as the observations of the past forward channel outputs sent to the transmitter through a feedback channel. DEF codes generalize Deepcode [1] in several ways: parity symbols are generated based on forward-channel output observations over longer time intervals in order to provide better error correction capability; and high-order modulation formats are deployed in the encoder so as to achieve increased spectral efficiency. Performance evaluations show that DEF codes have better performance compared to other DNN-based codes for channels with feedback.
Signal automata and hidden Markov models
Authors: Teodor Knapik
Date: 2021-05-04 00:00:00
Tasks: Time Series, Time Series Analysis

A generic method for inferring a dynamical hidden Markov model from a time series is proposed. Under reasonable hypothesis, the model is updated in constant time whenever a new measurement arrives.