pwc-archive/papers-with-abstracts huggingface dataset:
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?
Authors: Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, Kunal Talwar
Date: 2020-12-11 00:00:00
Tasks: Memorization, Prediction

Modern machine learning models are complex and frequently encode surprising amounts of information about individual inputs. In extreme cases, complex models appear to memorize entire input examples, including seemingly irrelevant information (social security numbers from text, for example). In this paper, we aim to understand whether this sort of memorization is necessary for accurate learning. We describe natural prediction problems in which every sufficiently accurate training algorithm must encode, in the prediction model, essentially all the information about a large subset of its training examples. This remains true even when the examples are high-dimensional and have entropy much higher than the sample size, and even when most of that information is ultimately irrelevant to the task at hand. Further, our results do not depend on the training algorithm or the class of models used for learning. Our problems are simple and fairly natural variants of the next-symbol prediction and the cluster labeling tasks. These tasks can be seen as abstractions of text- and image-related prediction problems. To establish our results, we reduce from a family of one-way communication problems for which we prove new information complexity lower bounds. Additionally, we present synthetic-data experiments demonstrating successful attacks on logistic regression and neural network classifiers.
Ensemble of Discriminators for Domain Adaptation in Multiple Sound Source 2D Localization
Authors: Guillaume Le Moing, Don Joven Agravante, Tadanobu Inoue, Jayakorn Vongkulbhisal, Asim Munawar, Ryuki Tachibana, Phongtharin Vinayavekhin
Date: 2020-12-10 00:00:00
Tasks: Domain Adaptation, Sound Source Localization

This paper introduces an ensemble of discriminators that improves the accuracy of a domain adaptation technique for the localization of multiple sound sources. Recently, deep neural networks have led to promising results for this task, yet they require a large amount of labeled data for training. Recording and labeling such datasets is very costly, especially because data needs to be diverse enough to cover different acoustic conditions. In this paper, we leverage acoustic simulators to inexpensively generate labeled training samples. However, models trained on synthetic data tend to perform poorly with real-world recordings due to the domain mismatch. For this, we explore two domain adaptation methods using adversarial learning for sound source localization which use labeled synthetic data and unlabeled real data. We propose a novel ensemble approach that combines discriminators applied at different feature levels of the localization model. Experiments show that our ensemble discrimination method significantly improves the localization performance without requiring any label from the real data.
SensiX: A Platform for Collaborative Machine Learning on the Edge
Authors: Chulhong Min, Akhil Mathur, Alessandro Montanari, Utku Gunay Acer, Fahim Kawsar
Date: 2020-12-04 00:00:00
Tasks: BIG-bench Machine Learning, Edge-computing

The emergence of multiple sensory devices on or near a human body is uncovering new dynamics of extreme edge computing. In this, a powerful and resource-rich edge device such as a smartphone or a Wi-Fi gateway is transformed into a personal edge, collaborating with multiple devices to offer remarkable sensory al eapplications, while harnessing the power of locality, availability, and proximity. Naturally, this transformation pushes us to rethink how to construct accurate, robust, and efficient sensory systems at personal edge. For instance, how do we build a reliable activity tracker with multiple on-body IMU-equipped devices? While the accuracy of sensing models is improving, their runtime performance still suffers, especially under this emerging multi-device, personal edge environments. Two prime caveats that impact their performance are device and data variabilities, contributed by several runtime factors, including device availability, data quality, and device placement. To this end, we present SensiX, a personal edge platform that stays between sensor data and sensing models, and ensures best-effort inference under any condition while coping with device and data variabilities without demanding model engineering. SensiX externalises model execution away from applications, and comprises of two essential functions, a translation operator for principled mapping of device-to-device data and a quality-aware selection operator to systematically choose the right execution path as a function of model accuracy. We report the design and implementation of SensiX and demonstrate its efficacy in developing motion and audio-based multi-device sensing systems. Our evaluation shows that SensiX offers a 7-13% increase in overall accuracy and up to 30% increase across different environment dynamics at the expense of 3mW power overhead.
Analyzing the Performance of Smart Industry 4.0 Applications on Cloud Computing Systems
Authors: Razin Farhan Hussain, Alireza Pakravan, Mohsen Amini Salehi
Date: 2020-12-11 00:00:00
Tasks: Cloud Computing, Descriptive

Cloud-based Deep Neural Network (DNN) applications that make latency-sensitive inference are becoming an indispensable part of Industry 4.0. Due to the multi-tenancy and resource heterogeneity, both inherent to the cloud computing environments, the inference time of DNN-based applications are stochastic. Such stochasticity, if not captured, can potentially lead to low Quality of Service (QoS) or even a disaster in critical sectors, such as Oil and Gas industry. To make Industry 4.0 robust, solution architects and researchers need to understand the behavior of DNN-based applications and capture the stochasticity exists in their inference times. Accordingly, in this study, we provide a descriptive analysis of the inference time from two perspectives. First, we perform an application-centric analysis and statistically model the execution time of four categorically different DNN applications on both Amazon and Chameleon clouds. Second, we take a resource-centric approach and analyze a rate-based metric in form of Million Instruction Per Second (MIPS) for heterogeneous machines in the cloud. This non-parametric modeling, achieved via Jackknife and Bootstrap re-sampling methods, provides the confidence interval of MIPS for heterogeneous cloud machines. The findings of this research can be helpful for researchers and cloud solution architects to develop solutions that are robust against the stochastic nature of the inference time of DNN applications in the cloud and can offer a higher QoS to their users and avoid unintended outcomes.
Recent Theoretical Advances in Non-Convex Optimization
Authors: Marina Danilova, Pavel Dvurechensky, Alexander Gasnikov, Eduard Gorbunov, Sergey Guminov, Dmitry Kamzolov, Innokentiy Shibaev
Date: 2020-12-11 00:00:00

Motivated by recent increased interest in optimization algorithms for non-convex optimization in application to training deep neural networks and other optimization problems in data analysis, we give an overview of recent theoretical results on global performance guarantees of optimization algorithms for non-convex optimization. We start with classical arguments showing that general non-convex problems could not be solved efficiently in a reasonable time. Then we give a list of problems that can be solved efficiently to find the global minimizer by exploiting the structure of the problem as much as it is possible. Another way to deal with non-convexity is to relax the goal from finding the global minimum to finding a stationary point or a local minimum. For this setting, we first present known results for the convergence rates of deterministic first-order methods, which are then followed by a general theoretical analysis of optimal stochastic and randomized gradient schemes, and an overview of the stochastic first-order methods. After that, we discuss quite general classes of non-convex problems, such as minimization of $\alpha$-weakly-quasi-convex functions and functions that satisfy Polyak--Lojasiewicz condition, which still allow obtaining theoretical convergence guarantees of first-order methods. Then we consider higher-order and zeroth-order/derivative-free methods and their convergence rates for non-convex optimization problems.
Structured Policy Representation: Imposing Stability in arbitrarily conditioned dynamic systems
Authors: Julen Urain, Davide Tateo, Tianyu Ren, Jan Peters
Date: 2020-12-11 00:00:00

We present a new family of deep neural network-based dynamic systems. The presented dynamics are globally stable and can be conditioned with an arbitrary context state. We show how these dynamics can be used as structured robot policies. Global stability is one of the most important and straightforward inductive biases as it allows us to impose reasonable behaviors outside the region of the demonstrations.
Analysis of Feature Representations for Anomalous Sound Detection
Authors: Robert Müller, Steffen Illium, Fabian Ritz, Kyrill Schmid
Date: 2020-12-11 00:00:00

In this work, we thoroughly evaluate the efficacy of pretrained neural networks as feature extractors for anomalous sound detection. In doing so, we leverage the knowledge that is contained in these neural networks to extract semantically rich features (representations) that serve as input to a Gaussian Mixture Model which is used as a density estimator to model normality. We compare feature extractors that were trained on data from various domains, namely: images, environmental sounds and music. Our approach is evaluated on recordings from factory machinery such as valves, pumps, sliders and fans. All of the evaluated representations outperform the autoencoder baseline with music based representations yielding the best performance in most cases. These results challenge the common assumption that closely matching the domain of the feature extractor and the downstream task results in better downstream task performance.
Detection of Adversarial Supports in Few-shot Classifiers Using Self-Similarity and Filtering
Authors: Yi Xiang Marcus Tan, Penny Chong, Jiamei Sun, Ngai-Man Cheung, Yuval Elovici, Alexander Binder
Date: 2020-12-09 00:00:00

Few-shot classifiers excel under limited training samples, making them useful in applications with sparsely user-provided labels. Their unique relative prediction setup offers opportunities for novel attacks, such as targeting support sets required to categorise unseen test samples, which are not available in other machine learning setups. In this work, we propose a detection strategy to identify adversarial support sets, aimed at destroying the understanding of a few-shot classifier for a certain class. We achieve this by introducing the concept of self-similarity of a support set and by employing filtering of supports. Our method is attack-agnostic, and we are the first to explore adversarial detection for support sets of few-shot classifiers to the best of our knowledge. Our evaluation of the miniImagenet (MI) and CUB datasets exhibits good attack detection performance despite conceptual simplicity, showing high AUROC scores. We show that self-similarity and filtering for adversarial detection can be paired with other filtering functions, constituting a generalisable concept.
Solving Inverse Problems for Spectral Energy Distributions with Deep Generative Networks
Authors: Agapi Rissaki, Orestis Pavlou, Dimitris Fotakis, Vicky Papadopoulou, Andreas Efstathiou
Date: 2020-12-09 00:00:00

We propose an end-to-end approach for solving inverse problems for a class of complex astronomical signals, namely Spectral Energy Distributions (SEDs). Our goal is to reconstruct such signals from scarce and/or unreliable measurements. We achieve that by leveraging a learned structural prior in the form of a Deep Generative Network. Similar methods have been tested almost exclusively for images which display useful properties (e.g., locality, periodicity) that are implicitly exploited. However, SEDs lack such properties which make the problem more challenging. We manage to successfully extend the methods to SEDs using a Generative Latent Optimization model trained with significantly fewer and corrupted data.
Distant Domain Transfer Learning for Medical Imaging
Authors: Shuteng Niu, Meryl Liu, Yongxin Liu, Jian Wang, Houbing Song
Date: 2020-12-10 00:00:00
Tasks: Classification, Computed Tomography (CT), COVID-19 Diagnosis, General Classification, image-classification, Image Classification, Medical Image Classification, Transfer Learning, UNET Segmentation

Medical image processing is one of the most important topics in the field of the Internet of Medical Things (IoMT). Recently, deep learning methods have carried out state-of-the-art performances on medical image tasks. However, conventional deep learning have two main drawbacks: 1) insufficient training data and 2) the domain mismatch between the training data and the testing data. In this paper, we propose a distant domain transfer learning (DDTL) method for medical image classification. Moreover, we apply our methods to a recent issue (Coronavirus diagnose). Several current studies indicate that lung Computed Tomography (CT) images can be used for a fast and accurate COVID-19 diagnosis. However, the well-labeled training data cannot be easily accessed due to the novelty of the disease and a number of privacy policies. Moreover, the proposed method has two components: Reduced-size Unet Segmentation model and Distant Feature Fusion (DFF) classification model. It is related to a not well-investigated but important transfer learning problem, termed Distant Domain Transfer Learning (DDTL). DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. In this study, we develop a DDTL model for COVID-19 diagnose using unlabeled Office-31, Catech-256, and chest X-ray image data sets as the source data, and a small set of COVID-19 lung CT as the target data. The main contributions of this study: 1) the proposed method benefits from unlabeled data collected from distant domains which can be easily accessed, 2) it can effectively handle the distribution shift between the training data and the testing data, 3) it has achieved 96\% classification accuracy, which is 13\% higher classification accuracy than "non-transfer" algorithms, and 8\% higher than existing transfer and distant transfer algorithms.
Machine Learning Prediction of Gamer's Private Networks
Authors: Chris Mazur, Jesse Ayers, Gaetan Hains, Youry Khmelevsky
Date: 2020-12-07 00:00:00
Tasks: BIG-bench Machine Learning, Minecraft, Prediction

The Gamer's Private Network (GPN) is a client/server technology created by WTFast for making the network performance of online games faster and more reliable. GPN s use middle-mile servers and proprietary algorithms to better connect online video-game players to their game's servers across a wide-area network. Online games are a massive entertainment market and network latency is a key aspect of a player's competitive edge. This market means many different approaches to network architecture are implemented by different competing companies and that those architectures are constantly evolving. Ensuring the optimal connection between a client of WTFast and the online game they wish to play is thus an incredibly difficult problem to automate. Using machine learning, we analyzed historical network data from GPN connections to explore the feasibility of network latency prediction which is a key part of optimization. Our next step will be to collect live data (including client/server load, packet and port information and specific game state information) from GPN Minecraft servers and bots. We will use this information in a Reinforcement Learning model along with predictions about latency to alter the clients' and servers' configurations for optimal network performance. These investigations and experiments will improve the quality of service and reliability of GPN systems.
Making Cross-Domain Recommendations by Associating Disjoint Users and Items Through the Affective Aware Pseudo Association Method
Authors: John Kalung Leung, Igor Griva, William G. Kennedy
Date: 2020-12-10 00:00:00
Tasks: Collaborative Filtering, Information Retrieval, Retrieval

This paper utilizes an ingenious text-based affective aware pseudo association method (AAPAM) to link disjoint users and items across different information domains and leverage them to make cross-domain content-based and collaborative filtering recommendations. This paper demonstrates that the AAPAM method could seamlessly join different information domain datasets to act as one without any additional cross-domain information retrieval protocols. Besides making cross-domain recommendations, the benefit of joining datasets from different information domains through AAPAM is that it eradicates cold start issues while making serendipitous recommendations.
KOSMOS: Knowledge-graph Oriented Social media and Mainstream media Overview System
Authors: Chua Hao Yang, Yong Shan Jie, Boon Kok Chin, Lander Chin, Lynnette Hui Xian Ng
Date: 2020-12-11 00:00:00
Tasks: Articles, Clustering, Entity Disambiguation, Retrieval

We introduce KOSMOS, a knowledge retrieval system based on the constructed knowledge graph of social media and mainstream media documents. The system first identifies key events from the documents at each time frame through clustering, extracting a document to represent each cluster, then describing the document in terms of 5W1H (Who, What, When, Where, Why, How). The event centric knowledge graph is enhanced by relation triplets and entity disambiguation from the representative document. This knowledge retrieval is supported by a web interface that presents a graph visualisation of related nodes and relevant articles based on a user query. The interface facilitates understanding relationships between events reported in mainstream and social media journalism through the KOSMOS information extraction pipeline, which is valuable to understand media slant and public opinions. Finally, we explore a use case in extracting events and relations from documents to understand the media and community's view to the 2020 COVID19 pandemic.
Market2Dish: Health-aware Food Recommendation
Authors: Wenjie Wang, Ling-Yu Duan, Hao Jiang, Peiguang Jing, Xuemeng Song, Liqiang Nie
Date: 2020-12-11 00:00:00
Tasks: Food recommendation, Nutrition, Retrieval

With the rising incidence of some diseases, such as obesity and diabetes, a healthy diet is arousing increasing attention. However, most existing food-related research efforts focus on recipe retrieval, user preference-based food recommendation, cooking assistance, or the nutrition and calorie estimation of dishes, ignoring the personalized health-aware food recommendation. Therefore, in this work, we present a personalized health-aware food recommendation scheme, namely Market2Dish, mapping the ingredients displayed in the market to the healthy dishes eaten at home. The proposed scheme comprises three components, namely recipe retrieval, user-health profiling, and health-aware food recommendation. In particular, recipe retrieval aims to acquire the ingredients available to the users, and then retrieve recipe candidates from a large-scale recipe dataset. User health profiling is to characterize the health conditions of users by capturing the textual health-related information crawled from social networks. Specifically, to solve the issue that the health-related information is extremely sparse, we incorporate a word-class interaction mechanism into the proposed deep model to learn the fine-grained correlations between the textual tweets and pre-defined health concepts. For the health-aware food recommendation, we present a novel category-aware hierarchical memory network-based recommender to learn the health-aware user-recipe interactions for better food recommendation. Moreover, extensive experiments demonstrate the effectiveness of the health-aware food recommendation scheme.
Learning from User Interactions with Rankings: A Unification of the Field
Authors: Harrie Oosterhuis
Date: 2020-12-09 00:00:00
Tasks: counterfactual, Learning-To-Rank

Ranking systems form the basis for online search engines and recommendation services. They process large collections of items, for instance web pages or e-commerce products, and present the user with a small ordered selection. The goal of a ranking system is to help a user find the items they are looking for with the least amount of effort. Thus the rankings they produce should place the most relevant or preferred items at the top of the ranking. Learning to rank is a field within machine learning that covers methods which optimize ranking systems w.r.t. this goal. Traditional supervised learning to rank methods utilize expert-judgements to evaluate and learn, however, in many situations such judgements are impossible or infeasible to obtain. As a solution, methods have been introduced that perform learning to rank based on user clicks instead. The difficulty with clicks is that they are not only affected by user preferences, but also by what rankings were displayed. Therefore, these methods have to prevent being biased by other factors than user preference. This thesis concerns learning to rank methods based on user clicks and specifically aims to unify the different families of these methods. As a whole, the second part of this thesis proposes a framework that bridges many gaps between areas of online, counterfactual, and supervised learning to rank. It has taken approaches, previously considered independent, and unified them into a single methodology for widely applicable and effective learning to rank from user clicks.
Limits of PageRank-based ranking methods in sports data
Authors: Yuhao Zhou, Ruijie Wang, Yi-Cheng Zhang, An Zeng, Matúš Medo
Date: 2020-12-11 00:00:00

While PageRank has been extensively used to rank sport tournament participants (teams or individuals), its superiority over simpler ranking methods has been never clearly demonstrated. We use sports results from 18 major leagues to calibrate a state-of-art model for synthetic sports results. Model data are then used to assess the ranking performance of PageRank in a controlled setting. We find that PageRank outperforms the benchmark ranking by the number of wins only when a small fraction of all games have been played. Increased randomness in the data, such as intrinsic randomness of outcomes or advantage of home teams, further reduces the range of PageRank's superiority. We propose a new PageRank variant which outperforms PageRank in all evaluated settings, yet shares its sensitivity to increased randomness in the data. Our main findings are confirmed by evaluating the ranking algorithms on real data. Our work demonstrates the danger of using novel metrics and algorithms without considering their limits of applicability.
Option Tracing: Beyond Binary Knowledge Tracing
Authors: Aritra Ghosh, Andrew S. Lan
Date: 2020-12-11 00:00:00
Tasks: Knowledge Tracing, Multiple-choice, Prediction

This paper details our solutions to Tasks 1&2 of the NeurIPS 2020 Education Challenge.1 Knowledge tracing, a family of methods to estimate each student’s mastery
levels on skills/knowledge components from their past responses to assessment
questions, is useful for progress monitoring, personalization, and helping teachers
to deliver personalized and targeted feedback to students to improve their learning
outcomes. One key limitation of current knowledge tracing methods is that they
can only estimate an overall knowledge level of a student since they analyze only
the binary-valued correctness of student responses. We adapt a series of popular
knowledge tracing methods to the task of option prediction in multiple choice
questions. Experimental results show that our method performs well on both option
prediction and correctness prediction.