pwc-archive/papers-with-abstracts huggingface dataset:
Hyperparameter Transfer Learning with Adaptive Complexity
Authors: Samuel Horváth, Aaron Klein, Peter Richtárik, Cédric Archambeau
Date: 2021-02-25 00:00:00
Tasks: Bayesian Optimization, Decision Making, Sequential Decision Making, Transfer Learning

Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models. In practice, one frequently has to solve similar hyperparameter tuning problems sequentially. For example, one might have to tune a type of neural network learned across a series of different classification problems. Recent work on multi-task BO exploits knowledge gained from previous tuning tasks to speed up a new tuning task. However, previous approaches do not account for the fact that BO is a sequential decision making procedure. Hence, there is in general a mismatch between the number of evaluations collected in the current tuning task compared to the number of evaluations accumulated in all previously completed tasks. In this work, we enable multi-task BO to compensate for this mismatch, such that the transfer learning procedure is able to handle different data regimes in a principled way. We propose a new multi-task BO method that learns a set of ordered, non-linear basis functions of increasing complexity via nested drop-out and automatic relevance determination. Experiments on a variety of hyperparameter tuning problems show that our method improves the sample ef
Vector quantile regression and optimal transport, from theory to numerics
Authors: Guillaume Carlier, Victor Chernozhukov, Gwendoline de Bie, Alfred Galichon
Date: 2021-02-25 00:00:00
Tasks: quantile regression, regression

In this paper, we first revisit the Koenker and Bassett variational approach to (univariate) quantile regression, emphasizing its link with latent factor representations and correlation maximization problems. We then review the multivariate extension due to Carlier et al. (2016, 2017) which relates vector quantile regression to an optimal transport problem with mean independence constraints. We introduce an entropic regularization of this problem, implement a gradient descent numerical method and illustrate its feasibility on univariate and bivariate examples.
SCD: A Stacked Carton Dataset for Detection and Segmentation
Authors: Jinrong Yang, Shengkai Wu, Lijun Gou, Hangcheng Yu, Chenxi Lin, Jiazhuo Wang, Minxuan Li, Xiaoping Li
Date: 2021-02-25 00:00:00
Tasks: Instance Segmentation, Semantic Segmentation

Carton detection is an important technique in the automatic logistics system and can be applied to many applications such as the stacking and unstacking of cartons, the unloading of cartons in the containers. However, there is no public large-scale carton dataset for the research community to train and evaluate the carton detection models up to now, which hinders the development of carton detection. In this paper, we present a large-scale carton dataset named Stacked Carton Dataset(SCD) with the goal of advancing the state-of-the-art in carton detection. Images are collected from the internet and several warehourses, and objects are labeled using per-instance segmentation for precise localization. There are totally 250,000 instance masks from 16,136 images. In addition, we design a carton detector based on RetinaNet by embedding Offset Prediction between Classification and Localization module(OPCL) and Boundary Guided Supervision module(BGS). OPCL alleviates the imbalance problem between classification and localization quality which boosts AP by 3.1% - 4.7% on SCD while BGS guides the detector to pay more attention to boundary information of cartons and decouple repeated carton textures. To demonstrate the generalization of OPCL to other datasets, we conduct extensive experiments on MS COCO and PASCAL VOC. The improvement of AP on MS COCO and PASCAL VOC is 1.8% - 2.2% and 3.4% - 4.3% respectively.
The Stochastic Balance Equation for the American Option Value Function and its Gradient
Authors: Malkhaz Shashiashvili
Date: 2021-02-25 00:00:00

In the paper we consider the problem of valuation and hedging of American options written on dividend-paying assets whose price dynamics follow the multidimensional diffusion model. We derive a stochastic balance equation for the American option value function and its gradient. We prove that the latter pair is the unique solution of the stochastic balance equation as a result of the uniqueness in the related adapted future-supremum problem.
Cognitive network science for understanding online social cognitions: A brief review
Authors: Massimo Stella
Date: 2021-02-25 00:00:00
Tasks: Data Integration

Social media are digitalising massive amounts of users' cognitions in terms of timelines and emotional content. Such Big Data opens unprecedented opportunities for investigating cognitive phenomena like perception, personality and information diffusion but requires suitable interpretable frameworks. Since social media data come from users' minds, worthy candidates for this challenge are cognitive networks, models of cognition giving structure to mental conceptual associations. This work outlines how cognitive network science can open new, quantitative ways for understanding cognition through online media, like: (i) reconstructing how users semantically and emotionally frame events with contextual knowledge unavailable to machine learning, (ii) investigating conceptual salience/prominence through knowledge structure in social discourse; (iii) studying users' personality traits like openness-to-experience, curiosity, and creativity through language in posts; (iv) bridging cognitive/emotional content and social dynamics via multilayer networks comparing the mindsets of influencers and followers. These advancements combine cognitive-, network- and computer science to understand cognitive mechanisms in both digital and real-world settings but come with limitations concerning representativeness, individual variability and data integration. Such aspects are discussed along the ethical implications of manipulating socio-cognitive data. In the future, reading cognitions through networks and social media can expose cognitive biases amplified by online platforms and relevantly inform policy making, education and markets about massive, complex cognitive trends.
Learning to Truncate Ranked Lists for Information Retrieval
Authors: Chen Wu, Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Xueqi Cheng
Date: 2021-02-25 00:00:00
Tasks: Information Retrieval, Retrieval

Ranked list truncation is of critical importance in a variety of professional information retrieval applications such as patent search or legal search. The goal is to dynamically determine the number of returned documents according to some user-defined objectives, in order to reach a balance between the overall utility of the results and user efforts. Existing methods formulate this task as a sequential decision problem and take some pre-defined loss as a proxy objective, which suffers from the limitation of local decision and non-direct optimization. In this work, we propose a global decision based truncation model named AttnCut, which directly optimizes user-defined objectives for the ranked list truncation. Specifically, we take the successful transformer architecture to capture the global dependency within the ranked list for truncation decision, and employ the reward augmented maximum likelihood (RAML) for direct optimization. We consider two types of user-defined objectives which are of practical usage. One is the widely adopted metric such as F1 which acts as a balanced objective, and the other is the best F1 under some minimal recall constraint which represents a typical objective in professional search. Empirical results over the Robust04 and MQ2007 datasets demonstrate the effectiveness of our approach as compared with the state-of-the-art baselines.
Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
Authors: Changyong Oh, Efstratios Gavves, Max Welling
Date: 2021-02-25 00:00:00
Tasks: Bayesian Optimization, Hyperparameter Optimization

The sample efficiency of Bayesian optimization(BO) is often boosted by Gaussian Process(GP) surrogate models. However, on mixed variable spaces, surrogate models other than GPs are prevalent, mainly due to the lack of kernels which can model complex dependencies across different types of variables. In this paper, we propose the frequency modulated (FM) kernel flexibly modeling dependencies among different types of variables, so that BO can enjoy the further improved sample efficiency. The FM kernel uses distances on continuous variables to modulate the graph Fourier spectrum derived from discrete variables. However, the frequency modulation does not always define a kernel with the similarity measure behavior which returns higher values for pairs of more similar points. Therefore, we specify and prove conditions for FM kernels to be positive definite and to exhibit the similarity measure behavior. In experiments, we demonstrate the improved sample efficiency of GP BO using FM kernels (BO-FM).On synthetic problems and hyperparameter optimization problems, BO-FM outperforms competitors consistently. Also, the importance of the frequency modulation principle is empirically demonstrated on the same problems. On joint optimization of neural architectures and SGD hyperparameters, BO-FM outperforms competitors including Regularized evolution(RE) and BOHB. Remarkably, BO-FM performs better even than RE and BOHB using three times as many evaluations.
Next Generation Models for Portfolio Risk Management: An Approach Using Financial Big Data
Authors: Kwangmin Jung, Donggyu Kim, Seunghyeon Yu
Date: 2021-02-25 00:00:00
Tasks: Management

This paper proposes a dynamic process of portfolio risk measurement to address potential information loss. The proposed model takes advantage of financial big data to incorporate out-of-target-portfolio information that may be missed when one considers the Value at Risk (VaR) measures only from certain assets of the portfolio. We investigate how the curse of dimensionality can be overcome in the use of financial big data and discuss where and when benefits occur from a large number of assets. In this regard, the proposed approach is the first to suggest the use of financial big data to improve the accuracy of risk analysis. We compare the proposed model with benchmark approaches and empirically show that the use of financial big data improves small portfolio risk analysis. Our findings are useful for portfolio managers and financial regulators, who may seek for an innovation to improve the accuracy of portfolio risk estimation.
Do Input Gradients Highlight Discriminative Features?
Authors: Harshay Shah, Prateek Jain, Praneeth Netrapalli
Date: 2021-02-25 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: image-classification, Image Classification

Post-hoc gradient-based interpretability methods [Simonyan et al., 2013, Smilkov et al., 2017] that provide instance-specific explanations of model predictions are often based on assumption (A): magnitude of input gradients -- gradients of logits with respect to input -- noisily highlight discriminative task-relevant features. In this work, we test the validity of assumption (A) using a three-pronged approach. First, we develop an evaluation framework, DiffROAR, to test assumption (A) on four image classification benchmarks. Our results suggest that (i) input gradients of standard models (i.e., trained on original data) may grossly violate (A), whereas (ii) input gradients of adversarially robust models satisfy (A). Second, we introduce BlockMNIST, an MNIST-based semi-real dataset, that by design encodes a priori knowledge of discriminative features. Our analysis on BlockMNIST leverages this information to validate as well as characterize differences between input gradient attributions of standard and robust models. Finally, we theoretically prove that our empirical findings hold on a simplified version of the BlockMNIST dataset. Specifically, we prove that input gradients of standard one-hidden-layer MLPs trained on this dataset do not highlight instance-specific signal coordinates, thus grossly violating assumption (A). Our findings motivate the need to formalize and test common assumptions in interpretability in a falsifiable manner [Leavitt and Morcos, 2020]. We believe that the DiffROAR evaluation framework and BlockMNIST-based datasets can serve as sanity checks to audit instance-specific interpretability methods; code and data available at https://github.com/harshays/inputgradients.
An Overview of Signal Processing Techniques for Joint Communication and Radar Sensing
Authors: J. Andrew Zhang, Fan Liu, Christos Masouros, Robert W. Heath Jr., Zhiyong Feng, Le Zheng, Athina Petropulu
Date: 2021-02-25 00:00:00

Joint communication and radar sensing (JCR) represents an emerging research field aiming to integrate the above two functionalities into a single system, sharing a majority of hardware and signal processing modules and, in a typical case, sharing a single transmitted signal. It is recognised as a key approach in significantly improving spectrum efficiency, reducing device size, cost and power consumption, and improving performance thanks to potential close cooperation of the two functions. Advanced signal processing techniques are critical for making the integration efficient, from transmission signal design to receiver processing. This paper provides a comprehensive overview of JCR systems from the signal processing perspective, with a focus on state-of-the-art. A balanced coverage on both transmitter and receiver is provided for three types of JCR systems, communication-centric, radar-centric, and joint design and optimization.
IIE-NLP-Eyas at SemEval-2021 Task 4: Enhancing PLM for ReCAM with Special Tokens, Re-Ranking, Siamese Encoders and Back Translation
Authors: Yuqiang Xie, Luxi Xing, Wei Peng, Yue Hu
Date: 2021-02-25 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Multiple-choice, Question Answering, Reading Comprehension, Re-Ranking

This paper introduces our systems for all three subtasks of SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning. To help our model better represent and understand abstract concepts in natural language, we well-design many simple and effective approaches adapted to the backbone model (RoBERTa). Specifically, we formalize the subtasks into the multiple-choice question answering format and add special tokens to abstract concepts, then, the final prediction of question answering is considered as the result of subtasks. Additionally, we employ many finetuning tricks to improve the performance. Experimental results show that our approaches achieve significant performance compared with the baseline systems. Our approaches achieve eighth rank on subtask-1 and tenth rank on subtask-2.
A New Neuromorphic Computing Approach for Epileptic Seizure Prediction
Authors: Fengshi Tian, Jie Yang, Shiqi Zhao, Mohamad Sawan
Date: 2021-02-25 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Prediction, Seizure prediction, Sensitivity, Specificity

Several high specificity and sensitivity seizure prediction methods with convolutional neural networks (CNNs) are reported. However, CNNs are computationally expensive and power hungry. These inconveniences make CNN-based methods hard to be implemented on wearable devices. Motivated by the energy-efficient spiking neural networks (SNNs), a neuromorphic computing approach for seizure prediction is proposed in this work. This approach uses a designed gaussian random discrete encoder to generate spike sequences from the EEG samples and make predictions in a spiking convolutional neural network (Spiking-CNN) which combines the advantages of CNNs and SNNs. The experimental results show that the sensitivity, specificity and AUC can remain 95.1%, 99.2% and 0.912 respectively while the computation complexity is reduced by 98.58% compared to CNN, indicating that the proposed Spiking-CNN is hardware friendly and of high precision.
$FM^2$: Field-matrixed Factorization Machines for Recommender Systems
Authors: Yang Sun, Junwei Pan, Alex Zhang, Aaron Flores
Date: 2021-02-20 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Click-through rate (CTR) prediction plays a critical role in recommender systems and online advertising. The data used in these applications are multi-field categorical data, where each feature belongs to one field. Field information is proved to be important and there are several works considering fields in their models. In this paper, we proposed a novel approach to model the field information effectively and efficiently. The proposed approach is a direct improvement of FwFM, and is named as Field-matrixed Factorization Machines (FmFM, or $FM^2$). We also proposed a new explanation of FM and FwFM within the FmFM framework, and compared it with the FFM. Besides pruning the cross terms, our model supports field-specific variable dimensions of embedding vectors, which acts as soft pruning. We also proposed an efficient way to minimize the dimension while keeping the model performance. The FmFM model can also be optimized further by caching the intermediate vectors, and it only takes thousands of floating-point operations (FLOPs) to make a prediction. Our experiment results show that it can out-perform the FFM, which is more complex. The FmFM model's performance is also comparable to DNN models which require much more FLOPs in runtime.
Deep NMF Topic Modeling
Authors: Jianyu Wang, Xiao-Lei Zhang
Date: 2021-02-24 00:00:00

Nonnegative matrix factorization (NMF) based topic modeling methods do not rely on model- or data-assumptions much. However, they are usually formulated as difficult optimization problems, which may suffer from bad local minima and high computational complexity. In this paper, we propose a deep NMF (DNMF) topic modeling framework to alleviate the aforementioned problems. It first applies an unsupervised deep learning method to learn latent hierarchical structures of documents, under the assumption that if we could learn a good representation of documents by, e.g. a deep model, then the topic word discovery problem can be boosted. Then, it takes the output of the deep model to constrain a topic-document distribution for the discovery of the discriminant topic words, which not only improves the efficacy but also reduces the computational complexity over conventional unsupervised NMF methods. We constrain the topic-document distribution in three ways, which takes the advantages of the three major sub-categories of NMF -- basic NMF, structured NMF, and constrained NMF respectively. To overcome the weaknesses of deep neural networks in unsupervised topic modeling, we adopt a non-neural-network deep model -- multilayer bootstrap network. To our knowledge, this is the first time that a deep NMF model is used for unsupervised topic modeling. We have compared the proposed method with a number of representative references covering major branches of topic modeling on a variety of real-world text corpora. Experimental results illustrate the effectiveness of the proposed method under various evaluation metrics.
A microservice-based framework for exploring data selection in cross-building knowledge transfer
Authors: Mouna Labiadh, Christian Obrecht, Catarina Ferreira da Silva, Parisa Ghodous
Date: 2021-02-23 00:00:00
Tasks: Domain Generalization, Transfer Learning

Supervised deep learning has achieved remarkable success in various applications. Successful machine learning application however depends on the availability of sufficiently large amount of data. In the absence of data from the target domain, representative data collection from multiple sources is often needed. However, a model trained on existing multi-source data might generalize poorly on the unseen target domain. This problem is referred to as domain shift. In this paper, we explore the suitability of multi-source training data selection to tackle the domain shift challenge in the context of domain generalization. We also propose a microservice-oriented methodology for supporting this solution. We perform our experimental study on the use case of building energy consumption prediction. Experimental results suggest that minimal building description is capable of improving cross-building generalization performances when used to select energy consumption data.
Preserved central model for faster bidirectional compression in distributed settings
Authors: Constantin Philippenko, Aymeric Dieuleveut
Date: 2021-02-24 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Model Compression

We develop a new approach to tackle communication constraints in a distributed learning problem with a central server. We propose and analyze a new algorithm that performs bidirectional compression and achieves the same convergence rate as algorithms using only uplink (from the local workers to the central server) compression. To obtain this improvement, we design MCM, an algorithm such that the downlink compression only impacts local models, while the global model is preserved. As a result, and contrary to previous works, the gradients on local servers are computed on perturbed models. Consequently, convergence proofs are more challenging and require a precise control of this perturbation. To ensure it, MCM additionally combines model compression with a memory mechanism. This analysis opens new doors, e.g. incorporating worker dependent randomized-models and partial participation.
Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning
Authors: Sheng Li, Yutai Zhou, Ross Allen, Mykel J. Kochenderfer
Date: 2021-02-24 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Communication is a important factor that enables agents work cooperatively in multi-agent reinforcement learning (MARL). Most previous work uses continuous message communication whose high representational capacity comes at the expense of interpretability. Allowing agents to learn their own discrete message communication protocol emerged from a variety of domains can increase the interpretability for human designers and other agents.This paper proposes a method to generate discrete messages analogous to human languages, and achieve communication by a broadcast-and-listen mechanism based on self-attention. We show that discrete message communication has performance comparable to continuous message communication but with much a much smaller vocabulary size.Furthermore, we propose an approach that allows humans to interactively send discrete messages to agents.
Benchmarking Graph Neural Networks on Link Prediction
Authors: Xing Wang, Alexander Vinel
Date: 2021-02-24 00:00:00
Tasks: Benchmarking, Graph Attention, Graph Neural Network, Link Prediction, Prediction

In this paper, we benchmark several existing graph neural network (GNN) models on different datasets for link predictions. In particular, the graph convolutional network (GCN), GraphSAGE, graph attention network (GAT) as well as variational graph auto-encoder (VGAE) are implemented dedicated to link prediction tasks, in-depth analysis are performed, and results from several different papers are replicated, also a more fair and systematic comparison are provided. Our experiments show these GNN architectures perform similarly on various benchmarks for link prediction tasks.