Miracle-dz/UniFAR-FTUs huggingface dataset:
Cumulants of Hawkes Processes are Robust to Observation Noise

Multivariate Hawkes processes (MHPs) are widely used in a variety of ﬁelds to model the occurrence of causally related discrete events in continuous time. Most state-of-the-art approaches address the problem of learning MHPs from perfect traces without noise. In practice, the process through which events are collected might introduce noise in the timestamps. In this work, we address the problem of learning the causal structure of MHPs when the observed timestamps of events are subject to random and unknown shifts, also known as random translations. We prove that the cumulants of MHPs are invariant to random translations, and therefore can be used to learn their underlying causal structure. Furthermore, we empirically characterize the effect of random translations on state-of-the-art learning methods. We show that maximum likelihood-based estimators are brittle, while cumulant-based estimators remain stable even in the presence of signiﬁcant time shifts.
Infinite-Dimensional Optimization for Zero-Sum Games via Variational Transport

Game optimization has been extensively studied when decision variables lie in a ﬁnite-dimensional space, of which solutions correspond to pure strategies at the Nash equilibrium (NE), and the gradient descent-ascent (GDA) method works widely in practice. In this paper, we consider inﬁnite-dimensional zero-sum games by a min-max distributional optimization problem over a space of probability measures deﬁned on a continuous variable set, which is inspired by ﬁnding a mixed NE for ﬁnite-dimensional zero-sum games. We then aim to answer the following question: Will GDA-type algorithms still be provably efﬁ-cient when extended to inﬁnite-dimensional zero-sum games? To answer this question, we propose a particle-based variational transport algorithm based on GDA in the functional spaces. Speciﬁcally, the algorithm performs multi-step functional gradient descent-ascent in the Wasserstein space via pushing two sets of particles in the variable space. By characterizing the gradient estimation error from variational form maximization and the convergence behavior of each player with different ob-jective landscapes, we prove that a theoretical version of the generalized GDA algorithm converges to the NE or the value of the game efﬁciently for a class of games under the Polyak-Łojasiewicz (PL) condition. To conclude, we provide complete statistical and convergence guarantees for solving an inﬁnite-dimensional zero-sum game via a provably efﬁcient particle-based method. Additionally, our work provides the ﬁrst thorough statistical analysis for the particle-based algorithm to learn an objective functional with a variational form us-ing universal approximators ( i.e. , neural networks
A Discriminative Technique for Multiple-Source Adaptation

We present a new discriminative technique for the multiple-source adaptation (MSA) problem. Unlike previous work, which relies on density estimation for each source domain, our solution only requires conditional probabilities that can be straightforwardly accurately estimated from unlabeled data from the source domains. We give a detailed analysis of our new technique, including general guarantees based on Rényi divergences, and learning bounds when conditional Maxent is used for estimating conditional probabilities for a point to belong to a source domain. We show that these guarantees compare favorably to those that can be derived for the generative solution, using kernel density estimation. Our experiments with real-world applications further demonstrate that our new discriminative MSA algorithm outperforms the previous generative solution as well as other domain adaptation baselines.
Novel On-Road Vehicle Detection System Using Multi-Stage Convolutional Neural Network

General object-detection methods based on deep learning have received considerable attention in the field of computer vision. However, when they are applied to vehicle detection (VD) in a straightforward manner to realize an intelligent vehicle (IV), a graphics processing unit (GPU) is required for their real-time implementation. The use of GPUs is unacceptable in commercial VD systems. A novel on-road VD method comprising the use of a multi-stage convolutional neural network (MSCNN) is proposed to solve this problem. In the MSCNN, the properties of the vehicles are exploited, and an efficient region proposal specialized for vehicles is developed. The proposed MSCNN comprises four stages: vehicle lower-boundary detection, vehicle upper-boundary detection, region proposal network (RPN), and vehicle classification. Effective anchor boxes are generated in the lower- and upper-boundary-detection stages with appropriate sizes for vehicles of all scales. The bounding box of the vehicle within the anchor box is determined in the RPN stage. In the last stage, the predicted bounding boxes are classified as vehicles or non-vehicles. Finally, the proposed method is applied to the KITTI, CrowdAI, and AUTTI datasets, and its advantages are demonstrated by comparing its performance with those presented in previous studies. The proposed MSCNN realizes an average precision (72.1%) on the KITTI dataset while running on a central processing unit (CPU).
Blind Pareto Fairness and Subgroup Robustness

With the wide adoption of machine learning algorithms across various application domains, there is a growing interest in the fairness properties of such algorithms. The vast majority of the activity in the field of group fairness addresses disparities between predeﬁned groups based on protected features such as gender, age, and race, which need to be available at train, and often also at test, time. These approaches are static and retrospective, since algorithms designed to protect groups identified a priori cannot anticipate and protect the needs of different at-risk groups in the future. In this work we analyze the space of solutions for worst-case fairness beyond demographics, and propose Blind Pareto Fairness (BPF), a method that leverages no-regret dynamics to recover a fair minimax classiﬁer that reduces worst-case risk of any potential subgroup of sufﬁcient size, and guarantees that the remaining population receives the best possible level of service. BPF addresses fairness beyond demographics, that is, it does not rely on predeﬁned notions of at-risk groups, neither at train nor at test time. Our experimental results show that the proposed framework improves worst-case risk in multiple standard datasets, while simultaneously providing better levels of service for the remaining population, in comparison to competing methods.
Evaluation of Congestion Aware Social Metrics for Centrality-Based Routing

Opportunistic networks utilize pocket switching for routing where each node forwards its messages to a suitable next node. The selection of the forwarder node is crucial for the efficient performance of a routing protocol. In any opportunistic network, some nodes have a paramount role in the routing process and these nodes could be identified with the assistance of the existing centrality measures available in network theory. However, the central nodes tend to suffer from congestion because a large number of nodes that are relatively less central attempt to forward their payload to the central nodes to increase the probability of the message delivery. This paper evaluates mechanisms to transform the social encounters into congestion aware metrics so that high-ranking central nodes are downgraded when they encounter congestion. The network transformations are aimed at aggregating the connectivity patterns of the nodes to implicitly accumulate the network information to be utilized by centrality measures for routing purposes. We have analyzed the performance of the metrics’ computed centrality measures using routing simulation on three real-world network traces. The results revealed that betweenness centrality along with the congestion aware network metrics holds the potential to deliver a competitive number of messages. Additionally, the proposed congestion aware metrics significantly balance the routing load among the central nodes.
Reserve Price Optimization for First Price Auctions

The display advertising industry has recently transitioned from second- to first-price auctions as its primary mechanism for ad allocation and pricing. In light of this, publishers need to re-evaluate and optimize their auction parameters, notably reserve prices. In this paper, we propose a gradient-based algorithm to adaptively update and optimize reserve prices based on estimates of bidders' responsiveness to experimental shocks in reserves. Our key innovation is to draw on the inherent structure of the revenue objective in order to reduce the variance of gradient estimates and improve convergence rates in both theory and practice. We show that revenue in a first-price auction can be usefully decomposed into a \emph{demand} component and a \emph{bidding} component, and introduce techniques to reduce the variance of each component. We characterize the bias-variance trade-offs of these techniques and validate the performance of our proposed algorithm through experiments on synthetic data and real display ad auctions data from Google ad exchange.
Troubleshooting an Intrusion Detection Dataset: the CICIDS2017 Case Study

Numerous studies have demonstrated the effectiveness of machine learning techniques in application to network intrusion detection. And yet, the adoption of machine learning for securing large-scale network environments remains challenging. The community acknowledges that network security presents unique challenges for machine learning, and the lack of training data representative of modern traffic remains one of the most intractable issues. New attempts are continuously made to develop high quality benchmark datasets and proper data collection methodologies. The CICIDS2017 dataset is one of the recent results, created to meet the demanding criterion of representativeness for network intrusion detection. In this paper we revisit CICIDS2017 and its data collection pipeline and analyze correctness, validity and overall utility of the dataset for the learning task. During this in-depth analysis, we uncover a series of problems with traffic generation, flow construction, feature extraction and labelling that severely affect the aforementioned properties. We investigate the causes of these shortcomings and address most of them by applying an improved data processing methodology. As a result, more than 20 percent of original traffic traces are reconstructed or relabelled. Machine learning benchmarks on the final dataset demonstrate significant improvements. Our study exemplifies how data collection issues may have enormous impact on model evaluation and provides recommendations for their anticipation and prevention.
LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs

Answering complex natural language questions on knowledge graphs (KGQA) is a challenging task. It requires reasoning with the input natu-ral language questions as well as a massive, incomplete heterogeneous KG. Prior methods obtain an abstract structured query graph/tree from the input question and traverse the KG for answers following the query tree. However, they inherently cannot deal with missing links in the KG. Here we present LEGO, a L atent E xecution-G uided reas O ning framework to handle this challenge in KGQA. LEGO works in an iterative way, which alternates between (1) a Query Synthesizer, which synthesizes a reasoning action and grows the query tree step-by-step, and (2) a Latent Space Executor that executes the reasoning action in the latent embedding space to combat against the missing information in KG. To learn the synthe-sizer without step-wise supervision, we design a generic latent execution guided bottom-up search procedure to ﬁnd good execution traces efﬁciently in the vast query space. Experimental results on several KGQA benchmarks demonstrate the effectiveness of our framework compared with previous state of the art.
Educating Managers and Future Managers in Sustainability: Experiences from the Fishpond Experiment at the Berner Fachhochschule

The paper presents an implementation of the fishpond experiment as a means to help the development of a first-hand experience of the implications of individual and group behavior to the tragedy of commons. Apart from its value to teaching purposes, we see the value for exploring the potential of scaling up and transfer of its deployment to a broader audience not only in the educational curricula but also for professional development of future generations of environmentally conscious and responsible managers and corporate decision-makers. Keywords—Sustainability, simulation, tragedy of the commons, fishpond experiment, executive learning, hands-on skills acquisition
DriftSurf: Stable-State / Reactive-State Learning under Concept Drift

When learning from streaming data, a change in the data distribution, also known as concept drift, can render a previously-learned model inaccurate and require training a new model. We present an adaptive learning algorithm that extends previous drift-detection-based methods by incorporating drift detection into a broader stable-state/reactivestate process. The advantage of our approach is that we can use aggressive drift detection in the stable state to achieve a high detection rate, but mitigate the false positive rate of standalone drift detection via a reactive state that reacts quickly to true drifts while eliminating most false positives. The algorithm is generic in its base learner and can be applied across a variety of supervised learning problems. Our theoretical analysis shows that the risk of the algorithm is (i) statistically better than standalone drift detection and (ii) competitive to an algorithm with oracle knowledge of when (abrupt) drifts occur. Experiments on synthetic and real datasets with concept drifts confrm our theoretical analysis.
Parameterless Transductive Feature Re-representation for Few-Shot Learning

Recent literature in few-shot learning (FSL) has shown that transductive methods often outperform their inductive counterparts. However, most trans-ductive solutions, particularly the meta-learning based ones, require inserting trainable parameters on top of some inductive baselines to facilitate transduction. In this paper, we propose a pa-rameterless transductive feature re-representation framework that differs from all existing solutions from the following perspectives. (1) It is widely compatible with existing FSL methods, including meta-learning and ﬁne tuning based models. (2) The framework is simple and introduces no extra training parameters when applied to any architecture. We conduct experiments on three benchmark datasets by applying the framework to both representative meta-learning baselines and state-of-the-art FSL methods. Our framework consistently improves performances in all experiments and refreshes the state-of-the-art FSL results.
Conservative Objective Models for Effective Offline Model-Based Optimization

Computational design problems arise in a number of settings, from synthetic biology to computer architectures. In this paper, we aim to solve data-driven model-based optimization (MBO) problems, where the goal is to find a design input that maximizes an unknown objective function provided access to only a static dataset of prior experiments. Such data-driven optimization procedures are the only practical methods in many real-world domains where active data collection is expensive (e.g., when optimizing over proteins) or dangerous (e.g., when optimizing over aircraft designs). Typical methods for MBO that optimize the design against a learned model suffer from distributional shift: it is easy to find a design that"fools"the model into predicting a high value. To overcome this, we propose conservative objective models (COMs), a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs, and uses it for optimization. Structurally, COMs resemble adversarial training methods used to overcome adversarial examples. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems, including optimizing protein sequences, robot morphologies, neural network weights, and superconducting materials.
Learning from Noisy Labels with No Change to the Training Process

There has been much interest in recent years in developing learning algorithms that can learn accurate classiﬁers from data with noisy labels. A widely-studied noise model is that of class-conditional noise (CCN), wherein a label y is ﬂipped to a label (cid:101) y with some associated noise probability that depends on both y and (cid:101) y . In the multiclass setting, all previously proposed algorithms under the CCN model involve changing the training process, by introducing a ‘noise-correction’ to the surrogate loss to be minimized over the noisy training examples. In this paper, we show that this is really unnecessary: one can simply perform class probability estimation (CPE) on the noisy examples, e.g. using a standard (multiclass) logistic regression algorithm, and then apply noise-correction only in the ﬁnal prediction step. This means that the training algorithm itself does not need any change, and one can simply use standard off-the-shelf implementations with no modiﬁcation to the code for training. Our approach can handle general multiclass loss matrices, including the usual 0-1 loss but also other losses such as those used for ordinal regression problems. We also provide a quantitative regret transfer bound, which bounds the target regret on the true distribution in terms of the CPE regret on the noisy distribution; in doing so, we extend the notion of strong properness introduced for binary losses by Agarwal (2014) to the multiclass case. Our bound suggests that the sample complexity of learning under CCN increases as the noise matrix approaches singularity. We also provide ﬁxes and potential improvements for noise estimation meth-ods that involve computing anchor points. Our experiments conﬁrm our theoretical ﬁndings.
Sharper Generalization Bounds for Clustering

Existing generalization analysis of clustering mainly focuses on speciﬁc instantiations, such as (kernel) k -means, and a uniﬁed framework for studying clustering performance is still lacking. Besides, the existing excess clustering risk bounds are mostly of order O ( K/ √ n ) provided that the underlying distribution has bounded support, where n is the sample size and K is the cluster numbers, or of order O ( K 2 /n ) under strong assumptions on the underlying distribution, where these assumptions are hard to be veriﬁed in general. In this paper, we propose a uniﬁed clustering learning framework and investigate its excess risk bounds, obtaining state-of-the-art upper bounds under mild assumptions. Speciﬁcally, we derive sharper bounds of order O ( K 2 /n ) under mild assumptions on the covering number of the hypothesis spaces, where these assumptions are easy to be veriﬁed. Moreover, for the hard clustering scheme, such as (kernel) k -means, if just assume the hypothesis functions to be bounded, we improve the upper bounds from the order O ( K/ √ n ) to O ( √ K/ √ n ) . Furthermore, state-of-the-art bounds of faster order O ( K/n ) are obtained with the covering number assumptions.
SiameseXML: Siamese Networks meet Extreme Classifiers with 100M Labels

Deep extreme multi-label learning (XML) requires training deep architectures that can tag a data point with its most relevant subset of labels from an extremely large label set. XML applications such as ad and product recommendation involve labels rarely seen during training but which nevertheless hold the key to recommendations that delight users. Effective utilization of label metadata and high quality predictions for rare labels at the scale of millions of labels are thus key challenges in contemporary XML research. To address these, this paper develops the SiameseXML framework based on a novel probabilistic model that naturally motivates a modular approach melding Siamese architectures with high-capacity extreme classiﬁers, and a training pipeline that effortlessly scales to tasks with 100 million labels. SiameseXML offers predictions 2–13% more accurate than leading XML methods on public benchmark datasets, as well as in live A/B tests on the Bing search engine, it offers signiﬁcant gains in click-through-rates, coverage, revenue and other online metrics over state-of-the-art techniques currently in production. Code for SiameseXML is available at https: //github.com/Extreme-classiﬁcation/siamesexml
Approximate Group Fairness for Clustering

We incorporate group fairness into the algorithmic centroid clustering problem, where $k$ centers are to be located to serve $n$ agents distributed in a metric space. We refine the notion of proportional fairness proposed in [Chen et al., ICML 2019] as {\em core fairness}, and $k$-clustering is in the core if no coalition containing at least $n/k$ agents can strictly decrease their total distance by deviating to a new center together. Our solution concept is motivated by the situation where agents are able to coordinate and utilities are transferable. A string of existence, hardness and approximability results is provided. Particularly, we propose two dimensions to relax core requirements: one is on the degree of distance improvement, and the other is on the size of deviating coalition. For both relaxations and their combination, we study the extent to which relaxed core fairness can be satisfied in metric spaces including line, tree and general metric space, and design approximation algorithms accordingly.
Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator

This paper quantifies the uncertainty of prediction risk for the min-norm least squares estimator in high-dimensional linear regression models. We establish the asymptotic normality of prediction risk when both the sample size and the number of features tend to infinity. Based on the newly established central limit theorems (CLTs), we derive the confidence intervals of the prediction risk under various scenarios. Our results demonstrate the sample-wise non-monotonicity of the prediction risk and confirm “more data hurt” phenomenon. Furthermore, the width of confidence intervals indicates that over-parameterization would enlarge the randomness of prediction performance.
Generative Adversarial Networks for Markovian Temporal Dynamics: Stochastic Continuous Data Generation

In this paper, we present a novel generative adversarial network (GAN) that can describe Markovian temporal dynamics. To generate stochastic sequential data, we introduce a novel stochastic differential equation-based conditional generator and spatial-temporal constrained discriminator networks. To stabilize the learning dynamics of the min-max type of the GAN objective function, we propose well-posed constraint terms for both networks. We also propose a novel conditional Markov Wasserstein distance to induce a pathwise Wasserstein distance. The experimental results demonstrate that our method outperforms state-of-the-art methods using several different types of data.