Miracle-dz/UniFAR-FTUs huggingface dataset:
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning

Continual learning agents experience a stream of (related) tasks. The main challenge is that the agent must not forget previous tasks and also adapt to novel tasks in the stream. We are interested in the intersection of two recent continual-learning scenarios. In meta-continual learning , the model is pre-trained using meta-learning to minimize catastrophic forgetting of previous tasks. In continual-meta learning , the aim is to train agents for faster remembering of previous tasks through adaptation. In their original formulations, both methods have limitations. We stand on their shoulders to propose a more general scenario, OSAKA, where an agent must quickly solve new (out-of-distribution) tasks, while also requiring fast remembering. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML , an online extension of the popular MAML algorithm as a strong baseline for this scenario. We show in an empirical study that Continual-MAML is better suited to the new scenario than the aforementioned methodologies including standard continual learning and meta-learning approaches.
Unsupervised Learning of Object Landmarks via Self-Training Correspondence

This paper addresses the problem of unsupervised discovery of object landmarks. We take a different path compared to existing works, based on 2 novel perspectives: (1) Self-training: starting from generic keypoints, we propose a self-training approach where the goal is to learn a detector that improves itself, becoming more and more tuned to object landmarks. (2) Correspondence: we identify correspondence as a key objective for unsupervised landmark discovery and propose an optimization scheme which alternates between recovering object landmark correspondence across different images via clustering and learning an object landmark descriptor without labels. Compared to previous works, our approach can learn landmarks that are more ﬂexible in terms of capturing large changes in viewpoint. We show the favourable properties of our method on a variety of dif-ﬁcult datasets including LS3D, BBCPose and Human3.6M. Code is available at https://github.com/malldimi1/UnsupervisedLandmarks .
Hybrid Variance-Reduced SGD Algorithms For Minimax Problems with Nonconvex-Linear Function

We develop a novel and single-loop variance-reduced algorithm to solve a class of stochastic nonconvex-convex minimax problems involving a nonconvex-linear objective function, which has various applications in different fields such as machine learning and robust optimization. This problem class has several computational challenges due to its nonsmoothness, nonconvexity, nonlinearity, and non-separability of the objective functions. Our approach relies on a new combination of recent ideas, including smoothing and hybrid biased variance-reduced techniques. Our algorithm and its variants can achieveO(T−2/3)-convergence rate and the best known oracle complexity under standard assumptions, where T is the iteration counter. They have several computational advantages compared to existing methods such as simple to implement and less parameter tuning requirements. They can also work with both single sample or mini-batch on derivative estimators, and with constant or diminishing step-sizes. We demonstrate the benefits of our algorithms over existing methods through two numerical examples, including a nonsmooth and nonconvex-non-strongly concave minimax model.
Certified Robustness of Graph Convolution Networks for Graph Classification under Topological Attacks

Graph convolution networks (GCNs) have become effective models for graph classiﬁcation. Similar to many deep networks, GCNs are vulnerable to adversarial attacks on graph topology and node attributes. Recently, a number of effective attack and defense algorithms have been developed, but certiﬁcates of robustness against topological perturbations are currently available only for PageRank and label/feature propagation, while none has been designed for GCNs. We propose the ﬁrst algorithm for certifying the robustness of GCNs to topological attacks in the application of graph classiﬁcation . Our method is based on Lagrange dual-ization and convex envelope, which result in tight approximation bounds that are efﬁciently computable by dynamic programming. When used in conjunction with robust training, it allows an increased number of graphs to be certiﬁed as robust.
Soft Contrastive Learning for Visual Localization

Localization by image retrieval is inexpensive and scalable due to simple mapping and matching techniques. Such localization, however, depends upon the quality of image features often obtained using Contrastive learning frameworks. Most contrastive learning strategies opt for features to distinguish different classes. In the context of localization, however, there is no natural deﬁnition of classes. Therefore, images are usually artiﬁcially separated into positive/negative classes, with respect to the chosen anchor images, based on some geometric proximity measure. In this paper, we show why such divisions are problematic for learning localization features. We argue that any artiﬁcial division based on proximity measure is undesirable, due to the inherently ambiguous supervision for images near proximity threshold. To this end, we propose a novel technique that uses soft positive/negative assignments of images for contrastive learning, avoiding the aforementioned problem. Our soft assignment makes a gradual distinction between close and far images in both geometric and feature spaces. Experiments on four large-scale benchmark datasets demonstrate the superiority of our soft contrastive learning over the state-of-the-art method for retrieval-based visual localization.
Factor Graph Neural Networks

In recent years, we have witnessed a surge of Graph Neural Networks (GNNs), most of which can learn powerful representations in an end-to-end fashion with great success in many real-world applications. They have resemblance to Probabilistic Graphical Models (PGMs), but break free from some limitations of PGMs. By aiming to provide expressive methods for representation learning instead of computing marginals or most likely configurations, GNNs provide flexibility in the choice of information flowing rules while maintaining good performance. Despite their success and inspirations, they lack efficient ways to represent and learn higher-order relations among variables/nodes. More expressive higher-order GNNs which operate on k-tuples of nodes need increased computational resources in order to process higher-order tensors. We propose Factor Graph Neural Networks (FGNNs) to effectively capture higher-order relations for inference and learning. To do so, we first derive an efficient approximate Sum-Product loopy belief propagation inference algorithm for discrete higher-order PGMs. We then neuralize the novel message passing scheme into a Factor Graph Neural Network (FGNN) module by allowing richer representations of the message update rules; this facilitates both efficient inference and powerful end-to-end learning. We further show that with a suitable choice of message aggregation operators, our FGNN is also able to represent Max-Product belief propagation, providing a single family of architecture that can represent both Max and Sum-Product loopy belief propagation. Our extensive experimental evaluation on synthetic as well as real datasets demonstrates the potential of the proposed model.
Can the Brain Do Backpropagation? - Exact Implementation of Backpropagation in Predictive Coding Networks

Backpropagation (BP) has been the most successful algorithm used to train artificial neural networks. However, there are several gaps between BP and learning in biologically plausible neuronal networks of the brain (learning in the brain, or simply BL, for short), in particular, (1) it has been unclear to date, if BP can be implemented exactly via BL, (2) there is a lack of local plasticity in BP, i.e., weight updates require information that is not locally available, while BL utilizes only locally available information, and (3) there is a lack of autonomy in BP, i.e., some external control over the neural network is required (e.g., switching between prediction and learning stages requires changes to dynamics and synaptic plasticity rules), while BL works fully autonomously. Bridging such gaps, i.e., understanding how BP can be approximated by BL, has been of major interest in both neuroscience and machine learning. Despite tremendous efforts, however, no previous model has bridged the gaps at a degree of demonstrating an equivalence to BP, instead, only approximations to BP have been shown. Here, we present for the first time a framework within BL that bridges the above crucial gaps. We propose a BL model that (1) produces exactly the same updates of the neural weights as BP, while (2) employing local plasticity, i.e., all neurons perform only local computations, done simultaneously. We then modify it to an alternative BL model that (3) also works fully autonomously. Overall, our work provides important evidence for the debate on the long-disputed question whether the brain can perform BP.
Approximation Algorithms for Generalized Path Scheduling

Scheduling problems where the machines can be represented as the edges of a network and each job needs to be processed by a sequence of machines that form a path in this network have been the subject of many research articles (e.g. ﬂow shop is the special case where the network as well as the sequence of machines for each job is a simple path). In this paper we consider one such problem, called Generalized Path Scheduling (GPS) problem, which can be deﬁned as follows. Given a set of non-preemptive jobs J and identical machines M ( | J | = n and | M | = m ). The machines are ordered on a path. Each job j = { P j = { l j , r j } , p j } is deﬁned by its processing time p j and a sub-path P j from machine with index l j to r j ( l j , r j ∈ M , and l j ≤ r j ) specifying the order of machines it must go through. We assume each machine has a queue of inﬁnite size where jobs can sit in the queue to resolve conﬂicts. Two objective functions, makespan and total completion time, are considered. Machines can be identical or unrelated. In the latter case, this problem generalizes the classical Flow shop problem (in which all jobs have to go through all machines from 1 to m in that order). Generalized Path Scheduling has been studied (e.g. see [11,4]). In this paper, we present several improved approximation algorithms for both objectives. For the case of number of machines being sub-logarithmic in the number of jobs we present a PTAS for both makespan and total completion time. The PTAS holds even on unrelated machines setting and therefore, generalizes the
Cherry Picking: Consumer Choices in Swarm Dynamics, Considering Price and Quality of Goods

This paper proposes a further development of the mathematical theory of swarms to behavioral dynamics of social and economic systems, with an application to the modeling of price series in a market. The complexity features of the system are properly described by modeling the asymmetric interactions between buyers and sellers, specifically considering the so-called cherry picking phenomenon, by which not only prices but also qualities are considered when buying a good. Finally, numerical simulations are performed to depict the predictive ability of the model and to show interesting emerging behaviors, as the coordination of buyers and their division in endogenous clusters.
Escaping the Gravitational Pull of Softmax

The softmax is the standard transformation used in machine learning to map realvalued vectors to categorical distributions. Unfortunately, this transform poses serious drawbacks for gradient descent (ascent) optimization. We reveal this difficulty by establishing two negative results: (1) optimizing any expectation with respect to the softmax must exhibit sensitivity to parameter initialization (“softmax gravity well”), and (2) optimizing log-probabilities under the softmax must exhibit slow convergence (“softmax damping”). Both findings are based on an analysis of convergence rates using the Non-uniform Łojasiewicz (NŁ) inequalities. To circumvent these shortcomings we investigate an alternative transformation, the escort mapping, that demonstrates better optimization properties. The disadvantages of the softmax and the effectiveness of the escort transformation are further explained using the concept of NŁ coefficient. In addition to proving bounds on convergence rates to firmly establish these results, we also provide experimental evidence for the superiority of the escort transformation.
Robustness Analysis of Non-Convex Stochastic Gradient Descent using Biased Expectations

This work proposes a novel analysis of stochastic gradient descent (SGD) for non-convex and smooth optimization. Our analysis sheds light on the impact of the probability distribution of the gradient noise on the convergence rate of the norm of the gradient. In the case of sub-Gaussian and centered noise, we prove that, with probability 1 − δ , the number of iterations to reach a precision ε for the squared gradient norm is O ( ε − 2 ln(1 /δ )) . In the case of centered and integrable heavy-tailed noise, we show that, while the expectation of the iterates may be inﬁnite, the squared gradient norm still converges with probability 1 − δ in O ( ε − p δ − q ) iterations, where p, q > 2 . This result shows that heavy-tailed noise on the gradient slows down the convergence of SGD without preventing it, proving that SGD is robust to gradient noise with unbounded variance, a setting of interest for Deep Learning. In addition, it indicates that choosing a step size proportional to T − 1 /b where b is the tail-parameter of the noise and T is the number of iterations leads to the best convergence rates. Both results are simple corollaries of a uniﬁed analysis using the novel concept of biased expectations , a simple and intuitive mathematical tool to obtain concentration inequalities. Using this concept, we propose a new quantity to measure the amount of noise added to the gradient, and discuss its value in multiple scenarios.
The Dilemma of TriHard Loss and an Element-Weighted TriHard Loss for Person Re-Identification

Triplet loss with batch hard mining (TriHard loss) is an important variation of triplet loss inspired by the idea that hard triplets improve the performance of metric leaning networks. However, there is a dilemma in the training process. The hard negative samples contain various quite similar characteristics compared with anchors and positive samples in a batch. Features of these characteristics should be clustered between anchors and positive samples while are also utilized to repel between anchors and hard negative samples. It is harmful for learning mutual features within classes. Several methods to alleviate the dilemma are designed and tested. In the meanwhile, an element-weighted TriHard loss is emphatically proposed to enlarge the distance between partial elements of feature vectors selectively which represent the different characteristics between anchors and hard negative samples. Extensive evaluations are conducted on Market1501 and MSMT17 datasets and the results achieve state-of-the-art on public baselines. The implementation of this work is available at https://github.com/LvWilliam/EWTH-Loss.
The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space

We revisit the problem of counting the number of distinct elements F 0 ( D ) in a data stream D , over a domain [ u ] . We propose an ( ε, δ ) -differentially private algorithm that approximates F 0 ( D ) within a factor of (1 ± γ ) , and with additive error of O ( (cid:112) ln(1 /δ ) /ε ) , using space O (ln(ln( u ) /γ ) /γ 2 ) . We improve on the prior work at least quadratically and up to exponentially, in terms of both space and additive error. Our additive error guarantee is optimal up to a factor of O ( (cid:112) ln(1 /δ )) , and the space bound is optimal up to a factor of O (cid:16) min (cid:110) ln (cid:16) ln( u ) γ (cid:17) , 1 γ 2 (cid:111)(cid:17) . We assume the existence of an ideal uniform random hash function, and ignore the space required to store it. We later relax this requirement by assuming pseudo-random functions and appealing to a computational variant of differential privacy, SIM-CDP. Our algorithm is built on top of the celebrated Flajolet-Martin (FM) sketch. We show that FM-sketch is differentially private as is, as long as there are ≈ (cid:112) ln(1 /δ ) / ( εγ ) distinct elements in the data set. Along the way, we prove a structural result showing that the maximum of k i.i. d. random variables is statistically close (in the sense of ε -differential privacy) to the maximum of ( k + 1) i.i. d. samples from the same distribution, as long as k = Ω (cid:0) 1 ε (cid:1) . Finally, experiments show that our algorithms introduces error within
Distributed Distillation for On-Device Learning

On-device learning promises collaborative training of machine learning models across edge devices without the sharing of user data. In state-of-the-art on-device learning algorithms, devices communicate their model weights over a decentralized communication network. Transmitting model weights requires huge communication overhead and means only devices with identical model architectures can be included. To overcome these limitations, we introduce a distributed distillation algorithm where devices communicate and learn from soft-decision (softmax) outputs, which are inherently architecture-agnostic and scale only with the number of classes. The communicated soft-decisions are each model’s outputs on a public, unlabeled reference dataset, which serves as a common vocabulary between devices. We prove that our algorithm converges with probability 1 to a stationary point where all devices in the communication network distill the entire network’s knowledge on the reference data, regardless of their local connections. Our analysis assumes smooth loss functions, which can be non-convex. Simulations support our theoretical ﬁndings and show that even a naive implementation of our algorithm signiﬁcantly reduces the communication overhead while achieving an overall comparable performance to state-of-the-art, depending on the regime. By requiring little communication overhead and allowing for cross-architecture training, we remove two main obstacles to scaling on-device learning.
Push or Pull for a Better Selectivity? A Study on the Electronic Effects of Substituents of the Pyridine Ring on the Enantiomeric Recognition of Chiral Pyridino-18-Crown-6 Ethers

Seven dimethyl-substituted optically active pyridino-18-crown-6 ethers containing various substituents at position 4 of the pyridine ring were studied with regards to the electron-donating and -withdrawing effects of substituents on enantiomeric recognition. In order to compile this set of compounds, we prepared four novel pyridino-18-crown-6 ethers, including an intermediate of a new synthetic route for a formerly reported crown ether. The discriminating ability of these pyridino-crown ethers with C2-symmetry toward the enantiomers of protonated primary amines was examined by isothermal titration calorimetry.
Reinforcement Learning for Control with Multiple Frequencies

Many real-world sequential decision problems involve multiple action variables whose control frequencies are different, such that actions take their effects at different periods. While these problems can be formulated with the notion of multiple action persistences in factored-action MDP (FA-MDP), it is non-trivial to solve them efﬁciently since an action-persistent policy constructed from a stationary policy can be arbitrarily suboptimal, rendering solution methods for the standard FA-MDPs hardly applicable. In this paper, we formalize the problem of multiple control frequencies in RL and provide its efﬁcient solution method. Our proposed method, Action-Persistent Policy Iteration (AP-PI), provides a theoretical guarantee on the convergence to an optimal solution while incurring only a factor of |A| increase in time complexity during policy improvement step, compared to the standard policy iteration for FA-MDPs. Extending this result, we present Action-Persistent Actor-Critic (AP-AC), a scalable RL algorithm for high-dimensional control tasks. In the experiments, we demonstrate that AP-AC signiﬁcantly out-performs the baselines on several continuous control tasks and a trafﬁc control simulation, which highlights the effectiveness of our method that directly optimizes the periodic non-stationary policy for tasks with multiple control frequencies.
A New Approach for Euler-Lagrange Orbits on Compact Manifolds with Boundary

Consider a compact manifold with boundary, homeomorphic to the N-dimensional disk, and a Tonelli Lagrangian function defined on the tangent bundle. In this paper, we study the multiplicity problem for Euler-Lagrange orbits that satisfy the conormal boundary conditions and that lay on the boundary only in their extreme points. In particular, for suitable values of the energy function and under mild hypotheses, if the Tonelli Lagrangian is reversible then the minimal number of Euler-Lagrange orbits with prescribed energy that satisfies the conormal boundary conditions is N. If L is not reversible, then this number is two.
Robust Multi-Agent Reinforcement Learning with Model Uncertainty

In this work, we study the problem of multi-agent reinforcement learning (MARL) with model uncertainty, which is referred to as robust MARL . This is naturally motivated by some multi-agent applications where each agent may not have perfectly accurate knowledge of the model, e.g., all the reward functions of other agents. Little a priori work on MARL has accounted for such uncertainties, neither in problem formulation nor in algorithm design. In contrast, we model the problem as a robust Markov game , where the goal of all agents is to ﬁnd policies such that no agent has the incentive to deviate, i.e., reach some equilibrium point, which is also robust to the possible uncertainty of the MARL model. We ﬁrst introduce the solution concept of robust Nash equilibrium in our setting, and develop a Q-learning algorithm to ﬁnd such equilibrium policies, with convergence guarantees under certain conditions. In order to handle possibly enormous state-action spaces in practice, we then derive the policy gradients for robust MARL, and develop an actor-critic algorithm with function approximation. Our experiments demonstrate that the proposed algorithm outperforms several baseline MARL methods that do not account for the model uncertainty, in several standard but uncertain cooperative and competitive MARL environments.