Miracle-dz/UniFAR-FTUs huggingface dataset:
AdamD: Improved bias-correction in Adam

Here I present a small update to the bias-correction term in the Adam optimizer that has the advantage of making smaller gradient updates in the first several steps of training. With the default bias-correction, Adam may actually make larger than requested gradient updates early in training. By only including the well-justified bias-correction of the second moment gradient estimate, $v_t$, and excluding the bias-correction on the first-order estimate, $m_t$, we attain these more desirable gradient update properties in the first series of steps. The default implementation of Adam may be as sensitive as it is to the hyperparameters $\beta_1, \beta_2$ partially due to the originally proposed bias correction procedure, and its behavior in early steps.
Neuro-Symbolic Reinforcement Learning with First-Order Logic

Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast convergence and interpretability for the policy in RL, we propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network, which can learn symbolic and interpretable rules in their differentiable network. The method is first to extract first-order logical facts from text observation and external word meaning network (ConceptNet), then train a policy in the network with directly interpretable logical operators. Our experimental results show RL training with the proposed method converges significantly faster than other state-of-the-art neuro-symbolic methods in a TextWorld benchmark.
Periodic DMP formulation for Quaternion Trajectories

Imitation learning techniques have been used as a way to transfer skills to robots. Among them, dynamic movement primitives (DMPs) have been widely exploited as an effective and an efficient technique to learn and reproduce complex discrete and periodic skills. While DMPs have been properly formulated for learning point-to-point movements for both translation and orientation, periodic ones are missing a formulation to learn the orientation. To address this gap, we propose a novel DMP formulation that enables encoding of periodic orientation trajectories. Within this formulation we develop two approaches: Riemannian metric-based projection approach and unit quaternion based periodic DMP. Both formulations exploit unit quaternions to represent the orientation. However, the first exploits the properties of Riemannian manifolds to work in the tangent space of the unit sphere. The second encodes directly the unit quaternion trajectory while guaranteeing the unitary norm of the generated quaternions. We validated the technical aspects of the proposed methods in simulation. Then we performed experiments on a real robot to execute daily tasks that involve periodic orientation changes (i.e., surface polishing/wiping and liquid mixing by shaking).
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation

Noise-contrastive estimation (NCE) is a statistically consistent method for learning unnormalized probabilistic models. It has been empirically observed that the choice of the noise distribution is crucial for NCE's performance. However, such observations have never been made formal or quantitative. In fact, it is not even clear whether the difficulties arising from a poorly chosen noise distribution are statistical or algorithmic in nature. In this work, we formally pinpoint reasons for NCE's poor performance when an inappropriate noise distribution is used. Namely, we prove these challenges arise due to an ill-behaved (more precisely, flat) loss landscape. To address this, we introduce a variant of NCE called"eNCE"which uses an exponential loss and for which normalized gradient descent addresses the landscape issues provably when the target and noise distributions are in a given exponential family.
Properties of Very Broad Line MgII Radio-Loud and Radio-Quiet Quasars

We perform an analysis of the properties of radio-loud (RL) and radio-quiet (RQ) quasars with MgII broad emission line (i-band magnitude ≤19.1 and z ≤1.9), selected from the parent sample of SDSS DR7 catalogue. For sources with full-width half maxima (FWHM) greater than 15,000 km s−1 (very broad line sample; VBL) we find the radio loud fraction (RLF) to be about 40%. To further investigate this result we compare the bolometric luminosity, optical continuum luminosity, black hole (BH) mass and Eddington ratios of our VBL sample of RL and RQ quasars. Our analysis shows that in our VBL sample space, RL quasars have higher luminosities and BH mass than RQ quasars. The similarity in the distribution of their covering fraction (CF) shows that there is no difference in dust distribution between VBL RL and RQ quasars and hence dust is not affecting our results. We also find that there is no correlation of RL quasar properties with optical continuum luminosity and BH mass.
Toward Accurate and Reliable Iris Segmentation Using Uncertainty Learning

Iris segmentation is a deterministic part of the iris recognition system. Unreliable segmentation of iris regions especially the limbic area is still the bottleneck problem, which impedes more accurate recognition. To make further efforts on accurate and reliable iris segmentation, we propose a bilateral self-attention module and design Bilateral Transformer (BiTrans) with hierarchical architecture by exploring spatial and visual relationships. The bilateral self-attention module adopts a spatial branch to capture spatial contextual information without resolution reduction and a visual branch with a large receptive field to extract the visual contextual features. BiTrans actively applies convolutional projections and cross-attention to improve spatial perception and hierarchical feature fusion. Besides, Iris Segmentation Uncertainty Learning is developed to learn the uncertainty map according to prediction discrepancy. With the estimated uncertainty, a weighting scheme and a regularization term are designed to reduce predictive uncertainty. More importantly, the uncertainty estimate reflects the reliability of the segmentation predictions. Experimental results on three publicly available databases demonstrate that the proposed approach achieves better segmentation performance using 20% FLOPs of the SOTA IrisParseNet.
Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation

Robust localization in dense urban scenarios using a low-cost sensor setup and sparse HD maps is highly relevant for the current advances in autonomous driving, but remains a challenging topic in research. We present a novel monocular localization approach based on a sliding-window pose graph that leverages predicted uncertainties for increased precision and robustness against challenging scenarios and per-frame failures. To this end, we propose an efficient multi-task uncertainty-aware perception module, which covers semantic segmentation, as well as bounding box detection, to enable the localization of vehicles in sparse maps, containing only lane borders and traffic lights. Further, we design differentiable cost maps that are directly generated from the estimated uncertainties. This opens up the possibility to minimize the reprojection loss of amorphous map elements in an association-free and uncertainty-aware manner. Extensive evaluation on the Lyft 5 dataset shows that, despite the sparsity of the map, our approach enables robust and accurate 6D localization in challenging urban scenarios using only monocular camera images and vehicle odometry.
Modeling Human-Human Collaboration: A Connection Between Inter-Personal Motor Synergy and Consensus Algorithms

Many day-to-day activities involve people working collaboratively toward reaching a desired outcome. Previous research in motor control and neuroscience have proposed inter-personal motor synergy (IPMS) as a mechanism of collaboration between people, referring to the idea of how two or more people may work together "as if they were one" to coordinate their motion. In motor control literature, the uncontrolled manifold (UCM) is used for quantifying IPMS. According to this approach, coordinated motion is achieved by stabilizing of a performance variable (e.g., an output in a collaborative output tracking task). We show that the UCM approach is closely related to the well-studied consensus approach in multi-agent systems that concerns processes by which a set of interacting agents agree on a shared objective. To explore the connection between these two approaches, in this paper, we provide a control-theoretic model that represents the systems-level behaviors in a collaborative task. In particular, we utilize the consensus protocol and show how the model can be systematically tuned to reproduce the behavior exhibited by human-human collaboration (HHC) experiments. We discuss the association between the proposed control law and the UCM approach and validate our model using experimental results previously collected from an inter-personal finger force production task.
A Gentle Introduction to Bayesian Hierarchical Linear Regression Models

Considering the flexibility and applicability of Bayesian modeling, in this work we revise the main characteristics of two hierarchical models in a regression setting. We study the full probabilistic structure of the models along with the full conditional distribution for each model parameter. Under our hierarchical extensions, we allow the mean of the second stage of the model to have a linear dependency on a set of covariates. The Gibbs sampling algorithms used to obtain samples when fitting the models are fully described and derived. In addition, we consider a case study in which the plant size is characterized as a function of nitrogen soil concentration and a grouping factor (farm).
Real-time Identification and Simultaneous Avoidance of Static and Dynamic Obstacles on Point Cloud for UAVs Navigation

Avoiding hybrid obstacles in unknown scenarios with an efficient flight strategy is a key challenge for unmanned aerial vehicle applications. In this paper, we introduce a more robust technique to distinguish and track dynamic obstacles from static ones with only point cloud input. Then, to achieve dynamic avoidance, we propose the forbidden pyramids method to solve the desired vehicle velocity with an efficient sampling-based method in iteration. The motion primitives are generated by solving a nonlinear optimization problem with the constraint of desired velocity and the waypoint. Furthermore, we present several techniques to deal with the position estimation error for close objects, the error for deformable objects, and the time gap between different submodules. The proposed approach is implemented to run onboard in real-time and validated extensively in simulation and hardware tests, demonstrating our superiority in tracking robustness, energy cost, and calculating time.
Layer-wise Adaptive Model Aggregation for Scalable Federated Learning

In Federated Learning (FL), a common approach for aggregating local solutions across clients is periodic full model averaging. It is, however, known that different layers of neural networks can have a different degree of model discrepancy across the clients. The conventional full aggregation scheme does not consider such a difference and synchronizes the whole model parameters at once, resulting in inefficient network bandwidth consumption. Aggregating the parameters that are similar across the clients does not make meaningful training progress while increasing the communication cost. We propose FedLAMA, a layer-wise adaptive model aggregation scheme for scalable FL. FedLAMA adjusts the aggregation interval in a layer-wise manner, jointly considering the model discrepancy and the communication cost. This fine-grained aggregation strategy enables to reduce the communication cost without significantly harming the model accuracy. Our extensive empirical study shows that, as the aggregation interval increases, FedLAMA shows a remarkably smaller accuracy drop than the periodic full aggregation, while achieving comparable communication efficiency.
Quantum batteries at the verge of a phase transition

Starting from the observation that the reduced state of a system strongly coupled to a bath is, in general, an athermal state, we introduce and study a cyclic battery–charger quantum device that is in thermal equilibrium, or in a ground state, during the charge storing stage. The cycle has four stages: the equilibrium storage stage is interrupted by disconnecting the battery from the charger, then work is extracted from the battery, and then the battery is reconnected with the charger; finally, the system is brought back to equilibrium. At no point during the cycle are the battery–charger correlations artificially erased. We study the case where the battery and charger together comprise a spin-1/2 Ising chain, and show that the main characteristics—the extracted energy and the thermodynamic efficiency—can be enhanced by operating the cycle close to the quantum phase transition point. When the battery is just a single spin, we find that the output work and efficiency show a scaling behavior at criticality and derive the corresponding critical exponents. Due to always present correlations between the battery and the charger, operations that are equivalent from the perspective of the battery can entail different energetic costs for switching the battery–charger coupling. This happens only when the coupling term does not commute with the battery’s bare Hamiltonian, and we use this purely quantum leverage to further optimize the performance of the device.
Modeling Performance in Open-Domain Dialogue with PARADISE

There has recently been an explosion of work on spoken dialogue systems, along with an increased interest in open-domain systems that engage in casual conversations on popular topics such as movies, books and music. These systems aim to socially engage, entertain, and even empathize with their users. Since the achievement of such social goals is hard to measure, recent research has used dialogue length or human ratings as evaluation metrics, and developed methods for automatically calculating novel metrics, such as coherence, consistency, relevance and engagement. Here we develop a PARADISE model for predicting the performance of Athena, a dialogue system that has participated in thousands of conversations with real users, while competing as a finalist in the Alexa Prize. We use both user ratings and dialogue length as metrics for dialogue quality, and experiment with predicting these metrics using automatic features that are both system dependent and independent. Our goal is to learn a general objective function that can be used to optimize the dialogue choices of any Alexa Prize system in real time and evaluate its performance. Our best model for predicting user ratings gets an R$^2$ of . 136 with a DistilBert model, and the best model for predicting length with system independent features gets an R$^2$ of . 865, suggesting that conversation length may be a more reliable measure for automatic training of dialogue systems.
A Row-Wise Update Algorithm for Sparse Stochastic Matrix Factorization

Nonnegative matrix factorization arises widely in machine learning and data analysis. In this paper, for a given factorization of rank r , we consider the sparse stochastic matrix factorization (SSMF) of decomposing a prescribed m -by- n stochastic matrix V into a product of an m -by- r stochastic matrix W and a sparse r -by- n stochastic matrix H . With the prescribed sparsity level, we reformulate the SSMF as an unconstrained nonvonvex-nonsmooth minimization problem and introduce a row-wise update algorithm for solving the minimization problem. We show that the proposed algorithm converges globally and the generated sequence converges to a special critical point of the cost function, which is a global mini-mizer over the W -factor as a whole and is nearly a global minimizer over each row vector of the H -factor. Numerical experiments on both synthetic and real data sets are given to demonstrate the eﬀectiveness of our proposed algorithm.
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

CLIP yielded impressive results on zero-shot transfer learning tasks and is considered as a foundation model like BERT or GPT3. CLIP vision models that have a rich representation are pre-trained using the InfoNCE objective and natural language supervision before they are fine-tuned on particular tasks. Though CLIP excels at zero-shot transfer learning, it suffers from an explaining away problem, that is, it focuses on one or few features, while neglecting other relevant features. This problem is caused by insufficiently extracting the covariance structure in the original multi-modal data. We suggest to use modern Hopfield networks to tackle the problem of explaining away. Their retrieved embeddings have an enriched covariance structure derived from co-occurrences of features in the stored embeddings. However, modern Hopfield networks increase the saturation effect of the InfoNCE objective which hampers learning. We propose to use the InfoLOOB objective to mitigate this saturation effect. We introduce the novel"Contrastive Leave One Out Boost"(CLOOB), which uses modern Hopfield networks for covariance enrichment together with the InfoLOOB objective. In experiments we compare CLOOB to CLIP after pre-training on the Conceptual Captions and the YFCC dataset with respect to their zero-shot transfer learning performance on other datasets. CLOOB consistently outperforms CLIP at zero-shot transfer learning across all considered architectures and datasets.
HALP: Hardware-Aware Latency Pruning

Structural pruning can simplify network architecture and improve inference speed. We propose Hardware-Aware Latency Pruning (HALP) that formulates structural pruning as a global resource allocation optimization problem, aiming at maximizing the accuracy while constraining latency under a predefined budget. For filter importance ranking, HALP leverages latency lookup table to track latency reduction potential and global saliency score to gauge accuracy drop. Both metrics can be evaluated very efficiently during pruning, allowing us to reformulate global structural pruning under a reward maximization problem given target constraint. This makes the problem solvable via our augmented knapsack solver, enabling HALP to surpass prior work in pruning efficacy and accuracy-efficiency trade-off. We examine HALP on both classification and detection tasks, over varying networks, on ImageNet and VOC datasets. In particular, for ResNet-50/-101 pruning on ImageNet, HALP improves network throughput by $1.60\times$/$1.90\times$ with $+0.3\%$/$-0.2\%$ top-1 accuracy changes, respectively. For SSD pruning on VOC, HALP improves throughput by $1.94\times$ with only a $0.56$ mAP drop. HALP consistently outperforms prior art, sometimes by large margins.
Prediction and comparative analysis of CTCF binding sites based on a first principle approach

We calculated the patterns for the CCCTC transcription factor (CTCF) binding sites across many genomes on a first principle approach. The validation of the first principle method was done on the human as well as on the mouse genome. The predicted human CTCF binding sites are consistent with the consensus sequence, ChIP-seq data for the K562 cell, nucleosome positions for IMR90 cell as well as the CTCF binding sites in the mouse HOXA gene. The analysis of Homo sapiens, Mus musculus, Sus scrofa, Capra hircus and Drosophila melanogaster whole genomes shows: binding sites are organized in cluster-like groups, where two consecutive sites obey a power-law with coefficient ranging from 0.3292 ± 0.0068 to 0.5409 ± 0.0064; the distance between these groups varies from 18.08 ± 0.52 kbp to 42.1 ± 2.0 kbp. The genome of Aedes aegypti does not show a power law, but 19.9% of binding sites are 144 ± 4 and 287 ± 5 bp distant of each other. We run negative tests, confirming the under-representation of CTCF binding sites in Caenorhabditis elegans, Plasmodium falciparum and Arabidopsis thaliana complete genomes.
Can Q-learning solve Multi Armed Bantids?

When a reinforcement learning (RL) method has to decide between several optional policies by solely looking at the received reward, it has to implicitly optimize a Multi-Armed-Bandit (MAB) problem. This arises the question: are current RL algorithms capable of solving MAB problems? We claim that the surprising answer is no. In our experiments we show that in some situations they fail to solve a basic MAB problem, and in many common situations they have a hard time: They suffer from regression in results during training, sensitivity to initialization and high sample complexity. We claim that this stems from variance differences between policies, which causes two problems: The first problem is the"Boring Policy Trap"where each policy have a different implicit exploration depends on its rewards variance, and leaving a boring, or low variance, policy is less likely due to its low implicit exploration. The second problem is the"Manipulative Consultant"problem, where value-estimation functions used in deep RL algorithms such as DQN or deep Actor Critic methods, maximize estimation precision rather than mean rewards, and have a better loss in low-variance policies, which cause the network to converge to a sub-optimal policy. Cognitive experiments on humans showed that noised reward signals may paradoxically improve performance. We explain this using the aforementioned problems, claiming that both humans and algorithms may share similar challenges in decision making. Inspired by this result, we propose the Adaptive Symmetric Reward Noising (ASRN) method, by which we mean equalizing the rewards variance across different policies, thus avoiding the two problems without affecting the environment's mean rewards behavior. We demonstrate that the ASRN scheme can dramatically improve the results.
Unified Style Transfer

Currently, it is hard to compare and evaluate different style transfer algorithms due to chaotic definitions of style and the absence of agreed objective validation methods in the study of style transfer. In this paper, a novel approach, the Unified Style Transfer (UST) model, is proposed. With the introduction of a generative model for internal style representation, UST can transfer images in two approaches, i.e., Domain-based and Image-based, simultaneously. At the same time, a new philosophy based on the human sense of art and style distributions for evaluating the transfer model is presented and demonstrated, called Statistical Style Analysis. It provides a new path to validate style transfer models' feasibility by validating the general consistency between internal style representation and art facts. Besides, the translation-invariance of AdaIN features is also discussed.