Miracle-dz/UniFAR-FTUs huggingface dataset:
Is Geometry Enough for Matching in Visual Localization?

In this paper, we propose to go beyond the well-established approach to vision-based localization that relies on visual descriptor matching between a query image and a 3D point cloud. While matching keypoints via visual descriptors makes localization highly accurate, it has significant storage demands, raises privacy concerns and requires update to the descriptors in the long-term. To elegantly address those practical challenges for large-scale localization, we present GoMatch, an alternative to visual-based matching that solely relies on geometric information for matching image keypoints to maps, represented as sets of bearing vectors. Our novel bearing vectors representation of 3D points, significantly relieves the cross-modal challenge in geometric-based matching that prevented prior work to tackle localization in a realistic environment. With additional careful architecture design, GoMatch improves over prior geometric-based matching work with a reduction of (10.67m,95.7deg) and (1.43m, 34.7deg) in average median pose errors on Cambridge Landmarks and 7-Scenes, while requiring as little as 1.5/1.7% of storage capacity in comparison to the best visual-based matching methods. This confirms its potential and feasibility for real-world localization and opens the door to future efforts in advancing city-scale visual localization methods that do not require storing visual descriptors.
Robust Coordinate Ascent Variational Inference with Markov chain Monte Carlo simulations

Variational Inference (VI) is a method that approximates a diﬃcult-to-compute posterior density using better behaved distributional families. VI is an alternative to the already well-studied Markov chain Monte Carlo (MCMC) method of approximating densities. With each algorithm, there are of course beneﬁts and drawbacks; does there exist a combination of the two that mitigates the ﬂaws of both? We propose a method to combine Coordinate Ascent Variational Inference (CAVI) with MCMC. This new methodology, termed Hybrid CAVI, seeks to improve the sensitivity to initialization and convergence problems of CAVI by proposing an initialization using method of moments estimates obtained from a short MCMC burn-in period. Unlike CAVI, Hybrid CAVI proves to also be eﬀective when the posterior is not from a conditionally conjugate exponential family.
On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields

The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random ﬁelds are mathematical structures that models the interaction between these variables by means of an inverse temperature parameter, responsible for controlling the spatial dependence structure along the ﬁeld. In this paper, we derive closed-form expressions for the Kullback-Leibler divergence between two pairwise isotropic Gaussian-Markov random ﬁelds in both univariate and multivariate cases. The proposed equation allows the development of novel similarity measures in image processing and machine learning applications, such as image denoising and unsupervised metric learning.
Lahjoita puhetta - a large-scale corpus of spoken Finnish with some benchmarks

The Donate Speech campaign has so far succeeded in gathering approximately 3600 hours of ordinary, colloquial Finnish speech into the Lahjoita puhetta (Donate Speech) corpus. The corpus includes over twenty thousand speakers from all the regions of Finland and from all age brackets. The primary goals of the collection were to create a representative, large-scale resource to study spontaneous spoken Finnish and to accelerate the development of language technology and speech-based services. In this paper, we present the collection process and the collected corpus, and showcase its versatility through multiple use cases. The evaluated use cases include: automatic speech recognition of spontaneous speech, detection of age, gender, dialect and topic and metadata analysis. We provide benchmarks for the use cases, as well down loadable, trained baseline systems with open-source code for reproducibility. One further use case is to verify the metadata and transcripts given in this corpus itself, and to suggest artificial metadata and transcripts for the part of the corpus where it is missing.
Neural Neighbor Style Transfer

We propose Neural Neighbor Style Transfer (NNST), a pipeline that offers state-of-the-art quality, generalization, and competitive efficiency for artistic style transfer. Our approach is based on explicitly replacing neural features extracted from the content input (to be stylized) with those from a style exemplar, then synthesizing the final output based on these rearranged features. While the spirit of our approach is similar to prior work, we show that our design decisions dramatically improve the final visual quality.
Multitasking Framework for Unsupervised Simple Definition Generation

The definition generation task can help language learners by providing explanations for unfamiliar words. This task has attracted much attention in recent years. We propose a novel task of Simple Definition Generation (SDG) to help language learners and low literacy readers. A significant challenge of this task is the lack of learner’s dictionaries in many languages, and therefore the lack of data for supervised training. We explore this task and propose a multitasking framework SimpDefiner that only requires a standard dictionary with complex definitions and a corpus containing arbitrary simple texts. We disentangle the complexity factors from the text by carefully designing a parameter sharing scheme between two decoders. By jointly training these components, the framework can generate both complex and simple definitions simultaneously. We demonstrate that the framework can generate relevant, simple definitions for the target words through automatic and manual evaluations on English and Chinese datasets. Our method outperforms the baseline model by a 1.77 SARI score on the English dataset, and raises the proportion of the low level (HSK level 1-3) words in Chinese definitions by 3.87%.
Kullback–Leibler control for discrete-time nonlinear systems on continuous spaces

Kullback–Leibler (KL) control enables efficient numerical methods for nonlinear optimal control problems. The crucial assumption of KL control is the full controllability of transition distributions. However, this assumption is often violated when the dynamics evolves in a continuous space. Consequently, applying KL control to problems with continuous spaces requires some approximation, which leads to the loss of the optimality. To avoid such an approximation, in this paper, we reformulate the KL control problem for continuous spaces so that it does not require unrealistic assumptions. The key difference between the original and reformulated KL control is that the former measures the control effort by the KL divergence between controlled and uncontrolled transition distributions while the latter replaces the uncontrolled transition by a noise-driven transition. We show that the reformulated KL control admits efficient numerical algorithms like the original one without unreasonable assumptions. Specifically, the associated value function can be computed by using a Monte Carlo method based on its path integral representation.
SD^2: Slicing and Dicing Scholarly Data for Interactive Evaluation of Academic Performance

Comprehensively evaluating and comparing researchers' academic performance is complicated due to the intrinsic complexity of scholarly data. Different scholarly evaluation tasks often require the publication and citation data to be investigated in various manners. In this paper, we present an interactive visualization framework, SD^2, to enable flexible data partition and composition to support various analysis requirements within a single system. SD^2 features the hierarchical histogram, a novel visual representation for flexibly slicing and dicing the data, allowing different aspects of scholarly performance to be studied and compared. We also leverage the state-of-the-art set visualization technique to select individual researchers or combine multiple scholars for comprehensive visual comparison. We conduct multiple rounds of expert evaluation to study the effectiveness and usability of SD^2 and revise the design and system implementation accordingly. The effectiveness of SD^2 is demonstrated via multiple usage scenarios with each aiming to answer a specific, commonly raised question.
Effective Explanations for Entity Resolution Models

Entity resolution (ER) aims at matching records that refer to the same real-world entity. Although widely studied for the last 50 years, ER still represents a challenging data management problem, and several recent works have started to investigate the opportunity of applying deep learning (DL) techniques to solve this problem. In this paper, we study the fundamental problem of explainability of the DL solution for ER. Understanding the matching predictions of an ER solution is indeed crucial to assess the trustworthiness of the DL model and to discover its biases. We treat the DL model as a black box classifier and - while previous approaches to provide explanations for DL predictions are agnostic to the classification task - we propose the CERTA approach that is aware of the semantics of the ER problem. Our approach produces both saliency explanations, which associate each attribute with a saliency score, and counterfactual explanations, which provide examples of values that can flip the prediction. CERTA builds on a probabilistic framework that aims at computing the explanations evaluating the outcomes produced by using perturbed copies of the input records. We experimentally evaluate CERTA'S explanations of state-of-the-art ER solutions based on DL models using publicly available datasets, and demonstrate the effectiveness of CERTA over recently proposed methods for this problem.
Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs

Recent advances in deep learning have enabled optimization of deep reactive policies (DRPs) for continuous MDP planning by encoding a parametric policy as a deep neural network and exploiting automatic differentiation in an end-to-end model-based gradient descent framework. This approach has proven effective for optimizing DRPs in nonlinear continuous MDPs, but it requires a large number of sampled trajectories to learn effectively and can suffer from high variance in solution quality. In this work, we revisit the overall model-based DRP objective and instead take a minorization-maximization perspective to iteratively optimize the DRP w.r. t. a locally tight lower-bounded objective. This novel formulation of DRP learning as iterative lower bound optimization (ILBO) is particularly appealing because (i) each step is structurally easier to optimize than the overall objective, (ii) it guarantees a monotonically improving objective under certain theoretical conditions, and (iii) it reuses samples between iterations thus lowering sample complexity. Empirical evaluation confirms that ILBO is significantly more sample-efficient than the state-of-the-art DRP planner and consistently produces better solution quality with lower variance. We additionally demonstrate that ILBO generalizes well to new problem instances (i.e., different initial states) without requiring retraining.
H\"older regularity for fractional $p$-Laplace equations

We give an alternative proof for H\"older regularity for weak solutions of nonlocal elliptic quasilinear equations modelled on the fractional p-Laplacian where we replace the discrete De Giorgi iteration on a sequence of concentric balls by a continuous iteration. This work can be viewed as the nonlocal counterpart to the ideas developed by Tiziano Granucci.
A Dearth of Close-in Stellar Companions to M-dwarf TESS Objects of Interest

TESS has proven to be a powerful resource for finding planets, including those that orbit the most prevalent stars in our galaxy: M dwarfs. Identification of stellar companions (both bound and unbound) has become a standard component of the transiting planet confirmation process in order to assess the level of light-curve dilution and the possibility of the target being a false positive. Studies of stellar companions have also enabled investigations into stellar multiplicity in planet-hosting systems, which has wide-ranging implications for both exoplanet detection and characterization, as well as for the formation and evolution of planetary systems. Speckle and AO imaging are some of the most efficient and effective tools for revealing close-in stellar companions; we therefore present observations of 58 M-dwarf TOIs obtained using a suite of speckle imagers at the 3.5 m WIYN telescope, the 4.3 m Lowell Discovery Telescope, and the 8.1 m Gemini North and South telescopes. These observations, as well as near-infrared adaptive optics images obtained for a subset (14) of these TOIs, revealed only two close-in stellar companions. Upon surveying the literature, and cross-matching our sample with Gaia, SUPERWIDE, and the catalog from El-Badry et al., we reveal an additional 15 widely separated common proper motion companions. We also evaluate the potential for undetected close-in companions. Taking into consideration the sensitivity of the observations, our findings suggest that the orbital period distribution of stellar companions to planet-hosting M dwarfs is shifted to longer periods compared to the expected distribution for field M dwarfs.
A New Virtual Oscillator-Based Grid-Forming Controller with Decoupled Control Over Individual Phases and Improved Performance of Unbalanced Fault Ride-Through

Virtual oscillator (VO) control is the latest and promising control technique for grid-forming inverters. VO controllers (VOCs) can simultaneously provide time-domain synchronization and steady-state droop functionalities. In recent developments, VOCs have been integrated into system-level control architecture to provide additional functionalities such as capacitor voltage regulation and overcurrent protection. This article has further refined the existing VO-based system-level grid-forming control architecture with two essential and specific research contributions. The first contribution is to improve the decoupling in control over individual phases, which is achieved by combining symmetrical-component-based VOC with nested voltage and current controls. The proposed phase decoupling control feature improves the grid-forming performance of existing VOCs in the presence of unbalanced grid voltages. The second contribution is to improve the fault ride-through performance under unbalanced faults by modifying the feedback of the existing VOC. Systematic development of the proposed grid-forming control architecture with analytical reasoning is presented. The proposed controller is successfully validated using simulations and experiments.
Evaluation of IoT Self-healing Mechanisms using Fault-Injection in Message Brokers

The widespread use of Internet-of-Things (IoT) across different application domains leads to an increased concern regarding their dependability, especially as the number of potentially mission-critical systems becomes considerable. Fault-tolerance has been used to reduce the impact of faults in systems, and their adoption in IoT is becoming a necessity. This work focuses on how to exercise fault-tolerance mechanisms by deliberately provoking its malfunction. We start by describing a proof-of-concept fault-injection add-on to a commonly used publish/subscribe broker. We then present several experiments mimicking real-world IoT scenarios, focusing on injecting faults in systems with (and without) active self-healing mechanisms and comparing their behavior to the baseline without faults. We observe evidence that fault-injection can be used to (a) exercise in-place fault-tolerance apparatus, and (b) detect when these mechanisms are not performing nominally, providing insights into enhancing in-place fault-tolerance techniques.
Bi-level Optimization for hyperparameters in Nonnegative Matrix Factorizations

Hyperparameters (HPs) Optimization in machine learning algorithms represents an open problem with a direct impact on algorithm performances and on the knowledge extraction process from data. Matrix Decompositions (MDs) has recently gained more attention in data science as mathematical techniques able to capture latent information embedded in large datasets. MDs can be formalized as penalized optimization problems in which the tuning of the penalization HPs represents an issue. Current literature panorama does not provide any general framework addressing optimally conditions for the best conﬁguration of HPs. In this work, we focus on the role the HPs play in the penalized Nonnegative Matrix Factorizations (NMF) context and on the importance of their correct selection. Using a bi-level optimization approach, HPs selection is considered from an optimization point of view and their choice is directly incorporated in the unsupervised algorithm as a part of the updating process. Either theorems of existence and convergence of numerical solutions, under appropriate assumptions, are studied and a novel algorithm, namely AltBi-Alternate bi-level Algorithm, is proposed to tune the penalization HP in NMF problems.
Synthesis of memristive one-port circuits with piecewise-smooth characteristics

A generalized approach for the implementation of memristive two-terminal circuits with piesewise-smooth characteristics is proposed on the example of a multifunctional circuit based on a transistor switch. Two versions of the circuit are taken into consideration: an experimental model of the piecewise-smooth memristor (Chua's memristor) and a piecewise-smooth memristive capacitor. Physical experiments are combined with numerical modelling of the discussed circuit models. Thus, it is demonstrated that the considered circuit is a flexible solution for synthesis of a wide range of memristive systems with tuneable characteristics.
Out-of-Core Edge Partitioning at Linear Run-Time

Graph edge partitioning is an important prepro-cessing step to optimize distributed computing jobs on graph-structured data. The edge set of a given graph is split into $k$ equally-sized partitions, such that the replication of vertices across partitions is minimized. Out-of-core edge partitioning algorithms are able to tackle the problem with low memory over-head. Existing out-of-core algorithms mainly work in a streaming manner and can be grouped into two types. While stateless streaming edge partitioning is fast and yields low partitioning quality, stateful streaming edge partitioning yields better quality, but is expensive, as it requires a scoring function to be evaluated for every edge on every partition, leading to a time complexity of O(|E| *k). In this paper, we propose 2PS-L, a novel out-of-core edge partitioning algorithm that builds upon the stateful streaming model, but achieves linear run-time i.e.,O(|E|)). 2PS-L consists of two phases. In the first phase, vertices are separated into clusters by a lightweight streaming clustering algorithm. In the second phase, the graph is re-streamed and vertex clustering from the first phase is exploited to reduce the search space of graph partitioning to only two target partitions for every edge. Our evaluations show that 2PS-L can achieve better partitioning quality than existing stateful streaming edge partitioners while having a much lower run-time. As a consequence, the total run-time of partitioning and subsequent distributed graph processing can be significantly reduced.
Possibility Before Utility: Learning And Using Hierarchical Affordances

Reinforcement learning algorithms struggle on tasks with complex hierarchical dependency structures. Humans and other intelligent agents do not waste time assessing the utility of every high-level action in existence, but instead only consider ones they deem possible in the first place. By focusing only on what is feasible, or"afforded", at the present moment, an agent can spend more time both evaluating the utility of and acting on what matters. To this end, we present Hierarchical Affordance Learning (HAL), a method that learns a model of hierarchical affordances in order to prune impossible subtasks for more effective learning. Existing works in hierarchical reinforcement learning provide agents with structural representations of subtasks but are not affordance-aware, and by grounding our definition of hierarchical affordances in the present state, our approach is more flexible than the multitude of approaches that ground their subtask dependencies in a symbolic history. While these logic-based methods often require complete knowledge of the subtask hierarchy, our approach is able to utilize incomplete and varying symbolic specifications. Furthermore, we demonstrate that relative to non-affordance-aware methods, HAL agents are better able to efficiently learn complex tasks, navigate environment stochasticity, and acquire diverse skills in the absence of extrinsic supervision -- all of which are hallmarks of human learning.
Supervised Training of Siamese Spiking Neural Networks with Earth Mover's Distance

This study adapts the highly-versatile siamese neural network model to the event data domain. We introduce a supervised training framework for optimizing Earth Mover's Distance (EMD) between spike trains with spiking neural networks (SNN). We train this model on images of the MNIST dataset converted into spiking domain with novel conversion schemes. The quality of the siamese embeddings of input images was evaluated by measuring the classifier performance for different dataset coding types. The models achieved performance similar to existing SNN-based approaches (F1-score of up to 0.9386) while using only about 15% of hidden layer neurons to classify each example. Furthermore, models which did not employ a sparse neural code were about 45% slower than their sparse counterparts. These properties make the model suitable for low energy consumption and low prediction latency applications.
Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature. Such prior work often require extensive trial-and-error training along with task-specific tuning of reward functions, which makes applying dexterous manipulation for general purpose problems quite impractical. A sample-efficient and practical alternate to trial-and-error learning is imitation learning. However, collecting and learning from demonstrations in dexterous manipulation is quite challenging due to the high-dimensional action-space involved with multi-finger control. In this work, we propose ‘Dexterous Imitation Made Easy’ (DIME) a new imitation learning framework for dexterous manipulation. DIME only requires a single RGB camera that observes a human operator to teleoperate a robotic hand. Once demonstrations are collected, DIME employs state-of-the-art imitation learning methods to train dexterous manipulation policies. On real robot benchmarks we demonstrate that DIME can be used to solve complex, in-hand manipulation tasks such as ‘flipping’, ‘spinning’, and ‘rotating’ objects with just 30 demonstrations and no additional robot training. Our code, pre-collected demonstrations, and robot videos are publicly available at: https://nyu-robot-learning.github. io/dime.