Miracle-dz/UniFAR-FTUs huggingface dataset:
A Unified Framework for Domain Adaptive Pose Estimation

While pose estimation is an important computer vision task, it requires expensive annotation and suffers from domain shift. In this paper, we investigate the problem of domain adaptive 2D pose estimation that transfers knowledge learned on a synthetic source domain to a target domain without supervision. While several domain adaptive pose estimation models have been proposed recently, they are not generic but only focus on either human pose or animal pose estimation, and thus their effectiveness is somewhat limited to specific scenarios. In this work, we propose a unified framework that generalizes well on various domain adaptive pose estimation problems. We propose to align representations using both input-level and output-level cues (pixels and pose labels, respectively), which facilitates the knowledge transfer from the source domain to the unlabeled target domain. Our experiments show that our method achieves state-of-the-art performance under various domain shifts. Our method outperforms existing baselines on human pose estimation by up to 4.5 percent points (pp), hand pose estimation by up to 7.4 pp, and animal pose estimation by up to 4.8 pp for dogs and 3.3 pp for sheep. These results suggest that our method is able to mitigate domain shift on diverse tasks and even unseen domains and objects (e.g., trained on horse and tested on dog). Our code will be publicly available at: https://github.com/VisionLearningGroup/UDA_PoseEstimation.
Learning Disentangled Representations of Negation and Uncertainty

Negation and uncertainty modeling are long-standing tasks in natural language processing. Linguistic theory postulates that expressions of negation and uncertainty are semantically independent from each other and the content they modify. However, previous works on representation learning do not explicitly model this independence. We therefore attempt to disentangle the representations of negation, uncertainty, and content using a Variational Autoencoder. We find that simply supervising the latent representations results in good disentanglement, but auxiliary objectives based on adversarial learning and mutual information minimization can provide additional disentanglement gains.
Random walks on the circle and Diophantine approximation

Random walks on the circle group R/Z$\mathbb {R}/\mathbb {Z}$ whose elementary steps are lattice variables with span α∉Q$\alpha \not\in \mathbb {Q}$ or p/q∈Q$p/q \in \mathbb {Q}$ taken mod Z$\mathbb {Z}$ exhibit delicate behavior. In the rational case, we have a random walk on the finite cyclic subgroup Zq$\mathbb {Z}_q$ , and the central limit theorem and the law of the iterated logarithm follow from classical results on finite state space Markov chains. In this paper, we extend these results to random walks with irrational span α$\alpha$ , and explicitly describe the transition of these Markov chains from finite to general state space as p/q→α$p/q \rightarrow \alpha$ along the sequence of best rational approximations. We also consider the rate of weak convergence to the stationary distribution in the Kolmogorov metric, and in the rational case observe a phase transition from polynomial to exponential decay after ≈q2$\approx q^2$ steps. This seems to be a new phenomenon in the theory of random walks on compact groups. In contrast, the rate of weak convergence to the stationary distribution in the total variation metric is purely exponential.
Efficient Fair Division with Minimal Sharing

When assets are to be divided among several partners, for example, a partnership split, fair division theory can be used to determine a fair allocation. The applicability of existing approaches is limited as they either treat assets as divisible resources that end up being shared among participants or deal with indivisible objects providing only approximate fairness. In practice, sharing is often possible but undesirable, and approximate fairness is not adequate, particularly for highly valuable assets. In “Efficient Fair Division with Minimal Sharing,” Sandomirskiy and Segal-Halevi introduce a novel approach offering a middle ground: the number of shared objects is minimized while maintaining exact fairness and economic efficiency. This minimization can be conducted in polynomial time for generic instances if the number of agents or objects is fixed. Experiments on real data demonstrate a substantial improvement over current methods.
Distributionally Robust Decision Making Leveraging Conditional Distributions

Distributionally robust optimization (DRO) is a powerful tool for decision making under uncertainty. It is particularly appealing because of its ability to leverage existing data. However, many practical problems call for decision-making with some auxiliary information, and DRO in the context of conditional distributions is not straightforward. We propose a conditional kernel distributionally robust optimization (CKDRO) method that enables robust decision making under conditional distributions through kernel DRO and the conditional mean operator in the reproducing kernel Hilbert space (RKHS). In particular, we consider problems where there is a correlation between the unknown variable y and an auxiliary observable variable x. Given past data of the two variables and a queried auxiliary variable, CKDRO represents the conditional distribution $\mathbb{P}\left( {y|x} \right)$ as the conditional mean operator in the RKHS space and quantifies the ambiguity set in the RKHS as well, which depends on the size of the dataset as well as the query point. To justify the use of RKHS, we demonstrate that the ambiguity set defined in RKHS can be viewed as a ball under a metric that is similar to the Wasserstein metric. The DRO is then dualized and solved via a finite dimensional convex program. The proposed CKDRO approach is applied to a generation scheduling problem and shows that the result of CKDRO is superior to common benchmarks in terms of quality and robustness.
Uncertainty Determines the Adequacy of the Mode and the Tractability of Decoding in Sequence-to-Sequence Models

In many natural language processing (NLP) tasks the same input (e.g. source sentence) can have multiple possible outputs (e.g. translations). To analyze how this ambiguity (also known as intrinsic uncertainty) shapes the distribution learned by neural sequence models we measure sentence-level uncertainty by computing the degree of overlap between references in multi-reference test sets from two different NLP tasks: machine translation (MT) and grammatical error correction (GEC). At both the sentence- and the task-level, intrinsic uncertainty has major implications for various aspects of search such as the inductive biases in beam search and the complexity of exact search. In particular, we show that well-known pathologies such as a high number of beam search errors, the inadequacy of the mode, and the drop in system performance with large beam sizes apply to tasks with high level of ambiguity such as MT but not to less uncertain tasks such as GEC. Furthermore, we propose a novel exact n-best search algorithm for neural sequence models, and show that intrinsic uncertainty affects model uncertainty as the model tends to overly spread out the probability mass for uncertain tasks and sentences.
LASER: LAtent SpacE Rendering for 2D Visual Localization

We present LASER, an image-based Monte Carlo Localization (MCL) framework for 2D floor maps. LASER introduces the concept of latent space rendering, where 2D pose hypotheses on the floor map are directly rendered into a geometrically-structured latent space by aggregating viewing ray features. Through a tightly coupled rendering codebook scheme, the viewing ray features are dynamically determined at rendering-time based on their geometries (i.e. length, incident-angle), endowing our representation with view-dependent fine-grain variability. Our codebook scheme effectively disentangles feature encoding from rendering, allowing the latent space rendering to run at speeds above 10KHz. Moreover, through metric learning, our geometrically-structured latent space is common to both pose hypotheses and query images with arbitrary field of views. As a result, LASER achieves state-of-the-art performance on large-scale indoor localization datasets (i. e. ZInD [5] and Structured3D [38]) for both panorama and perspective image queries, while significantly outperforming existing learning-based methods in speed.
Towards a Perron-Frobenius Theorem for Strongly Aperiodic Stochastic Chains

Perron-Frobenius theorem is a fundamental tool in matrix analysis that has applications to complex networks, population dynamics, social learning, and numerous physical, engineering, and economic phenomena. However, since this theorem and many of its extensions can be applied only to a single matrix at a time, their applications in networked dynamical systems are limited to static networks. To extend the applicability of these results to time-varying networks, we generalize two assertions of the Perron-Frobenius theorem to sequences as well as continua of row-stochastic matrices. Our results have potential applications in areas such as distributed averaging, optimization, and estimation.
Distributed Stochastic Nash Equilibrium Learning in Locally Coupled Network Games with Unknown Parameters

In stochastic Nash equilibrium problems (SNEPs), it is natural for players to be uncertain about their complex environments and have multi-dimensional unknown parameters in their models. Among various SNEPs, this paper focuses on locally coupled network games where the objective of each rational player is subject to the aggregate influence of its neighbors. We propose a distributed learning algorithm based on the proximal-point iteration and ordinary least-square estimator, where each player repeatedly updates the local estimates of neighboring decisions, makes its augmented best-response decisions given the current estimated parameters, receives the realized objective values, and learns the unknown parameters. Leveraging the Robbins-Siegmund theorem and the law of large deviations for M-estimators, we establish the almost sure convergence of the proposed algorithm to solutions of SNEPs when the updating step sizes decay at a proper rate.
LoCoV: low dimension covariance voting algorithm for portfolio optimization

Minimum-variance portfolio optimizations rely on accurate covariance estimator to obtain optimal portfolios. However, it usually suﬀers from large error from sample covariance matrix when the sample size n is not signiﬁcantly larger than the number of assets p . We analyze the random matrix aspects of portfolio optimization and iden-tify the order of errors in sample optimal portfolio weight and show portfolio risk are underestimated when using samples. We also provide LoCoV (low dimension covariance voting) algorithm to reduce error inherited from random samples. From various experiments, LoCoV is shown to outperform the classical method by a large margin.
Reachability Logic for Low-Level Programs

Automatic exploit generation is a relatively new area of research. Work in this area aims to automate the manual and labor intensive task of finding exploits in software. In this paper we present a novel program logic to support automatic exploit generation. We develop a program logic called Reachability Logic, which formally defines the relation between reachability of an assertion and the preconditions which allow them to occur. This relation is then used to calculate the search space of preconditions. We show that Reachability Logic is a powerful tool in automatically finding evidence that an assertion is reachable. We verify that the system works for small litmus tests, as well as real-world algorithms. An implementation has been developed, and the entire system is proven to be sound and complete in a theorem prover. This work represents an important step towards formally verified automatic exploit generation.
Intermediate-mass black holes born via repeated mergers are unlikely thermodynamically stable

Black hole (BH) thermodynamics is one of the most fascinating aspects of BH physics. While its direct proof is challenging to observe for astrophysical BHs, gravitational waves (GWs) have recently oﬀered a new indirect powerful method to probe that it is operating in the Universe. In this paper, we discuss the thermodynamic stability of IMBHs that are born through repeated mergers of stellar-mass BHs via GW emission in dense star clusters. We show that (cid:46) 20% of the IMBHs born via repeated mergers are thermodynamically unstable for a non-charged BH, while (cid:46) 40% are thermodynamically unstable for a moderately-charged BH. Only in the case of unlikely high charges, (cid:38) 60% of the IMBHs are thermodynamically stable. With the present and upcoming GW missions, such as LIGO/Virgo/KAGRA, LISA, DECIGO, and ET that promise to detect tens, or even hundreds, of IMBH mergers via GW emission in the next decade, IMBH thermodynamics may soon be tested.
Proactively Control Privacy in Recommender Systems

Recently, privacy issues in web services that rely on users’ personal data have raised great attention. Unlike existing privacy-preserving technologies such as federated learning and differential privacy, we explore another way to mitigate users’ privacy concerns, giving them control over their own data. For this goal, we propose a privacy aware recommendation framework that gives users delicate control over their personal data, including implicit behaviors, e.g., clicks and watches. In this new framework, users can proactively control which data to disclose based on the trade-off between anticipated privacy risks and potential utilities. Then we study users’ privacy decision making under different data disclosure mechanisms and recommendation models, and how their data disclosure decisions affect the recommender system’s performance. To avoid the high cost of real-world experiments, we apply simulations to study the effects of our proposed framework. Specifically, we propose a reinforcement learning algorithm to simulate users’ decisions (with various sensitivities) under three proposed platform mechanisms on two datasets with three representative recommendation models. The simulation results show that the platform mechanisms with finer split granularity and more unrestrained disclosure strategy can bring better results for both end users and platforms than the “all or nothing” binary mechanism adopted by most real-world applications. It also shows that our proposed framework can effectively protect users’ privacy since they can obtain comparable or even better results with much less disclosed data.
Ball 3D Localization From A Single Calibrated Image

Ball 3D localization in team sports has various applications including automatic offside detection in soccer, or shot release localization in basketball. Today, this task is either resolved by using expensive multi-views setups, or by restricting the analysis to ballistic trajectories. In this work, we propose to address the task on a single image from a calibrated monocular camera by estimating ball diameter in pixels and use the knowledge of real ball diameter in meters. This approach is suitable for any game situation where the ball is (even partly) visible. To achieve this, we use a small neural network trained on image patches around candidates generated by a conventional ball detector. Besides predicting ball diameter, our network outputs the con-ﬁdence of having a ball in the image patch. Validations on 3 basketball datasets reveals that our model gives remarkable predictions on ball 3D localization. In addition, through its conﬁdence output, our model improves the detection rate by ﬁltering the candidates produced by the detector. The contributions of this work are (i) the ﬁrst model to address 3D ball localization on a single image, (ii) an effective method for ball 3D annotation from single calibrated images, (iii) a high quality 3D ball evaluation dataset annotated from a single viewpoint. In addition, the code to reproduce this research is be made freely available at https://github. com/gabriel-vanzandycke/deepsport .
On-line Estimation of Stability and Passivity Metrics

We consider the problem of on-line evaluation of critical characteristic parameters such as the ℒ2-gain, input feedforward passivity index and output feedback passivity index of non-linear systems using their input-output data. Typically, having an accurate measure of such system indices enables the application of systematic control design techniques. Moreover, if such system indices can efficiently be evaluated on-line, they can be exploited to design intelligent control solutions. However, the existing off-line estimation methods of such system indices are computationally inefficient and require a large amount of input-output data. On the other hand, the existing on-line estimation methods take an averaging-based approach, which may be sub-optimal, computationally inefficient and susceptible to estimate saturation. To overcome these challenges in the on-line estimation of system indices, we establish and exploit several interesting theoretical results on a particular class of fractional function optimization problems. Finally, we provide several numerical examples to highlight our contributions.
Self-triggered MPC robust to bounded packet loss via a min-max approach*

Networked Control Systems typically come with a limited communication bandwidth and thus require special care when designing the underlying control and triggering law. A method that allows to consider hard constraints on the communication traffic as well as on states and inputs is self-triggered model predictive control (MPC). In this scheme, the optimal length of the sampling interval is determined proactively using predictions of the system behavior. However, previous formulations of self-triggered MPC have neglected the widespread phenomenon of packet loss, such that these approaches might fail in practice. In this paper, we present a novel self-triggered MPC scheme which is robust to bounded packet loss by virtue of a min-max optimization problem. We prove recursive feasibility, constraint satisfaction and convergence to the origin for any possible packet loss realization consistent with the boundedness constraint, and demonstrate the advantages of the proposed scheme in a numerical example.
Biocomputing Model Using Tripartite Synapses Provides Reliable Neuronal Logic Gating With Spike Pattern Diversity

Biocomputing technologies exploit biological communication mechanisms involving cell-cell signal propagation to perform computations. Researchers recently worked toward realising logic gates made by neurons to develop novel devices such as organic neuroprostheses or brain implants made by cells, herein termed living implants. Several challenges arise from this approach, mainly associated with the stochastic nature and noise of neuronal communication. Since astrocytes play a crucial role in the regulation of neurons activity, there is a possibility whereby astrocytes can be engineered to control synapses favouring reliable biocomputing. This work proposes a mathematical model of neuronal logic gates involving neurons and astrocytes, realising OR and AND gating. We use a shallow coupling of both the Izhikevich and Postnov models to characterise gating responses with spike pattern variability and astrocyte synaptic regulation. Logic operation error ratio and accuracy assess the AND and OR gates’ performances at different synaptic Gaussian noise levels. Our results demonstrate that the astrocyte regulating activity can effectively be used as a denoising mechanism, paving the way for highly reliable biocomputing implementations.
A Fast and Efficient Conditional Learning for Tunable Trade-Off between Accuracy and Robustness

Existing models that achieve state-of-the-art (SOTA) performance on both clean and adversarially-perturbed images rely on convolution operations conditioned with feature-wise linear modulation (FiLM) layers. These layers require many new parameters and are hyperparameter sensitive. They significantly increase training time, memory cost, and potential latency which can prove costly for resource-limited or real-time applications. In this paper, we present a fast learnable once-for-all adversarial training (FLOAT) algorithm, which instead of the existing FiLM-based conditioning, presents a unique weight conditioned learning that requires no additional layer, thereby incurring no significant increase in parameter count, training time, or network latency compared to standard adversarial training. In particular, we add configurable scaled noise to the weight tensors that enables a trade-off between clean and adversarial performance. Extensive experiments show that FLOAT can yield SOTA performance improving both clean and perturbed image classification by up to ~6% and ~10%, respectively. Moreover, real hardware measurement shows that FLOAT can reduce the training time by up to 1.43x with fewer model parameters of up to 1.47x on iso-hyperparameter settings compared to the FiLM-based alternatives. Additionally, to further improve memory efficiency we introduce FLOAT sparse (FLOATS), a form of non-iterative model pruning and provide detailed empirical analysis to provide a three way accuracy-robustness-complexity trade-off for these new class of pruned conditionally trained models.
ROBUST AND EFFICIENT AGGREGATION FOR DISTRIBUTED LEARNING

Distributed learning paradigms, such as federated and decentralized learning, allow for the coordination of models across a collection of agents, and without the need to exchange raw data. Instead, agents compute model updates locally based on their available data, and subsequently share the update model with a parameter server or their peers. This is followed by an aggregation step, which traditionally takes the form of a (weighted) average. Distributed learning schemes based on averaging are known to be susceptible to outliers. A single malicious agent is able to drive an averaging-based distributed learning algorithm to an arbitrarily poor model. This has motivated the development of robust aggregation schemes, which are based on variations of the median and trimmed mean. While such procedures ensure robustness to outliers and malicious behavior, they come at the cost of significantly reduced sample efficiency. This means that current robust aggregation schemes require significantly higher agent participation rates to achieve a given level of performance than their mean-based counterparts in non-contaminated settings. In this work we remedy this drawback by developing statistically efficient and robust aggregation schemes for distributed learning.
Curiosity Driven Self-supervised Tactile Exploration of Unknown Objects

Intricate behaviors an organism can exhibit is predicated on its ability to sense and effectively interpret complexities of its surroundings. Relevant information is often distributed between multiple modalities, and requires the organism to exhibit information assimilation capabilities in addition to information seeking behaviors. While biological beings leverage multiple sensing modalities for decision making, current robots are overly reliant on visual inputs. In this work, we want to augment our robots with the ability to leverage the (relatively under-explored) modality of touch. To focus our investigation, we study the problem of scene reconstruction where touch is the only available sensing modality. We present Tactile Slam (tSLAM) -- which prepares an agent to acquire information seeking behavior and use implicit understanding of common household items to reconstruct the geometric details of the object under exploration. Using the anthropomorphic `ADROIT' hand, we demonstrate that tSLAM is highly effective in reconstructing objects of varying complexities within 6 seconds of interactions. We also established the generality of tSLAM by training only on 3D Warehouse objects and testing on ContactDB objects.