Miracle-dz/UniFAR-FTUs huggingface dataset:
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey

State-of-the-art deep learning models are often trained with a large amount of costly labeled training data. However, requiring exhaustive manual annotations may degrade the model's generalizability in the limited-label regime.Semi-supervised learning and unsupervised learning offer promising paradigms to learn from an abundance of unlabeled visual data. Recent progress in these paradigms has indicated the strong benefits of leveraging unlabeled data to improve model generalization and provide better model initialization. In this survey, we review the recent advanced deep learning algorithms on semi-supervised learning (SSL) and unsupervised learning (UL) for visual recognition from a unified perspective. To offer a holistic understanding of the state-of-the-art in these areas, we propose a unified taxonomy. We categorize existing representative SSL and UL with comprehensive and insightful analysis to highlight their design rationales in different learning scenarios and applications in different computer vision tasks. Lastly, we discuss the emerging trends and open challenges in SSL and UL to shed light on future critical research directions.
Fast emulation of density functional theory simulations using approximate Gaussian processes

Fitting a theoretical model to experimental data in a Bayesian manner using Markov chain Monte Carlo typically requires one to evaluate the model thousands (or millions) of times. When the model is a slow-to-compute physics simulation, Bayesian model fitting becomes infeasible. To remedy this, a second statistical model that predicts the simulation output – an “emulator” – can be used in lieu of the full simulation during model fitting. A typical emulator of choice is the Gaussian process (GP), a flexible, non-linear model that provides both a predictive mean and variance at each input point. Gaussian process regression works well for small amounts of training data (n < 103), but becomes slow to train and use for prediction when the data set size becomes large. Various methods can be used to speed up the Gaussian process in the medium-to-large data set regime (n < 105), trading away predictive accuracy for drastically reduced runtime. This work examines the accuracy-runtime trade-off of several approximate Gaussian process models – the sparse variational GP, stochastic variational GP, and deep kernel learned GP – when emulating the predictions of density functional theory (DFT) models. Additionally, we use the emulators to calibrate, in a Bayesian manner, the DFT model parameters using observed data, resolving the computational barrier imposed by the data set size, and compare calibration results to previous work. The utility of these calibrated DFT models is to make predictions, based on observed data, about the properties of experimentally unobserved nuclides of interest e.g. super-heavy nuclei.
Mean-field synchronization model of turbulent thermoacoustic transitions

Thermoacoustic instabilities observed in turbulent combustion systems have disastrous consequences and are notoriously challenging to model, predict and control. Here, we introduce a mean-field model of thermoacoustic transitions, where the nonlinear flame response is modeled as the amplitude weighted response of an ensemble of phase oscillators constrained to collectively evolve at the rhythm of acoustic fluctuations. Starting from the acoustic wave equation coupled with the phase oscillators, we derive the evolution equations for the amplitude and phase and obtain the limit cycle solution. We show that the model captures abrupt and continuous transition to thermoacoustic instability observed in disparate combustors. We obtain quantitative insights into the model by estimating the model parameters from the experimental data using parameter optimisation. Importantly, our approach provides an explanation of spatiotemporal synchronization and pattern-formation underlying the transition to thermoacoustic instability while encapsulating the statistical properties of desynchronization, chimeras, and global phase synchronization. We further show using the model that continuous and abrupt transitions to limit cycle oscillations in turbulent combustors corresponds to synchronization transitions of \textit{second-order} and \textit{first-order}, respectively. The present formulation provides a highly interpretable model of thermoacoustic transitions: changes in empirical bifurcation parameters which lead to limit cycle oscillations amounts to an increase in the coupling strength of the phase oscillators, promoting global phase synchronization. The generality of the model in capturing different types of transitions and states of pattern-formation highlights the possibility of extending the present model to a broad range of fluid-dynamical phenomena beyond thermoacoustics.
Deep Hyperspectral and Multispectral Image Fusion With Inter-Image Variability

Hyperspectral image (HI) and multispectral image (MI) fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This article presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible image priors. The fusion problem is stated as an optimization problem in the maximum a posteriori framework. We introduce an original image fusion method that, on one hand, solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, that leverages lightweight convolutional neural network (CNN)-based networks to learn realistic image priors from data. In addition, we propose a zero-shot strategy to directly learn the image-specific prior of the latent images in an unsupervised manner. The performance of the algorithm is illustrated with real data subject to inter-image variability.
Robust Subspace Estimation Using Low-Rank Optimization: Theory and Applications

Various fundamental applications in computer vision and machine learning require finding the basis of a certain subspace. Examples of such applications include face detection, motion estimation, and activity recognition. An increasing interest has been recently placed on this area as a result of significant advances in the mathematics of matrix rank optimization. Interestingly, robust subspace estimation can be posed as a low-rank optimization problem, which can be solved efficiently using techniques such as the method of Augmented Lagrange Multiplier. In this book,the authorsdiscuss fundamental formulations and extensions for low-rank optimization-based subspace estimation and representation. By minimizing the rank of the matrix containing observations drawn from images, the authors demonstrate how to solve four fundamental computer vision problems, including video denosing, background subtraction, motion estimation, and activity recognition.
Dynamic Memory-based Curiosity: A Bootstrap Approach for Exploration

The sparsity of extrinsic rewards poses a serious challenge for reinforcement learning (RL). Currently, many efforts have been made on curiosity which can provide a representative intrinsic reward for effective exploration. However, the challenge is still far from being solved. In this paper, we present a novel curiosity for RL, named DyMeCu, which stands for Dynamic Memory-based Curiosity. Inspired by human curiosity and information theory, DyMeCu consists of a dynamic memory and dual online learners. The curiosity arouses if memorized information can not deal with the current state, and the information gap between dual learners can be formulated as the intrinsic reward for agents, and then such state information can be consolidated into the dynamic memory. Compared with previous curiosity methods, DyMeCu can better mimic human curiosity with dynamic memory, and the memory module can be dynamically grown based on a bootstrap paradigm with dual learners. On multiple benchmarks including DeepMind Control Suite and Atari Suite, large-scale empirical experiments are conducted and the results demonstrate that DyMeCu outperforms competitive curiosity-based methods with or without extrinsic rewards. We will release the code to enhance reproducibility.
Modeling Paragraph-Level Vision-Language Semantic Alignment for Multi-Modal Summarization

Most current multi-modal summarization methods follow a cascaded manner, where an off-the-shelf object detector is ﬁrst used to extract visual features, then these features are fused with language representations to generate the summary with an encoder-decoder model. The cascaded way cannot capture the semantic alignments between images and paragraphs, which are crucial to a precise summary. In this paper, we propose ViL-Sum to jointly model paragraph-level Vi sion-L anguage Semantic Alignment and Multi-Modal Sum marization. The core of ViL-Sum is a joint multi-modal encoder with two well-designed tasks, image reordering and image selection. The joint multi-modal encoder captures the interactions between modalities, where the reordering task guides the model to learn paragraph-level semantic alignment and the selection task guides the model to selected summary-related images in the ﬁnal summary. Experimental results show that our proposed ViL-Sum signiﬁcantly outperforms current state-of-the-art meth-ods. In further analysis, we ﬁnd that two well-designed tasks and joint multi-modal encoder can effectively guide the model to learn reasonable paragraphs-images and summary-images relations. 1
Scaling of the $^{19}$B two-neutron halo properties close to unitarity

We explore the description of the bound $^{19}$B isotope in terms of a $^{17}$B+n+n three-body system where the two-body subsystems $^{17}$B+n and neutron-neutron (nn) have virtual states close to the continuum. Dimensionless scaling functions for the root-mean-square (rms) radii are defined and studied for different parameters of the neutron-core potential and considering three different models for neutron-neutron interaction. The scaling functions for the radii are rooted on the universal behavior of three-body systems close to the Efimov limit and depend only on dimensionless quantities formed by the two-neutron separation energies and scattering lengths. Our results show in practice the model independence of these scaling functions close to unitarity. We provide an estimation of the different rms relative separation distances between the constituents, as well as of the proton and matter radii.
Detecting the unknown in Object Detection

Object detection methods have witnessed impressive improvements in the last years thanks to the design of novel neural network architectures and the availability of large scale datasets. However, current methods have a significant limitation: they are able to detect only the classes observed during training time, that are only a subset of all the classes that a detector may encounter in the real world. Furthermore, the presence of unknown classes is often not considered at training time, resulting in methods not even able to detect that an unknown object is present in the image. In this work, we address the problem of detecting unknown objects, known as open-set object detection. We propose a novel training strategy, called UNKAD, able to predict unknown objects without requiring any annotation of them, exploiting non annotated objects that are already present in the background of training images. In particular, exploiting the four-steps training strategy of Faster R-CNN, UNKAD first identifies and pseudo-labels unknown objects and then uses the pseudo-annotations to train an additional unknown class. While UNKAD can directly detect unknown objects, we further combine it with previous unknown detection techniques, showing that it improves their performance at no costs.
“It’s Just Part of Me:” Understanding Avatar Diversity and Self-presentation of People with Disabilities in Social Virtual Reality

In social Virtual Reality (VR), users are embodied in avatars and interact with other users in a face-to-face manner using avatars as the medium. With the advent of social VR, people with disabilities (PWD) have shown an increasing presence on this new social media. With their unique disability identity, it is not clear how PWD perceive their avatars and whether and how they prefer to disclose their disability when presenting themselves in social VR. We fill this gap by exploring PWD’s avatar perception and disability disclosure preferences in social VR. Our study involved two steps. We first conducted a systematic review of fifteen popular social VR applications to evaluate their avatar diversity and accessibility support. We then conducted an in-depth interview study with 19 participants who had different disabilities to understand their avatar experiences. Our research revealed a number of disability disclosure preferences and strategies adopted by PWD (e.g., reflect selective disabilities, present a capable self). We also identified several challenges faced by PWD during their avatar customization process. We discuss the design implications to promote avatar accessibility and diversity for future social VR platforms.
Time-to-Green predictions for fully-actuated signal control systems with supervised learning

Recently, efforts have been made to standardize signal phase and timing (SPaT) messages. These messages contain signal phase timings of all signalized intersection approaches. This information can thus be used for efficient motion planning, resulting in more homogeneous traffic flows and uniform speed profiles. Despite efforts to provide robust predictions for semi-actuated signal control systems, predicting signal phase timings for fully-actuated controls remains challenging. This paper proposes a time series prediction framework using aggregated traffic signal and loop detector data. We utilize state-of-the-art machine learning models to predict future signal phases' duration. The performance of a Linear Regression (LR), a Random Forest (RF), and a Long-Short-Term-Memory (LSTM) neural network are assessed against a naive baseline model. Results based on an empirical data set from a fully-actuated signal control system in Zurich, Switzerland, show that machine learning models outperform conventional prediction methods. Furthermore, tree-based decision models such as the RF perform best with an accuracy that meets requirements for practical applications.
Automatic music mixing with deep learning and out-of-domain data

Music mixing traditionally involves recording instruments in the form of clean, individual tracks and blending them into a final mixture using audio effects and expert knowledge (e.g., a mixing engineer). The automation of music production tasks has become an emerging field in recent years, where rule-based methods and machine learning approaches have been explored. Nevertheless, the lack of dry or clean instrument recordings limits the performance of such models, which is still far from professional human-made mixes. We explore whether we can use out-of-domain data such as wet or processed multitrack music recordings and repurpose it to train supervised deep learning models that can bridge the current gap in automatic mixing quality. To achieve this we propose a novel data preprocessing method that allows the models to perform automatic music mixing. We also redesigned a listening test method for evaluating music mixing systems. We validate our results through such subjective tests using highly experienced mixing engineers as participants.
Towards a Lower Bound for the Average Case Runtime of Simulated Annealing on TSP

We analyze simulated annealing (SA) for simple randomized instances of the Traveling Salesperson Problem. Our analysis shows that the theoretically optimal cooling schedule of Hajek [16] explores members of the solution set which are in expectation far from the global optimum. We obtain a lower bound on the expected length of the ﬁnal tour obtained by SA on these random instances. In addition, we also obtain an upper bound on the expected value of its variance. These bounds assume that the Markov chain that describes SA is stationary, a situation that does not truly hold in practice. Hence, we also formulate conditions under which the bounds extend to the nonstationary case. These bounds are obtained by comparing the tour length distribution to a related distribution. We furthermore provide numerical evidence for a stochastic dominance relation that appears to exist between these two distributions, and formulate a conjecture in this direction. If proved, this conjecture implies that SA stays far from the global optimum with high probability when executed for any sub-exponential number of iterations. This would show that SA requires at least exponentially many iterations to reach a global optimum with nonvanishing probability.
Toward a Reasoning and Learning Architecture for Ad Hoc Teamwork

We present an architecture for ad hoc teamwork , which refers to collaboration in a team of agents without prior coordination. State of the art methods for this problem often include a data-driven component that uses a long history of prior observations to model the behaviour of other agents (or agent types) and to determine the ad hoc agent’s behavior. In many practical domains, it is challenging to ﬁnd large training datasets, and necessary to understand and incrementally extend the existing models to account for changes in team composition or domain attributes. Our architecture combines the principles of knowledge-based and data-driven reasoning and learning. Speciﬁcally, we enable an ad hoc agent to perform non-monotonic logical reasoning with prior commonsense domain knowledge and incrementally-updated simple predictive models of other agents’ behaviour. We use the benchmark simulated multiagent collaboration domain Fort Attack to demonstrate that our architecture supports adaptation to unforeseen changes, incremental learning and revision of models of other agents’ behaviour from limited samples, transparency in the ad hoc agent’s decision making, and better performance than a data-driven baseline.
DPTDR: Deep Prompt Tuning for Dense Passage Retrieval

Deep prompt tuning (DPT) has gained great success in most natural language processing (NLP) tasks. However, it is not well-investigated in dense retrieval where fine-tuning (FT) still dominates. When deploying multiple retrieval tasks using the same backbone model (e.g., RoBERTa), FT-based methods are unfriendly in terms of deployment cost: each new retrieval model needs to repeatedly deploy the backbone model without reuse. To reduce the deployment cost in such a scenario, this work investigates applying DPT in dense retrieval. The challenge is that directly applying DPT in dense retrieval largely underperforms FT methods. To compensate for the performance drop, we propose two model-agnostic and task-agnostic strategies for DPT-based retrievers, namely retrieval-oriented intermediate pretraining and unified negative mining, as a general approach that could be compatible with any pre-trained language model and retrieval task. The experimental results show that the proposed method (called DPTDR) outperforms previous state-of-the-art models on both MS-MARCO and Natural Questions. We also conduct ablation studies to examine the effectiveness of each strategy in DPTDR. We believe this work facilitates the industry, as it saves enormous efforts and costs of deployment and increases the utility of computing resources. Our code is available at https://github.com/tangzhy/DPTDR.
Discovering Transferable Forensic Features for CNN-generated Images Detection

Visual counterfeits are increasingly causing an existential conundrum in mainstream media with rapid evolution in neural image synthesis methods. Though detection of such counterfeits has been a taxing problem in the image forensics community, a recent class of forensic detectors -- universal detectors -- are able to surprisingly spot counterfeit images regardless of generator architectures, loss functions, training datasets, and resolutions. This intriguing property suggests the possible existence of transferable forensic features (T-FF) in universal detectors. In this work, we conduct the first analytical study to discover and understand T-FF in universal detectors. Our contributions are 2-fold: 1) We propose a novel forensic feature relevance statistic (FF-RS) to quantify and discover T-FF in universal detectors and, 2) Our qualitative and quantitative investigations uncover an unexpected finding: color is a critical T-FF in universal detectors. Code and models are available at https://keshik6.github. io/transferable-forensic-features/
Driven Andreev molecule

We study the three terminal S-QD-S-QD-S Josephson junction biased with commensurate voltages. In the absence of an applied voltage, the Andreev bound states on each quantum dot hybridize forming an `Andreev molecule'. However, understanding of this system in a non-equilibrium setup is lacking. Applying a dc voltage on the bijunction makes the system time-periodic, and the equilibrium Andreev bound states evolve into a ladder of resonances with a finite lifetime due to multiple Andreev reflections (MAR). Starting from the time-periodic Bogoliubov-de Gennes equations we map the problem to a tight-binding chain in the (infinite) Floquet space. The resolvent of this non-Hermitian block matrix is obtained via a continued fraction method. We numerically calculate the Floquet-Andreev spectra which could be probed by local tunneling spectroscopy on the dots. We also consider the subgap current, and show that the Floquet resonances determine the position of the MAR steps. Proximity of the two dots causes splitting of the steps, while at large distances we observe interference effects which cause oscillations in the I-V curves. The latter effect should persist at very long distances.
Arbitrary-order asymptotic expansions of Gaussian quadrature rules with classical and generalised weight functions

Gaussian quadrature rules are a classical tool for the numerical approximation of integrals with smooth integrands and positive weight functions. We derive and expicitly list asymptotic expressions for the points and weights of Gaussian quadrature rules for three general classes of positive weight functions: analytic functions on a bounded interval with algebraic singularities at the endpoints, analytic weight functions on the halfline with exponential decay at infinity and an algebraic singularity at the finite endpoint, and analytic functions on the real line with exponential decay in both directions at infinity. The results include the Gaussian rules of classical orthogonal polynomials (Legendre, Jacobi, Laguerre and Hermite) as special cases. We present experiments indicating the range of the number of points at which these expressions achieve high precision. We provide an algorithm that can compute arbitrarily many terms in these expansions for the classical cases, and many though not all terms for the generalized cases.