Miracle-dz/UniFAR-FTUs huggingface dataset:
Disentangled Speech Embeddings Using Cross-Modal Self-Supervision

The objective of this paper is to learn representations of speaker identity without access to manually annotated data. To do so, we develop a self-supervised learning objective that exploits the natural cross-modal synchrony between faces and audio in video. The key idea behind our approach is to tease apart—without annotation—the representations of linguistic content and speaker identity. We construct a two-stream architecture which: (1) shares low-level features common to both representations; and (2) provides a natural mechanism for explicitly disentangling these factors, offering the potential for greater generalisation to novel combinations of content and identity and ultimately producing speaker identity representations that are more robust.We train our method on a large-scale audio-visual dataset of talking heads ‘in the wild’, and demonstrate its efficacy by evaluating the learned speaker representations for standard speaker recognition performance.
Regret Minimization in Stochastic Contextual Dueling Bandits

We consider the problem of stochastic $K$-armed dueling bandit in the contextual setting, where at each round the learner is presented with a context set of $K$ items, each represented by a $d$-dimensional feature vector, and the goal of the learner is to identify the best arm of each context sets. However, unlike the classical contextual bandit setup, our framework only allows the learner to receive item feedback in terms of their (noisy) pariwise preferences--famously studied as dueling bandits which is practical interests in various online decision making scenarios, e.g. recommender systems, information retrieval, tournament ranking, where it is easier to elicit the relative strength of the items instead of their absolute scores. However, to the best of our knowledge this work is the first to consider the problem of regret minimization of contextual dueling bandits for potentially infinite decision spaces and gives provably optimal algorithms along with a matching lower bound analysis. We present two algorithms for the setup with respective regret guarantees $\tilde O(d\sqrt{T})$ and $\tilde O(\sqrt{dT \log K})$. Subsequently we also show that $\Omega(\sqrt {dT})$ is actually the fundamental performance limit for this problem, implying the optimality of our second algorithm. However the analysis of our first algorithm is comparatively simpler, and it is often shown to outperform the former empirically. Finally, we corroborate all the theoretical results with suitable experiments.
Comparative Spectral Analysis of the Superluminous Supernova 2019neq

We present a spectroscopic analysis of the recently discovered fast-evolving Type I superluminous supernova (SLSN-I) SN 2019neq (at redshift z = 0.1059). We compare it to the well-studied slowly evolving SLSN-I SN 2010kd (z = 0.101). Our main goal is to search for spectroscopic differences between the two groups of SLSNe-I. Differences in the spectra may reveal different ejecta compositions and explosion mechanisms. Our investigation concentrates on optical spectra observed with the 10 m Hobby–Eberly Telescope Low Resolution Spectrograph-2 at McDonald Observatory during the photospheric phase. We apply the SYN++ code to model the spectra of SN 2019neq taken at −4 days, +5 days, and +29 days from maximum light. We examine the chemical evolution and ejecta composition of the SLSN by identifying the elements and ionization states in its spectra. We find that a spectral model consisting of O iii, Co iii, and Si iv gives a SYN++ fit that is comparable to the typical SLSN-I spectral model consisting of O ii, and conclude that the true identification of those lines, at least in the case of SN 2019neq, is ambiguous. Based on modeling the entire optical spectrum, we classify SN 2019neq as a fast-evolving SLSN-I having a photospheric velocity gradient of km s−1 day−1, which is among the highest velocity gradients observed for an SLSN-I. Inferring the velocity gradient from the proposed Fe ii λ5169 feature alone would result in km s−1 day−1, which is still within the observed range of fast-evolving SLSNe-I. In addition, we derive the number density of relevant ionization states for a variety of identified elements at the epoch of the three observations. Finally, we give constraints on the lower limit of the ejecta mass and find that both SLSNe have an ejecta mass at least one order of magnitude higher than normal SNe Ia, while the fast-evolving SN 2019neq has an ejecta mass a factor of two lower than the slowly evolving SN 2010kd. These mass estimates suggest the existence of a possible correlation between the evolution timescale and the ejected mass of SLSNe-I.
Implicit differentiation of Lasso-type models for hyperparameter optimization

Setting regularization parameters for Lasso-type estimators is notoriously difficult, though crucial in practice. The most popular hyperparameter optimization approach is grid-search using held-out validation data. Grid-search however requires to choose a predefined grid for each parameter, which scales exponentially in the number of parameters. Another approach is to cast hyperparameter optimization as a bi-level optimization problem, one can solve by gradient descent. The key challenge for these methods is the estimation of the gradient with respect to the hyperparameters. Computing this gradient via forward or backward automatic differentiation is possible yet usually suffers from high memory consumption. Alternatively implicit differentiation typically involves solving a linear system which can be prohibitive and numerically unstable in high dimension. In addition, implicit differentiation usually assumes smooth loss functions, which is not the case for Lasso-type problems. This work introduces an efficient implicit differentiation algorithm, without matrix inversion, tailored for Lasso-type problems. Our approach scales to high-dimensional data by leveraging the sparsity of the solutions. Experiments demonstrate that the proposed method outperforms a large number of standard methods to optimize the error on held-out data, or the Stein Unbiased Risk Estimator (SURE).
Scalable Constrained Bayesian Optimization

The global optimization of a high-dimensional black-box function under black-box constraints is a pervasive task in machine learning, control, and engineering. These problems are challenging since the feasible set is typically non-convex and hard to find, in addition to the curses of dimensionality and the heterogeneity of the underlying functions. In particular, these characteristics dramatically impact the performance of Bayesian optimization methods, that otherwise have become the de facto standard for sample-efficient optimization in unconstrained settings, leaving practitioners with evolutionary strategies or heuristics. We propose the scalable constrained Bayesian optimization (SCBO) algorithm that overcomes the above challenges and pushes the applicability of Bayesian optimization far beyond the state-of-the-art. A comprehensive experimental evaluation demonstrates that SCBO achieves excellent results on a variety of benchmarks. To this end, we propose two new control problems that we expect to be of independent value for the scientific community.
Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems

The stringent requirements of mobile edge computing (MEC) applications and functions fathom the high capacity and dense deployment of MEC hosts to the upcoming wireless networks. However, operating such high capacity MEC hosts can significantly increase energy consumption. Thus, a base station (BS) unit can act as a self-powered BS. In this article, an effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. First, a two-stage linear stochastic programming problem is formulated with the goal of minimizing the total energy consumption cost of the system while fulfilling the energy demand. Second, a semi-distributed data-driven solution is proposed by developing a novel multi-agent meta-reinforcement learning (MAMRL) framework to solve the formulated problem. In particular, each BS plays the role of a local agent that explores a Markovian behavior for both energy consumption and generation while each BS transfers time-varying features to a meta-agent. Sequentially, the meta-agent optimizes (i.e., exploits) the energy dispatch decision by accepting only the observations from each local agent with its own state information. Meanwhile, each BS agent estimates its own energy dispatch policy by applying the learned parameters from meta-agent. Finally, the proposed MAMRL framework is benchmarked by analyzing deterministic, asymmetric, and stochastic environments in terms of non-renewable energy usages, energy cost, and accuracy. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost (with 95.8% prediction accuracy), compared to other baseline methods.
A differentiable programming method for quantum control

Optimal control is highly desirable in many current quantum systems, especially to realize tasks in quantum information processing. We introduce a method based on differentiable programming to leverage explicit knowledge of the differential equations governing the dynamics of the system. In particular, a control agent is represented as a neural network that maps the state of the system at a given time to a control pulse. The parameters of this agent are optimized via gradient information obtained by direct differentiation through both the neural network and the differential equation of the system. This fully differentiable reinforcement learning approach ultimately yields time-dependent control parameters optimizing a desired figure of merit. We demonstrate the method’s viability and robustness to noise in eigenstate preparation tasks for three systems: a single qubit, a chain of qubits, and a quantum parametric oscillator.
Time-dependent radiation hydrodynamics on a moving mesh

We describe the structure and implementation of a radiation hydrodynamic solver for MANGA, the moving-mesh hydrodynamics module of the large-scale parallel code, Charm N-body GrAvity solver (ChaNGa). We solve the equations of time dependent radiative transfer using a reduced speed of light approximation following the algorithm of Jiang et al (2014). By writing the radiative transfer equations as a generalized conservation equation, we solve the transport part of these equations on an unstructured Voronoi mesh. We then solve the source part of the radiative transfer equations following Jiang et al (2014) using an implicit solver, and couple this to the hydrodynamic equations. The use of an implicit solver ensure reliable convergence and preserves the conservation properties of these equations even in situations where the source terms are stiff due to the small coupling timescales between radiation and matter. We present the results of a limited number of test cases (energy conservation, momentum conservation, dynamic diffusion, linear waves, crossing beams, and multiple shadows) to show convergence with analytic results and numerical stability. We also show that it produces qualitatively the correct results in the presence of multiple sources in the optically thin case.
Focus on Semantic Consistency for Cross-Domain Crowd Understanding

For pixel-level crowd understanding, it is time-consuming and laborious in data collection and annotation. Some domain adaptation algorithms try to liberate it by training models with synthetic data, and the results in some recent works have proved the feasibility. However, we found that a mass of estimation errors in the background areas impede the performance of the existing methods. In this paper, we propose a domain adaptation method to eliminate it. According to the semantic consistency, a similar distribution in deep layer’s features of the synthetic and real-world crowd area, we first introduce a semantic extractor to effectively distinguish crowd and background in high-level semantic information. Besides, to further enhance the adapted model, we adopt adversarial learning to align features in the semantic space. Experiments on three representative real datasets show that the proposed domain adaptation scheme achieves the state-of-the-art for cross-domain counting problems.
A search for the lenses in the Herschel Bright Sources (HerBS) sample

Verifying that sub-mm galaxies are gravitationally lensed requires time-expensive observations with oversubscribed high-resolution observatories. Here, we aim to strengthen the evidence of gravitational lensing within the Herschel Bright Sources (HerBS) by cross-comparing their positions to optical (SDSS) and near-infrared (VIKING) surveys, in order to search for the foreground lensing galaxy candidates. Resolved observations of the brightest HerBS sources have already shown that most are lensed, and a galaxy evolution model predicts that ∼76 per cent of the total HerBS sources are lensed, although with the SDSS survey we are only able to identify the likely foreground lenses for 25 per cent of the sources. With the near-infrared VIKING survey, however, we are able to identify the likely foreground lenses for 57 per cent of the sources, and we estimate that 82 per cent of the HerBS sources have lenses on the VIKING images even if we cannot identify the lens in every case. We find that the angular offsets between lens and Herschel source are larger than that expected if the lensing is done by individual galaxies. We also find that the fraction of HerBS sources that are lensed falls with decreasing 500-micron flux density, which is expected from the galaxy evolution model. Finally, we apply our statistical VIKING cross-identification to the entire Herschel-ATLAS catalogue, where we also find that the number of lensed sources falls with decreasing 500-micron flux density.
Ruppeiner geometry for charged AdS black holes surrounded by quintessence with a cloud of strings background

In this letter, we present a study of Ruppeiner’s thermodynamic geometry for charged AdS black holes with a cloud of strings and surrounded by quintessence with the motivation to probe the nature of interactions between black hole microstructures. We report that the scalar curvature of the Ruppeiner metric can assume both positive and negative values depending on certain parameters of the black hole indicating the presence of both repulsion and attraction dominated regions. The point at which the Ruppeiner curvature changes sign is determined by the electric charge and the coupling constants associated with quintessence and the cloud of strings. Such a crossing of curvature is also observed in the neutral case in this system.
Corona-heated Accretion-disk Reprocessing: A Physical Model to Decipher the Melody of AGN UV/Optical Twinkling

Active galactic nuclei (AGNs) have long been observed to “twinkle” (i.e., their brightness varies with time) on timescales from days to years in the UV/optical bands. Such AGN UV/optical variability is essential for probing the physics of supermassive black holes (SMBHs), the accretion disk, and the broad-line region. Here, we show that the temperature fluctuations of an AGN accretion disk, which is magnetically coupled with the corona, can account for observed high-quality AGN optical light curves. We calculate the temperature fluctuations by considering the gas physics of the accreted matter near the SMBH. We find that the resulting simulated AGN UV/optical light curves share the same statistical properties as the observed ones as long as the dimensionless viscosity parameter α, which is widely believed to be controlled by magnetohydrodynamic (MHD) turbulence in the accretion disk, is about 0.01–0.2. Moreover, our model can simultaneously explain the larger-than-expected accretion disk sizes and the dependence of UV/optical variability upon wavelength for NGC 5548. Our model also has the potential to explain some other observational facts of AGN UV/optical variability, including the timescale-dependent bluer-when-brighter color variability and the dependence of UV/optical variability on AGN luminosity and black-hole mass. Our results also demonstrate a promising way to infer the black-hole mass, the accretion rate, and the radiative efficiency, thereby facilitating understanding of the gas physics and MHD turbulence near the SMBH and its cosmic mass growth history by fitting the AGN UV/optical light curves in the era of time-domain astronomy.
Input Perturbation: A New Paradigm between Central and Local Differential Privacy

Traditionally, there are two models on differential privacy: the central model and the local model. The central model focuses on the machine learning model and the local model focuses on the training data. In this paper, we study the \textit{input perturbation} method in differentially private empirical risk minimization (DP-ERM), preserving privacy of the central model. By adding noise to the original training data and training with the `perturbed data', we achieve ($\epsilon$,$\delta$)-differential privacy on the final model, along with some kind of privacy on the original data. We observe that there is an interesting connection between the local model and the central model: the perturbation on the original data causes the perturbation on the gradient, and finally the model parameters. This observation means that our method builds a bridge between local and central model, protecting the data, the gradient and the model simultaneously, which is more superior than previous central methods. Detailed theoretical analysis and experiments show that our method achieves almost the same (or even better) performance as some of the best previous central methods with more protections on privacy, which is an attractive result. Moreover, we extend our method to a more general case: the loss function satisfies the Polyak-Lojasiewicz condition, which is more general than strong convexity, the constraint on the loss function in most previous work.
Syndrome-aware Herb Recommendation with Multi-Graph Convolution Network

Herb recommendation plays a crucial role in the therapeutic process of Traditional Chinese Medicine (TCM), which aims to recommend a set of herbs to treat the symptoms of a patient. While several machine learning methods have been developed for herb recommendation, they are limited in modeling only the interactions between herbs and symptoms, and ignoring the intermediate process of syndrome induction. When performing TCM diagnostics, an experienced doctor typically induces syndromes from the patient’s symptoms and then suggests herbs based on the induced syndromes. As such, we believe the induction of syndromes — an overall description of the symptoms — is important for herb recommendation and should be properly handled. However, due to the ambiguity and complexity of syndrome induction, most prescriptions lack the explicit ground truth of syndromes.In this paper, we propose a new method that takes the implicit syndrome induction process into account for herb recommendation. Specifically, given a set of symptoms to treat, we aim to generate an overall syndrome representation by effectively fusing the embeddings of all the symptoms in the set, so as to mimic how a doctor induces the syndromes. Towards symptom embedding learning, we additionally construct a symptom-symptom graph from the input prescriptions for capturing the relations (cooccurred patterns) between symptoms; we then build graph convolution networks (GCNs) on both symptom-symptom and symptom-herb graphs to learn symptom embedding. Similarly, we construct a herb-herb graph and build GCNs on both herbherb and symptom-herb graphs to learn herb embedding, which is finally interacted with the syndrome representation to predict the scores of herbs. The advantage of such a Multi-Graph GCN architecture is that more comprehensive representations can be obtained for symptoms and herbs. We conduct extensive experiments on a public TCM dataset, demonstrating significant improvements over state-of-the-art herb recommendation methods. Further studies justify the effectiveness of our design of syndrome representation and multiple graphs.
Towards a Soft Faceted Browsing Scheme for Information Access

Faceted browsing is a commonly supported feature of user interfaces for access to information. Existing interfaces generally treat facet values selected by a user as hard filters and respond to the user by only displaying information items strictly satisfying the filters and in their original ranking order. We propose a novel alternative strategy for faceted browsing, called soft faceted browsing, where the system also includes some possibly relevant items outside the selected filter in a non-intrusive way and re-ranks the items to better satisfy the user's information need. Such a soft faceted browsing strategy can be beneficial when the user does not have a very confident and strict preference for the selected facet values, and is especially appropriate for applications such as e-commerce search where the user would like to explore a larger space before finalizing a purchasing decision. We propose a probabilistic framework for modeling and solving the soft faceted browsing problem, and apply the framework to study the case of facet filter selection in e-commerce search engines. Preliminary experiment results demonstrate the soft faceted browsing scheme is better than the traditional faceted browsing scheme in terms of its efficiency in helping users navigate in the information space.
A survey on Semi-, Self- and Unsupervised Techniques in Image Classification

While deep learning strategies achieve outstanding results in computer vision tasks, one issue remains: The current strategies rely heavily on a huge amount of labeled data. In many real-world problems, it is not feasible to create such an amount of labeled training data. Therefore, it is common to incorporate unlabeled data into the training process to reach equal results with fewer labels. Due to a lot of concurrent research, it is difficult to keep track of recent developments. In this survey, we provide an overview of often used ideas and methods in image classification with fewer labels. We compare 25 methods in detail. In our analysis, we identify three major trends. 1. State-of-theart methods are scaleable to real-world applications based on their accuracy. 2. The degree of supervision which is needed to achieve comparable results to the usage of all labels is decreasing. 3. All methods share common ideas while only a few methods combine these ideas to achieve better performance. Based on all of these three trends we discover future research opportunities.
SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion

Certain facial parts are salient (unique) in appearance, which substantially contribute to the holistic recognition of a subject. Occlusion of these salient parts deteriorates the performance of face recognition algorithms. In this paper, we propose a generative model to reconstruct the missing parts of the face which are under occlusion. The proposed generative model (SD-GAN) reconstructs a face preserving the illumination variation and identity of the face. A novel adversarial training algorithm has been designed for a bimodal mutually exclusive Generative Adversarial Network (GAN) model, for faster convergence. A novel adversarial "structural" loss function is also proposed, comprising of two components: a holistic and a local loss, characterized by SSIM and patch-wise MSE. Ablation studies on real and synthetically occluded face datasets reveal that our proposed technique outperforms the competing methods by a considerable margin, even for boosting the performance of Face Recognition.