Miracle-dz/UniFAR-FTUs huggingface dataset:
Self Supervision Does Not Help Natural Language Supervision at Scale

Self supervision and natural language supervision have emerged as two exciting ways to train general purpose image encoders which excel at a variety of downstream tasks. Recent works such as M3AE [31] and SLIP [64] have suggested that these approaches can be eﬀectively combined, but most notably their results use small ( < 20M examples) pre-training datasets and don’t eﬀectively reﬂect the large-scale regime ( > 100M samples) that is commonly used for these approaches. Here we investigate whether a similar approach can be eﬀec-tive when trained with a much larger amount of data. We ﬁnd that a combination of two state of the art approaches: masked auto-encoders, MAE [38] and contrastive language image pre-training, CLIP [68] provides a beneﬁt over CLIP when trained on a corpus of 11.3M image-text pairs, but little to no beneﬁt (as evaluated on a suite of common vision tasks) over CLIP when trained on a large corpus of 1.4B images. Our work provides some much needed clarity into the eﬀectiveness (or lack thereof) of self supervision for large-scale image-text training.
"My Zelda Cane": Strategies Used by Blind Players to Play Visual-Centric Digital Games

Mainstream games are typically designed around the visual experience, with behaviors and interactions highly dependent on vision. Despite this, blind people are playing mainstream games while dealing with and overcoming inaccessible content, often together with friends and audiences. In this work, we analyze over 70 hours of YouTube videos, where blind content-creators play visual-centric games. We point out the various strategies employed by players to overcome barriers that permeate mainstream games. We reflect on ways to enable and improve blind players’ experience with these games, shedding light on the positive and negative consequences of apparently benign design choices. Our observations underline how game elements are appropriated for accessibility, the incidental consequences of audio design, and the trade-offs between accessibility, agency, and engagement.
Rings close to periodic with applications to matrix, endomorphism and group rings

We examine those matrix rings whose entries lie in periodic rings equipped with some additional properties. Specifically, we prove that the famous Diesl's question whether or not $R$ being nil-clean implies that $\mathbb{M}_n(R)$ is nil-clean for all $n\geq 1$ is paralleling to the corresponding implication for (Abelian, local) periodic rings. Besides, we study when the endomorphism ring $\mathrm{E}(G)$ of an Abelian group $G$ is periodic. Concretely, we establish that $\mathrm{E}(G)$ is periodic exactly when $G$ is finite as well as we find a complete necessary and sufficient condition when the endomorphism ring over an Abelian group is strongly $m$-nil clean for some natural number $m$ thus refining an"old"result concerning strongly nil-clean endomorphism rings. Responding to a question when a group ring is periodic, we show that if $R$ is a right (resp., left) perfect periodic ring and $G$ is a locally finite group, then the group ring $RG$ is periodic, too. We finally find some criteria under certain conditions when the tensor product of two periodic algebras over a commutative ring is again periodic. In addition, some other sorts of rings very close to periodic rings, namely the so-called weakly periodic rings, are also investigated.
Obstacle avoidance using Raycasting and Riemannian Motion Policies at kHz rates for MAVs

This paper presents a novel method for using Riemannian Motion Policies on volumetric maps, shown in the example of obstacle avoidance for Micro Aerial Vehicles (MAVs), Today, most robotic obstacle avoidance algorithms rely on sampling or optimization-based planners with volumetric maps. However, they are computationally expensive and often have inflexible monolithic architectures. Riemannian Motion Policies are a modular, parallelizable, and efficient navigation alternative but are challenging to use with the widely used voxel-based environment representations. We propose using GPU raycasting and tens of thousands of concurrent policies to provide direct obstacle avoidance using Riemannian Motion Policies in voxelized maps without needing map smoothing or pre-processing. Additionally, we present how the same method can directly plan on LiDAR scans without any intermediate map. We show how this reactive approach compares favorably to traditional planning methods and can evaluate up to 200 million rays per second. We demonstrate the planner successfully on a real MAV for static and dynamic obstacles. The presented planner is made available as an open-source package11https://github.com/ethz-asl/reactive_avoidance.
Astronomical Detection of the Interstellar Anion C10H− toward TMC-1 from the GOTHAM Large Program on the Green Bank Telescope

Using data from the Green Bank Telescope (GBT) Observations of TMC-1: Hunting for Aromatic Molecules (GOTHAM) survey, we report the first astronomical detection of the C10H− anion. The astronomical observations also provided the necessary data to refine the spectroscopic parameters of C10H−. From the velocity stacked data and the matched filter response, C10H− is detected at >9σ confidence level at a column density of 4.04−2.23+10.67×1011 cm−2. A dedicated search for the C10H radical was also conducted toward TMC-1. In this case, the stacked molecular emission of C10H was detected at a ∼3. 2σ confidence interval at a column density of 2.02−0.82+2.68×1011 cm−2. However, as the determined confidence level is currently <5σ, we consider the identification of C10H as tentative. The full GOTHAM data set was also used to better characterize the physical parameters including column density, excitation temperature, line width, and source size for the C4H, C6H, and C8H radicals and their respective anions, and the measured column densities were compared to the predictions from a gas/grain chemical formation model and from a machine learning analysis. Given the measured values, the C10H−/C10H column density ratio is ∼ 2.0−1.6+5.9 —the highest value measured between an anion and neutral species to date. Such a high ratio is at odds with current theories for interstellar anion chemistry. For the radical species, both models can reproduce the measured abundances found from the survey; however, the machine learning analysis matches the detected anion abundances much better than the gas/grain chemical model, suggesting that the current understanding of the formation chemistry of molecular anions is still highly uncertain.
Learning stability of partially observed switched linear systems

This paper deals with learning stability of partially observed switched linear systems under arbitrary switching. Such systems are widely used to describe cyber-physical systems which arise by combining physical systems with digital components. In many real-world applications, the internal states cannot be observed directly. It is thus more realistic to conduct system analysis using the outputs of the system. Stability is one of the most frequent requirement for safety and robustness of cyber-physical systems. Existing methods for analyzing stability of switched linear systems often require the knowledge of the parameters and/or all the states of the underlying system. In this paper, we propose an algorithm for deciding stability of switched linear systems under arbitrary switching based purely on observed output data. The proposed algorithm essentially relies on an output-based Lyapunov stability framework and returns an estimate of the joint spectral radius (JSR). We also prove a probably approximately correct error bound on the quality of the estimate of the JSR from the perspective of statistical learning theory.
Which differential equations correspond to the Lindblad equation?

The Lindblad master equation can always be transformed into a first-order linear ordinary differential equation (1ODE) for the coherence vector. We pose the inverse problem: given a finite-dimensional, non-homogeneous 1ODE, does a corresponding Lindblad equation exist? If so, what are the corresponding Hamiltonian and Lindblad operators? We provide a general solution to this problem, including a complete positivity test in terms of the parameters of the 1ODE. We also derive a host of properties relating the two representations (master equation and 1ODE), which are of independent interest.
A Survey of Meta-Reinforcement Learning

While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficiency and the limited generality of the policies it produces. A promising approach for alleviating these limitations is to cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible. In this survey, we describe the meta-RL problem setting in detail as well as its major variations. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. Using these clusters, we then survey meta-RL algorithms and applications. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.
Preparing quantum many-body scar states on quantum computers

Quantum many-body scar states are highly excited eigenstates of many-body systems that exhibit atypical entanglement and correlation properties relative to typical eigenstates at the same energy density. Scar states also give rise to infinitely long-lived coherent dynamics when the system is prepared in a special initial state having finite overlap with them. Many models with exact scar states have been constructed, but the fate of scarred eigenstates and dynamics when these models are perturbed is difficult to study with classical computational techniques. In this work, we propose state preparation protocols that enable the use of quantum computers to study this question. We present protocols both for individual scar states in a particular model, as well as superpositions of them that give rise to coherent dynamics. For superpositions of scar states, we present both a system-size-linear depth unitary and a finite-depth nonunitary state preparation protocol, the latter of which uses measurement and postselection to reduce the circuit depth. For individual scarred eigenstates, we formulate an exact state preparation approach based on matrix product states that yields quasipolynomial-depth circuits, as well as a variational approach with a polynomial-depth ansatz circuit. We also provide proof of principle state-preparation demonstrations on superconducting quantum hardware.
Fast Inference in Denoising Diffusion Models via MMD Finetuning

Denoising Diffusion Models (DDMs) have become a popular tool for generating high-quality samples from complex data distributions. These models are able to capture sophisticated patterns and structures in the data, and can generate samples that are highly diverse and representative of the underlying distribution. However, one of the main limitations of diffusion models is the complexity of sample generation, since a large number of inference timesteps is required to faithfully capture the data distribution. In this paper, we present MMD-DDM, a novel method for fast sampling of diffusion models. Our approach is based on the idea of using the Maximum Mean Discrepancy (MMD) to finetune the learned distribution with a given budget of timesteps. This allows the finetuned model to significantly improve the speed-quality trade-off, by substantially increasing fidelity in inference regimes with few steps or, equivalently, by reducing the required number of steps to reach a target fidelity, thus paving the way for a more practical adoption of diffusion models in a wide range of applications. We evaluate our approach on unconditional image generation with extensive experiments across the CIFAR-10, CelebA, ImageNet and LSUN-Church datasets. Our findings show that the proposed method is able to produce high-quality samples in a fraction of the time required by widely-used diffusion models, and outperforms state-of-the-art techniques for accelerated sampling. Code is available at: https://github.com/diegovalsesia/MMD-DDM.
Spectral Submanifolds of the Navier-Stokes Equations

Spectral subspaces of a linear dynamical system identify a large class of invariant structures that highlight/isolate the dynamics associated to select subsets of the spectrum. The corresponding notion for nonlinear systems is that of spectral submanifolds -- manifolds invariant under the full nonlinear dynamics that are determined by their tangency to spectral subspaces of the linearized system. In light of the recently-emerged interest in their use as tools in model reduction, we propose an extension of the relevant theory to the realm of fluid dynamics. We show the existence of a large (and the most pertinent) subclass of spectral submanifolds and foliations - describing the behaviour of nearby trajectories - about fixed points and periodic orbits of the Navier-Stokes equations. Their uniqueness and smoothness properties are discussed in detail, due to their significance from the perspective of model reduction. The machinery is then put to work via a numerical algorithm developed along the lines of the parameterization method, that computes the desired manifolds as power series expansions. Results are shown within the context of 2D channel flows.
Multimodal Video Adapter for Parameter Efficient Video Text Retrieval

State-of-the-art video-text retrieval (VTR) methods usually fully fine-tune the pre-trained model (e.g. CLIP) on specific datasets, which may suffer from substantial storage costs in practical applications since a separate model per task needs to be stored. To overcome this issue, we present the premier work on performing parameter-efficient VTR from the pre-trained model, i.e., only a small number of parameters are tunable while freezing the backbone. Towards this goal, we propose a new method dubbed Multimodal Video Adapter (MV-Adapter) for efficiently transferring the knowledge in the pre-trained CLIP from image-text to video-text. Specifically, MV-Adapter adopts bottleneck structures in both video and text branches and introduces two novel components. The first is a Temporal Adaptation Module employed in the video branch to inject global and local temporal contexts. We also learn weights calibrations to adapt to the dynamic variations across frames. The second is a Cross-Modal Interaction Module that generates weights for video/text branches through a shared parameter space, for better aligning between modalities. Thanks to above innovations, MV-Adapter can achieve on-par or better performance than standard fine-tuning with negligible parameters overhead. Notably, on five widely used VTR benchmarks (MSR-VTT, MSVD, LSMDC, DiDemo, and ActivityNet), MV-Adapter consistently outperforms various competing methods in V2T/T2V tasks with large margins. Codes will be released.
Morphological analysis of the polarized synchrotron emission with WMAP and Planck

The bright polarized synchrotron emission, away from the Galactic plane, originates mostly from filamentary structures. We implement a filament finder algorithm which allows the detection of bright elongated structures in polarized intensity maps. We analyse the sky at 23 and 30 GHz as observed respectively by WMAP and Planck. We identify 19 filaments, 13 of which have been previously observed. For each filament, we study the polarization fraction, finding values typically larger than for the areas outside the filaments, excluding the Galactic plane, and a fraction of about 30% is reached in two filaments. We study the polarization spectral indices of the filaments, and find a spectral index consistent with the values found in previous analysis (about -3.1) for more diffuse regions. Decomposing the polarization signals into the E and B families, we find that most of the filaments are detected in PE , but not in PB . We then focus on understanding the statistical properties of the diffuse regions of the synchrotron emission at 23 GHz. Using Minkowski functionals and tensors, we analyse the non-Gaussianity and statistical isotropy of the polarized intensity maps. For a sky coverage corresponding to 80% of the fainter emission, and on scales smaller than 6 degrees (ℓ > 30), the deviations from Gaussianity and isotropy are significantly higher than 3σ. The level of deviation decreases for smaller scales, however, it remains significantly high for the lowest analised scale (∼ 1.5°). When 60% sky coverage is analysed, we find that the deviations never exceed 3σ. Finally, we present a simple data-driven model to generate non-Gaussian and anisotropic simulations of the synchrotron polarized emission. The simulations are fitted in order to match the spectral and statistical properties of the faintest 80% sky coverage of the data maps.
Music Playlist Title Generation Using Artist Information

Automatically generating or captioning music playlist titles given a set of tracks is of significant interest in music streaming services as customized playlists are widely used in personalized music recommendation, and well-composed text titles attract users and help their music discovery. We present an encoder-decoder model that generates a playlist title from a sequence of music tracks. While previous work takes track IDs as tokenized input for playlist title generation, we use artist IDs corresponding to the tracks to mitigate the issue from the long-tail distribution of tracks included in the playlist dataset. Also, we introduce a chronological data split method to deal with newly-released tracks in real-world scenarios. Comparing the track IDs and artist IDs as input sequences, we show that the artist-based approach significantly enhances the performance in terms of word overlap, semantic relevance, and diversity.
Semiparametric inference using fractional posteriors

We establish a general Bernstein--von Mises theorem for approximately linear semiparametric functionals of fractional posterior distributions based on nonparametric priors. This is illustrated in a number of nonparametric settings and for different classes of prior distributions, including Gaussian process priors. We show that fractional posterior credible sets can provide reliable semiparametric uncertainty quantification, but have inflated size. To remedy this, we further propose a \textit{shifted-and-rescaled} fractional posterior set that is an efficient confidence set having optimal size under regularity conditions. As part of our proofs, we also refine existing contraction rate results for fractional posteriors by sharpening the dependence of the rate on the fractional exponent.
FECANet: Boosting Few-Shot Semantic Segmentation With Feature-Enhanced Context-Aware Network

Few-shot semantic segmentation is the task of learning to locate each pixel of the novel class in the query image with only a few annotated support images. The current correlation-based methods construct pair-wise feature correlations to establish the many-to-many matching because the typical prototype-based approaches cannot learn fine-grained correspondence relations. However, the existing methods still suffer from the noise contained in naive correlations and the lack of context semantic information in correlations. To alleviate these problems mentioned above, we propose a Feature-Enhanced Context-Aware Network (FECANet). Specifically, a feature enhancement module is proposed to suppress the matching noise caused by inter-class local similarity and enhance the intra-class relevance in the naive correlation. In addition, we propose a novel correlation reconstruction module that encodes extra correspondence relations between foreground and background and multi-scale context semantic features, significantly boosting the encoder to capture a reliable matching pattern. Experiments on PASCAL-$5^{i}$ and COCO-$20^{i}$ datasets demonstrate that our proposed FECANet leads to remarkable improvement compared to previous state-of-the-arts, demonstrating its effectiveness.
Global Mapping of Fragmented Rocks on the Moon with a Neural Network: Implications for the Failure Mode of Rocks on Airless Surfaces

Failure modes of lunar boulders depend both on rheology and the erosion agents acting in the lunar surface environment. Here, we address the failure modes of lunar boulders and their variations at a quasi-global scale (60°N to S). We deploy a neural network and map a total of ∼130,000 fragmented boulders (width > ∼10 m) scattered across the lunar surface and visually identify a dozen different disintegration morphologies corresponding to different failure modes. Our findings suggest that before a boulder is catastrophically shattered by an impact, there is an internal weakening period with minor morphological evidence of damage at the rock scale at the resolution of the used imagery. We find that some of the rare pre-shattering morphologies (e.g., fractures) are equivalent to morphologies observed on asteroid Bennu, suggesting that these morphologies on the Moon and on asteroids are likely not diagnostic of their formation mechanism (e.g., meteoroid impact, thermal stresses). In addition, we identify new morphologies such as breccia boulders with an advection-like erosion style. We publicly release the produced fractured boulder catalog along with this paper.
An Efficient Block Rational Krylov Solver for Sylvester Equations with Adaptive Pole Selection

We present an algorithm for the solution of Sylvester equations with right-hand side of low rank. The method is based on projection onto a block rational Krylov subspace, with two key contributions with respect to the state-of-the-art. First, we show how to maintain the last pole equal to infinity throughout the iteration, by means of pole reodering. This allows for a cheap evaluation of the true residual at every step. Second, we extend the convergence analysis in [Beckermann B. , An error analysis for rational Galerkin projection applied to the Sylvester equation, SINUM, 2011] to the block case. This extension allows to link the convergence with the problem of minimizing the norm of a small rational matrix over the spectra or field-of-values of the involved matrices. This is in contrast with the non-block case, where the minimum problem is scalar, instead of matrix-valued. Replacing the norm of the objective function with an easier to evaluate function yields several adaptive pole selection strategies, providing a theoretical analysis for known heuristics, as well as effective novel techniques.