Miracle-dz/UniFAR-FTUs huggingface dataset:
CAE v2: Context Autoencoder with CLIP Target

Masked image modeling (MIM) learns visual representation by masking and reconstructing image patches. Applying the reconstruction supervision on the CLIP representation has been proven effective for MIM. However, it is still under-explored how CLIP supervision in MIM influences performance. To investigate strategies for refining the CLIP-targeted MIM, we study two critical elements in MIM, i.e., the supervision position and the mask ratio, and reveal two interesting perspectives, relying on our developed simple pipeline, context autodecoder with CLIP target (CAE v2). Firstly, we observe that the supervision on visible patches achieves remarkable performance, even better than that on masked patches, where the latter is the standard format in the existing MIM methods. Secondly, the optimal mask ratio positively correlates to the model size. That is to say, the smaller the model, the lower the mask ratio needs to be. Driven by these two discoveries, our simple and concise approach CAE v2 achieves superior performance on a series of downstream tasks. For example, a vanilla ViT-Large model achieves 81.7% and 86.7% top-1 accuracy on linear probing and fine-tuning on ImageNet-1K, and 55.9% mIoU on semantic segmentation on ADE20K with the pre-training for 300 epochs. We hope our findings can be helpful guidelines for the pre-training in the MIM area, especially for the small-scale models.
Electronic signature of the vacancy ordering in NbO (Nb3O3)

We investigated the electronic structure of the vacancy-ordered 4d-transition metal monoxide NbO (Nb3O3) using angle-integrated soft- and hard-x-ray photoelectron spectroscopy as well as ultra-violet angle-resolved photoelectron spectroscopy. We found that density-functional-based band structure calculations can describe the spectral features accurately provided that self-interaction effects are taken into account. In the angle-resolved spectra we were able to identify the so-called vacancy band that characterizes the ordering of the vacancies. This together with the band structure results indicates the important role of the very large inter-Nb-4d hybridization for the formation of the ordered vacancies and the high thermal stability of the ordered structure of niobium monoxide.
Entanglement properties of the time-periodic Kitaev chain

The entanglement properties of the time-periodic Kitaev chain with nearest-neighbor and next-nearest-neighbor hopping is studied. The cases of the exact eigenstate of the time-periodic Hamiltonian, referred to as the Floquet ground state (FGS), as well as a physical state obtained from time evolving an initial state unitarily under the influence of the time-periodic drive are explored. Topological phases are characterized by different numbers of Majorana zero (${\mathbb{Z}}_{0}$) and $\ensuremath{\pi}$ (${\mathbb{Z}}_{\ensuremath{\pi}}$) modes, where the zero modes are present even in the absence of the drive, while the $\ensuremath{\pi}$ modes arise due to resonant driving. The entanglement spectrum (ES) of the FGS as well as the physical state show topological Majorana modes whose number is different from that of the quasienergy spectrum. The number of Majorana edge modes in the ES of the FGS vary in time from $|{\mathbb{Z}}_{0}\ensuremath{-}{\mathbb{Z}}_{\ensuremath{\pi}}|$ to ${\mathbb{Z}}_{0}+{\mathbb{Z}}_{\ensuremath{\pi}}$ within one drive cycle, with the maximal ${\mathbb{Z}}_{0}+{\mathbb{Z}}_{\ensuremath{\pi}}$ modes appearing at a special time-reversal-symmetric point of the cycle. For the physical state, on the other hand, only the modes inherited from the initial wave function, namely the ${\mathbb{Z}}_{0}$ modes, appear in the ES. The ${\mathbb{Z}}_{\ensuremath{\pi}}$ modes are absent in the physical state as they merge with the bulk excitations that are simultaneously created due to resonant driving. The topological properties of the Majorana zero and $\ensuremath{\pi}$ modes in the ES are also explained by mapping the parent wave function to a Bloch sphere.
Unified Question Answering in Slovene

Question answering is one of the most challenging tasks in language understanding. Most approaches are developed for English, while less-resourced languages are much less researched. We adaptasuccessful English question-answering approach, called UniﬁedQA, to the less-resourced Slovene language. Our adaptation uses the encoder-decoder transformer SloT5 and mT5 models to handle four question-answering formats: yes/no, multiple-choice, abstractive, and extractive. We use existing Slovene adap-tations offourdatasets, and machine translate the MCTestdataset. We show that a general model can answer questions in diﬀerent formats at least as well as specialized models. The results are further improved using cross-lingual transfer from English. While we produce state-of-the-art results for Slovene, the performance still lags behind English.
Analysis of CN Coma Morphology Features of Comet 21P/Giacobini–Zinner

We analyze jet features found in the coma of comet 21P/Giacobini–Zinner (21P/GZ) during its 2018 perihelion passage using narrowband CN photometric imaging in order to determine the comet’s rotational period, and we constrain the CN gas outflow velocity and rotational state through the analysis of azimuthally enhanced morphological features. We find that 21P/GZ has a period of either 7.39 ± 0.01 hr or 10.66 ± 0.01 hr. We measure a lower limit to the outflow velocity for the northern jet of 730 ± 30 m s−1 and for the southern jet of 740 ± 30 m s−1. We analyze the morphologies of the jet features and determine that the northern jet possesses a corkscrew pattern, and we utilize that knowledge to determine a rotational pole position at an R.A. of 169−23+28 deg and a decl. of 73−11+5 deg, with an undetermined sense of rotation.
Machine Learning for Software Engineering: A Tertiary Study

Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009 and 2022, covering 6,117 primary studies. The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML. We propose a number of ML for SE research challenges and actions, including conducting further empirical validation and industrial studies on ML, reconsidering deficient SE methods, documenting and automating data collection and pipeline processes, reexamining how industrial practitioners distribute their proprietary data, and implementing incremental ML approaches.
Topological insulators in longitudinally driven waveguides: Lieb and kagome lattices

Topological insulators are studied via tight-binding approximations of longitudinally driven photonic lattices with three lattice sites per unit cell. Two cases are considered in detail: Lieb and Kagome lattices. The lattice is decomposed into three sublattices each of which are allowed move independently of one another. Emphasis is placed on periodic driving induced by laser-etched helical coils along the direction of propagation. The linear Floquet bands are constructed for various inter-sublattice rotation patterns such as: different radii, different frequency, phase offset and quasi one-dimensional motion. Depending on the nature of the band structure, bulk spectral bands with nonzero Chern number are found to support topologically protected edge states which can move unidirectionally. In this case, the modes move scatter-free around defects due to underlying topological protection. Intriguing mode dynamics are found including bi-directional topological modes and bulk-edge leakage i.e. excitation of bulk modes at a defect for edge modes with dispersion frequencies nearby the bulk bands. Finally, certain nonlinear edge modes are also found to propagate unidirectionally and scatter-free around lattice defects.
TrafficCAM: A Versatile Dataset for Traffic Flow Segmentation

Trafﬁc ﬂow analysis is revolutionising trafﬁc management. Qualifying trafﬁc ﬂow data, trafﬁc control bureaus could provide drivers with real-time alerts, advising the fastest routes and therefore optimising transportation logistics and reducing congestion. The existing trafﬁc ﬂow datasets have two major limitations. They feature a limited number of classes, usually limited to one type of vehicle, and the scarcity of unlabelled data. In this paper, we introduce a new benchmark trafﬁc ﬂow image dataset called TrafﬁcCAM. Our dataset distinguishes itself by two major highlights. Firstly, TrafﬁcCAM provides both pixel-level and instance-level semantic labelling along with a large range of types of vehicles and pedestrians. It is composed of a large and diverse set of video sequences recorded in streets from eight Indian cities with stationary cameras. Secondly, TrafﬁcCAM aims to establish a new benchmark for developing fully-supervised tasks, and im-portantly, semi-supervised learning techniques. It is the ﬁrst dataset that provides a vast amount of unlabelled data, helping to better capture trafﬁc ﬂow qualiﬁcation under a low cost annotation requirement. More precisely, our dataset has 4,402 image frames with semantic and instance annotations along with 59,944 unlabelled image frames. We validate our new dataset through a large and comprehensive range of experiments on several state-of-the-art approaches under four different settings: fully-supervised semantic and instance segmentation, and semi-supervised semantic and instance segmentation tasks. Our benchmark dataset will be released in https://anonymous_ submission .
Reverse test and quantum analogue of classical fidelity and generalized fidelity

The aim of the present paper is to give axiomatic characterization of quantum relative entropy utilizing resource conversion scenario. We consider two sets of axioms: non-asymptotic and asymptotic. In the former setting, we prove that the upperbound and the lowerbund of D^{Q}({\rho}||{\sigma}) is D^{R}({\rho}||{\sigma}):=tr{\rho}ln{\sigma}^{1/2}{\rho}^{-1}{\sigma}^{1/2} and D({\rho}||{\sigma}):= tr{\rho}(ln{\rho}-ln{\sigma}), respectively. In the latter setting, we prove uniqueness of quantum relative entropy, that is, D^{Q}({\rho}||{\sigma}) should equal a constant multiple of D({\rho}||{\sigma}). In the analysis, we define and use reverse test and asymptotic reverse test, which are natural inverse of hypothesis test.
Simple method for determining asymptotic states of fast neutrino-flavor conversion

Neutrino-neutrino forward scatterings potentially induce collective neutrino oscillation in dense neutrino gases in astrophysical sites such as core-collapse supernovae (CCSN) and binary neutron star mergers (BNSM). In this paper, we present a detailed study of fast neutrino-flavor conversion (FFC), paying special attention to asymptotic states, by linear stability analysis and local simulations with a periodic boundary condition. We find that asymptotic states can be characterized by two key properties of FFC: (1) the conservation of lepton number for each flavor of neutrinos and (2) the disappearance of ELN(electron neutrino-lepton number)-XLN(heavy-leptonic one) angular crossings in the spatial- or time-averaged distributions. The system which initially has the positive (negative) ELN-XLN density reaches a flavor equipartition in the negative (positive) ELN-XLN angular directions, and the other part compensates it to preserve the conservation laws. These properties of FFCs offer an approximate scheme determining the survival probability of neutrinos in asymptotic states without solving quantum kinetic equations. We also demonstrate that the total amount of flavor conversion can vary with species-dependent neutrino distributions for identical ELN-XLN ones. Our results suggest that even shallow or narrow ELN angular crossings have the ability to drive large flavor conversion, exhibiting the need for including the effects of FFCs in the modeling of CCSN and BNSM.
Heart Abnormality Detection from Heart Sound Signals using MFCC Feature and Dual Stream Attention Based Network

—Cardiovascular diseases are one of the leading cause of death in today’s world and early screening of heart condition plays a crucial role in preventing them. The heart sound signal is one of the primary indicator of heart condition and can be used to detect abnormality in the heart. The acquisition of heart sound signal is non-invasive, cost effective and requires minimum equipment. But currently the detection of heart abnormality from heart sound signal depends largely on the expertise and experience of the physician. As such an automatic detection system for heart abnormality detection from heart sound signal can be a great asset for the people living in underdeveloped areas. In this paper we propose a novel deep learning based dual stream network with attention mechanism that uses both the raw heart sound signal and the MFCC features to detect abnormality in heart condition of a patient. The deep neural network has a convolutional stream that uses the raw heart sound signal and a recurrent stream that uses the MFCC features of the signal. The features from these two streams are merged together using a novel attention network and passed through the classiﬁcation network. The model is trained on the largest publicly available dataset of PCG signal and achieves an accuracy of 87 . 11% , sensitivity of 82 . 41% , speciﬁcty of 91 . 8% and a MACC of 87 . 12% .
Charge-dependent flow induced by magnetic and electric fields in heavy ion collisions

We investigate the charge-dependent flow induced by magnetic and electric fields in heavy-ion collisions. We simulate the evolution of the expanding cooling droplet of strongly coupled plasma hydrodynamically, using the iEBE-VISHNU framework, and add the magnetic and electric fields as well as the electric currents they generate in a perturbative fashion. We confirm the previously reported effect of the electromagnetically induced currents [Gursoy et al., Phys. Rev. C 89, 054905 (2014)], that is a charge-odd directed flow Δv1 that is odd in rapidity, noting that it is induced by magnetic fields (a laFaraday and Lorentz) and by electric fields (the Coulomb field from the charged spectators). In addition, we find a charge-odd Δv3 that is also odd in rapidity and that has a similar physical origin. We furthermore show that the electric field produced by the net charge density of the plasma drives rapidity-even charge-dependent contributions to the radial flow ⟨pT⟩ and the elliptic flow Δv2. Although their magnitudes are comparable to the charge-odd Δv1 and Δv3, they have a different physical origin, namely the Coulomb forces within the plasma.
Minimum Path Cover in Parameterized Linear Time

A minimum path cover (MPC) of a directed acyclic graph (DAG) $G = (V,E)$ is a minimum-size set of paths that together cover all the vertices of the DAG. Computing an MPC is a basic polynomial problem, dating back to Dilworth's and Fulkerson's results in the 1950s. Since the size $k$ of an MPC (also known as the width) can be small in practical applications, research has also studied algorithms whose running time is parameterized on $k$. We obtain a new MPC parameterized algorithm for DAGs running in time $O(k^2|V| + |E|)$. Our algorithm is the first solving the problem in parameterized linear time. Additionally, we obtain an edge sparsification algorithm preserving the width of a DAG but reducing $|E|$ to less than $2|V|$. This algorithm runs in time $O(k^2|V|)$ and requires an MPC of a DAG as input, thus its total running time is the same as the running time of our MPC algorithm.
Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks

Short-term memory in standard, general-purpose, sequence-processing recurrent neural networks (RNNs) is stored as activations of nodes or"neurons. "Generalising feedforward NNs to such RNNs is mathematically straightforward and natural, and even historical: already in 1943, McCulloch and Pitts proposed this as a surrogate to"synaptic modifications"(in effect, generalising the Lenz-Ising model, the first non-sequence processing RNN architecture of the 1920s). A lesser known alternative approach to storing short-term memory in"synaptic connections"-- by parameterising and controlling the dynamics of a context-sensitive time-varying weight matrix through another NN -- yields another"natural"type of short-term memory in sequence processing NNs: the Fast Weight Programmers (FWPs) of the early 1990s. FWPs have seen a recent revival as generic sequence processors, achieving competitive performance across various tasks. They are formally closely related to the now popular Transformers. Here we present them in the context of artificial NNs as an abstraction of biological NNs -- a perspective that has not been stressed enough in previous FWP work. We first review aspects of FWPs for pedagogical purposes, then discuss connections to related works motivated by insights from neuroscience.
Vibronic order and emergent magnetism in cubic $d^1$ double perovskites

The synergistic interplay of different interactions in materials leads to the emergence of novel quantum phenomena. Spin-orbit and vibronic couplings usually counteract each other, however, in cubic $d^1$ double perovskites they coexist and give rise to spin-orbit-lattice entanglement with unquenched dynamic Jahn-Teller effect on the metal sites. The correlation of these entangled states induced by intersite interactions was not assessed so far. Here, we investigate the joint cooperative effect of spin-orbit and vibronic interactions on the formation of the ordered phases in $d^1$ double perovskites. We found that the magnetic ordered states in these systems coexist with a dynamic vibronic order characterized by the ordering of vibronic quadrupole moments on sites. This treatment allows the rationalization of a number of unexplained features of experimentally investigated phases.
BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection

3D object detection from multiple image views is a fundamental and challenging task for visual scene understanding. Owing to its low cost and high efficiency, multi-view 3D object detection has demonstrated promising application prospects. However, accurately detecting objects through perspective views is extremely difficult due to the lack of depth information. Current approaches tend to adopt heavy backbones for image encoders, making them inapplicable for real-world deployment. Different from the images, LiDAR points are superior in providing spatial cues, resulting in highly precise localization. In this paper, we explore the incorporation of LiDAR-based detectors for multi-view 3D object detection. Instead of directly training a depth prediction network, we unify the image and LiDAR features in the Bird-Eye-View (BEV) space and adaptively transfer knowledge across non-homogenous representations in a teacher-student paradigm. To this end, we propose \textbf{BEVDistill}, a cross-modal BEV knowledge distillation (KD) framework for multi-view 3D object detection. Extensive experiments demonstrate that the proposed method outperforms current KD approaches on a highly-competitive baseline, BEVFormer, without introducing any extra cost in the inference phase. Notably, our best model achieves 59.4 NDS on the nuScenes test leaderboard, achieving new state-of-the-art in comparison with various image-based detectors. Code will be available at https://github.com/zehuichen123/BEVDistill.
VeLO: Training Versatile Learned Optimizers by Scaling Up

While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the same scaling approach behind the success of deep learning to learn versatile optimizers. We train an optimizer for deep learning which is itself a small neural network that ingests gradients and outputs parameter updates. Meta-trained with approximately four thousand TPU-months of compute on a wide variety of optimization tasks, our optimizer not only exhibits compelling performance, but optimizes in interesting and unexpected ways. It requires no hyperparameter tuning, instead automatically adapting to the specifics of the problem being optimized. We open source our learned optimizer, meta-training code, the associated train and test data, and an extensive optimizer benchmark suite with baselines at velo-code.github. io.
Machine learning opens a doorway for microrheology with optical tweezers in living systems

It has been argued that linear microrheology with optical tweezers (MOT) of living systems “is not an option” because of the wide gap between the observation time required to collect statistically valid data and the mutational times of the organisms under study. Here, we have explored modern machine learning (ML) methods to reduce the duration of MOT measurements from tens of minutes down to one second by focusing on the analysis of computer simulated experiments. For the first time in the literature, we explicate the relationship between the required duration of MOT measurements (Tm) and the fluid relative viscosity (ηr) to achieve an uncertainty as low as 1% by means of conventional analytical methods, i.e., Tm≅17ηr3 minutes, thus revealing why conventional MOT measurements commonly underestimate the materials’ viscoelastic properties, especially in the case of high viscous fluids or soft-solids. Finally, by means of real experimental data, we have developed and corroborated an ML algorithm to determine the viscosity of Newtonian fluids from trajectories of only one second in duration, yet capable of returning viscosity values carrying an error as low as ∼0.3% at best, hence opening a doorway for MOT in living systems.
Applying a Particle-only Model to the HL Tau Disk

Observations have revealed rich structures in protoplanetary disks, offering clues about their embedded planets. Due to the complexities introduced by the abundance of gas in these disks, modeling their structure in detail is computationally intensive, requiring complex hydrodynamic codes and substantial computing power. It would be advantageous if computationally simpler models could provide some preliminary information on these disks. Here we apply a particle-only model (that we developed for gas-poor debris disks) to the gas-rich disk, HL Tauri, to address the question of whether such simple models can inform the study of these systems. Assuming three potentially embedded planets, we match HL Tau’s radial profile fairly well and derive best-fit planetary masses and orbital radii (0.40, 0.02, 0.21 Jupiter masses for the planets orbiting a 0.55 M⊙ star at 11.22, 29.67, 64.23 au). Our derived parameters are comparable to those estimated by others, except for the mass of the second planet. Our simulations also reproduce some narrower gaps seen in the ALMA image away from the orbits of the planets. The nature of these gaps is debated but, based on our simulations, we argue they could result from planet–disk interactions via mean-motion resonances, and need not contain planets. Our results suggest that a simple particle-only model can be used as a first step to understanding dynamical structures in gas disks, particularly those formed by planets, and determine some parameters of their hidden planets, serving as useful initial inputs to hydrodynamic models which are needed to investigate disk and planet properties more thoroughly.