Miracle-dz/UniFAR-FTUs huggingface dataset:
Domain Generalization: A Survey

Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce. This is because most learning algorithms strongly rely on the i.i. d. assumption on source/target data, which is often violated in practice due to domain shift. Domain generalization (DG) aims to achieve OOD generalization by using only source data for model learning. Over the last ten years, research in DG has made great progress, leading to a broad spectrum of methodologies, e.g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, to name a few; DG has also been studied in various application areas including computer vision, speech recognition, natural language processing, medical imaging, and reinforcement learning. In this paper, for the first time a comprehensive literature review in DG is provided to summarize the developments over the past decade. Specifically, we first cover the background by formally defining DG and relating it to other relevant fields like domain adaptation and transfer learning. Then, we conduct a thorough review into existing methods and theories. Finally, we conclude this survey with insights and discussions on future research directions.
Yaglom limit for critical neutron transport

We consider the classical Yaglom limit theorem for the Neutron Branching Process (NBP) in the setting that the mean semigroup is critical, i.e. its leading eigenvalue is zero. We show that the law of the process conditioned on survival is asymptotically equivalent to an exponential distribution. As part of the proof, we also show that the probability of survival decays inversely proportionally to time. Although Yaglom limit theorems have recently been handled in the setting of spatial branching processes and superprocesses, as well as in the setting of isotropic homogeneous Neutron Branching Processes back in the 1970s, a large component of our proof appeals to a completely new combinatorial approach which is necessary as, unlike the aforementioned literature, the branching mechanism is non-local and anisotropic. Our new approach and the main novelty of this work is based around a coupled pair of inductive hypotheses, which describe the convergence of all of the associated moments of the NBP. For the proof of the latter, we make use of the recently demonstrated spine decomposition, cf. \cite{SNTE}, pulling out the leading order terms using a non-trivial combinatorial decomposition.
Cosmic ray protons and electrons from supernova remnants

Context. The spectrum of cosmic ray protons and electrons released by supernova remnants throughout their evolution is poorly known because of the difficulty in accounting for particle escape and confinement downstream of a shock front, where both adiabatic and radiative losses are present. Since electrons lose energy mainly through synchrotron losses, it is natural to ask whether the spectrum released into the interstellar medium may be different from that of their hadronic counterpart. Independent studies of cosmic ray transport through the Galaxy require that the source spectrum of electrons and protons be very different. Therefore, the above question acquires a phenomenological relevance. Aims. Here we calculate the spectrum of cosmic ray protons released during the evolution of supernovae of different types, accounting for the escape from the upstream region and for adiabatic losses of particles advected downstream of the shock and liberated at later times. The same calculation is carried out for electrons, where in addition to adiabatic losses we take the radiative losses suffered behind the shock into account. These electrons are dominated by synchrotron losses in the magnetic field, which most likely is self-generated by cosmic rays accelerated at the shock. Methods. We use standard temporal evolution relations for supernova shocks expanding in different types of interstellar media together with an analytic description of particle acceleration and magnetic field amplification to determine the density and spectrum of cosmic ray particles. Their evolution in time is derived by numerically solving the equation describing advection with adiabatic and radiative losses for electrons and protons. The flux from particles continuously escaping the supernova remnants is also accounted for. Results. The magnetic field in the post-shock region is calculated by using an analytic treatment of the magnetic field amplification due to nonresonant and resonant streaming instability and their saturation. The resulting field is compared with the available set of observational results concerning the dependence of the magnetic field strength upon shock velocity. We find that when the field is the result of the growth of the cosmic-ray-driven nonresonant instability alone, the spectrum of electrons and protons released by a supernova remnant are indeed different; however, such a difference becomes appreciable only at energies ≳100−1000 GeV, while observations of the electron spectrum require such a difference to be present at energies as low as ∼10 GeV. An effect at such low energies requires substantial magnetic field amplification in the late stages of supernova remnant evolution (shock velocity ≪1000 km s−1); this may not be due to streaming instability but rather hydrodynamical processes. We comment on the feasibility of such conditions and speculate on the possibility that the difference in spectral shape between electrons and protons may reflect either some unknown acceleration effect or additional energy losses in cocoons around the sources.
OAG-BERT: Pre-train Heterogeneous Entity-augmented Academic Language Models

To enrich language models with domain knowledge is crucial but difficult. Based on the world’s largest public academic graph Open Academic Graph (OAG), we pre-train an academic language model, namely OAG-BERT, which integrates massive heterogeneous entities including paper, author, concept, venue, and affiliation. To better endow OAG-BERT with the ability to capture entity information, we develop novel pre-training strategies including heterogeneous entity type embedding, entity-aware 2D positional encoding, and span-aware entity masking. For zero-shot inference, we design a special decoding strategy to allow OAG-BERT to generate entity names from scratch. We evaluate the OAG-BERT on various downstream academic tasks, including NLP benchmarks, zero-shot entity inference, heterogeneous graph link prediction and author name disambiguation. Results demonstrate the effectiveness of the proposed pre-training approach to both comprehending academic texts and modeling knowledge from heterogeneous entities. OAG-BERT has been deployed to multiple real-world applications, such as reviewer recommendations and paper tagging in the AMiner system. OAG-BERT 1 is also available to the public through the CogDL package.
An area law for 2d frustration-free spin systems

We prove that the entanglement entropy of the ground state of a locally gapped frustration-free 2D lattice spin system satisfies an area law with respect to a vertical bipartition of the lattice into left and right regions. We first establish that the ground state projector of any locally gapped frustration-free 1D spin system can be approximated to within error є by a degree O(√nlog(є−1)) multivariate polynomial in the interaction terms of the Hamiltonian. This generalizes the optimal bound on the approximate degree of the boolean AND function, which corresponds to the special case of commuting Hamiltonian terms. For 2D spin systems we then construct an approximate ground state projector (AGSP) that employs the optimal 1D approximation in the vicinity of the boundary of the bipartition of interest. This AGSP has sufficiently low entanglement and error to establish the area law using a known technique.
Asymptotic approximation of the likelihood of stationary determinantal point processes

Continuous determinantal point processes (DPPs) are a class of repulsive point processes on ℝd$$ {\mathbb{R}}^d $$ with many statistical applications. Although an explicit expression of their density is known, it is too complicated to be used directly for maximum likelihood estimation. In the stationary case, an approximation using Fourier series has been suggested, but it is limited to rectangular observation windows and no theoretical results support it. In this contribution, we investigate a different way to approximate the likelihood by looking at its asymptotic behavior when the observation window grows toward ℝd$$ {\mathbb{R}}^d $$ . This new approximation is not limited to rectangular windows, is faster to compute than the previous one, does not require any tuning parameter, and some theoretical justifications are provided. It moreover provides an explicit formula for estimating the asymptotic variance of the associated estimator. The performances are assessed in a simulation study on standard parametric models on ℝd$$ {\mathbb{R}}^d $$ and compare favorably to common alternative estimation methods for continuous DPPs.
Neural 3D Video Synthesis

TIANYE LI∗†, University of Southern California, USA and Facebook Reality Labs Research, USA MIRA SLAVCHEVA∗, Facebook Reality Labs Research, USA MICHAEL ZOLLHOEFER, Facebook Reality Labs Research, USA SIMON GREEN, Facebook Reality Labs Research, USA CHRISTOPH LASSNER, Facebook Reality Labs Research, USA CHANGIL KIM, Facebook, USA TANNER SCHMIDT, Facebook Reality Labs Research, USA STEVEN LOVEGROVE, Facebook Reality Labs Research, USA MICHAEL GOESELE, Facebook Reality Labs Research, USA ZHAOYANG LV, Facebook Reality Labs Research, USA
Sequential Place Learning: Heuristic-Free High-Performance Long-Term Place Recognition

Sequential matching using hand-crafted heuristics has been standard practice in route-based place recognition for enhancing pairwise similarity results for nearly a decade. However, precision-recall performance of these algorithms dramatically degrades when searching on short temporal window (TW) lengths, while demanding high compute and storage costs on large robotic datasets for autonomous navigation research. Here, influenced by biological systems that robustly navigate spacetime scales even without vision, we develop a joint visual and positional representation learning technique, via a sequential process, and design a learning-based CNN+LSTM architecture, trainable via backpropagation through time, for viewpoint- and appearance-invariant place recognition. Our approach, Sequential Place Learning (SPL), is based on a CNN function that visually encodes an environment from a single traversal, thus reducing storage capacity, while an LSTM temporally fuses each visual embedding with corresponding positional data -- obtained from any source of motion estimation -- for direct sequential inference. Contrary to classical two-stage pipelines, e.g., match-then-temporally-filter, our network directly eliminates false-positive rates while jointly learning sequence matching from a single monocular image sequence, even using short TWs. Hence, we demonstrate that our model outperforms 15 classical methods while setting new state-of-the-art performance standards on 4 challenging benchmark datasets, where one of them can be considered solved with recall rates of 100% at 100% precision, correctly matching all places under extreme sunlight-darkness changes. In addition, we show that SPL can be up to 70x faster to deploy than classical methods on a 729 km route comprising 35,768 consecutive frames. Extensive experiments demonstrate the. Baseline code available at https://github.com/mchancan/deepseqslam
Generalized collisional fluid theory for multi-component, multi-temperature plasma using the linearized Boltzmann collision operator for scrape-off layer/edge applications

Grad’s method is used on the linearized Boltzmann collision operator to derive the most general expressions for the collision coefficients for a multi-component, multi-temperature plasma up to rank-2. In doing so, the collision coefficients then get expressed as series sum of pure coefficients of temperature and mass ratios multiplied by the cross-section dependent Chapman–Cowling integrals. These collisional coefficients are compared to previously obtained coefficients by Zhdanov (2002 Transport Processes in Multicomponent Plasma (London: Taylor and Francis)) for 13N-moment multi-temperature scheme. First, the differences in coefficients are compared directly, and then the differences in first approximation to viscosity and friction force are compared. For the 13N-moment multi-temperature coefficients, it is found that they behave reasonably similarly for small temperature differences, but display substantial differences in the coefficients when the temperature differences are high, both for the coefficients and for viscosity and friction force values. Furthermore, the obtained coefficients are compared to the 21N-moment single-temperature approximation provided by Zhdanov et al, and it is seen that the differences are higher than the 13N-moment multi-temperature coefficients, and have substantial differences even in the vicinity of equal temperatures, especially for the viscosity and friction force calculations.
On the Just-In-Time Discovery of Profit-Generating Transactions in DeFi Protocols

Decentralized Finance (DeFi) is a blockchain-asset-enabled finance ecosystem with millions of daily USD transaction volume, billions of locked up USD, as well as a plethora of newly emerging protocols (for lending, staking, and exchanges). Because all transactions, user balances, and total value locked in DeFi are publicly readable, a natural question that arises is: how can we automatically craft profitable transactions across the intertwined DeFi platforms? In this paper, we investigate two methods that allow us to automatically create profitable DeFi trades, one well-suited to arbitrage and the other applicable to more complicated settings. We first adopt the Bellman-Ford-Moore algorithm with DeFiPoser-ARB and then create logical DeFi protocol models for a theorem prover in DeFiPoser-SMT. While DeFiPoser-ARB focuses on DeFi transactions that form a cycle and performs very well for arbitrage, DeFiPoser-SMT can detect more complicated profitable transactions. We estimate that DeFiPoser-ARB and DeFiPoser-SMT can generate an average weekly revenue of 191.48 ETH (76,592 USD) and 72.44 ETH (28,976 USD) respectively, with the highest transaction revenue being 81.31 ETH (32,524 USD) and 22.40 ETH (8,960 USD) respectively. We further show that DeFiPoser-SMT finds the known economic bZx attack from February 2020, which yields 0.48M USD. Our forensic investigations show that this opportunity existed for 69 days and could have yielded more revenue if exploited one day earlier. Our evaluation spans 150 days, given 96 DeFi protocol actions, and 25 assets.Looking beyond the financial gains mentioned above, forks deteriorate the blockchain consensus security, as they increase the risks of double-spending and selfish mining. We explore the implications of DeFiPoser-ARB and DeFiPoser-SMT on blockchain consensus. Specifically, we show that the trades identified by our tools exceed the Ethereum block reward by up to 874×. Given optimal adversarial strategies provided by a Markov Decision Process (MDP), we quantify the value threshold at which a profitable transaction qualifies as Miner Extractable Value (MEV) and would incentivize MEV-aware miners to fork the blockchain. For instance, we find that on Ethereum, a miner with a hash rate of 10% would fork the blockchain if an MEV opportunity exceeds 4× the block reward.
An Empirical Study of Compound PCFGs

Compound probabilistic context-free grammars (C-PCFGs) have recently established a new state of the art for phrase-structure grammar induction. However, due to the high time-complexity of chart-based representation and inference, it is difficult to investigate them comprehensively. In this work, we rely on a fast implementation of C-PCFGs to conduct evaluation complementary to that of (CITATION). We highlight three key findings: (1) C-PCFGs are data-efficient, (2) C-PCFGs make the best use of global sentence-level information in preterminal rule probabilities, and (3) the best configurations of C-PCFGs on English do not always generalize to morphology-rich languages.
Multi-View Audio And Music Classification

We propose in this work a multi-view learning approach for audio and music classification. Considering four typical low-level representations (i.e. different views) commonly used for audio and music recognition tasks, the proposed multi-view network consists of four subnetworks, each handling one input types. The learned embedding in the subnetworks are then concatenated to form the multi-view embedding for classification similar to a simple concatenation network. However, apart from the joint classification branch, the network also maintains four classification branches on the single-view embedding of the subnetworks. A novel method is then proposed to keep track of the learning behavior on the classification branches and adapt their weights to proportionally blend their gradients for network training. The weights are adapted in such a way that learning on a branch that is generalizing well will be encouraged whereas learning on a branch that is overfitting will be slowed down. Experiments on three different audio and music classification tasks show that the proposed multi-view network not only outperforms the single-view baselines but also is superior to the multi-view baselines based on concatenation and late fusion.
Extracting Optimal Explanations for Ensemble Trees via Logical Reasoning

Ensemble trees are a popular machine learning model which often yields high prediction performance when analysing structured data. Although individual small decision trees are deemed explainable by nature, an ensemble of large trees is often difficult to understand. In this work, we propose an approach called optimised explanation (OptExplain) that faithfully extracts global explanations of ensemble trees using a combination of logical reasoning, sampling and optimisation. Building on top of this, we propose a method called the profile of equivalent classes (ProClass), which uses MAX-SAT to simplify the explanation even further. Our experimental study on several datasets shows that our approach can provide high-quality explanations to large ensemble trees models, and it betters recent top-performers.
Pseudo-labeling for Scalable 3D Object Detection

To safely deploy autonomous vehicles, onboard perception systems must work reliably at high accuracy across a diverse set of environments and geographies. One of the most common techniques to improve the efficacy of such systems in new domains involves collecting large labeled datasets, but such datasets can be extremely costly to obtain, especially if each new deployment geography requires additional data with expensive 3D bounding box annotations. We demonstrate that pseudo-labeling for 3D object detection is an effective way to exploit less expensive and more widely available unlabeled data, and can lead to performance gains across various architectures, data augmentation strategies, and sizes of the labeled dataset. Overall, we show that better teacher models lead to better student models, and that we can distill expensive teachers into efficient, simple students. Specifically, we demonstrate that pseudo-label-trained student models can outperform supervised models trained on 3-10 times the amount of labeled examples. Using PointPillars [24], a two-year-old architecture, as our student model, we are able to achieve state of the art accuracy simply by leveraging large quantities of pseudo-labeled data. Lastly, we show that these student models generalize better than supervised models to a new domain in which we only have unlabeled data, making pseudo-label training an effective form of unsupervised domain adaptation.
Introduction to a low-mass dark matter project, ALETHEIA: A Liquid hElium Time projection cHambEr In dArk matter

Dark Matter (DM) is one of the most critical questions to be understood and answered in fundamental physics today. Plenty of astronomical and cosmological observations have already pinned down that DM exists in the Universe, the Milky Way, and the Solar System. However, understanding DM with the language of elementary physics is still in progress. DM direct detection tests the interactive cross-section between galactic DM particles and an underground detector's nucleons. WIMPs is the most discussed DM candidate. After decades of hunting, a convincing WIMPs signal is still at large. Relatively, the low-mass WIMPs region ($\sim$ 10 MeV/c$^2$ - 10 GeV/c$^2$) has not been fully exploited compared to high-mass WIMPs ($\sim$ 10 GeV/c$^2$ - 10 TeV/c$^2$). By filling the arguably cleanest bulk material, LHe, into the arguably most competitive detector in the field, TPCs, ALETHEIA is supposed to achieve an extremely low-level background; therefore, to help answer one of the most pressing physical questions today: the nature of DM. In this paper, we briefly go through the physics motivation of low-mass DM, the ALETHEIA detector's design, possible analysis channels available for DM searches, and the progress we have made since the project launched in the summer of 2020.
Distribution and availability of copper, iron, manganese and zinc in the archaeological black earth profile from the Amazon region

Os solos de terra preta arqueologica sao ricos em materia orgânica, contem fragmentos cerâmicos e artefatos liticos e apresentam nutrientes em concentracoes mais elevadas do que outros tipos de solos. Com o intuito de contribuir com informacoes sobre concentracoes de micronutrientes disponiveis em solos de terra preta, foram avaliadas atraves de extracoes quimicas sequenciais, a distribuicao e disponibilidade de Cu, Fe, Mn e Zn em um perfil de terra preta arqueologica no municipio de Juruti, estado do Para. As maiores concentracoes de Cu, Fe, Mn e Zn no perfil sao encontradas na fracao residual. Este estudo mostrou que ferro, manganes e zinco sao preferencialmente disponiveis a partir das fracoes, associados aos oxidos de Fe-Mn, variando de 1265,39 a 1818,12 mg kg-1, 0,83 a 48,51 mg kg-1 e 1,92 a 12,05 mg kg-1, respectivamente, e o cobre a partir da materia orgânica, variando de 0,13 a 0,45 mg kg-1.
Auditory Attention Decoding from EEG using Convolutional Recurrent Neural Network

The auditory attention decoding (AAD) approach was proposed to determine the identity of the attended talker in a multi-talker scenario by analyzing electroencephalography (EEG) data. Although the linear model-based method has been widely used in AAD, the linear assumption was considered oversimplified and the decoding accuracy remained lower for shorter decoding windows. Recently, nonlinear models based on deep neural networks (DNN) have been proposed to solve this problem. However, these models did not fully utilize both the spatial and temporal features of EEG, and the interpretability of DNN models was rarely investigated. In this paper, we proposed novel convolutional recurrent neural network (CRNN) based regression model and classification model, and compared them with both the linear model and the state-of-the-art DNN models. Results showed that, our proposed CRNN-based classification model outperformed others for shorter decoding windows (around 90% for 2 s and 5 s). Although worse than classification models, the decoding accuracy of the proposed CRNN - based regression model was about 5% greater than other regression models. The interpretability of DNN models was also investigated by visualizing layers' weight.