Miracle-dz/UniFAR-FTUs huggingface dataset:
GP-NAS-ensemble: a model for NAS Performance Prediction

It is of great significance to estimate the performance of a given model architecture without training in the application of Neural Architecture Search (NAS) as it may take a lot of time to evaluate the performance of an architecture. In this paper, a novel NAS framework called GP-NAS-ensemble is proposed to predict the performance of a neural network architecture with a small training dataset. We make several improvements on the GP-NAS model to make it share the advantage of ensemble learning methods. Our method ranks second in the CVPR2022 second lightweight NAS challenge performance prediction track.
FRAME: Fast and Robust Autonomous 3D Point Cloud Map-Merging for Egocentric Multi-Robot Exploration

This article presents a 3D point cloud map-merging framework for egocentric heterogeneous multi-robot exploration, based on overlap detection and alignment, that is independent of a manual initial guess or prior knowledge of the robots' poses. The novel proposed solution utilizes state-of-the-art place recognition learned descriptors, that through the framework's main pipeline, offer a fast and robust region overlap estimation, hence eliminating the need for the time-consuming global feature extraction and feature matching process that is typically used in 3D map integration. The region overlap estimation provides a homogeneous rigid transform that is applied as an initial condition in the point cloud registration algorithm Fast-GICP, which provides the final and refined alignment. The efficacy of the proposed framework is experimentally evaluated based on multiple field multi-robot exploration missions in underground environments, where both ground and aerial robots are deployed, with different sensor configurations.
FInC Flow: Fast and Invertible k×k Convolutions for Normalizing Flows

Invertible convolutions have been an essential element for building expressive normalizing flow-based generative models since their introduction in Glow. Several attempts have been made to design invertible $k \times k$ convolutions that are efficient in training and sampling passes. Though these attempts have improved the expressivity and sampling efficiency, they severely lagged behind Glow which used only $1 \times 1$ convolutions in terms of sampling time. Also, many of the approaches mask a large number of parameters of the underlying convolution, resulting in lower expressivity on a fixed run-time budget. We propose a $k \times k$ convolutional layer and Deep Normalizing Flow architecture which i. ) has a fast parallel inversion algorithm with running time O$(n k^2)$ ($n$ is height and width of the input image and k is kernel size), ii. ) masks the minimal amount of learnable parameters in a layer. iii. ) gives better forward pass and sampling times comparable to other $k \times k$ convolution-based models on real-world benchmarks. We provide an implementation of the proposed parallel algorithm for sampling using our invertible convolutions on GPUs. Benchmarks on CIFAR-10, ImageNet, and CelebA datasets show comparable performance to previous works regarding bits per dimension while significantly improving the sampling time.
Compact Objects from Spontaneous Symmetry Breaking

Gravitational waves from merging compact objects provides the opportunity to explore the properties of black holes and neutron stars in the strong regime of gravity. It is therefore of interest to explore the theoretical model that accurately describes them. Using the coset construction, we build a worldline Effective Field Theory that is derived from symmetry principles, does not involve additional degrees of freedom, and describes the most general compact object allowed in an effective Einstein-Maxwell vierbein theory. Such extended object can be described by its mass, spin, charge and size effects: tides, polarization and dissipation. By recognizing the symmetry breaking pattern, we derive all the covariant building blocks and constraints to build up the relevant invariant operators in the action to all orders. The developed theory elucidates the description of compact objects as an Effective Field Theory.
SpArX: Sparse Argumentative Explanations for Neural Networks

Neural networks (NNs) have various applications in AI, but explaining their decisions remains challenging. Existing approaches often focus on explaining how changing individual inputs affects NNs' outputs. However, an explanation that is consistent with the input-output behaviour of an NN is not necessarily faithful to the actual mechanics thereof. In this paper, we exploit relationships between multi-layer perceptrons (MLPs) and quantitative argumentation frameworks (QAFs) to create argumentative explanations for the mechanics of MLPs. Our SpArX method first sparsifies the MLP while maintaining as much of the original structure as possible. It then translates the sparse MLP into an equivalent QAF to shed light on the underlying decision process of the MLP, producing global and/or local explanations. We demonstrate experimentally that SpArX can give more faithful explanations than existing approaches, while simultaneously providing deeper insights into the actual reasoning process of MLPs.
DeepCOVID-Fuse: A Multi-modality Deep Learning Model Fusing Chest X-Radiographs and Clinical Variables to Predict COVID-19 Risk Levels

Propose: To present DeepCOVID-Fuse, a deep learning fusion model to predict risk levels in patients with confirmed coronavirus disease 2019 (COVID-19) and to evaluate the performance of pre-trained fusion models on full or partial combination of chest x-ray (CXRs) or chest radiograph and clinical variables. Materials and Methods: The initial CXRs, clinical variables and outcomes (i.e., mortality, intubation, hospital length of stay, ICU admission) were collected from February 2020 to April 2020 with reverse-transcription polymerase chain reaction (RT-PCR) test results as the reference standard. The risk level was determined by the outcome. The fusion model was trained on 1657 patients (Age: 58.30 +/- 17.74; Female: 807) and validated on 428 patients (56.41 +/- 17.03; 190) from Northwestern Memorial HealthCare system and was tested on 439 patients (56.51 +/- 17.78; 205) from a single holdout hospital. Performance of pre-trained fusion models on full or partial modalities were compared on the test set using the DeLong test for the area under the receiver operating characteristic curve (AUC) and the McNemar test for accuracy, precision, recall and F1. Results: The accuracy of DeepCOVID-Fuse trained on CXRs and clinical variables is 0.658, with an AUC of 0.842, which significantly outperformed (p < 0.05) models trained only on CXRs with an accuracy of 0.621 and AUC of 0.807 and only on clinical variables with an accuracy of 0.440 and AUC of 0.502. The pre-trained fusion model with only CXRs as input increases accuracy to 0.632 and AUC to 0.813 and with only clinical variables as input increases accuracy to 0.539 and AUC to 0.733. Conclusion: The fusion model learns better feature representations across different modalities during training and achieves good outcome predictions even when only some of the modalities are used in testing.
Gender-Segmented Labor Markets and Foreign Demand Shocks

Gender segmentation in labor markets shapes the local effects of international trade. We develop a theory that combines exports with gender-segmented labor markets and show that, in this framework, foreign demand shocks may either increase or decrease the female-to-male employment ratio. If a foreign demand shock happens in a female-intensive (male-intensive) sector, the model predicts that the female-to-male employment ratio should increase (decrease). We then use plausibly exogenous variation in the exposure of Tunisian local labor markets to foreign demand shocks and show that the empirical results are consistent with the theoretical prediction. In Tunisia, a developing country with a high degree of gender segmentation in labor markets, foreign-demand shocks have been relatively larger in male-intensive sectors. This induced a decrease in the female-to-male employment ratio, with households likely substituting female for male labor supply.
Enhancing quality and speed in database-free neural network reconstructions of undersampled MRI with SCAMPI

Purpose: We present SCAMPI (Sparsity Constrained Application of deep Magnetic resonance Priors for Image reconstruction), an untrained deep Neural Network for MRI reconstruction without previous training on datasets. It expands the Deep Image Prior approach with a multi-domain, sparsity-enforcing loss function to achieve higher image quality at a faster convergence speed than previously reported methods. Methods: Two-dimensional MRI data from the FastMRI dataset with Cartesian undersampling in phase-encoding direction were reconstructed for different acceleration rates for single coil and multicoil data. Results: The performance of our architecture was compared to state-of-the-art Compressed Sensing methods and ConvDecoder, another untrained Neural Network for 2D MRI reconstruction. SCAMPI outperforms these by better reducing undersampling artifacts and yielding lower error metrics in multicoil imaging. In comparison to ConvDecoder, the U-Net architecture combined with an elaborated loss-function allows for much faster convergence at higher image quality. SCAMPI can reconstruct multicoil data without explicit knowledge of coil sensitivity profiles.Moreover, it is a novel tool for reconstructing undersampled single coil k-space data. Conclusion: The presented approach avoids overfitting to dataset features, that can occur in Neural Networks trained on databases, because the network parameters are tuned only on the reconstruction data. It allows better results and faster reconstruction than the baseline untrained Neural Network approach.
Quantum-Electrodynamical Corrections to Energy-Levels of Helium

Quantum-electrodynamical corrections to energy levels of helium are calculated with an accuracy of a5mc~ on the basis of the Bethe-Salpeter equation according to the perturbation method of Fulton and Karplus. The interaction kernel of the equation is derived according to the method of Martin and Schwinger. The corrections are carefully calculated by the method which is free from any doubt. The result is expressed in terms of ·non-relativistic eigenfunctions. It is found for a:5mC! corrEctions that correction factors for spin-orbit and spin-spin couplings are in complete agreement with tf:ose derived by G. Araki on the ·basis of a simple consideration. Since a4mi! corrections often gave rise to cqnfusions and discussions among many authors these corrections are carefully examined. It is ' found on the firm basis that the a 4mi! terms involve a contact interaction but not the Breit surplus term. Further it is found that the energy corrEction for the . excited l::ound s~ate includes an imaginary part. Of course the ground-state energy is real. In fact the al:solute value of the imaginary part represents half the width of the level in question. A discussion for this fact is given.
Cellular Network Speech Enhancement: Removing Background and Transmission Noise

The primary objective of speech enhancement is to reduce background noise while preserving the target’s speech. A common dilemma occurs when a speaker is conﬁned to a noisy envi-ronment and receives a call with high background and transmission noise. To address this problem, the Deep Noise Suppression (DNS) Challenge focuses on removing the background noise with the next-generation deep learning models to en-hance the target’s speech; however, researchers fail to consider Voice Over IP (VoIP) applications their transmission noise. Fo-cusing on Google Meet and its cellular application, our work achieves state-of-the-art performance on the Google Meet To Phone Track of the VoIP DNS Challenge. This paper demonstrates how to beat industrial performance and achieve 1.92 PESQ and 0.88 STOI, as well as superior acoustic ﬁdelity, perceptual quality, and intelligibility in various metrics.
Provable Unrestricted Adversarial Training without Compromise with Generalizability

—Adversarial training (AT) is widely considered as the most promising strategy to defend against adversarial attacks and has drawn increasing interest from researchers. However, the existing AT methods still suffer from two challenges. First, they are unable to handle unrestricted adversarial examples (UAEs), which are built from scratch, as opposed to restricted adversarial examples (RAEs), which are created by adding perturbations bound by an l p norm to observed examples. Second, the existing AT methods often achieve adversarial robustness at the expense of standard generalizability (i.e., the accuracy on natural examples) because they make a tradeoff between them. To overcome these challenges, we propose a unique viewpoint that understands UAEs as imperceptibly perturbed unobserved examples. Also, we ﬁnd that the tradeoff results from the separation of the distributions of adversarial examples and natural examples. Based on these ideas, we propose a novel AT approach called Provable Unrestricted Adversarial Training (PUAT), which can provide a target classiﬁer with comprehensive adversarial robustness against both UAE and RAE, and simultaneously improve its standard generalizability. Particularly, PUAT utilizes partially labeled data to achieve effective UAE generation by accurately capturing the natural data distribution through a novel augmented triple-GAN. At the same time, PUAT extends the traditional AT by introducing the supervised loss of the target classiﬁer into the adversarial loss and achieves the alignment between the UAE distribution, the natural data distribution, and the distribution learned by the classiﬁer, with the collaboration of the augmented triple-GAN. Finally, the solid theoretical analysis and extensive experiments conducted on widely-used benchmarks demonstrate the superiority of PUAT.
A conditional abundance matching method of extending simulated halo merger trees to resolve low-mass progenitors and sub-halos

We present an algorithm to extend subhalo merger trees in a low-resolution dark-matter-only simulation by conditionally matching them to those in a high-resolution simulation. The algorithm is general and can be applied to simulation data with different resolutions using different target variables. We instantiate the algorithm by a case in which trees from ELUCID, a constrained simulation of (500 h−1 Mpc)3 volume of the local universe, are extended by matching trees from TNGDark, a simulation with much higher resolution. Our tests show that the extended trees are statistically equivalent to the high-resolution trees in the joint distribution of subhalo quantities and in important summary statistics relevant to modeling galaxy formation and evolution in halos. The extended trees preserve certain information of individual systems in the target simulation, including properties of resolved satellite subhalos, and shapes and orientations of their host halos. With the extension, subhalo merger trees in a cosmological scale simulation are extrapolated to a mass resolution comparable to that in a higher-resolution simulation carried out in a smaller volume, which can be used as the input for (sub)halo-based models of galaxy formation. The source code of the algorithm, and halo merger trees extended to a mass resolution of ∼2 × 108 h−1 M⊙ in the entire ELUCID simulation, are available.
Ensemble Transfer Learning for Multilingual Coreference Resolution

Entity coreference resolution is an important research problem with many applications, including information extraction and question answering. Coreference resolution for English has been studied extensively. However, there is relatively little work for other languages. A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. To overcome this challenge, we design a simple but effective ensemble-based framework that combines various transfer learning (TL) techniques. We first train several models using different TL methods. Then, during inference, we compute the unweighted average scores of the models’ predictions to extract the final set of predicted clusters. Furthermore, we also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts. Leveraging the idea that the coreferential links naturally exist between anchor texts pointing to the same article, our method builds a sizeable distantly-supervised dataset for the target language that consists of tens of thousands of documents. We can pre-train a model on the pseudo-labeled dataset before finetuning it on the final target dataset. Experimental results on two benchmark datasets, OntoNotes and SemEval, confirm the effectiveness of our methods. Our best ensembles consistently outperform the baseline approach of simple training by up to 7.68% in the F1 score. These ensembles also achieve new state-of-the-art results for three languages: Arabic, Dutch, and Spanish.
Self-Supervised Image Representation Learning: Transcending Masking with Paired Image Overlay

Self-supervised learning has become a popular approach in recent years for its ability to learn meaningful representations without the need for data annotation. This paper proposes a novel image augmentation technique, overlaying images, which has not been widely applied in self-supervised learning. This method is designed to provide better guidance for the model to understand underlying information, resulting in more useful representations. The proposed method is evaluated using contrastive learning, a widely used self-supervised learning method that has shown solid performance in downstream tasks. The results demonstrate the effectiveness of the proposed augmentation technique in improving the performance of self-supervised models.
Slice Transformer and Self-supervised Learning for 6DoF Localization in 3D Point Cloud Maps

Precise localization is critical for autonomous vehicles. We present a self-supervised learning method that employs transformers for the first time for the task of outdoor localization using LiDAR data. We propose a pre-text task that reorganizes the slices of a 360° LiDAR scan to leverage its axial properties. Our model, called Slice Transformer, employs multi-head attention while systematically processing the slices. To the best of our knowledge, this is the first instance of leveraging multi-head attention for outdoor point clouds. We additionally introduce the Perth-Wadataset, which provides a large-scale LiDAR map of Perth city in Western Australia, covering ~4km2area. Localization annotations are provided for Perth - Wa.The proposed localization method is thoroughly evaluated on Perth-WA and Appollo-SouthBay datasets. We also establish the efficacy of our self-supervised learning approach for the common downstream task of object classification using ModelNet40 and ScanNN datasets. The code and Perth-WA data will be publicly released.
Improving Accuracy of Zero-Shot Action Recognition with Handcrafted Features

With the development of machine learning, datasets for models are getting increasingly larger. This leads to increased data annotation costs and training time, which undoubtedly hinders the development of machine learning. To solve this problem, zero-shot learning is gaining considerable attention. With zero-shot learning, objects can be recognized or classiﬁed, even without having been seen before. Nevertheless, the accuracy of this method is still low, thus limiting its practical application. To solve this problem, we propose a video-text matching model, which can learn from handcrafted features. Our model can be used alone to predict the action classes and can also be added to any other model to improve its accuracy. Moreover, our model can be continuously optimized to improve its accuracy. We only need to manually annotate some features, which incurs some labor costs; in many situations, the costs are worth it. The results with UCF101 and HMDB51 show that our model achieves the best accuracy and also improves the accuracies of other models.
Reacceleration of Galactic Cosmic Rays beyond the Knee at the Termination Shock of a Cosmic-Ray-driven Galactic Wind

The origin of cosmic rays (CRs) above the knee in the spectrum is an unsolved problem. We present a wind model in which interstellar gas flows along a nonrotating, expanding flux tube with a changing speed and cross-sectional area. CRs from Galactic sources, such as supernova remnants, which are coupled to the plasma via Alfvén waves, provide the main pressure source for driving this outflow. These CRs are then subject to diffusive shock reacceleration at the Galactic wind termination shock, which is located at a distance ∼200 kpc. Some of the highest-energy reaccelerated particles propagate upstream against the wind and can contribute to the petaelectronvolt to exaelectronvolt range of the spectrum. We analyze the conditions under which efficient reacceleration can occur and find that rigidities ∼10–40 PV can be obtained and that the termination shock may account for between 10% and 50% of the proton spectrum measured in IceCube/IceTop experiment. In our model, the termination shock is unable to fully explain the CR spectrum in the petaelectronvolt to exaelectronvolt range. The highest-energy particles that escape downstream from our termination shock, and similar shocks surrounding most galaxies, can be further accelerated by intergalactic shock fronts.
Investigation of single event upset subject to protons of intermediate energy range (RAM)

Nuclear reaction models to reproduce p+Si nuclear reactions precisely in the incident proton energy region of below 50 MeV are investigated, and a computer code based on exciton models is developed. Si irradiation experiments in the intermediate energy region were performed to measure energy deposited by p+Si nuclear reactions with two totally depleted Si detectors in face-to-face arrangement. Coincident signals were analyzed by a two-dimensional pulse height analyzer. This method is shown to be effective in discriminating signals of contaminating particles. The experimental observations are in good agreement with the computed predictions. The single-event upset (SEU) cross section of the 93L422 (RAM) is also compared with the calculated values. >
Sampling-based Nyström Approximation and Kernel Quadrature

We analyze the Nystr\"om approximation of a positive definite kernel associated with a probability measure. We first prove an improved error bound for the conventional Nystr\"om approximation with i.i. d. sampling and singular-value decomposition in the continuous regime; the proof techniques are borrowed from statistical learning theory. We further introduce a refined selection of subspaces in Nystr\"om approximation with theoretical guarantees that is applicable to non-i.i. d. landmark points. Finally, we discuss their application to convex kernel quadrature and give novel theoretical guarantees as well as numerical observations.
Polynomial spline estimation for nonparametric (auto-)regressive models

In this paper we consider nonparametric (auto-)regressive models with conditional variance function. Based on polynomial spline estimation, we construct the estimators of conditional mean function and conditional variance function. Consistency of these estimators is proved under the α -mixing condition. Meanwhile, uniform and global rates of convergence of these estimators are established, and global rates of convergence can attain to the optimal ones. Moreover, a method for selecting knot number is developed based on Bayes Information Criterion. The methodology is illustrated by a simulation study and is applied to CNY/USD middle exchange rate.