Miracle-dz/UniFAR-FTUs huggingface dataset:
GOOD: A Graph Out-of-Distribution Benchmark

Out-of-distribution (OOD) learning deals with scenarios in which training and test data follow different distributions. Although general OOD problems have been intensively studied in machine learning, graph OOD is only an emerging area of research. Currently, there lacks a systematic benchmark tailored to graph OOD method evaluation. In this work, we aim at developing an OOD benchmark, known as GOOD, for graphs specifically. We explicitly make distinctions between covariate and concept shifts and design data splits that accurately reflect different shifts. We consider both graph and node prediction tasks as there are key differences in designing shifts. Overall, GOOD contains 11 datasets with 17 domain selections. When combined with covariate, concept, and no shifts, we obtain 51 different splits. We provide performance results on 10 commonly used baseline methods with 10 random runs. This results in 510 dataset-model combinations in total. Our results show significant performance gaps between in-distribution and OOD settings. Our results also shed light on different performance trends between covariate and concept shifts by different methods. Our GOOD benchmark is a growing project and expects to expand in both quantity and variety of resources as the area develops. The GOOD benchmark can be accessed via https://github.com/divelab/GOOD/.
Optimal quasi-Bayesian reduced rank regression with incomplete response

The aim of reduced rank regression is to connect multiple response variables to multiple predictors. This model is very popular, especially in biostatistics where multiple measurements on individuals can be re-used to predict multiple outputs. Unfortunately, there are often missing data in such datasets, making it diﬃcult to use standard estimation tools. In this paper, we study the problem of reduced rank regression where the response matrix is incomplete. We propose a quasi-Bayesian approach to this problem, in the sense that the likelihood is replaced by a quasi-likelihood. We provide a tight oracle inequality, proving that our method is adaptive to the rank of the coeﬃcient matrix. We describe a Langevin Monte Carlo algorithm for the computation of the posterior mean. Numerical comparison on synthetic and real data show that our method are competitive to the state-of-the-art where the rank is chosen by cross validation, and sometimes lead to an improvement.
Testing chiral unitary models for the $\Lambda(1405)$ in $K^{+}\pi\Sigma$ photoproduction

We adopt a novel formalism for the low-energy analysis of the γp → K + π Σ photoproduction reaction to calculate the π Σ invariant mass distributions in the Λ(1405) resonance region. The approach adheres to constraints arising from unitarity, gauge invariance and chiral perturbation theory, and is used without adjusting any model parameters. It is found that the meson-baryon rescattering in the ﬁnal state has a major impact on the magnitude and structure of the generated spectra that are compared with the experimental data from the CLAS collaboration. We demonstrate a large sensitivity of the theoretical predictions to the choice of the coupled-channel π Σ − ¯ KN model amplitudes which should enable one to constrain the parameter space of these models.
Efficient WiFi LiDAR SLAM for Autonomous Robots in Large Environments

Autonomous robots operating in indoor and GPS denied environments can use LiDAR for SLAM instead. However, LiDARs do not perform well in geometrically-degraded environments, due to the challenge of loop closure detection and computational load to perform scan matching. Existing WiFi infrastructure can be exploited for localization and mapping with low hardware and computational cost. Yet, accurate pose estimation using WiFi is challenging as different signal values can be measured at the same location due to the unpredictability of signal propagation. Therefore, we introduce the use of WiFi fingerprint sequence for pose estimation (i.e. loop closure) in SLAM. This approach exploits the spatial coherence of location fingerprints obtained while a mobile robot is moving. This has better capability of correcting odometry drift. The method also incorporates LiDAR scans and thus, improving computational efficiency for large and geometrically-degraded environments while maintaining the accuracy of LiDAR SLAM. We conducted experiments in an indoor environment to illustrate the effectiveness of the method. The results are evaluated based on Root Mean Square Error (RMSE) and it has achieved an accuracy of 0.88m for the test environment.
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-Level Product Retrieval

Our goal in this research is to study a more realistic environment in which we can conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories. We first contribute the Product1M datasets and define two real practical instance-level retrieval tasks that enable evaluations on price comparison and personalized recommendations. For both instance-level tasks, accurately identifying the intended product target mentioned in visual-linguistic data and mitigating the impact of irrelevant content are quite challenging. To address this, we devise a more effective cross-modal pretraining model capable of adaptively incorporating key concept information from multi-modal data. This is accomplished by utilizing an entity graph, where nodes represented entities and edges denoted the similarity relations between them. Specifically, a novel Entity-Graph Enhanced Cross-Modal Pretraining (EGE-CMP) model is proposed for instance-level commodity retrieval, which explicitly injects entity knowledge in both node-based and subgraph-based ways into the multi-modal networks via a self-supervised hybrid-stream transformer. This could reduce the confusion between different object contents, thereby effectively guiding the network to focus on entities with real semantics. Experimental results sufficiently verify the efficacy and generalizability of our EGE-CMP, outperforming several SOTA cross-modal baselines like CLIP Radford et al. 2021, UNITER Chen et al. 2020 and CAPTURE Zhan et al. 2021.
How You Start Matters for Generalization

Characterizing the remarkable generalization properties of over-parameterized neural networks remains an open problem. In this paper, we promote a shift of focus towards initialization rather than neural architecture or (stochastic) gradient descent to explain this implicit regularization. Through a Fourier lens, we derive a general result for the spectral bias of neural networks and show that the generalization of neural networks is heavily tied to their initialization. Further, we empirically solidify the developed theoretical insights using practical, deep networks. Finally, we make a case against the controversial flat-minima conjecture and show that Fourier analysis grants a more reliable framework for understanding the generalization of neural networks.
Reconstructing vehicles from orthographic drawings using deep neural networks

This paper explores the current state-of-the-art of object reconstruction from multiple orthographic drawings using deep neural networks. It proposes two algorithms to extract multiple views from a single image. The paper proposes a system based on pixel-aligned implicit functions (PIFu) and develops an advanced sampling strategy to generate signed distance samples. It also compares this approach to depth map regression from multiple views. Additionally, the paper uses a novel dataset for vehicle reconstruction from the racing game Assetto Corsa, which features higher quality models than the commonly used ShapeNET dataset. The trained neural network generalizes well to real-world inputs and creates plausible and detailed reconstructions.
Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets

Deep neural networks usually perform poorly when the training dataset suffers from extreme class imbalance. Recent studies found that directly training with out-of-distribution data (i.e., open-set samples) in a semi-supervised manner would harm the generalization performance. In this work, we theoretically show that out-of-distribution data can still be leveraged to augment the minority classes from a Bayesian perspective. Based on this motivation, we propose a novel method called Open-sampling, which utilizes open-set noisy labels to re-balance the class priors of the training dataset. For each open-set instance, the label is sampled from our pre-defined distribution that is complementary to the distribution of original class priors. We empirically show that Open-sampling not only re-balances the class priors but also encourages the neural network to learn separable representations. Extensive experiments demonstrate that our proposed method significantly outperforms existing data re-balancing methods and can boost the performance of existing state-of-the-art methods.
Reviewer Preferences and Gender Disparities in Aesthetic Judgments

Aesthetic preferences are considered highly subjective resulting in inherently noisy judgements of aesthetic objects, yet certain aspects of aesthetic judgement display convergent trends over time. This paper present a study that uses literary reviews as a proxy for aesthetic judgement in order to identify systematic components that can be attributed to bias. Specifically we find that judgement of literary quality in newspapers displays a gender bias in preference of male writers. Male reviewers have a same gender preference while female reviewer show an opposite gender preference. While alternative accounts exist of this apparent gender disparity, we argue that it reflects a cultural gender antagonism.
DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification

Multiple Instance Learning (MIL) is widely used in analyz-ing histopathological Whole Slide Images (WSIs). However, existing MIL methods do not explicitly model the data distribution, and instead they only learn a bag-level or instance-level decision boundary discriminatively by training a classiﬁer. In this paper, we propose DGMIL: a feature distribution guided deep MIL framework for WSI classiﬁcation and positive patch localization. Instead of designing complex discriminative network architectures, we reveal that the inherent feature distribution of histopathological image data can serve as a very eﬀective guide for instance classiﬁcation. We propose a cluster-conditioned feature distribution modeling method and a pseudo label-based iterative feature space reﬁnement strategy so that in the ﬁnal feature space the positive and negative instances can be easily separated. Experiments on the CAMELYON16 dataset and the TCGA Lung Cancer dataset show that our method achieves new SOTA for both global classiﬁcation and positive patch localization tasks.
Learning Implicit Feature Alignment Function for Semantic Segmentation

Integrating high-level context information with low-level details is of central importance in semantic segmentation. Towards this end, most existing segmentation models apply bilinear up-sampling and convolutions to feature maps of different scales, and then align them at the same resolution. However, bilinear up-sampling blurs the precise information learned in these feature maps and convolutions incur extra computation costs. To address these issues, we propose the Implicit Feature Alignment function (IFA). Our method is inspired by the rapidly expanding topic of implicit neural representations, where coordinate-based neural networks are used to designate fields of signals. In IFA, feature vectors are viewed as representing a 2D field of information. Given a query coordinate, nearby feature vectors with their relative coordinates are taken from the multi-level feature maps and then fed into an MLP to generate the corresponding output. As such, IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. We demonstrate the efficacy of IFA on multiple datasets, including Cityscapes, PASCAL Context, and ADE20K. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art computation-accuracy trade-off on common benchmarks. Code will be made available at https://github.com/hzhupku/IFA.
Sparse Double Descent: Where Network Pruning Aggravates Overfitting

People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity. However, our work surprisingly discovers that network pruning sometimes even aggravates overfitting. We report an unexpected sparse double descent phenomenon that, as we increase model sparsity via network pruning, test performance first gets worse (due to overfitting), then gets better (due to relieved overfitting), and gets worse at last (due to forgetting useful information). While recent studies focused on the deep double descent with respect to model overparameterization, they failed to recognize that sparsity may also cause double descent. In this paper, we have three main contributions. First, we report the novel sparse double descent phenomenon through extensive experiments. Second, for this phenomenon, we propose a novel learning distance interpretation that the curve of $\ell_{2}$ learning distance of sparse models (from initialized parameters to final parameters) may correlate with the sparse double descent curve well and reflect generalization better than minima flatness. Third, in the context of sparse double descent, a winning ticket in the lottery ticket hypothesis surprisingly may not always win.
Distances for Comparing Multisets and Sequences

Measuring the distance between data points is fundamental to many statistical techniques, such as dimension reduction or clustering algorithms. However, improvements in data collection technologies has led to a growing versatility of structured data for which standard distance measures are inap-plicable. In this paper, we consider the problem of measuring the distance between sequences and multisets of points lying within a metric space, mo-tivated by the analysis of an in-play football data set. Drawing on the wider literature, including that of time series analysis and optimal transport, we discuss various distances which are available in such an instance. For each distance, we state and prove theoretical properties, proposing possible extensions where they fail. Finally, via an example analysis of the in-play football data, we illustrate the usefulness of these distances in practice.
Enriched purity and presentability in Banach spaces

Abstract The category of Banach spaces and linear maps of norm is locally -presentable but not locally finitely presentable. We prove, however, that is locally finitely presentable in the enriched sense over complete metric spaces. Moreover, in this sense, pure morphisms are just ideals of Banach spaces. We characterize classes of Banach spaces approximately injective with respect to sets of morphisms having finite-dimensional domains and separable codomains.
SimA: Simple Softmax-free Attention for Vision Transformers

Recently, vision transformers have become very popular. However, deploying them in many applications is computationally expensive partly due to the Softmax layer in the attention block. We introduce a simple but effective, Softmax-free attention block, SimA, which normalizes query and key matrices with simple $\ell_1$-norm instead of using Softmax layer. Then, the attention block in SimA is a simple multiplication of three matrices, so SimA can dynamically change the ordering of the computation at the test time to achieve linear computation on the number of tokens or the number of channels. We empirically show that SimA applied to three SOTA variations of transformers, DeiT, XCiT, and CvT, results in on-par accuracy compared to the SOTA models, without any need for Softmax layer. Interestingly, changing SimA from multi-head to single-head has only a small effect on the accuracy, which simplifies the attention block further. The code is available here: https://github.com/UCDvision/sima
ATOMS: ALMA Three-millimeter Observations of Massive Star-forming regions – XI. From inflow to infall in hub-filament systems

We investigate the presence of hub-filament systems in a large sample of 146 active proto-clusters, using H13CO+ J=1-0 molecular line data obtained from the ATOMS survey. We find that filaments are ubiquitous in proto-clusters, and hub-filament systems are very common from dense core scales (∼0.1 pc) to clump/cloud scales (∼1–10 pc). The proportion of proto-clusters containing hub-filament systems decreases with increasing dust temperature (Td) and luminosity-to-mass ratios (L/M) of clumps, indicating that stellar feedback from H ii regions gradually destroys the hub-filament systems as proto-clusters evolve. Clear velocity gradients are seen along the longest filaments with a mean velocity gradient of 8.71 km s−1 pc−1 and a median velocity gradient of 5.54 km s−1 pc−1. We find that velocity gradients are small for filament lengths larger than ∼1 pc, probably hinting at the existence of inertial inflows, although we cannot determine whether the latter are driven by large-scale turbulence or large-scale gravitational contraction. In contrast, velocity gradients below ∼1 pc dramatically increase as filament lengths decrease, indicating that the gravity of the hubs or cores starts to dominate gas infall at small scales. We suggest that self-similar hub-filament systems and filamentary accretion at all scales may play a key role in high-mass star formation.
Large non-reciprocal charge transport in Pt2MnGe up to room temperature

Non-reciprocal charge transport that is strongly associated with the structural or magnetic chirality of the quantum materials system is one of the most exotic properties of condensed matter physics. Here, using magnetic alloys film Pt2MnGe, we have realized the large non-reciprocal charge transport up to room temperature, which roots in the organic combination of chirality dependent carrier scattering and special magnetic configurations. In this framework, the conduction electrons are scattered asymmetrically by the emerging non-zero vector spin chirality under in-plane magnetic field, resulting in robust non-reciprocal response. More astonishingly, the vector spin chirality in Pt2MnGe film can be reversed by a spin-polarized current through spin Hall effect in a junction with Pt layer. Our work resolves the general limitation of non-reciprocal charge transport to cryogenic temperatures, and paves the way for extending its applications in the emerging field of chiral spintronics.
Mass Hierarchies and Quantum Gravity Constraints in DKMM‐refined KKLT

We carefully revisit the mass hierarchies for the KKLT scenario with an uplift term from an anti D3‐brane in a strongly warped throat. First, we derive the bound resulting from what is usually termed “the throat fitting into the bulk” directly from the Klebanov‐Strassler geometry. Generating the small value of the superpotential W0 via the mechanism proposed by Demirtas, Kim, McAllister and Moritz (DKMM), we identify two possible DKMM‐refined KKLT scenarios for stabilizing the light axio‐dilaton modulus. The first scenario spoils the expected hierarchy between the bulk and the throat mass scales and implies that the energy scale of the uplift is larger than the species scale of the effective theory in the throat. Moreover it requires an unnaturally large tadpole N∼107−8$N\sim 10^{7-8}$ that is certainly in conflict with tadpole cancellation. For the less restricted second scenario, the hierarchy can be controlled at the expense of, under the most optimistic assumptions, an only moderately large tadpole N∼102−3$N\sim 10^{2-3}$ .
NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates

Conventionally, audio super-resolution models fixed the initial and the target sampling rates, which necessitate the model to be trained for each pair of sampling rates. We introduce NU-Wave 2, a diffusion model for neural audio upsampling that enables the generation of 48 kHz audio signals from inputs of various sampling rates with a single model. Based on the architecture of NU-Wave, NU-Wave 2 uses short-time Fourier convolution (STFC) to generate harmonics to resolve the main failure modes of NU-Wave, and incorporates bandwidth spectral feature transform (BSFT) to condition the bandwidths of inputs in the frequency domain. We experimentally demonstrate that NU-Wave 2 produces high-resolution audio regardless of the sampling rate of input while requiring fewer parameters than other models. The official code and the audio samples are available at https://mindslab-ai.github. io/nuwave2.
Exploring Jupiter's Polar Deformation Lengths with High-resolution Shallow Water Modeling

The polar regions of Jupiter host a myriad of dynamically interesting phenomena, including vortex configurations, folded-filamentary regions (FFRs), and chaotic flows. Juno observations have provided unprecedented views of the high latitudes, allowing for more constraints to be placed upon the troposphere and the overall atmospheric energy cycle. Moist convective events are believed to be the primary drivers of energetic storm behavior as observed on the planet. Here we introduce a novel single-layer shallow water model to investigate the effects of polar moist convective events at high resolution, the presence of dynamical instabilities over long timescales, and the emergence of FFRs at high latitudes. We use a flexible, highly parallelizable, finite difference hydrodynamic code to explore the parameter space set up by previous models. We study the long-term effects of deformation length (L d ), injected pulse size, and injected geopotential. We find that models with L d beyond 1500 km (planetary Burger number, Bu = 4.4 × 10−4) tend to homogenize their potential vorticity in the form of dominant stable polar cyclones, while lower-L d cases tend to show less stability with regard to Arnol’d-type flows. We also find that large turbulent forcing scales consistently lead to the formation of high-latitude FFRs. Our findings support the idea that moist convection occurring at high latitudes may be sufficient to produce the dynamical variety seen at the Jovian poles. Additionally, derived values of localized horizontal shear and L d may constrain FFR formation and evolution.