Miracle-dz/UniFAR-FTUs huggingface dataset:
Eight Things to Know about Large Language Models

The widespread public deployment of large language models (LLMs) in recent months has prompted a wave of new attention and engagement from advocates, policymakers, and scholars from many fields. This attention is a timely response to the many urgent questions that this technology raises, but it can sometimes miss important considerations. This paper surveys the evidence for eight potentially surprising such points: 1. LLMs predictably get more capable with increasing investment, even without targeted innovation. 2. Many important LLM behaviors emerge unpredictably as a byproduct of increasing investment. 3. LLMs often appear to learn and use representations of the outside world. 4. There are no reliable techniques for steering the behavior of LLMs. 5. Experts are not yet able to interpret the inner workings of LLMs. 6. Human performance on a task isn't an upper bound on LLM performance. 7. LLMs need not express the values of their creators nor the values encoded in web text. 8. Brief interactions with LLMs are often misleading.
Learning Similarity between Scene Graphs and Images with Transformers

Scene graph generation is conventionally evaluated by (mean) Recall@K, which measures the ratio of correctly predicted triplets that appear in the ground truth. However, such triplet-oriented metrics cannot capture the global semantic information of scene graphs, and measure the similarity between images and generated scene graphs. The usability of scene graphs is therefore limited in downstream tasks. To address this issue, a framework that can measure the similarity of scene graphs and images is urgently required. Motivated by the successful application of Contrastive Language-Image Pre-training (CLIP), we propose a novel contrastive learning framework consisting of a graph Transformer and an image Transformer to align scene graphs and their corresponding images in the shared latent space. To enable the graph Transformer to comprehend the scene graph structure and extract representative features, we introduce a graph serialization technique that transforms a scene graph into a sequence with structural encoding. Based on our framework, we introduce R-Precision measuring image retrieval accuracy as a new evaluation metric for scene graph generation and establish new benchmarks for the Visual Genome and Open Images datasets. A series of experiments are further conducted to demonstrate the effectiveness of the graph Transformer, which shows great potential as a scene graph encoder.
Multilevel CNNs for Parametric PDEs

We combine concepts from multilevel solvers for partial differential equations (PDEs) with neural network based deep learning and propose a new methodology for the efficient numerical solution of high-dimensional parametric PDEs. An in-depth theoretical analysis shows that the proposed architecture is able to approximate multigrid V-cycles to arbitrary precision with the number of weights only depending logarithmically on the resolution of the finest mesh. As a consequence, approximation bounds for the solution of parametric PDEs by neural networks that are independent on the (stochastic) parameter dimension can be derived. The performance of the proposed method is illustrated on high-dimensional parametric linear elliptic PDEs that are common benchmark problems in uncertainty quantification. We find substantial improvements over state-of-the-art deep learning-based solvers. As particularly challenging examples, random conductivity with high-dimensional non-affine Gaussian fields in 100 parameter dimensions and a random cookie problem are examined. Due to the multilevel structure of our method, the amount of training samples can be reduced on finer levels, hence significantly lowering the generation time for training data and the training time of our method.
RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding

We propose a lightweight and scalable Regional Point-Language Contrastive learning framework, namely \textbf{RegionPLC}, for open-world 3D scene understanding, aiming to identify and recognize open-set objects and categories. Specifically, based on our empirical studies, we introduce a 3D-aware SFusion strategy that fuses 3D vision-language pairs derived from multiple 2D foundation models, yielding high-quality, dense region-level language descriptions without human 3D annotations. Subsequently, we devise a region-aware point-discriminative contrastive learning objective to enable robust and effective 3D learning from dense regional language supervision. We carry out extensive experiments on ScanNet, ScanNet200, and nuScenes datasets, and our model outperforms prior 3D open-world scene understanding approaches by an average of 17.2\% and 9.1\% for semantic and instance segmentation, respectively, while maintaining greater scalability and lower resource demands. Furthermore, our method has the flexibility to be effortlessly integrated with language models to enable open-ended grounded 3D reasoning without extra task-specific training. Code will be released.
Strouhal numbers of rectangular cylinders

Experiments on the vortex-shedding frequencies of various rectangular cylinders were conducted in a wind tunnel and in a water tank. The results show how Strouhal number varies with a width-to-height ratio of the cylinders in the range of Reynolds number between 70 and 2 × l04. There is found to exist a certain range of Reynolds number for the cylinders with the width-to-height ratios of 2 and 3 where flow pattern abruptly changes with a sudden discontinuity in Strouhal number. The changes in flow pattern corresponding to the discontinuity of Strouhal number have been confirmed by means of measurements of velocity distribution and flow visualization. These data are compared with those of other investigators. The experimental results have been found to show a good agreement with those of numerical calculations.
Revaluing Low-Wage Work

Labor and community activists have organized around raising wages for low-wage workers, notably through the “Fight for 15” campaign among fast food workers. This study uses skills data together with manager survey data to estimate occupational wages. Quantile regression was used to estimate a skills-based wage. The analysis provides empirical support for a US$15 an hour minimum wage. The results indicate that low-wage workers possess skills that are undervalued in the labor market.
Wave-averaged motion of small particles in surface gravity waves: Effect of particle shape on orientation, drift, and dispersion

Particles such as microplastics and phytoplankton suspended in the water column in the natural environment are often subject to the action of surface gravity waves. By modelling such anisotropic particles as small spheroids that slowly settle (or rise) in a wavy environment, we consider how the particle shape and buoyancy couple to the background wave-driven flow to influence the particle orientation, drift, and dispersion. A multiscale expansion allows the wave-induced oscillations to be separated from the wave-averaged particle motion. Using the wave-averaged equations of particle motion, we demonstrate that spheroidal particles have a wave-induced preferential orientation with different stable solutions for prolate and oblate particles. The resulting preferential orientation positions particles with their longest axis pointing in the direction of wave propagation and upwards against gravity. The angle at which the longest axis points upwards is a function of particle aspect ratio. In this orientation, particles drift in the direction opposite to wave propagation, weakening and potentially even reversing their Stokes drift. The wave-induced stable orientation also results in a reduced settling velocity relative to a random (isotropic) orientation. The dispersion of a particle cloud is controlled by the distribution of orientations. For a cloud of particles released together with random (isotropic) orientation, the initial cloud growth rate is ballistic in all directions. Wave action acts to suppress the vertical dispersion, but enhances horizontal dispersion into a super-ballistic state when the Stokes drift shear acts on a particle cloud that has expanded in the vertical direction.
Discovering and Explaining the Noncausality of Deep Learning in SAR ATR

In recent years, deep learning has been widely used in synthetic aperture radar (SAR) automatic target recognition (ATR) and achieved excellent performance on the moving and stationary target acquisition and recognition (MSTAR) dataset. However, due to constrained imaging conditions, MSTAR has data biases such as background correlation, that is, background clutter properties have a spurious correlation with target classes. Deep learning can overfit clutter to reduce training errors. Therefore, the degree of overfitting for clutter reflects the noncausality of deep learning in SAR ATR. Existing methods only qualitatively analyze this phenomenon. In this letter, we quantify the contributions of different regions to target recognition based on the Shapley value. The Shapley value of clutter measures the degree of overfitting. Moreover, we explain how data bias and model bias contribute to noncausality. Concisely, data bias leads to comparable signal-to-clutter ratios (SCR) and clutter textures in training and test sets. And various model structures have different degrees of overfitting for these biases. The experimental results of various models under standard operating conditions (SOCs) on the MSTAR dataset support our conclusions. Our code is available at https://github.com/waterdisappear/Data-Bias-in-MSTAR.
Dichotomies for Maximum Matching Cut: H-Freeness, Bounded Diameter, Bounded Radius

The (Perfect) Matching Cut problem is to decide if a graph $G$ has a (perfect) matching cut, i.e., a (perfect) matching that is also an edge cut of $G$. Both Matching Cut and Perfect Matching Cut are known to be NP-complete. A perfect matching cut is also a matching cut with maximum number of edges. To increase our understanding of the relationship between the two problems, we introduce the Maximum Matching Cut problem. This problem is to determine a largest matching cut in a graph. We generalize and unify known polynomial-time algorithms for Matching Cut and Perfect Matching Cut restricted to graphs of diameter at most $2$ and to $(P_6+sP_2)$-free graphs. We also show that the complexity of Maximum Matching Cut differs from the complexities of Matching Cut and Perfect Matching Cut by proving NP-hardness of Maximum Matching Cut for $2P_3$-free quadrangulated graphs of diameter $3$ and radius $2$ and for subcubic line graphs of triangle-free graphs. In this way, we obtain full dichotomies of Maximum Matching Cut for graphs of bounded diameter, bounded radius and $H$-free graphs. Finally, we apply our techniques to get a dichotomy for the Maximum Disconnected Perfect Matching problem for $H$-free graphs. A disconnected perfect matching of a graph $G$ is a perfect matching that contains a matching cut of $G$. The Maximum Disconnected Perfect Matching problem asks to determine for a connected graph $G$, a disconnected perfect matching with a largest matching cut over all disconnected perfect matchings of $G$. Our dichotomy result implies that the original decision problem Disconnected Perfect Matching is polynomial-time solvable for $(P_6+sP_2)$-free graphs for every $s\geq 0$, which resolves an open problem of Bouquet and Picouleau (arXiv, 2020).
Variations of Orthonormal Basis Matrices of Subspaces

An orthonormal basis matrix $X$ of a subspace ${\cal X}$ is known not to be unique, unless there are some kinds of normalization requirements. One of them is to require that $X^{\rm T}D$ is positive semi-definite, where $D$ is a constant matrix of apt size. It is a natural one in multi-view subspace learning models in which $X$ serves as a projection matrix and is determined by a maximization problem over the Stiefel manifold whose objective function contains and increases with tr$(X^{\rm T}D)$. This paper is concerned with bounding the change in orthonormal basis matrix $X$ as subspace ${\cal X}$ varies under the requirement that $X^{\rm T}D$ stays positive semi-definite. The results are useful in convergence analysis of the NEPv approach (nonlinear eigenvalue problem with eigenvector dependency) to solve the maximization problem.
Enormous variation in homogeneity of room temperature superconductor samples: a Comment on Nature 615, 244 (2023)

The resistive transition width of a recently discovered room temperature near-ambient-pressure superconductor [1] changes by more than three orders of magnitude between diﬀerent samples, with the transition temperature nearly unchanged. For the narrowest transitions, the transition width relative to T c is only 0 . 014%. This anomalous behavior and other issues indicate that this system is not a superconductor.
Vision-Language Models for Vision Tasks: A Survey

Most visual recognition studies rely heavily on crowd-labelled data in deep neural networks (DNNs) training, and they usually train a DNN for each single visual recognition task, leading to a laborious and time-consuming visual recognition paradigm. To address the two challenges, Vision-Language Models (VLMs) have been intensively investigated recently, which learns rich vision-language correlation from web-scale image-text pairs that are almost infinitely available on the Internet and enables zero-shot predictions on various visual recognition tasks with a single VLM. This paper provides a systematic review of visual language models for various visual recognition tasks, including: (1) the background that introduces the development of visual recognition paradigms; (2) the foundations of VLM that summarize the widely-adopted network architectures, pre-training objectives, and downstream tasks; (3) the widely-adopted datasets in VLM pre-training and evaluations; (4) the review and categorization of existing VLM pre-training methods, VLM transfer learning methods, and VLM knowledge distillation methods; (5) the benchmarking, analysis and discussion of the reviewed methods; (6) several research challenges and potential research directions that could be pursued in the future VLM studies for visual recognition. A project associated with this survey has been created at https://github.com/jingyi0000/VLM_survey.
FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction

Click-through rate (CTR) prediction is one of the fundamental tasks in online advertising and recommendation. Multi-layer perceptron (MLP) serves as a core component in many deep CTR prediction models, but it has been widely shown that applying a vanilla MLP network alone is ineffective in learning complex feature interactions. As such, many two-stream models (e.g., Wide&Deep, DeepFM, and DCN) have recently been proposed, aiming to integrate two parallel sub-networks to learn feature interactions from two different views for enhanced CTR prediction. In addition to one MLP stream that learns feature interactions implicitly, most of the existing research focuses on designing another stream to complement the MLP stream with explicitly enhanced feature interactions. Instead, this paper presents a simple two-stream feature interaction model, namely FinalMLP, which employs only MLPs in both streams yet achieves surprisingly strong performance. In contrast to sophisticated network design in each stream, our work enhances CTR modeling through a feature selection module, which produces differentiated feature inputs to two streams, and a group-wise bilinear fusion module, which effectively captures stream-level interactions across two streams. We show that FinalMLP achieves competitive or even better performance against many existing two-stream CTR models on four open benchmark datasets and also brings significant CTR improvements during an online A/B test in our industrial news recommender system. We envision that the simple yet effective FinalMLP model could serve as a new strong baseline for future development of two-stream CTR models. Our source code will be available at MindSpore/models and FuxiCTR/model_zoo.
DRIP: deep regularizers for inverse problems

In this paper we consider inverse problems that are mathematically ill-posed. That is, given some (noisy) data, there is more than one solution that approximately fits the data. In recent years, deep neural techniques that find the most appropriate solution, in the sense that it contains a-priori information, were developed. However, they suffer from several shortcomings. First, most techniques cannot guarantee that the solution fits the data at inference. Second, while the derivation of the techniques is inspired by the existence of a valid scalar regularization function, such techniques do not in practice rely on such a function, and therefore veer away from classical variational techniques. In this work we introduce a new family of neural regularizers for the solution of inverse problems. These regularizers are based on a variational formulation and are guaranteed to fit the data. We demonstrate their use on a number of highly ill-posed problems, from image deblurring to limited angle tomography.
Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tails

In this work, we study the convergence \emph{in high probability} of clipped gradient methods when the noise distribution has heavy tails, ie. , with bounded $p$th moments, for some $1<p\le2$. Prior works in this setting follow the same recipe of using concentration inequalities and an inductive argument with union bound to bound the iterates across all iterations. This method results in an increase in the failure probability by a factor of $T$, where $T$ is the number of iterations. We instead propose a new analysis approach based on bounding the moment generating function of a well chosen supermartingale sequence. We improve the dependency on $T$ in the convergence guarantee for a wide range of algorithms with clipped gradients, including stochastic (accelerated) mirror descent for convex objectives and stochastic gradient descent for nonconvex objectives. This approach naturally allows the algorithms to use time-varying step sizes and clipping parameters when the time horizon is unknown, which appears impossible in prior works. We show that in the case of clipped stochastic mirror descent, problem constants, including the initial distance to the optimum, are not required when setting step sizes and clipping parameters.
Interstellar objects

ABSTRACT Since 2017, two macroscopic interstellar objects have been discovered in the inner Solar System, both of which are distinct in nature. The first interstellar object, 1I/‘Oumuamua, passed within lunar distances of the Earth, appeared asteroidal lacking detectable levels of gas or dust loss, yet exhibited a nongravitational acceleration. 1I/‘Oumuamua's brief visit left open questions regarding its provenance which has given rise to many theoretical hypotheses, including an icy comet lacking a dust coma, an elongated fragment of a planet or planetesimal that was tidally disrupted, and an ultra-porous fractal aggregate. The second interstellar object, 2I/Borisov, was distinct from 1I/‘Oumuamua in terms of its bulk physical properties and displayed a definitive cometary tail. We review the discoveries of these objects, the subsequent observations and characterisations, and the theoretical hypotheses regarding their origins. We describe 1I/‘Oumuamua and 2I/Borisov in the context of active asteroids and comets in the Solar System. The discovery of these two objects implies a galactic-wide population of similar bodies. Forthcoming observatories should detect many more interstellar planetesimals which may offer new insights into how planetary formation processes vary throughout the Galaxy.
A 1.3% Distance to M33 from Hubble Space Telescope Cepheid Photometry

We present a low-dispersion period–luminosity relation (PL) based on 154 Cepheids in Messier 33 (M33) with Hubble Space Telescope (HST) photometry from the PHATTER survey. Using high-quality ground-based light curves, we recover Cepheid phases and amplitudes for multi-epoch HST data and we perform template fitting to derive intensity-averaged mean magnitudes. HST observations in the SH0ES near-infrared Wesenheit system significantly reduce the effect of crowding relative to ground-based data, as seen in the final PL scatter of σ = 0.11 mag. We adopt the absolute calibration of the PL based on HST observations in the Large Magellanic Cloud and a distance derived using late-type detached eclipsing binaries to obtain a distance modulus for M33 of μ = 24.622 ± 0.030 mag (d = 840 ± 11 kpc), a best-to-date precision of 1.3%. We find very good agreement with past Cepheid-based measurements. Several tip of the red giant branch estimates bracket our result while disagreeing with each other. Finally, we show that the flux contribution from star clusters hosting Cepheids in M33 does not impact the distance measurement and we find only ∼3.7% of the sample is located in (or nearby) young clusters. M33 offers one of the best sites for the cross-calibration of many primary distance indicators. Thus, a precise independent geometric determination of its distance would provide a valuable new anchor to measure the Hubble constant.
New astrometric observations of Triton in 2007–2009

Astrometric positions of the Neptunian satellite Triton with a visual magnitude of 13.5 were obtained during three successive oppositions in 2007, 2008 and 2009. A total of 1095 new observed positions of Triton were collected during 46 nights of observations, involving eight missions and three telescopes. We compared our observations to the best ephemerides of Triton available now. This comparison has shown that our observations present a high level of accuracy as they provide standard deviations of residuals hardly higher than 50 mas and mean residuals lower than 30 mas, corresponding to about only 500 km in the position of the very distant satellite Triton. Moreover, we have compared most of the different planetary ephemerides of Neptune available now as well as two recent orbit models of Triton. These new comparisons have clearly shown the differences between all of these ephemerides which can be significant and that are presented in this work.
Fundamental Limits of Non-Centered Non-Separable Channels and Their Application in Holographic MIMO Communications

The classical Rician Weichselberger channel and the emerging holographic multiple-input multiple-output (MIMO) channel share a common characteristic of non-separable correlation, which captures the interdependence between transmit and receiver antennas. However, this correlation structure makes it very challenging to characterize the fundamental limits of non-centered (Rician), non-separable MIMO channels. In fact, there is a dearth of existing literature that addresses this specific aspect, underscoring the need for further research in this area. In this paper, we investigate the mutual information (MI) of non-centered non-separable MIMO channels, where both the line-of-sight and non-line-of-sight components are considered. By utilizing random matrix theory (RMT), we set up a central limit theorem for the MI and give the closed-form expressions for its mean and variance. The derived results are then utilized to approximate the ergodic MI and outage probability of holographic MIMO channels. Numerical simulations validate the accuracy of the theoretical results.