Miracle-dz/UniFAR-FTUs huggingface dataset:
Stellar mass spectrum within massive collapsing clumps. II. Thermodynamics and tidal forces of the first Larson core. A robust mechanism for the peak of the IMF

We investigate the dependence of the peak of the IMF on the physics of the so-called first Larson core, which corresponds to the point where the dust becomes opaque to its own radiation. We perform numerical simulations of collapsing clouds of $1000 M_\odot$ for various gas equation of state (eos), paying great attention to the numerical resolution and convergence. The initial conditions of these numerical experiments are varied in the companion paper. We also develop analytical models that we confront to our numerical results. If an isothermal eos is used, we show that the peak of the IMF shifts to lower masses with improved numerical resolution. When an adiabatic eos is employed, numerical convergence is obtained. The peak position varies with the eos and we find that the peak position is about ten times the mass of the first Larson core. By analyzing the stability of non-linear density fluctuations in the vicinity of a point mass and then summing over a reasonable density distribution, we find that tidal forces exert a strong stabilizing effect and likely lead to a preferential mass several times larger than that of the first Larson core. We propose that in a sufficiently massive and cold cloud, the peak of the IMF is determined by the thermodynamics of the high density adiabatic gas as well as the stabilizing influence of tidal forces. The resulting characteristic mass is about ten times the mass of the first Larson core, which altogether leads to a few tenths of solar masses. Since these processes are not related to the large scale physical conditions and to the environment, our results suggest a possible explanation for the apparent universality of the peak of the IMF.
Are Current Decoding Strategies Capable of Facing the Challenges of Visual Dialogue?

Decoding strategies play a crucial role in natural language generation systems. They are usually designed and evaluated in open-ended text-only tasks, and it is not clear how different strategies handle the numerous challenges that goal-oriented multimodal systems face (such as grounding and informativeness). To answer this question, we compare a wide variety of different decoding strategies and hyper-parameter configurations in a Visual Dialogue referential game. Although none of them successfully balance lexical richness, accuracy in the task, and visual grounding, our in-depth analysis allows us to highlight the strengths and weaknesses of each decoding strategy. We believe our findings and suggestions may serve as a starting point for designing more effective decoding algorithms that handle the challenges of Visual Dialogue tasks.
Metadata Privacy Beyond Tunneling for Instant Messaging

Transport layer data leaks metadata unintentionally -- such as who communicates with whom. While tools for strong transport layer privacy exist, they have adoption obstacles, including performance overheads incompatible with mobile devices. We posit that by changing the objective of metadata privacy for $\textit{all traffic}$, we can open up a new design space for pragmatic approaches to transport layer privacy. As a first step in this direction, we propose using techniques from information flow control and present a principled approach to constructing formal models of systems with metadata privacy for $\textit{some}$, deniable, traffic. We prove that deniable traffic achieves metadata privacy against strong adversaries -- this constitutes the first bridging of information flow control and anonymous communication to our knowledge. Additionally, we show that existing state-of-the-art protocols can be extended to support metadata privacy, by designing a novel protocol for $\textit{deniable instant messaging}$ (DenIM), which is a variant of the Signal protocol. To show the efficacy of our approach, we implement and evaluate a proof-of-concept instant messaging system running DenIM on top of unmodified Signal. We empirically show that the DenIM on Signal can maintain low-latency for unmodified Signal traffic without breaking existing features, while at the same time supporting deniable Signal traffic.
Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination

Large-scale pretrained language models have made significant advances in solving downstream language understanding tasks. However, they generally suffer from reporting bias, the phenomenon describing the lack of explicit commonsense knowledge in written text, e.g., ”an orange is orange”. To overcome this limitation, we develop a novel approach, Z-LaVI, to endow language models with visual imagination capabilities. Specifically, we leverage two complementary types of ”imaginations”: (i) recalling existing images through retrieval and (ii) synthesizing nonexistent images via text-to-image generation. Jointly exploiting the language inputs and the imagination, a pretrained vision-language model (e.g., CLIP) eventually composes a zero-shot solution to the original language tasks. Notably, fueling language models with imagination can effectively leverage visual knowledge to solve plain language tasks. In consequence, Z-LaVI consistently improves the zero-shot performance of existing language models across a diverse set of language tasks.
Spectrum-BERT: Pre-training of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors

Spectral detection technology, as a non-invasive method for rapid detection of substances, combined with deep learning algorithms, has been widely used in food detection. However, in real scenarios, acquiring and labeling spectral data is an extremely labor-intensive task, which makes it impossible to provide enough high-quality data for training efficient supervised deep learning models. To better leverage limited samples, we apply pre-training&fine-tuning paradigm to the field of spectral detection for the first time and propose a pre-training method of deep bidirectional transformers for spectral classification of Chinese liquors, abbreviated as Spectrum-BERT. Specifically, first, to retain the model's sensitivity to the characteristic peak position and local information of the spectral curve, we innovatively partition the curve into multiple blocks and obtain the embeddings of different blocks, as the feature input for the next calculation. Second, in the pre-training stage, we elaborately design two pre-training tasks, Next Curve Prediction (NCP) and Masked Curve Model (MCM), so that the model can effectively utilize unlabeled samples to capture the potential knowledge of spectral data, breaking the restrictions of the insufficient labeled samples, and improving the applicability and performance of the model in practical scenarios. Finally, we conduct a large number of experiments on the real liquor spectral dataset. In the comparative experiments, the proposed Spectrum-BERT significantly outperforms the baselines in multiple metrics and this advantage is more significant on the imbalanced dataset. Moreover, in the parameter sensitivity experiment, we also analyze the model performance under different parameter settings, to provide a reference for subsequent research.
Linear Systems Approach to Describing and Classifying Fano Resonances (Postprint)

Abstract : We have shown that asymmetric resonant line shapes derived elsewhere from rigorous electromagnetic calculations and from the two-oscillators model can also be obtained by treating the photonic resonance as a response of a first-order linear system. We have reformulated the line shape equation into a weighted sum of the Fano and Lorentzian line shapes and proposed a two-parameter classification scheme with one parameter being the asymmetry factor in the original Fano formula and the other one being the relative weight of the Fano and Lorentzian line shapes.
Federated and Meta learning over Non-Wireless and Wireless Networks: A Tutorial

—In recent years, various machine learning (ML) so- lutions have been developed to solve resource management, interference management, autonomy, and decision-making problems in non-wireless and wireless networks. Standard ML approaches require collecting data at a central server for training, which cannot preserve the data privacy of devices. To address this issue, federated learning (FL) is an effective method to allow edge devices to collaboratively train ML models without sharing local datasets for data privacy. Typically, FL focuses on learning a global model for a given task and all devices and hence cannot adapt the model to devices with different data distributions. In such cases, meta learning can be employed to adapt learning models to different data distributions using a few data samples. In this tutorial, we conduct a comprehensive review on FL, meta learning, and federated meta learning (FedMeta). Compared to other tutorial papers, our objective is to leverage how FL/meta-learning/FedMeta can be designed, optimized, and evolved over non-wireless and wireless networks. Furthermore, we analyze not only the relationship among these learning algorithms but also their advantages and disadvantages in real-world applications.
Full-disc Ca ii K observations—A window to past solar magnetism

Full-disc observations of the Sun in the Ca ii K line provide one of the longest collections of solar data. First such observations were made in 1892 and since then various sites around the world have carried out regular observations, with Kodaikanal, Meudon, Mt Wilson, and Coimbra being some of the most prominent ones. By now, Ca ii K observations from over 40 different sites allow an almost complete daily coverage of the last century. Ca ii K images provide direct information on plage and network regions on the Sun and, through their connection to solar surface magnetic field, offer an excellent opportunity to study solar magnetism over more than a century. This makes them also extremely important, among others, for solar irradiance reconstructions and studies of the solar influence on Earth’s climate. However, these data also suffer from numerous issues, which for a long time have hampered their analysis. Without properly addressing these issues, Ca ii K data cannot be used to their full potential. Here, we first provide an overview of the currently known Ca ii K data archives and sources of the inhomogeneities in the data, before discussing existing processing techniques, followed by a recap of the main results derived with such data so far.
Semi-Parametric Neural Image Synthesis

Novel architectures have recently improved generative image synthesis leading to excellent visual quality in various tasks. Much of this success is due to the scalability of these architectures and hence caused by a dramatic increase in model complexity and in the computational resources invested in training these models. Our work questions the underlying paradigm of compressing large training data into ever growing parametric representations. We rather present an orthogonal, semi-parametric approach. We complement comparably small diffusion or autoregressive models with a separate image database and a retrieval strategy. During training we retrieve a set of nearest neighbors from this external database for each training instance and condition the generative model on these informative samples. While the retrieval approach is providing the (local) content, the model is focusing on learning the composition of scenes based on this content. As demonstrated by our experiments, simply swapping the database for one with different contents transfers a trained model post-hoc to a novel domain. The evaluation shows competitive performance on tasks which the generative model has not been trained on, such as class-conditional synthesis, zero-shot stylization or text-to-image synthesis without requiring paired text-image data. With negligible memory and computational overhead for the external database and retrieval we can significantly reduce the parameter count of the generative model and still outperform the state-of-the-art.
A simple probabilistic neural network for machine understanding

We discuss the concept of probabilistic neural networks with a fixed internal representation being models for machine understanding. Here, ‘understanding’ is interpretted as the ability to map data to an already existing representation which encodes an a priori organisation of the feature space. We derive the internal representation by requiring that it satisfies the principles of maximal relevance and of maximal ignorance about how different features are combined. We show that, when hidden units are binary variables, these two principles identify a unique model—the hierarchical feature model—which is fully solvable and provides a natural interpretation in terms of features. We argue that learning machines with this architecture possess a number of interesting properties, such as the continuity of the representation with respect to changes in parameters and data, the possibility of controlling the level of compression and the ability to support functions that go beyond generalisation. We explore the behaviour of the model with extensive numerical experiments and argue that models in which the internal representation is fixed reproduce a learning modality which is qualitatively different from that of traditional models, such as restricted Boltzmann machines.
CMU-NeT: A Strong Convmixer-Based Medical Ultrasound Image Segmentation Network

U-Net and its extensions have achieved great success in medical image segmentation. However, due to the inherent local characteristics of ordinary convolution operations, U-Net encoder cannot effectively extract global context information. In addition, simple skip connections cannot capture salient features. In this work, we propose a fully convolutional segmentation network (CMU-Net) which incorporates hybrid convolutions and multi-scale attention gate. The ConvMixer module extracts global context information by mixing features at distant spatial locations. Moreover, the multi-scale attention gate emphasizes valuable features and achieves efficient skip connections. We evaluate the proposed method using both breast ultrasound datasets and a thyroid ultrasound image dataset; and CMU-Net achieves average Intersection over Union (IoU) values of 73.27% and 84.75%, and F1 scores of 84.16% and 91.71%. The code is available at https://github.com/FengheTan9/CMU-Net.
Box dimension of the graphs of the generalized Weierstrass-type functions

For a Lipschitz $\mathbb{Z}-$periodic function $\phi:\mathbb{R}\to \mathbb{R}^2$ satisfied that $\mathbb{R}^2\setminus\{\phi(x):x\in\mathbb{R}\}$ is not connected, an integer $b\ge 2$ and $\lambda\in (c/{b^{\frac12}},1)$, we prove the following for the generalized Weierstrass-type function $W(x)=\sum\limits_{n=0}^{\infty}{{\lambda}^n\phi(b^nx)}$: the box dimension of its graph is equal to $3+2\log_b\lambda$, where $c$ is a constant depending on $\phi$.
Enhancing Tabular Reasoning with Pattern Exploiting Training

Recent methods based on pre-trained language models have exhibited superior performance over tabular tasks (e.g., tabular NLI), despite showing inherent problems such as not using the right evidence and inconsistent predictions across inputs while reasoning over the tabular data (Gupta et al., 2021). In this work, we utilize Pattern-Exploiting Training (PET) (i.e., strategic MLM) on pre-trained language models to strengthen these tabular reasoning models’ pre-existing knowledge and reasoning abilities. Our upgraded model exhibits a superior understanding of knowledge facts and tabular reasoning compared to current baselines. Additionally, we demonstrate that such models are more effective for underlying downstream tasks of tabular inference on INFOTABS. Furthermore, we show our model’s robustness against adversarial sets generated through various character and word level perturbations.
A perturbation study of axisymmetric strongly magnetic degenerate stars : the case of super-Chandrasekhar white dwarfs

In the presence of a strong magnetic field a stellar equilibrium configuration, aided by the Lorentz force, can support a larger mass than a non-magnetic one. This has been considered a possible explanation of the super-Chandrasekhar mass white dwarfs giving rise to over-luminous Type-Ia supernovae. We present here linear and non-linear perturbation studies of such strongly magetised configurations and show that axisymmetric configurations with poloidal or toroidal fields are unstable. The numerical evolution of the perturbations shows instability after about an Alfven crossing time. This time scale is very short for the magnetically supported super-Chandrasekhar mass white dwarfs. Uniform rotation about the symmetry axis can reduce the growth rate but can not stabilize the super-massive configurations. It is concluded that long-lived super-Chandrasekhar mass white dwarfs supported by magnetic field are unlikely to occur in Nature.
The Curious Case of Absolute Position Embeddings

Transformer language models encode the notion of word order using positional information. Most commonly, this positional information is represented by absolute position embeddings (APEs), that are learned from the pretraining data. However, in natural language, it is not absolute position that matters, but relative position, and the extent to which APEs can capture this type of information has not been investigated. In this work, we observe that models trained with APE over-rely on positional information to the point that they break-down when subjected to sentences with shifted position information. Specifically, when models are subjected to sentences starting from a non-zero position (excluding the effect of priming), they exhibit noticeably degraded performance on zero to full-shot tasks, across a range of model families and model sizes. Our findings raise questions about the efficacy of APEs to model the relativity of position information, and invite further introspection on the sentence and word order processing strategies employed by these models.
Distance-to-Set Priors and Constrained Bayesian Inference

Constrained learning is prevalent in many statistical tasks. Recent work proposes distance-to-set penalties to derive estimators under general constraints that can be specified as sets, but focuses on obtaining point estimates that do not come with corresponding measures of uncertainty. To remedy this, we approach distance-to-set regularization from a Bayesian lens. We consider a class of smooth distance-to-set priors, showing that they yield well-defined posteriors toward quantifying uncertainty for constrained learning problems. We discuss relationships and advantages over prior work on Bayesian constraint relaxation. Moreover, we prove that our approach is optimal in an information geometric-sense for finite penalty parameters $\rho$, and enjoys favorable statistical properties when $\rho\to\infty$. The method is designed to perform effectively within gradient-based MCMC samplers, as illustrated on a suite of simulated and real data applications.
Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models

In the era of billion-parameter-sized Language Models (LMs), start-ups have to follow trends and adapt their technology accordingly. Nonetheless, there are open challenges since the development and deployment of large models comes with a need for high computational resources and has economical consequences. In this work, we follow the steps of the R{&D group of a modern legal-tech start-up and present important insights on model development and deployment. We start from ground zero by pre-training multiple domain-specific multi-lingual LMs which are a better fit to contractual and regulatory text compared to the available alternatives (XLM-R). We present benchmark results of such models in a half-public half-private legal benchmark comprising 5 downstream tasks showing the impact of larger model size. Lastly, we examine the impact of a full-scale pipeline for model compression which includes: a) Parameter Pruning, b) Knowledge Distillation, and c) Quantization: The resulting models are much more efficient without sacrificing performance at large.
Controlled Text Reduction

Producing a reduced version of a source text, as in generic or focused summarization, inherently involves two distinct subtasks: deciding on targeted content and generating a coherent text conveying it. While some popular approaches address summarization as a single end-to-end task, prominent works support decomposed modeling for individual subtasks. Further, semi-automated text reduction is also very appealing, where users may identify targeted content while models would generate a corresponding coherent summary.In this paper, we focus on the second subtask, of generating coherent text given pre-selected content. Concretely, we formalize Controlled Text Reduction as a standalone task, whose input is a source text with marked spans of targeted content (“highlighting”). A model then needs to generate a coherent text that includes all and only the target information.We advocate the potential of such models, both for modular fully-automatic summarization, as well as for semi-automated human-in-the-loop use cases.Facilitating proper research, we crowdsource high-quality dev and test datasets for the task. Further, we automatically generate a larger “silver” training dataset from available summarization benchmarks, leveraging a pretrained summary-source alignment model.Finally, employing these datasets, we present a supervised baseline model, showing promising results and insightful analyses.
The physics of multiphase gas flows: fragmentation of a radiatively cooling gas cloud in a hot wind

Galactic winds exhibit a multiphase structure that consists of hot-diffuse and cold-dense phases. Here we present high-resolution idealised simulations of the interaction of a hot supersonic wind with a cold cloud with the moving-mesh code arepo in setups with and without radiative cooling. We demonstrate that cooling causes clouds with sizes larger than the cooling length to fragment in two- and three-dimensional simulations (2D and 3D). We confirm earlier 2D simulations by McCourt et al. 2018 and highlight differences of the shattering processes of 3D clouds that are exposed to a hot wind. The fragmentation process is quantified with a friends-of-friends analysis of shattered cloudlets and density power spectra. Those show that radiative cooling causes the power spectral index to gradually increase when the initial cloud radius is larger than the cooling length and with increasing time until the cloud is fully dissolved in the hot wind. A resolution of around 1 pc is required to reveal the effect of cooling-induced fragmentation of a 100 pc outflowing cloud. Thus, state-of-the-art cosmological zoom simulations of the circumgalactic medium (CGM) fall short by orders of magnitudes from resolving this fragmentation process. This physics is, however, necessary to reliably model observed column densities and covering fractions of Lyman-$\alpha$ haloes, high-velocity clouds, and broad-line regions of active galactic nuclei.