Miracle-dz/UniFAR-FTUs huggingface dataset:
The physical drivers of gas turbulence in simulated disc galaxies

We use the EAGLE cosmological simulations to study the evolution of the vertical velocity dispersion of cold gas, $\sigma_{z}$, in central disc galaxies and its connection to stellar feedback, gravitational instabilities, cosmological gas accretion and galaxy mergers. To isolate the impact of feedback, we analyse runs that turn off stellar and (or) AGN feedback in addition to a run that includes both. The evolution of $\sigma_z$ and its dependence on stellar mass and star formation rate in EAGLE are in good agreement with observations. Galaxies hosted by haloes of similar virial mass, $\rm M_{200}$, have similar $\sigma_z$ values even in runs where feedback is absent. The prevalence of local instabilities in discs is uncorrelated with $\sigma_z$ at low redshift and becomes only weakly correlated at high redshifts and in galaxies hosted by massive haloes. $\sigma_z$ correlates most strongly with the specific gas accretion rate onto the disc as well as with the degree of misalignment between the inflowing gas and the disc's rotation axis. These correlations are significant across all redshifts and halo masses, with misaligned accretion being the primary driver of high gas turbulence at redshifts $z \lesssim 1$ and for halo masses $\rm M_{200} \lesssim 10^{11.5} M_{\odot}$. Galaxy mergers increase $\sigma_z$, but because they are rare in our sample, they play only a minor role in its evolution. Our results suggest that the turbulence of cold gas in EAGLE discs results from a complex interplay of different physical processes whose relative importance depends on halo mass and redshift.
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models

In Multi-Task Learning (MTL), tasks may compete and limit the performance achieved on each other, rather than guiding the optimization to a solution, superior to all its single-task trained counterparts. Since there is often not a unique solution optimal for all tasks, practitioners have to balance tradeoffs between tasks' performance, and resort to optimality in the Pareto sense. Most MTL methodologies either completely neglect this aspect, and instead of aiming at learning a Pareto Front, produce one solution predefined by their optimization schemes, or produce diverse but discrete solutions. Recent approaches parameterize the Pareto Front via neural networks, leading to complex mappings from tradeoff to objective space. In this paper, we conjecture that the Pareto Front admits a linear parameterization in parameter space, which leads us to propose \textit{Pareto Manifold Learning}, an ensembling method in weight space. Our approach produces a continuous Pareto Front in a single training run, that allows to modulate the performance on each task during inference. Experiments on multi-task learning benchmarks, ranging from image classification to tabular datasets and scene understanding, show that \textit{Pareto Manifold Learning} outperforms state-of-the-art single-point algorithms, while learning a better Pareto parameterization than multi-point baselines.
Understanding CNN Fragility When Learning With Imbalanced Data

Convolutional neural networks (CNNs) have achieved impressive results on imbalanced image data, but they still have difficulty generalizing to minority classes and their decisions are difficult to interpret. These problems are related because the method by which CNNs generalize to minority classes, which requires improvement, is wrapped in a black-box. To demystify CNN decisions on imbalanced data, we focus on their latent features. Although CNNs embed the pattern knowledge learned from a training set in model parameters, the effect of this knowledge is contained in feature and classification embeddings (FE and CE). These embeddings can be extracted from a trained model and their global, class properties (e.g., frequency, magnitude and identity) can be analyzed. We find that important information regarding the ability of a neural network to generalize to minority classes resides in the class top-K CE and FE. We show that a CNN learns a limited number of class top-K CE per category, and that their magnitudes vary based on whether the same class is balanced or imbalanced. We hypothesize that latent class diversity is as important as the number of class examples, which has important implications for re-sampling and cost-sensitive methods. These methods generally focus on rebalancing model weights, class numbers and margins; instead of diversifying class latent features. We also demonstrate that a CNN has difficulty generalizing to test data if the magnitude of its top-K latent features do not match the training set. We use three popular image datasets and two cost-sensitive algorithms commonly employed in imbalanced learning for our experiments.
Towards Efficient and Effective Self-Supervised Learning of Visual Representations

Self-supervision has emerged as a propitious method for visual representation learning after the recent paradigm shift from handcrafted pretext tasks to instance-similarity based approaches. Most state-of-the-art methods enforce similarity between various augmentations of a given image, while some methods additionally use contrastive approaches to explicitly ensure diverse representations. While these approaches have indeed shown promising direction, they require a significantly larger number of training iterations when compared to the supervised counterparts. In this work, we explore reasons for the slow convergence of these methods, and further propose to strengthen them using well-posed auxiliary tasks that converge significantly faster, and are also useful for representation learning. The proposed method utilizes the task of rotation prediction to improve the efficiency of existing state-of-the-art methods. We demonstrate significant gains in performance using the proposed method on multiple datasets, specifically for lower training epochs.
Deep Scattering Spectrum Germaneness for Fault Detection and Diagnosis for Component-Level Prognostics and Health Management (PHM)

Most methodologies for fault detection and diagnosis in prognostics and health management (PHM) systems use machine learning (ML) or deep learning (DL), in which either some features are extracted beforehand (in the case of typical ML approaches) or the filters are used to extract features autonomously (in the case of DL) to perform the critical classification task. In particular, in the fault detection and diagnosis of industrial robots where the primary sources of information are electric current, vibration, or acoustic emissions signals that are rich in information in both the temporal and frequency domains, techniques capable of extracting meaningful information from non-stationary frequency-domain signals with the ability to map the signals into their constituent components with compressed information are required. This has the potential to minimise the complexity and size of traditional ML- and DL-based frameworks. The deep scattering spectrum (DSS) is one of the approaches that use the Wavelet Transform (WT) analogy for separating and extracting information embedded in a signal’s various temporal and frequency domains. Therefore, the primary focus of this work is the investigation of the efficacy and applicability of the DSS’s feature domain relative to fault detection and diagnosis for the mechanical components of industrial robots. For this, multiple industrial robots with distinct mechanical faults were studied. Data were collected from these robots under different fault conditions and an approach was developed for classifying the faults using DSS’s low-variance features extracted from input signals. The presented approach was implemented on the practical test benches and demonstrated satisfactory performance in fault detection and diagnosis for simple and complex classification problems with a classification accuracy of 99.7% and 88.1%, respectively. The results suggest that, similarly to other ML techniques, the DSS offers significant potential in addressing fault classification challenges, especially for cases where the data are in the form of signals.
FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences

In the recent paper FLECS (Agafonov et al, FLECS: A Federated Learning Second-Order Framework via Compression and Sketching), the second-order framework FLECS was proposed for the Federated Learning problem. This method utilize compression of sketched Hessians to make communication costs low. However, the main bottleneck of FLECS is gradient communication without compression. In this paper, we propose the modiﬁcation of FLECS with compressed gradient differences, which we call FLECS-CGD (FLECS with C ompressed G radient D ifferences) and make it applicable for stochastic optimization. Convergence guarantees are provided in strongly convex and nonconvex cases. Experiments show the practical beneﬁt of proposed approach.
Searching for Compact Objects in Binaries with Gaia DR3

We search for compact objects in binaries based on Gaia DR3. A sample of 10 targets is derived under the following conditions: the radial velocity variable, low temperature (T eff < 6000 K), high mass function (f(M 2) > 1M ⊙), and ellipsoidal-like light curves. Two targets have LAMOST spectroscopic observations, one of which is a double-lined spectroscopic binary. The observational data of seven targets are not self-consistent, since their photometric periods are even shorter than the theoretical minimum orbital periods calculated by the stellar parameters from Gaia DR3. After excluding these seven inconsistent targets and another target contaminated by a near-bright star, the remaining two targets may contain compact objects worth follow-up observations. This work may serve as an example to demonstrate the feasibility of searching for compact objects in the massive Gaia data.
Domain Specific Sub-network for Multi-Domain Neural Machine Translation

This paper presents Domain-Specific Sub-network (DoSS). It uses a set of masks obtained through pruning to define a sub-network for each domain and finetunes the sub-network parameters on domain data. This performs very closely and drastically reduces the number of parameters compared to finetuning the whole network on each domain. Also a method to make masks unique per domain is proposed and shown to greatly improve the generalization to unseen domains. In our experiments on German to English machine translation the proposed method outperforms the strong baseline of continue training on multi-domain (medical, tech and religion) data by 1.47 BLEU points. Also continue training DoSS on new domain (legal) outperforms the multi-domain (medical, tech, religion, legal) baseline by 1.52 BLEU points.
Global existence and large time behavior for primitive equations with free boundary

In the present paper, the primitive equations, which can be used to simulate the large scale motion of ocean and atmosphere, are considered in the three-dimensional domain bounded below by a fixed solid boundary and above by a free moving boundary. The global existence and uniqueness of strong solutions are established and the long time convergence to the equilibrium state is showed either at exponential rate for horizontal periodic domain or at algebraic rate for horizontal whole space.
Universal hidden monotonic trend estimation with contrastive learning

In this paper, we describe a universal method for extracting the underlying monotonic trend factor from time series data. We propose an approach related to the Mann-Kendall test, a standard monotonic trend detection method and call it contrastive trend estimation (CTE). We show that the CTE method identifies any hidden trend underlying temporal data while avoiding the standard assumptions used for monotonic trend identification. In particular, CTE can take any type of temporal data (vector, images, graphs, time series, etc.) as input. We finally illustrate the interest of our CTE method through several experiments on different types of data and problems.
A crack in the track of the Hubble Constant

Measuring the rate at which the universe expands at a given time -- the 'Hubble constant' -- has been a topic of controversy since the first measure of its expansion by Edwin Hubble in the 1920's. As early as the 1970's, Sandage et de Vaucouleurs have been arguing about the adequate methodology for such a measurement. Should astronomers focus only on their best indicators, e.g., the Cepheids, and improve the precision of this measurement based on a unique object to the best possible? Or should they 'spread the risks', i.e., multiply the indicators and methodologies before averaging over their results? Is a robust agreement across several uncertain measures, as is currently argued to defend the existence of a 'Hubble crisis' more telling than a single one percent precision measurement? This controversy, I argue, stems from a misconception of what managing the uncertainties associated with such experimental measurements require. Astrophysical measurements, such as the measure of the Hubble constant, require a methodology that permits both to reduce the known uncertainties and to track the unknown unknowns. Based on the lessons drawn from the so-called Hubble crisis, I sketch a methodological guide for identifying, quantifying and reducing uncertainties in astrophysical measurements, hoping that such a guide can not only help to re-frame the current Hubble tension, but serve as a starting point for future fruitful discussions between astrophysicists, astronomers and philosophers.
IF-GAN: A Novel Generator Architecture with Information Feedback

This paper presents an alternative generator architecture for image generation, having a novel information feedback system. Contrary to conventional methods in which the latent space unilaterally aﬀectsthefeaturespaceinthegenerator,theproposedmethodtrainsnotonlythefeaturespacebutalsothelatentonebyinterchangingtheirinformation.Tothisend,weintroduceanovelmodule,called informationfeedback(IF)block,whichjointlyupdatesthelatentandfeaturespaces.Toshowthesuperiorityoftheproposedmethod,wepresentextensiveexperimentsonvariousdatasetsincluding subsetsofLSUNandFFHQ.Experimentalresultsrevealthattheproposedmethodcandramaticallyimprovetheimagegenerationperformance,intermsofFrechetinceptiondistance(FID),kernel inceptiondistance(KID),andPrecisionandRecall(P&R).
Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder

Discourse Parsing is a key NLP task for processing multi-sentential natural language. Most research in the area thereby focuses on one of the two main discourse theories – RST (Mann and Thompson, 1988) or PDTB (Prasad et al., 2008). While these discourse theories are of great value for the field of discourse parsing and have been guiding its progress ever since, there are two major problems when data is annotated according to those theories: (1) Due to the inherently ambiguous nature of natural language, the inter-annotator agreement when following the annotation guidelines is relatively low (Carlson et al., 2003) and (2) The annotation process itself is expensive and tedious, limiting the size of available gold-standard datasets. With a growing need for robust and general discourse structures in many downstream tasks and real-world applications (e.g. Gerani et al. (2014); Nejat et al. (2017); Ji and Smith (2017)), the current lack of high-quality, high-quantity discourse trees poses a severe shortcoming. Fortunately, there are more data-driven alternatives to infer discourse structures. For example, the recently proposed MEGA-DT treebank (Huber and Carenini, 2019), whose discourse structures and nuclearity attributes are automatically inferred from large-scale sentiment datasets, reaching state-of-the-art performance on the inter-domain discourse parsing task. Similarly, another approach by Liu et al. (2019) infers latent discourse trees from the downstream summarization task using a transformer model. Outside the area of discourse parsing, syntactic trees have previously been inferred according to several diverse strategies, e.g. Socher et al. (2011); Yogatama et al. (2016); Choi et al. (2018); Maillard et al. (2019). In general, the approaches mentioned above have been shown to capture valuable information. Some models outperform baselines trained on human datasets (see Huber and Carenini (2019)), others are proven to enhance diverse downstream tasks (Liu et al., 2019; Choi et al., 2018). However, despite these initial successes, one limitation that all aforementioned models share is the taskspecificity, oftentimes only capturing downstream task-related components and potentially compromising the generality of the resulting trees, as for instance shown for the model using summarization data (Liu et al., 2019) in Ferracane et al. (2019). In order the alleviate the limitation described above, we propose a new strategy to generate tree structures in a task-agnostic, unsupervised fashion by extending a latent tree induction framework (Choi et al., 2018) with an auto-encoding objective. The proposed approach can be applied to any tree-structured objective, such as syntactic parsing, discourse parsing and others. However, due to the especially difficult annotation process to generate discoursetrees, we initially intend to develop such method to complement task-specific models in generating much larger and more diverse discourse treebanks.
Clustering Categorical Data: Soft Rounding k-modes

Over the last three decades, researchers have intensively explored various clustering tools for categorical data analysis. Despite the proposal of various clustering algorithms, the classical k-modes algorithm remains a popular choice for unsupervised learning of categorical data. Surprisingly, our first insight is that in a natural generative block model, the k-modes algorithm performs poorly for a large range of parameters. We remedy this issue by proposing a soft rounding variant of the k-modes algorithm (SoftModes) and theoretically prove that our variant addresses the drawbacks of the k-modes algorithm in the generative model. Finally, we empirically verify that SoftModes performs well on both synthetic and real-world datasets.
Stability analysis of planetary systems via second-order R\'enyi entropy

The long-term dynamical evolution is a crucial point in recent planetary research. Although the amount of observational data is continuously growing and the precision allows us to obtain accurate planetary orbits, the canonical stability analysis still requires N-body simulations and phase space trajectory investigations. We propose a method for stability analysis of planetary motion based on the generalized Rényi entropy obtained from a scalar measurement. The radial velocity data of the central body in the gravitational three-body problem is used as the basis of a phase space reconstruction procedure. Then, Poincaré’s recurrence theorem contributes to ﬁnding a natural partitioning in the reconstructed phase space to obtain the Rényi entropy. It turns out that the entropy-based stability analysis is in good agreement with other chaos detection methods, and it requires only a few tens of thousands of orbital period integration time.
Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation

We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. It also helps reduce the surface variations across different expressions and languages. Unlike most prior work that only evaluates the ability to measure semantic similarity, we present a thorough evaluation of existing multilingual sentence embeddings and our improved versions, which include a collection of five transfer tasks in different downstream applications. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic textual similarity and transfer tasks.
A novel statistical methodology for quantifying the spatial arrangements of axons in peripheral nerves

A thorough understanding of the neuroanatomy of peripheral nerves is required for a better insight into their function and the development of neuromodulation tools and strategies. In biophysical modeling, it is commonly assumed that the complex spatial arrangement of myelinated and unmyelinated axons in peripheral nerves is random, however, in reality the axonal organization is inhomogeneous and anisotropic. Present quantitative neuroanatomy methods analyze peripheral nerves in terms of the number of axons and the morphometric characteristics of the axons, such as area and diameter. In this study, we employed spatial statistics and point process models to describe the spatial arrangement of axons and Sinkhorn distances to compute the similarities between these arrangements (in terms of first- and second-order statistics) in various vagus and pelvic nerve cross-sections. We utilized high-resolution transmission electron microscopy (TEM) images that have been segmented using a custom-built high-throughput deep learning system based on a highly modified U-Net architecture. Our findings show a novel and innovative approach to quantifying similarities between spatial point patterns using metrics derived from the solution to the optimal transport problem. We also present a generalizable pipeline for quantitative analysis of peripheral nerve architecture. Our data demonstrate differences between male- and female-originating samples and similarities between the pelvic and abdominal vagus nerves.
Dark matter from Monogem

As a supernova shock expands into space, it may collide with dark matter particles, scattering them up to velocities more than an order of magnitude larger than typical dark matter velocities in the Milky Way. If a supernova remnant is close enough to Earth, and the appropriate age, this flux of high-velocity dark matter could be detectable in direct detection experiments, particularly if the dark matter interacts via a velocity-dependent operator. This could make it easier to detect light dark matter that would otherwise have too little energy to be detected. We show that the Monogem Ring supernova remnant is both close enough and the correct age to produce such a flux, and thus we produce novel direct detection constraints and sensitivities for future experiments.