Miracle-dz/UniFAR-FTUs huggingface dataset:
Neural Random Projection: From the Initial Task To the Input Similarity Problem

In this paper, we propose a novel approach for implicit data representation to evaluate similarity of input data using a trained neural network. In contrast to the previous approach, which uses gradients for representation, we utilize only the outputs from the last hidden layer of a neural network and do not use a backward step. The proposed technique explicitly takes into account the initial task and significantly reduces the size of the vector representation, as well as the computation time. The key point is minimization of information loss between layers. Generally, a neural network discards information that is not related to the problem, which makes the last hidden layer representation useless for input similarity task. In this work, we consider two main causes of information loss: correlation between neurons and insufficient size of the last hidden layer. To reduce the correlation between neurons we use orthogonal weight initialization for each layer and modify the loss function to ensure orthogonality of the weights during training. Moreover, we show that activation functions can potentially increase correlation. To solve this problem, we apply modified Batch-Normalization with Dropout. Using orthogonal weight matrices allow us to consider such neural networks as an application of the Random Projection method and get a lower bound estimate for the size of the last hidden layer. We perform experiments on MNIST and physical examination datasets. In both experiments, initially, we split a set of labels into two disjoint subsets to train a neural network for binary classification problem, and then use this model to measure similarity between input data and define hidden classes. Our experimental results show that the proposed approach achieves competitive results on the input similarity task while reducing both computation time and the size of the input representation.
Evidence for galaxy assembly bias in BOSS CMASS redshift-space galaxy correlation function

Building accurate and flexible galaxy-halo connection models is crucial in modeling galaxy clustering on non-linear scales. Recent studies have found that halo concentration by itself cannot capture the full galaxy assembly bias effect and that the local environment of the halo can be an excellent indicator of galaxy assembly bias. In this paper, we propose an extended halo occupation distribution model (HOD) that includes both a concentration-based assembly bias term and an environment-based assembly bias term. We use this model to achieve a good fit (chi^2/DoF = 1.35) on the 2D redshift-space 2-point correlation function (2PCF) of the Baryon Oscillation Spectroscopic Survey (BOSS) CMASS galaxy sample. We find that the inclusion of both assembly bias terms is strongly favored by the data and the standard 5-parameter HOD is strongly rejected. More interestingly, the redshift-space 2PCF drives the assembly bias parameters in a way that preferentially assigns galaxies to lower mass halos. This results in galaxy-galaxy lensing predictions that are within 1sigma agreement with the observation, alleviating the perceived tension between galaxy clustering and lensing. We also showcase a consistent 3-5sigma preference for a positive environment-based assembly bias that persists over variations in the fit. We speculate that the environmental dependence might be driven by underlying processes such as mergers and feedback, but might also be indicative of a larger halo boundaries such as the splashback radius. Regardless, this work highlights the importance of building flexible galaxy-halo connection models and demonstrates the extra constraining power of the redshift-space 2PCF.
Controllable Continuous Gaze Redirection

In this work, we present interpGaze, a novel framework for controllable gaze redirection that achieves both precise redirection and continuous interpolation. Given two gaze images with different attributes, our goal is to redirect the eye gaze of one person into any gaze direction depicted in the reference image or to generate continuous intermediate results. To accomplish this, we design a model including three cooperative components: an encoder, a controller and a decoder. The encoder maps images into a well-disentangled and hierarchically-organized latent space. The controller adjusts the magnitudes of latent vectors to the desired strength of corresponding attributes by altering a control vector. The decoder converts the desired representations from the attribute space to the image space. To facilitate covering the full space of gaze directions, we introduce a high-quality gaze image dataset with a large range of directions, which also benefits researchers in related areas. Extensive experimental validation and comparisons to several baseline methods show that the proposed interpGaze outperforms state-of-the-art methods in terms of image quality and redirection precision.
Eigenmode analysis of membrane stability in inviscid flow

We study the instability of a thin membrane (of zero bending rigidity) to out-of-plane deflections, when the membrane is immersed in an inviscid fluid flow and sheds a trailing vortex-sheet wake. We solve the nonlinear eigenvalue problem iteratively with large ensembles of initial guesses, for three canonical boundary conditions---both ends fixed, one end fixed and one free, and both free. Over several orders of magnitude of membrane mass density, we find instability by divergence or flutter (particularly at large mass density, or with one or both ends free). The most unstable eigenmodes generally become "wavier" at smaller mass density and smaller tension, but with regions of nonmonotonic behavior. We find good quantitative agreement with unsteady time-stepping simulations at small amplitude, but only qualitative similarities with the eventual steady-state large-amplitude motions.
On the relationship between mean observations, spatial averages and the Dyer-Roeder approximation in Einstein-Straus models

The redshift and redshift-distance relation in different Einstein-Straus models are considered. Specifically, the mean of these observables along 1000 light rays in different specific models are compared with predictions based on the Dyer-Roeder approximation and relations based on spatial averaging. It is shown that in certain limits, including those studied earlier in the literature, the Dyer-Roeder approximation and relations based on spatial averages agree with each other to a good precision regarding the redshift and redshift-distance relation and make good predictions of the mean of the exact relations. In limits where the two methods disagree, the Dyer-Roeder approximation clearly yields the better approximation of the true mean. This is explained by demonstrating the effect of boundary terms and integrated Sachs-Wolfe contributions but it is pointed out that the result seems to be valid for other Swiss-cheese models as well. An expression for the redshift drift in Einstein-Straus models is presented and used for studying the behavior of this quantity in particular Einstein-Straus models.
Recurrent babbling: evaluating the acquisition of grammar from limited input data

Recurrent Neural Networks (RNNs) have been shown to capture various aspects of syntax from raw linguistic input. In most previous experiments, however, learning happens over unrealistic corpora, which do not reflect the type and amount of data a child would be exposed to. This paper remedies this state of affairs by training an LSTM over a realistically sized subset of child-directed input. The behaviour of the network is analysed over time using a novel methodology which consists in quantifying the level of grammatical abstraction in the model’s generated output (its ‘babbling’), compared to the language it has been exposed to. We show that the LSTM indeed abstracts new structures as learning proceeds.
The impact of disturbed galaxy clusters on the kinematics of active galactic nuclei

We produce a kinematic analysis of AGN-hosting cluster galaxies from a sample 33 galaxy clusters selected using the X-ray Clusters Database (BAX) and populated with galaxies from the Sloan Digital Sky Survey (SDSS) Data Release 8 (DR8). The 33 galaxy clusters are delimited by their relative intensity of member galaxy substructuring as a proxy to core merging to derive two smaller sub-samples of 8 dynamically active (merging) and 25 dynamically relaxed (non-merging) states. The AGN were selected for each cluster sub-sample by employing the WHAN diagram to the strict criteria of log$_{10}$([NII]/H$\alpha$)$\geq-0.32$ and EW$_{\mathrm{H}\alpha}$ $\geq$ 6{\AA}, providing pools of 70 merging and 225 non-merging AGN sub-populations. By co-adding the clusters to their respective dynamical states to improve the signal-to-noise of our AGN sub-populations we find that merging galaxy clusters on average host kinematically active AGN between 0-1.5 $r_{200}$ as $r_{200}\rightarrow0$, where their velocity dispersion profile (VDP) presents a significant deviation from the non-AGN sub-population VDP by $\gtrsim3\sigma$. This result is indicative that the AGN-hosting cluster galaxies have recently coalesced onto a common potential. Further analysis of the composite distributions illustrate non-merging AGN-hosting sub-populations have, on average, already been accreted and predominantly lie within backsplash regions of the projected phase-space. This suggests merging cluster dynamical states hold relatively younger AGN sub-populations kinematically compared with those found in non-merging cluster dynamical states.
The next common and widespread bunting to go? Global population decline in the Rustic Bunting Emberiza rustica

Summary Populations of several long-distance migratory songbirds in Eurasia are in peril, drastically illustrated by the recent range-wide population collapse in the Yellow-breasted Bunting Emberiza aureola. There are signals of a strong decline also in the Rustic Bunting E. rustica, but no range-wide assessment of population trends in this superabundant and widespread bunting species has yet been undertaken. The conservation status of Rustic Bunting is ‘Least Concern’ on the global IUCN Red List, but it has recently been upgraded to ‘Vulnerable’ on the European Red List. To assess the Rustic Bunting’s global conservation status we compiled, for the first time, population data across its breeding and wintering ranges. The analysis reveals a 75–87% decline in overall population size over the last 30 years and a 32–91% decline over the last 10 years. The trend estimates indicate that the long-term (30-year) range-wide population decline in the Rustic Bunting is of similar magnitude to two well-known examples of declining species within the same genus, the Yellow-breasted Bunting and the Ortolan Bunting E. hortulana. The magnitude of the range-wide population decline over the last 10 years suggests that the Rustic Bunting could be upgraded from ‘Least Concern’ to ‘Vulnerable’ or ‘Endangered’ on the IUCN global Red List. Agricultural intensification in the wintering range and intensified levels of disturbance, including logging and fire, in the breeding range could be important drivers of the range-wide population decline, and persecution could also contribute. Untangling threat factors and their interactions on Rustic Bunting is necessary for conservation, but hampered by our currently limited understanding of the relationships between population dynamics and different threats.
Provable Fictitious Play for General Mean-Field Games

We propose a reinforcement learning algorithm for stationary mean-field games, where the goal is to learn a pair of mean-field state and stationary policy that constitutes the Nash equilibrium. When viewing the mean-field state and the policy as two players, we propose a fictitious play algorithm which alternatively updates the mean-field state and the policy via gradient-descent and proximal policy optimization, respectively. Our algorithm is in stark contrast with previous literature which solves each single-agent reinforcement learning problem induced by the iterates mean-field states to the optimum. Furthermore, we prove that our fictitious play algorithm converges to the Nash equilibrium at a sublinear rate. To the best of our knowledge, this seems the first provably convergent single-loop reinforcement learning algorithm for mean-field games based on iterative updates of both mean-field state and policy.
What Have We Achieved on Text Summarization?

Deep learning has led to significant improvement in text summarization with various methods investigated and improved ROUGE scores reported over the years. However, gaps still exist between summaries produced by automatic summarizers and human professionals. Aiming to gain more understanding of summarization systems with respect to their strengths and limits on a fine-grained syntactic and semantic level, we consult the Multidimensional Quality Metric (MQM) and quantify 8 major sources of errors on 10 representative summarization models manually. Primarily, we find that 1) under similar settings, extractive summarizers are in general better than their abstractive counterparts thanks to strength in faithfulness and factual-consistency; 2) milestone techniques such as copy, coverage and hybrid extractive/abstractive methods do bring specific improvements but also demonstrate limitations; 3) pre-training techniques, and in particular sequence-to-sequence pre-training, are highly effective for improving text summarization, with BART giving the best results.
Augmenting physical models with deep networks for complex dynamics forecasting

Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling-based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framework, a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists of decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model. The learning problem is carefully formulated such that the physical model explains as much of the data as possible, while the data-driven component only describes information that cannot be captured by the physical model; no more, no less. This not only provides the existence and uniqueness for this decomposition, but also ensures interpretability and benefit generalization. Experiments made on three important use cases, each representative of a different family of phenomena, i.e. reaction–diffusion equations, wave equations and the non-linear damped pendulum, show that APHYNITY can efficiently leverage approximate physical models to accurately forecast the evolution of the system and correctly identify relevant physical parameters. The code is available at https://github.com/yuan-yin/APHYNITY.
High-order Semantic Role Labeling

Semantic role labeling is primarily used to identify predicates, arguments, and their semantic relationships. Due to the limitations of modeling methods and the conditions of pre-identified predicates, previous work has focused on the relationships between predicates and arguments and the correlations between arguments at most, while the correlations between predicates have been neglected for a long time. High-order features and structure learning were very common in modeling such correlations before the neural network era. In this paper, we introduce a high-order graph structure for the neural semantic role labeling model, which enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models and further boost our baseline to achieve new state-of-the-art results.
Query-Key Normalization for Transformers

Low-resource language translation is a challenging but socially valuable NLP task. Building on recent work adapting the Transformer’s normalization to this setting, we propose QKNorm, a normalization technique that modifies the attention mechanism to make the softmax function less prone to arbitrary saturation without sacrificing expressivity. Specifically, we apply l2-normalization along the head dimension of each query and key matrix prior to multiplying them and then scale up by a learnable parameter instead of dividing by the square root of the embedding dimension. We show improvements averaging 0.928 BLEU over state-of-the-art bilingual benchmarks for 5 low-resource translation pairs from the TED Talks corpus and IWSLT’15.
Metal-poor Stars Observed with the Southern African Large Telescope

We present the first release of a large-scale study of relatively bright (V < 13.5) metal-poor stars observed with the Southern African Large Telescope (SALT), based on high-resolution spectra of 50 stars with a resolving power of R ∼ 40,000 and S/N ∼ 20 per pixel at 4300 Å. The elemental abundances of C, Sr, Ba, and Eu are reported, as well as several α-elements (Mg, Ca, Sc, Ti, and V) and iron-peak elements (Mn, Co, Ni, and Zn). We find a diverse array of abundance patterns, including several consistent with the signatures of carbon-enhanced metal-poor CEMP-i and CEMP-r stars. We find that 15 of 50 (30%) are carbon enhanced (with [C/Fe] > +0.70), and that a large fraction (26 of 50, 52%) are enhanced in r-process elements. Among the r-process-enhanced stars, five are strongly enhanced r-II ([Eu/Fe] > +1.0) stars (two of which are newly discovered) and 21 are newly discovered moderately enhanced r-I (+0.3 ≤ [Eu/Fe] ≤ +1.0) stars. There are eight stars in our sample that, on the basis of their abundances and kinematics, are possible members of the metal-weak thick-disk population. We also compare our measured abundances to progenitor-enrichment models, and find that the abundance patterns for the majority of our stars can be attributed to a single (rather than multiple) enrichment event.
Epitaxy of new layered materials: 2D chalcogenides and challenges of weak van der Waals interactions.

The application of new materials in nanotechnology opens new perspectives and enables ground-breaking innovations. Two-dimensional van der Waals materials and more specific, 2D chalcogenides are a promising class of new materials awaiting their usage in the semiconductor industry. However, the integration of van der Waals materials relying on industry-compatible manufacturing processes is still a major challenge. This is currently restricting the application of these new materials to the research laboratories environment only. The large-area and single-crystalline growth of van der Waals materials is one of the most important requirements to meet the challenging demands implied by the semiconductor industry. This review contributes to a more generalized understanding on the integration of van der Waals materials - and in more specific 2D chalcogenides - through the growth process of epitaxy. This, can pursue further the aspiration of large-area, single-crystalline and defect-free epitaxial integration of (quasi) van der Waals homo- and heterostructures into the great world of the semiconductor industry.
Contrastive Learning with Hard Negative Samples

How can you sample good negative examples for contrastive learning? We argue that, as with metric learning, contrastive learning of representations benefits from hard negative samples (i.e., points that are difficult to distinguish from an anchor point). The key challenge toward using hard negatives is that contrastive methods must remain unsupervised, making it infeasible to adopt existing negative sampling strategies that use true similarity information. In response, we develop a new family of unsupervised sampling methods for selecting hard negative samples where the user can control the hardness. A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible. The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.
On the Crowdsourcing of Behaviors for Autonomous Agents

This letter is concerned with the problem of designing, from data, agents that are able to craft their behavior from a set of contributors in order to fulfill some agent-specific task. This is not necessarily known to the contributors. After formalizing this crowdsourcing process as a control problem, we present a result to synthesize behaviors from the information made available by the contributors. The result is turned into an algorithm and the theoretical findings are complemented via an example.
SOLUTION OF THREE-DIMENSIONAL CONVECTION-DIFFUSION PROBLEMS USING TETRAHEDRAL ELEMENTS AND FLOW-ORIENTED UPWIND INTERPOLATION FUNCTIONS

A numerical method for three-dimensional elliptic convection-diffusion problems is presented. The formulation is based on four-node tetrahedral elements and flow-oriented interpolation functions that respond appropriately to an element Peclet number and the direction of the element-average velocity vector. A control-Volume approach is used to derive the discretization equations. The method has been applied successfully to several test problems.
Inferring Properties from Categories versus Inferring Categories from Properties: The Case of Gender.

GELMAN, SUSAN A. ; COLLMAN, PAMELA; and MACCOBY, ELEANOR E. Inferring Properties from Categories versus Inferring Categories from Properties: The Case of Gender. CHILD DEVELOPMENT, 1986, 57, 396-404. The present study tested a distinction between inferring new categories on the basis of property information (predicted to be difficult) and inferring new properties on the basis of category information (predicted to be easier). One group of preschoolers learned new properties for specific boys and girls and was asked to say which property a new child would have, given a gender label that conflicted with the child's appearance. Other children saw the identical stimuli but were to classify them as "boy" or "girl" when given a sex-linked property that conflicted with appearance. All children were also tested on gender constancy. As predicted, children performed poorly on gender constancy and the classification task but accurately inferred many sex-linked properties on the basis of category membership, ignoring conflicting perceptual information. Control conditions support the claim that these effects are not due to differential memory demands in the different conditions. Future research should distinguish between using a category as the basis for making property inferences and the developmentally later skill of classifying an object by using property information. Preschoolers can ignore conflicting perceptual information much more easily on the former than on the latter task.