Miracle-dz/UniFAR-FTUs huggingface dataset:
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders

While pretrained encoders have achieved success in various natural language understanding (NLU) tasks, there is a gap between these pretrained encoders and natural language generation (NLG). NLG tasks are often based on the encoder-decoder framework, where the pretrained encoders can only benefit part of it. To reduce this gap, we introduce DeltaLM, a pretrained multilingual encoder-decoder model that regards the decoder as the task layer of off-the-shelf pretrained encoders. Specifically, we augment the pretrained multilingual encoder with a decoder and pre-train it in a self-supervised way. To take advantage of both the large-scale monolingual data and bilingual data, we adopt the span corruption and translation span corruption as the pre-training tasks. Experiments show that DeltaLM outperforms various strong baselines on both natural language generation and translation tasks, including machine translation, abstractive text summarization, data-to-text, and question generation. The code and pretrained models are available at \url{https://aka.ms/deltalm}.
byteSteady: Fast Classification Using Byte-Level n-Gram Embeddings

This article introduces byteSteady -- a fast model for classification using byte-level n-gram embeddings. byteSteady assumes that each input comes as a sequence of bytes. A representation vector is produced using the averaged embedding vectors of byte-level n-grams, with a pre-defined set of n. The hashing trick is used to reduce the number of embedding vectors. This input representation vector is then fed into a linear classifier. A straightforward application of byteSteady is text classification. We also apply byteSteady to one type of non-language data -- DNA sequences for gene classification. For both problems we achieved competitive classification results against strong baselines, suggesting that byteSteady can be applied to both language and non-language data. Furthermore, we find that simple compression using Huffman coding does not significantly impact the results, which offers an accuracy-speed trade-off previously unexplored in machine learning.
Influence of Enrichment of Wheat Bread with Pomegranate ( Punica granatum L) Peels by-Products

The influence of the enrichment of wheat bread with pomegranate peels by-products at 5%. 7.5% and 10% levels were investigated. The results show that pomegranate peel contained contain 15.91%, 7.56%, 18.1%, 15.67% and 42.76% protein, lipids, fiber, ash and carbohydrates, respectively. The rheological properties of the dough was tested and revealed that the extensibility decreased from 250 mm in wheat flour to 210, 196 and 170 mm in wheat flour containing 5%, 7,5% and 10% pomegranate peels flour, respectively. While increasing the addition of pomegranate flour caused a decrease in dough stability to 196 and 170 mm, respectively. The addition of pomegranate peels to wheat flour increase the water absorption of the dough from 55.5 to 70.4, 79.2 and 88.1% 5%, 7.5% and 10% of pomegranate peels, respectively. The addition of pomegranate peels also caused an increase in arrived time and a decrease in dough stability. Loaf bread weight increased with increasing pomegranate peels concentration, in contrast, loaf volume and specific volume decreased by increasing pomegranate peels. Enrichment of bread with various levels of pomegranate peels significantly (P < 0.05) decreased acceptance of its organoleptic properties by human panelists. Generally, fiber as a food industry by-product is recommended to be used as food additives to gain nutritional and healthy benefit.
NP-DRAW: A Non-Parametric Structured Latent Variable Modelfor Image Generation

In this paper, we present a non-parametric structured latent variable model for image generation, called NP-DRAW, which sequentially draws on a latent canvas in a part-by-part fashion and then decodes the image from the canvas. Our key contributions are as follows. 1) We propose a non-parametric prior distribution over the appearance of image parts so that the latent variable ``what-to-draw'' per step becomes a categorical random variable. This improves the expressiveness and greatly eases the learning compared to Gaussians used in the literature. 2) We model the sequential dependency structure of parts via a Transformer, which is more powerful and easier to train compared to RNNs used in the literature. 3) We propose an effective heuristic parsing algorithm to pre-train the prior. Experiments on MNIST, Omniglot, CIFAR-10, and CelebA show that our method significantly outperforms previous structured image models like DRAW and AIR and is competitive to other generic generative models. Moreover, we show that our model's inherent compositionality and interpretability bring significant benefits in the low-data learning regime and latent space editing. Code is available at https://github.com/ZENGXH/NPDRAW.
Task-Driven Out-of-Distribution Detection with Statistical Guarantees for Robot Learning

: Our goal is to perform out-of-distribution (OOD) detection , i.e., to detect when a robot is operating in environments that are drawn from a different distribution than the environments used to train the robot. We leverage Probably Approximately Correct (PAC)-Bayes theory in order to train a policy with a guaranteed bound on performance on the training distribution. Our key idea for OOD detection then relies on the following intuition: violation of the performance bound on test environments provides evidence that the robot is operating OOD. We formalize this via statistical techniques based on p-values and concentration inequalities. The resulting approach (i) provides guaranteed conﬁdence bounds on OOD detection, and (ii) is task-driven and sensitive only to changes that impact the robot’s performance. We demonstrate our approach on a simulated example of grasping objects with unfamiliar poses or shapes. We also present both simulation and hardware experiments for a drone performing vision-based obstacle avoidance in unfamiliar environments (including wind disturbances and different obstacle densities). Our examples demonstrate that we can perform task-driven OOD detection within just a handful of trials. Comparisons with baselines also demonstrate the advantages of our approach in terms of providing statistical guarantees and being insensitive to task-irrelevant distribution shifts.
Subgraph Federated Learning with Missing Neighbor Generation

Graphs have been widely used in data mining and machine learning due to their unique representation of real-world objects and their interactions. As graphs are getting bigger and bigger nowadays, it is common to see their subgraphs separately collected and stored in multiple local systems. Therefore, it is natural to consider the subgraph federated learning setting, where each local system holds a small subgraph that may be biased from the distribution of the whole graph. Hence, the subgraph federated learning aims to collaboratively train a powerful and generalizable graph mining model without directly sharing their graph data. In this work, towards the novel yet realistic setting of subgraph federated learning, we propose two major techniques: (1) FedSage, which trains a GraphSage model based on FedAvg to integrate node features, link structures, and task labels on multiple local subgraphs; (2) FedSage+, which trains a missing neighbor generator along FedSage to deal with missing links across local subgraphs. Empirical results on four real-world graph datasets with synthesized subgraph federated learning settings demonstrate the effectiveness and efficiency of our proposed techniques. At the same time, consistent theoretical implications are made towards their generalization ability on the global graphs.
The Hot Circumgalactic Medium of the Milky Way: Evidence for Supervirial, Virial, and Subvirial Temperatures; Nonsolar Chemical Composition; and Nonthermal Line Broadening

For the first time, we present the simultaneous detection and characterization of three distinct phases at >105 K in z = 0 absorption using deep Chandra observations toward Mrk 421. The extraordinarily high signal-to-noise ratio (≥60) of the spectra has allowed us to detect a hot phase of the Milky Way circumgalactic medium (CGM) at 3.2−0.5+1.5× 107 K coexisting with a warm-hot phase at 1.5 ± 0.1 × 106 K and a warm phase at 3.0 ± 0.4 × 105 K. The warm-hot phase is at the virial temperature of the Galaxy, and the warm phase may have cooled from the warm-hot phase, but the supervirial hot phase remains a mystery. We find that [C/O] in the warm and warm-hot phases, [Mg/O] in the warm-hot phase, and [Ne/O] in the hot phase are supersolar, and the hot and the warm-hot phases are α-enhanced. Nonthermal line broadening is evident in the warm-hot and the hot phases, and it dominates the total line broadening. Our results indicate that the >105 K CGM is a complex ecosystem. It provides insights on the thermal and chemical history of the Milky Way CGM and theories of galaxy evolution.
Promises and Pitfalls of Black-Box Concept Learning Models

Machine learning models that incorporate concept learning as an intermediate step in their decision making process can match the performance of black-box predictive models while retaining the ability to explain outcomes in human understandable terms. However, we demonstrate that the concept representations learned by these models encode information beyond the pre-defined concepts, and that natural mitigation strategies do not fully work, rendering the interpretation of the downstream prediction misleading. We describe the mechanism underlying the information leakage and suggest recourse for mitigating its effects.
Efficient algorithms for computing rank‐revealing factorizations on a GPU

Standard rank‐revealing factorizations such as the singular value decomposition (SVD) and column pivoted QR factorization are challenging to implement efficiently on a GPU. A major difficulty in this regard is the inability of standard algorithms to cast most operations in terms of the Level‐3 BLAS. This article presents two alternative algorithms for computing a rank‐revealing factorization of the form A=UTV∗$$ \mathbf{\mathsf{A}}=\mathbf{\mathsf{UT}}{\mathbf{\mathsf{V}}}^{\ast } $$ , where U$$ \mathbf{\mathsf{U}} $$ and V$$ \mathbf{\mathsf{V}} $$ are orthogonal and T$$ \mathbf{\mathsf{T}} $$ is trapezoidal (or triangular if A$$ \mathbf{\mathsf{A}} $$ is square). Both algorithms use randomized projection techniques to cast most of the flops in terms of matrix‐matrix multiplication, which is exceptionally efficient on the GPU. Numerical experiments illustrate that these algorithms achieve significant acceleration over finely tuned GPU implementations of the SVD while providing low rank approximation errors close to that of the SVD.
$\ell^p$-Distances on Multiparameter Persistence Modules

Motivated both by theoretical and practical considerations in topological data analysis, we generalize the p-Wasserstein distance on barcodes to multiparameter persistence modules. For each p ∈ [1,∞], we in fact introduce two such generalizations dp I and dp M , such that d∞I equals the interleaving distance and d∞M equals the matching distance. We show that d p M ≤ d p I for all p ∈ [1,∞], extending an observation of Landi in the p = ∞ case. We observe that the distances d p M can be efficiently approximated. Finally, we show that on 1or 2-parameter persistence modules over prime fields, dp I is the universal (i.e., largest) metric satisfying a natural stability property; our result extends a stability result of Skraba and Turner for the p-Wasserstein distance on barcodes in the 1-parameter case, and is also a close analogue of a universality property for the interleaving distance given by the second author. In a companion paper, we apply some of these results to study the stability of (2-parameter) multicover persistent homology.
Distributed Nash Equilibrium Seeking Algorithm Design for Multi-Cluster Games with High-Order Players

In this paper, a multi-cluster game with high-order players is investigated. Different from the well-known multi-cluster games, the dynamics of players are taken into account in our problem. Due to the high-order dynamics of players, existing algorithms for multi-cluster games cannot solve the problem. For purpose of seeking the Nash equilibrium of the game, we design a distributed algorithm based on gradient descent and state feedback, where a distributed estimator is embedded for the players to estimate the decisions of other players. Furthermore, we analyze the exponential convergence of the algorithm via variational analysis and Lyapunov stability theory. Finally, a numerical simulation verifies the effectiveness of our method.
Scalable Perception-Action-Communication Loops With Convolutional and Graph Neural Networks

In this paper, we present a perception-action-communication loop design using Vision-based Graph Aggregation and Inference (VGAI). This multi-agent decentralized learning-to-control framework maps raw visual observations to agent actions, aided by local communication among neighboring agents. Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN/GNN), addressing agent-level visual perception and feature learning, as well as swarm-level communication, local information aggregation and agent action inference, respectively. By jointly training the CNN and GNN, image features and communication messages are learned in conjunction to better address the specific task. We use imitation learning to train the VGAI controller in an offline phase, relying on a centralized expert controller. This results in a learned VGAI controller that can be deployed in a distributed manner for online execution. Additionally, the controller exhibits good scaling properties, with training in smaller teams and application in larger teams. Through a multi-agent flocking application, we demonstrate that VGAI yields performance comparable to or better than other decentralized controllers, using only the visual input modality and without accessing precise location or motion state information.
Multi-Domain Active Learning: A Comparative Study

Multi-domain learning (MDL) refers to learning a set of models simultaneously, with each one specialized to perform a task in a certain domain. Generally, high labeling effort is required in MDL, as data needs to be labeled by human experts for every domain. To address the above issue, Active learning (AL) can be utilized to reduce the labeling effort by only using the most informative data. The resultant paradigm is termed multi-domain active learning (MDAL). However, despite the practical signiﬁcance of MDAL, there exists little research on it, not to mention any off-the-shelf solu-tion. To ﬁll this gap, we construct a simple pipeline of MDAL, and present a comprehensive comparative study of 30 different MDAL algorithms, which are established by combining 6 representative MDL models (equipped with various information-sharing schemes) and 5 well-used AL strategies. We evaluate the algorithms on 6 datasets, involving textual and visual classiﬁcation tasks. In most cases, AL brings notable improvements to MDL, and surprisingly, the naive best vs second best (BvSB) uncertainty strategy could perform competitively to the state-of-the-art AL strategies. Besides, among the MDL models, MAN and SDL-joint achieve the top performance when applied to vector features and raw images, respectively. Furthermore, we qualitatively analyze the behaviors of these strategies and models, shedding light on their superior performance in the comparison. Overall, some guidelines are provided, which could help to choose MDL models and AL strategies for particular applications
Post Selections Using Test Sets (PSUTS) and How Developmental Networks Avoid Them

—This paper raises a rarely reported practice in Artiﬁcial Intelligence (AI) called Post Selection Using Test Sets (PSUTS). Consequently, the popular error-backprop methodology in deep learning lacks an acceptable generalization power. All AI methods fall into two broad schools, connectionist and symbolic. The PSUTS fall into two kinds, machine PSUTS and human PSUTS. The connectionist school received criticisms for its “scrufﬁness” due to a huge number of network parameters and now the worse machine PSUTS; but the seemingly “clean” symbolic school seems more brittle because of a weaker generalization power using human PSUTS. This paper formally deﬁnes what PSUTS is, analyzes why error-backprop methods with random initial weights suffer from severe local minima, why PSUTS violates well-established research ethics, and how every paper that used PSUTS should have at least transparently reported PSUTS. For improved transparency in future publications, this paper proposes a new standard for performance evaluation of AI, called developmental errors for all networks trained, along with Three Learning Conditions: (1) an incremental learning architecture, (2) a training experience and (3) a limited amount of computational resources. Developmental Networks avoid PSUTS and are not “scruffy” because they drive Emergent Turing Machines and are optimal in the sense of maximum-likelihood across lifetime.
Generalized Unsupervised Clustering of Hyperspectral Images of Geological Targets in the Near Infrared

The application of infrared hyperspectral imagery to geological problems is becoming more popular as data become more accessible and cost-effective. Clustering and classifying spectrally similar materials is often a first step in applications ranging from economic mineral exploration on Earth to planetary exploration on Mars. Semi-manual classification guided by expertly developed spectral parameters can be time consuming and biased, while supervised methods require abundant labeled data and can be difficult to generalize. Here we develop a fully unsupervised workflow for feature extraction and clustering informed by both expert spectral geologist input and quantitative metrics. Our pipeline uses a lightweight autoencoder followed by Gaussian mixture modeling to map the spectral diversity within any image. We validate the performance of our pipeline at submillimeter-scale with expert-labelled data from the Oman ophiolite drill core and evaluate performance at meters-scale with partially classified orbital data of Jezero Crater on Mars (the landing site for the Perseverance rover). We additionally examine the effects of various preprocessing techniques used in traditional analysis of hyperspectral imagery. This pipeline provides a fast and accurate clustering map of similar geological materials and consistently identifies and separates major mineral classes in both laboratory imagery and remote sensing imagery. We refer to our pipeline as "Generalized Pipeline for Spectroscopic Unsupervised clustering of Minerals (GyPSUM). "
Scaling Properties of Galaxy Groups

Galaxy groups and poor clusters are more common than rich clusters, and host the largest fraction of matter content in the Universe. Hence, their studies are key to understand the gravitational and thermal evolution of the bulk of the cosmic matter. Moreover, because of their shallower gravitational potential, galaxy groups are systems where non-gravitational processes (e.g., cooling, AGN feedback, star formation) are expected to have a higher impact on the distribution of baryons, and on the general physical properties, than in more massive objects, inducing systematic departures from the expected scaling relations. Despite their paramount importance from the astrophysical and cosmological point of view, the challenges in their detection have limited the studies of galaxy groups. Upcoming large surveys will change this picture, reassigning to galaxy groups their central role in studying the structure formation and evolution in the Universe, and in measuring the cosmic baryonic content. Here, we review the recent literature on various scaling relations between X-ray and optical properties of these systems, focusing on the observational measurements, and the progress in our understanding of the deviations from the self-similar expectations on groups’ scales. We discuss some of the sources of these deviations, and how feedback from supernovae and/or AGNs impacts the general properties and the reconstructed scaling laws. Finally, we discuss future prospects in the study of galaxy groups.
Exposing individual differences through network topology

Social animals, including humans, have a broad range of personality traits, which can be used to predict individual behavioral responses and decisions. Current methods to quantify individual personality traits in humans rely on self-report questionnaires, which require time and effort to collect and rely on active cooperation. However, personality differences naturally manifest in social interactions such as online social networks. Here, we demonstrate that the topology of an online social network can be used to characterize the personality traits of its members. We analyzed the directed social graph formed by the users of the LiveJournal (LJ) blogging platform. Individual users personality traits, inferred from their self-reported domains of interest (DOIs), were associated with their network measures. Empirical clustering of DOIs by topological similarity exposed two main self-emergent DOI groups that were in alignment with the personality meta-traits plasticity and stability. Closeness, a global topological measure of network centrality, was significantly higher for bloggers associated with plasticity (vs. stability). A local network motif (a triad of 3 connected bloggers) that correlated with closeness also separated the personality meta-traits. Finally, topology-based classification of DOIs (without analyzing the content of the blogs) attained>70% accuracy (average AUC of the test-set). These results indicate that personality traits are evident and detectable in network topology. This has serious implications for user privacy. But, if used responsibly, network identification of personality traits could aid in early identification of health-related risks, at the population level.
Three K2 Campaigns Yield Rotation Periods for 1013 Stars in Praesepe

We use three campaigns of K2 observations to complete the census of rotation in low-mass members of the benchmark, ≈670 Myr old open cluster Praesepe. We measure new rotation periods (P rot) for 220 ≲1.3 M ⊙ Praesepe members and recovery periods for 97% (793/812) of the stars with a P rot in the literature. Of the 19 stars for which we do not recover a P rot, 17 were not observed by K2. As K2’s three Praesepe campaigns took place over the course of 3 yr, we test the stability of our measured P rot for stars observed in more than one campaign. We measure P rot consistent to within 10% for >95% of the 331 likely single stars with ≥2 high-quality observations; the median difference in P rot is 0.3%, with a standard deviation of 2%. Nearly all of the exceptions are stars with discrepant P rot measurements in Campaign 18, K2’s last, which was significantly shorter than the earlier two (≈50 days rather than ≈75 days). This suggests that, despite the evident morphological evolution we observe in the light curves of 38% of the stars, P rot measurements for low-mass stars in Praesepe are stable on timescales of several years. A P rot can therefore be taken to be representative even if measured only once.