Miracle-dz/UniFAR-FTUs huggingface dataset:
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

As the training of giant dense models hits the boundary on the availability and capability of the hardware resources today, Mixture-of-Experts (MoE) models become one of the most promising model architectures due to their significant training cost reduction compared to a quality-equivalent dense model. Its training cost saving is demonstrated from encoder-decoder models (prior works) to a 5x saving for auto-aggressive language models (this work along with parallel explorations). However, due to the much larger model size and unique architecture, how to provide fast MoE model inference remains challenging and unsolved, limiting its practical usage. To tackle this, we present DeepSpeed-MoE, an end-to-end MoE training and inference solution as part of the DeepSpeed library, including novel MoE architecture designs and model compression techniques that reduce MoE model size by up to 3.7x, and a highly optimized inference system that provides 7.3x better latency and cost compared to existing MoE inference solutions. DeepSpeed-MoE offers an unprecedented scale and efficiency to serve massive MoE models with up to 4.5x faster and 9x cheaper inference compared to quality-equivalent dense models. We hope our innovations and systems help open a promising path to new directions in the large model landscape, a shift from dense to sparse MoE models, where training and deploying higher-quality models with fewer resources becomes more widely possible.
FEEDBACK from the NGC7538 Hii region

Context. How do expanding Hii regions interact with their environmental cloud? This is one of the central questions driving the SOFIA legacy program FEEDBACK. Here, we present a case study toward the prototypical Hii region NGC7538. Aims. We want to understand the interaction of the NGC7538 Hii region with the neighboring molecular cloud that hosts several active star-forming regions. Methods. With the Stratospheric Observatory for Infrared Astronomy (SOFIA) we mapped an area of ∼210′2 (∼125 pc2) around NGC7538 in the velocity-resolved ionized carbon fine-structure line [CII] at 1.9 THz (158 μm). Complementary observed atomic carbon [CI] at 492 GHz and high-J CO(8–7) data as well as archival near-/far-infrared, cm continuum, CO(3–2) and HI data are folded into the analysis. Results. The ionized carbon [CII] data reveal rich morphological and kinematic structures. While the overall morphology follows the general ionized gas that is also visible in the radio continuum emission, the channel maps show multiple bubble-like structures with sizes on the order of ∼80–100′′ (∼1.0–1.28 pc). While at least one of them may be an individual feedback bubble driven by the main exciting sources of the NGC7538 Hii region (the O3 and O9 stars IRS6 & IRS5), the other bubble-like morphologies may also be due to the intrinsically porous structure of the Hii region. An analysis of the expansion velocities around 10 km s−1 indicates that thermal expansion is not sufficient but that wind-driving from the central O-stars is required. The region exhibits a general velocity gradient across, but we also identify several individual velocity components. The most blue-shifted [CII] component has barely any molecular or atomic counterparts. At the interface to the molecular cloud, we find a typical photon-dominated region (PDR) with a bar-shape. Ionized C, atomic C0 and molecular carbon CO show a layered structure in this PDR. The carbon in the PDR is dominated by its ionized C form with atomic C0 and molecular CO masses of ∼0.45±0.1 M and ∼1.2±0.1 M , respectively, compared to the ionized carbon C in the range of 3.6 − 9.7 M . This bar-shaped PDR exhibits a velocity-gradient across, indicating motions along the line-of-sight towards the observer. Conclusions. Even if dominated by two nearby exciting sources (IRS6 & IRS5), the NGC7538 Hii region exhibits a diverse set of sub-structures that interact with each other as well as with the adjacent cloud. Compared to other recent [CII] observations of Hii regions (e.g., Orion Veil, RCW120, RCW49), bubble-shape morphologies revealed in [CII] emission, indicative of expanding shells, are recurring structures of PDRs.
Eﬃciently Detecting Multiple Structural Breaks in Systems of Linear Regression Equations with Integrated and Stationary Regressors

In this paper, we propose a two-step procedure based on the group LASSO estimator in combination with a backward elimination algorithm to eﬃciently detect multiple structural breaks in linear regressions with multivariate responses. Applying the two-step estimator, we jointly detect the number and location of change points, and provide consistent estimates of the coeﬃcients. Our framework is ﬂexible enough to allow for a mix of integrated and stationary regressors, as well as deterministic terms. Using simulation experiments, we show that the proposed two-step estimator performs competitively against the likelihood-based approach (Qu and Perron, 2007; Li and Perron, 2017; Oka and Perron, 2018) when trying to detect common breaks in ﬁnite samples. However, the two-step estimator is computationally much more eﬃcient. An economic application to the identiﬁcation of structural breaks in the term structure of interest rates illustrates this methodology.
SympOCnet: Solving optimal control problems with applications to high-dimensional multi-agent path planning problems

Solving high-dimensional optimal control problems in real-time is an important but challenging problem, with applications to multi-agent path planning problems, which have drawn increased attention given the growing popularity of drones in recent years. In this paper, we propose a novel neural network method called SympOCnet that applies the Symplectic network to solve high-dimensional optimal control problems with state constraints. We present several numerical results on path planning problems in two-dimensional and three-dimensional spaces. Specifically, we demonstrate that our SympOCnet can solve a problem with more than 500 dimensions in 1.5 hours on a single GPU, which shows the effectiveness and efficiency of SympOCnet. The proposed method is scalable and has the potential to solve truly high-dimensional path planning problems in real-time.
Stellar dating using chemical clocks and Bayesian inference

Context. Dating stars is a major challenge with a deep impact on many astrophysical ﬁelds. One of the most promising techniques for this is using chemical abundances. Recent space- and ground-based facilities have improved the quantity of stars with accurate observations. This has opened the door for using Bayesian inference tools to maximise the information we can extract from them. Aims. Our aim is to present accurate and reliable stellar age estimates of FGK stars using chemical abundances and stellar parameters. Methods. We used one of the most ﬂexible Bayesian inference techniques (hierarchical Bayesian models) to exceed current possi- bilities in the use of chemical abundances for stellar dating. Our model is a data-driven model. We used a training set that has been presented in the literature with ages estimated with isochrones and accurate stellar abundances and general characteristics. The core of the model is a prescription of certain abundance ratios as linear combinations of stellar properties including age. We gathered four di ﬀ erent testing sets to assess the accuracy, precision, and limits of our model. We also trained a model using chemical abundances alone. Results. We found that our age estimates and those coming from asteroseismology, other accurate sources, and also with ten Gaia benchmark stars agree well. The mean absolute di ﬀ erence of our estimates compared with those used as reference is 0.9Ga, with a mean di ﬀ erence of 0.01Ga. When using open clusters, we reached a very good agreement for Hyades, NGC2632, Ruprecht 147, and IC4651. We also found outliers that are a reﬂection of chemical peculiarities and / or stars at the limit of the validity ranges of the training set. The model that only uses chemical abundances shows slightly worse mean absolute di ﬀ erence (1.18Ga) and mean di ﬀ erence ( − 0 . 12Ga).
Study of Frequency domain exponential functional link network filters

The exponential functional link network (EFLN) filter has attracted tremendous interest due to its enhanced nonlinear modeling capability. However, the computational complexity will dramatically increase with the dimension growth of the EFLN-based filter. To improve the computational efficiency, we propose a novel frequency domain exponential functional link network (FDEFLN) filter in this paper. The idea is to organize the samples in blocks of expanded input data, transform them from time domain to frequency domain, and thus execute the filtering and adaptation procedures in frequency domain with the overlap-save method. A FDEFLN-based nonlinear active noise control (NANC) system has also been developed to form the frequency domain exponential filtered-s least mean-square (FDEFsLMS) algorithm. Moreover, the stability, steady-state performance and computational complexity of algorithms are analyzed. Finally, several numerical experiments corroborate the proposed FDEFLN-based algorithms in nonlinear system identification, acoustic echo cancellation and NANC implementations, which demonstrate much better computational efficiency.
Deep Learning for Agile Effort Estimation Have We Solved the Problem Yet?

—In the last decade, several studies have proposed the use of automated techniques to estimate the effort of agile software development. In this paper we perform a close replication and extension of a seminal work proposing the use of Deep Learning for agile effort estimation (namely Deep-SE), which has set the state-of-the-art since. Speciﬁcally, we replicate three of the original research questions aiming at investigating the effectiveness of Deep-SE for both within-project and cross-project effort estimation. We benchmark Deep-SE against three baseline techniques (i.e., Random, Mean and Median effort prediction) and a previously proposed method to estimate agile software project development effort (dubbed TF/IDF-SE), as done in the original study. To this end, we use both the data from the original study and a new larger dataset of 31,960 issues, which we mined from 29 open-source projects. Using more data allows us to strengthen our conﬁdence in the results and further mitigate the threat to the external validity of the study. We also extend the original study by investigating two additional research questions. One evaluates the accuracy of Deep-SE when the training set is augmented with issues from all other projects available in the repository at the time of estimation, and the other examines whether an expensive pre-training step used by the original Deep-SE, has any beneﬁcial effect on its accuracy and convergence speed. The results of our replication show that Deep-SE outperforms the Median baseline estimator and TF/IDF-SE in only very few cases with statistical signiﬁcance (8/42 and 9/32 cases, respectively), thus confounding previous ﬁndings on the efﬁcacy of Deep-SE. The two additional RQs revealed that neither augmenting the training set nor pre-training Deep-SE play a role in improving its accuracy and convergence speed. These results suggest that using semantic similarity is not enough to differentiate user stories with respect to their story points; thus, future work has yet to explore and ﬁnd new techniques and features that obtain accurate agile software development estimates.
The evolution of cooperation: an evolutionary advantage of individuals impedes the evolution of the population

Range expansion is a universal process in biological systems, and therefore plays a part in biological evolution. Using a quantitative individual-based method based on the stochastic process, we identify that enhancing the inherent self-proliferation advantage of cooperators relative to defectors is a more effective channel to promote the evolution of cooperation in range expansion than weakening the benefit acquisition of defectors from cooperators. With this self-proliferation advantage, cooperators can rapidly colonize virgin space and establish spatial segregation more readily, which acts like a protective shield to further promote the evolution of cooperation in return. We also show that lower cell density and migration rate have a positive effect on the competition of cooperators with defectors. Biological evolution is based on competition between individuals and should therefore favor selfish behaviors. However, we observe a counterintuitive phenomenon that the evolution of a population is impeded by the fitness-enhancing chemotactic movement of individuals. This highlights a conflict between the interests of the individual and the population. The short-sighted selfish behavior of individuals may not be that favored in the competition between populations. Such information provides important implications for the handling of cooperation.
Towards fully automated robotic platform for remote auscultation

Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an ageing population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost‐effective screening tool for detecting abnormal clinical signs.
Learning-Based MIMO Channel Estimation under Spectrum Efficient Pilot Allocation and Feedback

Wireless links using massive MIMO transceivers are vital for next generation wireless communications networks networks. Precoding in Massive MIMO transmission requires accurate downlink channel state information (CSI). Many recent works have effectively applied deep learning (DL) to jointly train UE-side compression networks for delay domain CSI and a BS-side decoding scheme. Vitally, these works assume that the full delay domain CSI is available at the UE, but in reality, the UE must estimate the delay domain based on a limited number of frequency domain pilots. In this work, we propose a linear pilot-to-delay (P2D) estimator that transforms sparse frequency pilots to the truncated delay CSI. We show that the P2D estimator is accurate under frequency downsampling, and we demonstrate that the P2D estimate can be effectively utilized with existing autoencoder-based CSI estimation networks. In addition to accounting for pilot-based estimates of downlink CSI, we apply unrolled optimization networks to emulate iterative solutions to compressed sensing (CS), and we demonstrate better estimation performance than prior autoencoder-based DL networks. Finally, we investigate the efficacy of trainable CS networks for in a differential encoding network for time-varying CSI estimation, and we propose a new network, MarkovNet-ISTA-ENet, comprised of both a CS network for initial CSI estimation and multiple autoencoders to estimate the error terms. We demonstrate that this heterogeneous network has better asymptotic performance than networks comprised of only one type of network.
An Intelligent System for Multi-topic Social Spam Detection in Microblogging

The communication revolution has perpetually reshaped the means through which people send and receive information. Social media is an important pillar of this revolution and has brought profound changes to various aspects of our lives. However, the open environment and popularity of these platforms inaugurate windows of opportunities for various cyber threats, thus social networks have become a fertile venue for spammers and other illegitimate users to execute their malicious activities. These activities include phishing hot and trendy topics and posting a wide range of contents in many topics. Hence, it is crucial to continuously introduce new techniques and approaches to detect and stop this category of users. This article proposes a novel and effective approach to detect social spammers. An investigation into several attributes to measure topic-dependent and topic-independent users’ behaviours on Twitter is carried out. The experiments of this study are undertaken on various machine learning classifiers. The performance of these classifiers is compared and their effectiveness is measured via a number of robust evaluation measures. Furthermore, the proposed approach is benchmarked against state-of-the-art social spam and anomalous detection techniques. These experiments report the effectiveness and utility of the proposed approach and embedded modules.
Dark Energy Survey Year 3 Results: Three-point shear correlations and mass aperture moments

We present high signal-to-noise measurements of three-point shear correlations and the third moment of the mass aperture statistic using the first 3 years of data from the Dark Energy Survey. We additionally obtain the first measurements of the configuration and scale dependence of the four three-point shear correlations which carry cosmological information. With the third-order mass aperture statistic, we present tomographic measurements over angular scales of 4 to 60 arcminutes with a combined statistical significance of 15.0$\sigma$. Using the tomographic information and measuring also the second-order mass aperture, we additionally obtain a skewness parameter and its redshift evolution. We find that the amplitudes and scale-dependence of these shear 3pt functions are in qualitative agreement with measurements in a mock galaxy catalog based on N-body simulations, indicating promise for including them in future cosmological analyses. We validate our measurements by showing that B-modes, parity-violating contributions and PSF modeling uncertainties are negligible, and determine that the measured signals are likely to be of astrophysical and gravitational origin.
CurvingLoRa to Boost LoRa Network Capacity via Concurrent Transmission

LoRaWAN has emerged as an appealing technology to connect IoT devices but it functions without explicit coordination among transmitters, which can lead to many packet collisions as the network scales. State-of-the-art work proposes various approaches to deal with these collisions, but most functions only in high signal-to-interference ratio (SIR) conditions and thus does not scale to many scenarios where weak receptions are easily buried by stronger receptions from nearby transmitters. In this paper, we take a fresh look at LoRa's physical layer, revealing that its underlying linear chirp modulation fundamentally limits the capacity and scalability of concurrentLoRa transmissions. We show that by replacing linear chirps with their non-linear counterparts, we can boost the capacity of concurrent LoRa transmissions and empower the LoRa receiver to successfully receive weak transmissions in the presence of strong colliding signals. Such a non-linear chirp design further enables the receiver to demodulate fully aligned collision symbols - a case where none of the existing approaches can deal with. We implement these ideas in a holistic LoRaWAN stack based on the USRP N210 software-defined radio platform. Our head-to-head comparison with two state-of-the-art research systems and a standard LoRaWAN baseline demonstrates that CurvingLoRa improves the network throughput by 1.6-7.6x while simultaneously sacrificing neither power efficiency nor noise resilience. An open-source dataset and code will be made available before publication.
Thermal Properties of 1847 WISE-observed Asteroids

We present new thermophysical model fits of 1847 asteroids, deriving thermal inertia, diameter, and Bond and visible geometric albedo. We use thermal flux measurements obtained by the Wide-field Infrared Survey Explorer (WISE) during its fully cryogenic phase, when both the 12 μm (W3) and 22 μm (W4) bands were available. We take shape models and spin information from the Database of Asteroid Models from Inversion Techniques (DAMIT) and derive new shape models through lightcurve inversion and combining WISE photometry with existing DAMIT lightcurves. When we limit our sample to the asteroids with the most reliable shape models and thermal flux measurements, we find broadly consistent thermal inertia relations with recent studies. We apply fits to the diameters D (km) and thermal inertia Γ (J m−2 s−0.5 K−1) normalized to 1 au with a linear relation of the form log[Γ]=α+βlog[D] , where we find α = 2.667 ± 0.059 and β = −0.467 ± 0.044 for our sample alone and α = 2.509 ± 0.017 and β = −0.352 ± 0.012 when combined with other literature estimates. We find little evidence of any correlation between rotation period and thermal inertia, owing to the small number of slow rotators to consider in our sample. While the large uncertainties on the majority of our derived thermal inertia only allow us to identify broad trends between thermal inertia and other physical parameters, we can expect a significant increase in high-quality thermal flux measurements and asteroid shape models with upcoming infrared and wide-field surveys, enabling even more thermophysical modeling of higher precision in the future.
FARSI: Facebook AR System Investigator for Agile Domain-Specific System-on-Chip Exploration

BEHZAD BOROUJERDIAN, University of Texas at Austin and Facebook Inc., USA YING JING, University of Illinois Urbana-Champaign , USA AMIT KUMAR, Facebook Inc., USA LAVANYA SUBRAMANIAN, Facebook Inc., USA LUKE YEN, Tenstorrent Inc., USA VINCENT LEE, Facebook Inc., USA VIVEK VENKATESAN, Facebook Inc., USA AMIT JINDAL, Peloton Interactive Inc., USA ROBERT SHEARER, Facebook Inc., USA VIJAY JANAPA REDDI, Harvard University and The University of Texas at Austin , USA
The homotopy category of acyclic complexes of pure-projective modules

Abstract Let R be any ring with identity. We show that the homotopy category of all acyclic chain complexes of pure-projective R-modules is a compactly generated triangulated category. We do this by constructing abelian model structures that put this homotopy category into a recollement with two other compactly generated triangulated categories: The usual derived category of R and the pure derived category of R.
Galaxy Correlation Function and Local Density from Photometric Redshifts Using the Stochastic Order Redshift Technique (SORT)

The stochastic order redshift technique (SORT) is a simple, efficient, and robust method to improve cosmological redshift measurements. The method relies upon having a small ($\sim$10 per cent) reference sample of high-quality redshifts. Within pencil-beam-like sub-volumes surrounding each galaxy, we use the precise dN/d$z$ distribution of the reference sample to recover new redshifts and assign them one-to-one to galaxies such that the original rank order of redshifts is preserved. Preserving the rank order is motivated by the fact that random variables drawn from Gaussian probability density functions with different means but equal standard deviations satisfy stochastic ordering. The process is repeated for sub-volumes surrounding each galaxy in the survey. This results in every galaxy with an uncertain redshift being assigned multiple"recovered"redshifts from which a new redshift estimate can be determined. An earlier paper applied SORT to a mock Sloan Digital Sky Survey at $z \lesssim$ 0.2 and accurately recovered the two-point correlation function on scales $\gtrsim$4 $h^{-1}$Mpc. In this paper, we test the performance of SORT in surveys spanning the redshift range 0.75$<z<$2.25. We used two mock surveys extracted from the Small MultiDark-Planck and Bolshoi-Planck N-body simulations with dark matter haloes that were populated by the Santa Cruz semi-analytic model. We find that SORT is able to improve redshift estimates and recover distinctive large-scale features of the cosmic web. Further, it provides unbiased estimates of the redshift-space two-point correlation function $\xi(s)$ on scales $\gtrsim$2.5 $h^{-1}$Mpc, as well as local densities in regions of average or higher density. This may allow improved understanding of how galaxy properties relate to their local environments.