Miracle-dz/UniFAR-FTUs huggingface dataset:
Majorana neutrinos in the triple gauge boson coupling $ZZZ^*$

Confirmed by the measurement of neutrino oscillations, neutrino mass is recognized as a genuine manifestation of physics beyond the Standard Model, while its originating mechanism remains a mystery. Moreover, the proper field-theory description of neutrinos, whether they are Majorana or Dirac type, must be linked to such a mechanism. The present work addresses the calculation, estimation, and analysis of one-loop contributions from virtual Majorana neutrinos, light and heavy as well, to the neutral gauge boson coupling $ZZZ$, which participates in $Z$-boson pair production from $e^+e^-$ collisions. This task is carried out in the framework defined by a seesaw variant in which light neutrinos remain massless at tree level, then becoming massive radiatively. The $ZZZ^*$ coupling, with $Z^*$ an off-shell $Z$ boson, is defined by two form factors, namely, $f_4$, characterizing CP-odd effects, and $f_5$, which is CP-even. Constraints from the Large Hadron Collider on both these quantities are currently ${\cal O}(10^{-4})$. Our calculation yields CP-nonpreserving contributions to $ZZZ$, which are absent in the framework of the sole Standard Model. Our estimations show that the $f_4$ contribution might be as large as ${\cal O}(10^{-7})$ for heavy-neutrino masses $\sim1\,{\rm TeV}$. CP-even contributions $f_5$ are also generated, which are, in general, larger than their CP-odd counterparts. We estimate them to be as large as ${\cal O}(10^{-4})$ at a center-of-mass energy of $500\,{\rm GeV}$, in $e^+e^-$ collisions.
Persona-aware Generative Model for Code-mixed Language

Code-mixing and script-mixing are prevalent across online social networks and multilingual societies. However, a user's preference toward code-mixing depends on the socioeconomic status, demographics of the user, and the local context, which existing generative models mostly ignore while generating code-mixed texts. In this work, we make a pioneering attempt to develop a persona-aware generative model to generate texts resembling real-life code-mixed texts of individuals. We propose a Persona-aware Generative Model for Code-mixed Generation, PARADOX, a novel Transformer-based encoder-decoder model that encodes an utterance conditioned on a user's persona and generates code-mixed texts without monolingual reference data. We propose an alignment module that re-calibrates the generated sequence to resemble real-life code-mixed texts. PARADOX generates code-mixed texts that are semantically more meaningful and linguistically more valid. To evaluate the personification capabilities of PARADOX, we propose four new metrics -- CM BLEU, CM Rouge-1, CM Rouge-L and CM KS. On average, PARADOX achieves 1.6 points better CM BLEU, 47% better perplexity and 32% better semantic coherence than the non-persona-based counterparts.
Investigation of nonlinear flame response to dual-frequency disturbances

The two-way interaction between the unsteady flame heat release rate and acoustic waves can lead to combustion instability within combustors. To understand and quantify the flame response to oncoming acoustic waves, previous studies have typically considered the flame dynamic response to pure tone forcing and assumed a dynamically linear or weakly nonlinear response. In this study, the introduction of excitation with two distinct frequencies denoted $St_1$ and $St_2$ is considered, including the effect of excitation amplitude in order to gain more insight into the nature of flame nonlinearities and their link with combustion instabilities. The investigation considers laminar flames and combines a low-order asymptotic analysis (up to third order in normalised excitation amplitude) with numerical methods based on the model framework of the $G$-equation. The importance of the propagation speed of the disturbance and its variation with frequency on the nonlinear response of the flame is highlighted. The influence path of the disturbance at one of the forcing frequencies, say $St_2$, on the flame dynamic response at the other forcing frequency $St_1$ is studied in detail. In concrete terms, the perturbation at $St_2$ acts in conjunction with the perturbation at $St_1$ to induce third-order nonlinear interactions in the flame kinematics, significantly altering the behavior of the flame response at $St_1$, as compared to the case where the flame is only subjected to the excitation at $St_1$. Particularly, when the normalised forcing amplitudes at the two frequencies are 0.2 and 0.3 respectively, the heat release rate response at the former frequency is attenuated by over 40 \% compared to the single-frequency response. This provides important insights into how nonlinearity due to frequency interactions can act to reduce the flame response.
A Comparison of Residual-based Methods on Fault Detection

An important initial step in fault detection for complex industrial systems is gaining an understanding of their health condition. Subsequently, continuous monitoring of this health condition becomes crucial to observe its evolution, track changes over time, and isolate faults. As faults are typically rare occurrences, it is essential to perform this monitoring in an unsupervised manner. Various approaches have been proposed not only to detect faults in an unsupervised manner but also to distinguish between different potential fault types. In this study, we perform a comprehensive comparison between two residual-based approaches: autoencoders, and theinput-output models that establish a mapping between operating conditions and sensor readings. We explore the sensorwise residuals and aggregated residuals for the entire system in both methods. The performance evaluation focuses on three tasks: health indicator construction, fault detection, and health indicator interpretation. To perform the comparison, we utilize the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dynamical model, specifically a subset of the turbofan engine dataset containing three different fault types. All models are trained exclusively on healthy data. Fault detection is achieved by applying a threshold that is determined based on the healthy condition. The detection results reveal that both models are capable of detecting faults with an average delay of around 20 cycles and maintain a low false positive rate. While the fault detection performance is similar for both models, the input-output model provides better interpretability regarding potential fault types and the possible faulty components.
A new data adaptive elastic net predictive model using hybridized smoothed covariance estimators with information complexity

ABSTRACT We develop a novel Adaptive Elastic Net (AEN) modelling using a new covariance-regularization approach via the Hybridized Smoothed Covariance Estimators (HSCEs) to identify and select the best subset of predictors in undersized high-dimensional data sets. We introduce and score the Consistent and Misspecification Resistant Information Measure of Complexity criterion, and the Extended Consistent Akaike's Information Criterion with Fisher Information in AEN models for the first time. We carry out a large Monte Carlo simulation study using the medianmean-squared-error (MMSE) to demonstrate and compare the performance of the MMSE prediction. This is done using Cross-validated Fit Adaptive Elastic Net (CV-AEN) to avoid double shrinkage by varying both the error variance and the correlation structure of the model. Later, the new proposed AEN model is applied to a real undersized benchmark data set to predict the Riboflavin (Vitamin B2) production to select the best subset of predictors to predict the production rate of vitamin B2 and provide the best predictive model. The proposed new approach enables a simple and reliable identification of the best subset of predictive genes of the production rate of Riboflavin (Vitamin B2) without an exhaustive search of all possible subset selection in undersized high-dimensional data. It is a new and novel approach that has generalizability to other regularized General Linear Regression (GLM) models to determine the best predictor space for undersized data.
Dual Adversarial Alignment for Realistic Support-Query Shift Few-shot Learning

Support-query shift few-shot learning aims to classify unseen examples (query set) to labeled data (support set) based on the learned embedding in a low-dimensional space under a distribution shift between the support set and the query set. However, in real-world scenarios the shifts are usually unknown and varied, making it difficult to estimate in advance. Therefore, in this paper, we propose a novel but more difficult challenge, RSQS, focusing on Realistic Support-Query Shift few-shot learning. The key feature of RSQS is that the individual samples in a meta-task are subjected to multiple distribution shifts in each meta-task. In addition, we propose a unified adversarial feature alignment method called DUal adversarial ALignment framework (DuaL) to relieve RSQS from two aspects, i.e., inter-domain bias and intra-domain variance. On the one hand, for the inter-domain bias, we corrupt the original data in advance and use the synthesized perturbed inputs to train the repairer network by minimizing distance in the feature level. On the other hand, for intra-domain variance, we proposed a generator network to synthesize hard, i.e., less similar, examples from the support set in a self-supervised manner and introduce regularized optimal transportation to derive a smooth optimal transportation plan. Lastly, a benchmark of RSQS is built with several state-of-the-art baselines among three datasets (CIFAR100, mini-ImageNet, and Tiered-Imagenet). Experiment results show that DuaL significantly outperforms the state-of-the-art methods in our benchmark.
Census for the rest-frame optical and UV morphologies of galaxies at z = 4–10: First phase of inside-out galaxy formation

We present the rest-frame optical and UV surface brightness (SB) profiles for 149 galaxies with Mopt < −19.4 mag at z = 4–10 (29 of which are spectroscopically confirmed with JWST NIRSpec), securing high signal-to-noise ratios of 10–135 with deep JWST NIRCam 1–$5\, \mu$m images obtained by the CEERS survey. We derive morphologies of our high-z galaxies, carefully evaluating the systematics of SB profile measurements with Monte Carlo simulations as well as the impacts of a) AGNs, b) multiple clumps including galaxy mergers, c) spatial resolution differences with previous HST studies, and d) strong emission lines, e.g., Hα and [O iii], on optical morphologies with medium-band F410M images. Conducting Sérsic profile fitting for our high-z galaxy SBs with GALFIT, we obtain effective radius ranges for optical re,opt and UV re,UV wavelengths of re, opt = 0.05–1.6 kpc and re,UV = 0.03–1.7 kpc that are consistent with previous results within large scatters in the size–luminosity relations. However, we find that the effective radius ratio, re,opt$/$re,UV, is almost unity, $1.01^{+0.35}_{-0.22}$, over z = 4–10 with no signatures of past inside-out star formation such as found at z ∼ 0–2. There are no spatial offsets exceeding 3σ between the optical and UV morphology centers in cases of no mergers, indicative of major star-forming activity only being found near mass centers of galaxies at z ≳ 4 probably experiencing the first phase of inside-out galaxy formation.
Constraining the halo-ISM connection through multi-transition carbon monoxide line-intensity mapping

Line-intensity mapping (LIM) surveys will characterise the cosmological large-scale structure of emissivity in a range of atomic and molecular spectral lines, but existing literature rarely considers whether these surveys can recover excitation properties of the tracer gas species, such as the carbon monoxide (CO) molecule. Combining basic empirical and physical assumptions with the off-the-shelf Radex radiative transfer code or a Gaussian process emulator of Radex outputs, we devise a basic dark matter halo model for CO emission by tying bulk CO properties to halo properties, exposing physical variables governing CO excitation as free parameters. The CO Mapping Array Project (COMAP) is working towards a multi-band survey programme to observe both CO(1–0) and CO(2–1) at z ∼ 7. We show that this programme, as well as a further `Triple Deluxe' extension to higher frequencies covering CO(3–2), is fundamentally capable of successfully recovering the connection between halo mass and CO abundances, and constraining the molecular gas kinetic temperature and density within the star-forming interstellar medium in ways that single-transition CO LIM cannot. Given a fiducial thermal pressure of ∼ 104 K cm-3 for molecular gas in halos of ∼ 1010 M ⊙, simulated multi-band COMAP surveys successfully recover the thermal pressure within 68% interval half-widths of 0.5–0.6 dex. Construction of multi-frequency LIM instrumentation to access multiple CO transitions is crucial in harnessing this capability, as part of a cosmic statistical probe of gas metallicity, dust chemistry, and other physical parameters in star-forming regions of the first galaxies and proto-galaxies out of reionisation.
PCFGaze: Physics-Consistent Feature for Appearance-based Gaze Estimation

Although recent deep learning based gaze estimation approaches have achieved much improvement, we still know little about how gaze features are connected to the physics of gaze. In this paper, we try to answer this question by analyzing the gaze feature manifold. Our analysis revealed the insight that the geodesic distance between gaze features is consistent with the gaze differences between samples. According to this finding, we construct the Physics- Consistent Feature (PCF) in an analytical way, which connects gaze feature to the physical definition of gaze. We further propose the PCFGaze framework that directly optimizes gaze feature space by the guidance of PCF. Experimental results demonstrate that the proposed framework alleviates the overfitting problem and significantly improves cross-domain gaze estimation accuracy without extra training data. The insight of gaze feature has the potential to benefit other regression tasks with physical meanings.
TiAVox: Time-aware Attenuation Voxels for Sparse-view 4D DSA Reconstruction

Four-dimensional Digital Subtraction Angiography (4D DSA) plays a critical role in the diagnosis of many medical diseases, such as Arteriovenous Malformations (AVM) and Arteriovenous Fistulas (AVF). Despite its significant application value, the reconstruction of 4D DSA demands numerous views to effectively model the intricate vessels and radiocontrast flow, thereby implying a significant radiation dose. To address this high radiation issue, we propose a Time-aware Attenuation Voxel (TiAVox) approach for sparse-view 4D DSA reconstruction, which paves the way for high-quality 4D imaging. Additionally, 2D and 3D DSA imaging results can be generated from the reconstructed 4D DSA images. TiAVox introduces 4D attenuation voxel grids, which reflect attenuation properties from both spatial and temporal dimensions. It is optimized by minimizing discrepancies between the rendered images and sparse 2D DSA images. Without any neural network involved, TiAVox enjoys specific physical interpretability. The parameters of each learnable voxel represent the attenuation coefficients. We validated the TiAVox approach on both clinical and simulated datasets, achieving a 31.23 Peak Signal-to-Noise Ratio (PSNR) for novel view synthesis using only 30 views on the clinically sourced dataset, whereas traditional Feldkamp-Davis-Kress methods required 133 views. Similarly, with merely 10 views from the synthetic dataset, TiAVox yielded a PSNR of 34.32 for novel view synthesis and 41.40 for 3D reconstruction. We also executed ablation studies to corroborate the essential components of TiAVox. The code will be publically available.
Domain Adaptation for Satellite-Borne Hyperspectral Cloud Detection

The advent of satellite-borne machine learning hardware accelerators has enabled the on-board processing of payload data using machine learning techniques such as convolutional neural networks (CNN). A notable example is using a CNN to detect the presence of clouds in hyperspectral data captured on Earth observation (EO) missions, whereby only clear sky data is downlinked to conserve bandwidth. However, prior to deployment, new missions that employ new sensors will not have enough representative datasets to train a CNN model, while a model trained solely on data from previous missions will underperform when deployed to process the data on the new missions. This underperformance stems from the domain gap, i.e., differences in the underlying distributions of the data generated by the different sensors in previous and future missions. In this paper, we address the domain gap problem in the context of on-board hyperspectral cloud detection. Our main contributions lie in formulating new domain adaptation tasks that are motivated by a concrete EO mission, developing a novel algorithm for bandwidth-efficient supervised domain adaptation, and demonstrating test-time adaptation algorithms on space deployable neural network accelerators. Our contributions enable minimal data transmission to be invoked (e.g., only 1% of the weights in ResNet50) to achieve domain adaptation, thereby allowing more sophisticated CNN models to be deployed and updated on satellites without being hampered by domain gap and bandwidth limitations.
DMKD: Improving Feature-based Knowledge Distillation for Object Detection Via Dual Masking Augmentation

Recent mainstream masked distillation methods function by reconstructing selectively masked areas of a student network from the feature map of its teacher counterpart. In these methods, the masked regions need to be properly selected, such that reconstructed features encode sufficient discrimination and representation capability like the teacher feature. However, previous masked distillation methods only focus on spatial masking, making the resulting masked areas biased towards spatial importance without encoding informative channel clues. In this study, we devise a Dual Masked Knowledge Distillation (DMKD) framework which can capture both spatially important and channel-wise informative clues for comprehensive masked feature reconstruction. More specifically, we employ dual attention mechanism for guiding the respective masking branches, leading to reconstructed feature encoding dual significance. Furthermore, fusing the reconstructed features is achieved by self-adjustable weighting strategy for effective feature distillation. Our experiments on object detection task demonstrate that the student networks achieve performance gains of 4.1% and 4.3% with the help of our method when RetinaNet and Cascade Mask R-CNN are respectively used as the teacher networks, while outperforming the other state-of-the-art distillation methods.
Tuning flat bands by interlayer interaction, spin-orbital coupling, and external fields in twisted homotrilayer 
MoS2

Ultraflat bands have already been detected in twisted bilayer graphene and twisted bilayer transition-metal dichalcogenides, which provide a platform to investigate strong correlations. In this paper, the electronic properties of twisted trilayer molybdenum disulfide (TTM) are investigated via an accurate tight-binding Hamiltonian. We find that the highest valence bands are derived from the $\Gamma$-point of the constituent monolayer, and they exhibit a graphenelike dispersion or become isolated flat bands that are dependent on the starting stacking arrangements. The lattice relaxation, local deformation, and external fields can significantly tune the electronic structures of TTM. After introducing the spin-orbital coupling effect, we find a spin-valley-layer locking effect at the minimum of the conduction band at the $K$- and $K^\prime$-point of the Brillouin zone, which may provide a platform to study optical properties and magnetoelectric effects.
An improvement to the Kelley-Meka bounds on three-term arithmetic progressions

In a recent breakthrough Kelley and Meka proved a quasipolynomial upper bound for the density of sets of integers without non-trivial three-term arithmetic progressions. We present a simple modification to their method that strengthens their conclusion, in particular proving that if $A\subset\{1,\ldots,N\}$ has no non-trivial three-term arithmetic progressions then \[\lvert A\rvert \leq \exp(-c(\log N)^{1/9})N\] for some $c>0$.
SeisCLIP: A Seismology Foundation Model Pre-Trained by Multimodal Data for Multipurpose Seismic Feature Extraction

In seismology, while training a specific deep learning model for each task is common, it often faces challenges such as the scarcity of labeled data and limited regional generalization. Addressing these issues, we introduce SeisCLIP: a foundation model for seismology, leveraging contrastive learning during pretraining on multimodal data of seismic waveform spectra and the corresponding local and global event information. SeisCLIP consists of a transformer-based spectrum encoder and an multilayer perceptron (MLP)-based information encoder that are jointly pre-trained on massive data. During pretraining, contrastive learning aims to enhance representations by training two encoders to bring the corresponding waveform spectra and event information closer in the feature space, while distancing uncorrelated pairs. Remarkably, the pre-trained spectrum encoder offers versatile features, enabling its application across diverse tasks and regions. Thus, it requires only modest datasets for fine-tuning to specific downstream tasks. Our evaluations demonstrate SeisCLIP’s superior performance over baseline methods in tasks like event classification, localization, and focal mechanism analysis, even when using distinct datasets from various regions. In essence, SeisCLIP emerges as a promising foundational model for seismology, potentially revolutionizing foundation-model-based research in the domain.
A Theoretical Explanation of Activation Sparsity through Flat Minima and Adversarial Robustness

A recent empirical observation (Li et al., 2022b) of activation sparsity in MLP blocks offers an opportunity to drastically reduce computation costs for free. Although having attributed it to training dynamics, existing theoretical explanations of activation sparsity are restricted to shallow networks, small training steps and special training, despite its emergence in deep models standardly trained for a large number of steps. To fill these gaps, we propose the notion of gradient sparsity as one source of activation sparsity and a theoretical explanation based on it that sees sparsity a necessary step to adversarial robustness w.r. t. hidden features and parameters, which is approximately the flatness of minima for well-learned models. The theory applies to standardly trained LayerNorm-ed MLPs, and further to Transformers or other architectures trained with weight noises. Eliminating other sources of flatness except for sparsity, we discover the phenomenon that the ratio between the largest and smallest non-zero singular values of weight matrices is small. When discussing the emergence of this spectral concentration, we use random matrix theory (RMT) as a powerful tool to analyze stochastic gradient noises. Validational experiments are conducted to verify our gradient-sparsity-based explanation. We propose two plug-and-play modules for both training and finetuning for sparsity. Experiments on ImageNet-1k and C4 demonstrate their 50% sparsity improvements, indicating further potential cost reduction in both training and inference.