Miracle-dz/UniFAR-FTUs huggingface dataset:
Including stress relaxation in point-process model for seismic occurrence

Physics-based and statistic-based models for describing seismic occurrence are two sides of the same coin. In this article we compare the temporal organization of events obtained in a spring-block model for the seismic fault with the one predicted by probabilistic models for seismic occurrence. Thanks to the optimization of the parameters, by means of a Maximum Likelihood Estimation, it is possible to identify the statistical model which fits better the physical one. The results show that the best statistical model must take into account the non trivial interplay between temporal clustering, related to aftershock occurrence, and the stress discharge following the occurrence of high magnitude mainshocks. The two mechanisms contribute in different ways according to the minimum magnitude considered in the data fitting catalog.
A Dynamical Survey of the Trans-Neptunian Region. I. Mean-motion Resonances with Neptune

In this paper, we present a large-scale dynamical survey of the trans-Neptunian region, with particular attention to mean-motion resonances (MMRs). We study a set of 4121 trans-Neptunian objects (TNOs), a sample far larger than in previous works. We perform direct long-term numerical integrations that enable us to examine the overall dynamics of the individual TNOs as well as to identify all MMRs. For the latter purpose, we apply an own-developed FAIR method that allows the semi-automatic identification of even very high-order MMRs. Apart from searching for the more frequent eccentricity-type resonances that previous studies concentrated on, we set our method to allow the identification of inclination-type MMRs, too. Furthermore, we distinguish between TNOs that are locked in a given MMR throughout the whole integration time span (108 yr) and those that are only temporarily captured in resonances. For a more detailed dynamical analysis of the trans-Neptunian space, we also construct dynamical maps using test particles. Observing the fine structure of the 34–80 au region underlines the stabilizing role of the MMRs, with the regular regions coinciding with the positions of real TNOs.
Effect of Soaking Time on Phase Formation and Electrical Properties of PLZT Based Ceramics

Lead lanthanum zirconate titanate ceramics (PLZT) are well known for their excellent dielectric, piezoelectric and ferroelectric properties and electro-optic characteristics. The solid solutions of compounds are expected to possess very interesting properties, making them promising candidates for electromechanical devices. This study describes processing and sintering conditions and electrical properties of the PLZT ceramics (9/65/35) that were successfully prepared from the respective starting materials by vibro-milling mixed-oxide method at various soaking time conditions for optimized properties. A composition and microstructure of the ceramics were investigated using X-ray diffraction and scanning electron microscopy. The dielectric properties of the densified samples were examined as a function of temperature and frequency to verify the relationships between soaking time conditions, phase formation, densification, microstructural development, dielectric, and ferroelectric characteristics.
Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt

Inspired by DETR variants, query-based end-to-end instance segmentation (QEIS) methods have recently outperformed CNN-based models on large-scale datasets. Yet they would lose efficacy when only a small amount of training data is available since it's hard for the crucial queries/kernels to learn localization and shape priors. To this end, this work offers a novel unsupervised pre-training solution for low-data regimes. Inspired by the recent success of the Prompting technique, we introduce a new pre-training method that boosts QEIS models by giving Saliency Prompt for queries/kernels. Our method contains three parts: 1) Saliency Masks Proposal is responsible for generating pseudo masks from unlabeled images based on the saliency mechanism. 2) Prompt-Kernel Matching transfers pseudo masks into prompts and injects the corresponding localization and shape priors to the best-matched kernels. 3) Kernel Supervision is applied to supply supervision at the kernel level for robust learning. From a practical perspective, our pre-training method helps QEIS models achieve a similar convergence speed and comparable performance with CNN-based models in low-data regimes. Experimental results show that our method significantly boosts several QEIS models on three datasets.11Code: https://github.com/lifuguan/saliency.prompt
QR-CLIP: Introducing Explicit Open-World Knowledge for Location and Time Reasoning

Daily images may convey abstract meanings that require us to memorize and infer profound information from them. To encourage such human-like reasoning, in this work, we teach machines to predict where and when it was taken rather than performing basic tasks like traditional segmentation or classification. Inspired by Horn's QR theory, we designed a novel QR-CLIP model consisting of two components: 1) the Quantity module first retrospects more open-world knowledge as the candidate language inputs; 2) the Relevance module carefully estimates vision and language cues and infers the location and time. Experiments show our QR-CLIP's effectiveness, and it outperforms the previous SOTA on each task by an average of about 10% and 130% relative lift in terms of location and time reasoning. This study lays a technical foundation for location and time reasoning and suggests that effectively introducing open-world knowledge is one of the panaceas for the tasks.
Mnemosyne: Learning to Train Transformers with Transformers

In this work, we propose a new class of learnable optimizers, called \textit{Mnemosyne}. It is based on the novel spatio-temporal low-rank implicit attention Transformers that can learn to train entire neural network architectures, including other Transformers, without any task-specific optimizer tuning. We show that Mnemosyne: (a) outperforms popular LSTM optimizers (also with new feature engineering to mitigate catastrophic forgetting of LSTMs), (b) can successfully train Transformers while using simple meta-training strategies that require minimal computational resources, (c) matches accuracy-wise SOTA hand-designed optimizers with carefully tuned hyper-parameters (often producing top performing models). Furthermore, Mnemosyne provides space complexity comparable to that of its hand-designed first-order counterparts, which allows it to scale to training larger sets of parameters. We conduct an extensive empirical evaluation of Mnemosyne on: (a) fine-tuning a wide range of Vision Transformers (ViTs) from medium-size architectures to massive ViT-Hs (36 layers, 16 heads), (b) pre-training BERT models and (c) soft prompt-tuning large 11B+ T5XXL models. We complement our results with a comprehensive theoretical analysis of the compact associative memory used by Mnemosyne which we believe was never done before.
Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense

Recent advancements in masked image modeling (MIM) have made it a prevailing framework for self-supervised visual representation learning. The MIM pretrained models, like most deep neural network methods, are still vulnerable to adversarial attacks, limiting their practical application, and this issue has received little research attention. In this paper, we investigate how this powerful self-supervised learning paradigm can provide adversarial robustness to downstream classifiers. During the exploration, we find that noisy image modeling (NIM), a simple variant of MIM that adopts denoising as the pre-text task, reconstructs noisy images surprisingly well despite severe corruption. Motivated by this observation, we propose an adversarial defense method by exploiting the pretrained decoder for denoising, referred to as De^3, through which NIM is able to enhance adversarial robustness beyond providing pretrained features. Furthermore, we incorporate a simple modification, sampling the noise scale hyperparameter from random distributions, and enable the defense to achieve a better and tunable trade-off between accuracy and robustness. Experimental results demonstrate that, in terms of adversarial robustness, NIM is superior compared to MIM thanks to its effective denoising capability. Moreover, the defense provided by NIM achieves performance on par with adversarial training while offering the extra tunability advantage. Source code and models will be made available.
Convergence condition of simulated quantum annealing with a non-stoquastic catalyst

The Ising model with a transverse field and an antiferromagnetic transverse interaction is represented as a matrix in the computational basis with non-zero off-diagonal elements with both positive and negative signs and thus may be regarded to be non-stoquastic. We show that the local Boltzmann factors of such a system under an appropriate Suzuki–Trotter representation can be chosen non-negative and thus may potentially be simulated classically without a sign problem if the parameter values are limited to a subspace of the whole parameter space. We then derive conditions for parameters to satisfy asymptotically in order that simulated quantum annealing of this system converges to thermal equilibrium in the long-time limit.
The big challenge for livestock genomics is to make sequence data pay

This paper will argue that one of the biggest challenges for livestock genomics is to make whole-genome sequencing and functional genomics applicable to breeding practice. It discusses potential explanations for why it is so difficult to consistently improve the accuracy of genomic prediction by means of whole-genome sequence data, and three potential attacks on the problem.
The Cool Circumgalactic Medium of Low-redshift Star-forming Galaxies. I. Empirical Model and Mean Properties

We present an analytic model for the cool, T ∼ 104 K, circumgalactic medium (CGM), describing the gas distribution, and thermal and ionization states. Our model assumes (total) pressure equilibrium with the ambient warm/hot CGM, photoionization by the metagalactic radiation, and allows for nonthermal pressure support, parameterized by the ratio of thermal pressures, η = P hot,th/P cool,th. We apply the model to the COS-Halos measurements and find that a nominal model with η = 3, gas distribution out to r ≈ 0.6R vir, and M cool = 3 × 109 M ⊙, corresponding to a volume filling fraction of f V,cool ≈ 1%, reproduces the H i and low/intermediate metal ions (C ii, C iii, Si ii, Si iii, and Mg ii) mean column densities. Variation of ±0.5 dex in η or M cool encompasses ∼2/3 of the scatter between objects. Our nominal model underproduces the measured C iv and Si iv columns, and these can be reproduced with (i) a cool phase with M cool ∼ 1010 M ⊙ and η ≈ 5, or (ii) cooling or mixing gas at intermediate temperatures, with M ∼ 1.5 × 1010 M ⊙ and occupying ∼1/2 of the total CGM volume. For cool gas with f V,cool ≈ 1%, we estimate an upper limit on the cloud sizes, R cl ≲ 0.5 kpc. Our results suggest that for the average galaxy CGM, the mass and nonthermal support in the cool phase are lower than previously estimated, and extreme scenarios are not necessary. We estimate the rates of cool gas depletion and replenishment, and find accretion onto the galaxy can be offset, allowing Ṁcool≈0 over long timescales.
Diagrammatization: Rationalizing with diagrammatic AI explanations for abductive reasoning on hypotheses

Many visualizations have been developed for explainable AI (XAI), but they often require further reasoning by users to interpret. We argue that XAI should support abductive reasoning — inference to the best explanation — with diagrammatic reasoning to convey hypothesis generation and evaluation. Inspired by Peircean diagrammatic reasoning and the 5-step abduction process, we propose Diagrammatization, an approach to provide diagrammatic, abductive explanations based on domain hypotheses. We implemented DiagramNet for a clinical application to predict diagnoses from heart auscultation, and explain with shape-based murmur diagrams. In modeling studies, we found that DiagramNet not only provides faithful murmur shape explanations, but also has better prediction performance than baseline models. We further demonstrate the usefulness of diagrammatic explanations in a qualitative user study with medical students, showing that clinically-relevant, diagrammatic explanations are preferred over technical saliency map explanations. This work contributes insights into providing domain-conventional abductive explanations for user-centric XAI.
Scale up with Order: Finding Good Data Permutations for Distributed Training

Gradient Balancing (GraB) is a recently proposed technique that ﬁnds provably better data permutations when training models with multiple epochs over a ﬁnite dataset. It converges at a faster rate than the widely adopted Random Reshuﬄing, by minimizing the discrepancy of the gradients on adjacently selected examples. However, GraB only operates under critical assumptions such as small batch sizes and centralized data, leaving open the question of how to order examples at large scale—i.e. distributed learning with decentralized data. To alleviate the limitation, in this paper we propose D-GraB, an algorithm that orders the examples in a parallel setting with negligible overhead, which enjoys linear speed up at rate ˜ O (( mnT ) − 2 / 3 ) on smooth non-convex objectives and ˜ O (( mnT ) − 2 ) under PL condition, where n denotes the number of parallel workers, m denotes the number of examples per worker and T denotes the number of epochs. D-GraB beneﬁts from both data ordering and parallelism. Empirically, we show on various applications including GLUE, CIFAR10 and WikiText-2 that D-GraB outperforms naive parallel GraB and Distributed Random Reshuﬄing in terms of both training and validation performance.
SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling

Time series analysis is widely used in extensive areas. Recently, to reduce labeling expenses and benefit various tasks, self-supervised pre-training has attracted immense interest. One mainstream paradigm is masked modeling, which successfully pre-trains deep models by learning to reconstruct the masked content based on the unmasked part. However, since the semantic information of time series is mainly contained in temporal variations, the standard way of randomly masking a portion of time points will seriously ruin vital temporal variations of time series, making the reconstruction task too difficult to guide representation learning. We thus present SimMTM, a Simple pre-training framework for Masked Time-series Modeling. By relating masked modeling to manifold learning, SimMTM proposes to recover masked time points by the weighted aggregation of multiple neighbors outside the manifold, which eases the reconstruction task by assembling ruined but complementary temporal variations from multiple masked series. SimMTM further learns to uncover the local structure of the manifold, which is helpful for masked modeling. Experimentally, SimMTM achieves state-of-the-art fine-tuning performance compared to the most advanced time series pre-training methods in two canonical time series analysis tasks: forecasting and classification, covering both in- and cross-domain settings.
Sketched Ridgeless Linear Regression: The Role of Downsampling

Overparametrization often helps improve the generalization performance. This paper presents a dual view of overparametrization suggesting that downsampling may also help generalize. Focusing on the proportional regime $m\asymp n \asymp p$, where $m$ represents the sketching size, $n$ is the sample size, and $p$ is the feature dimensionality, we investigate two out-of-sample prediction risks of the sketched ridgeless least square estimator. Our findings challenge conventional beliefs by showing that downsampling does not always harm generalization but can actually improve it in certain cases. We identify the optimal sketching size that minimizes out-of-sample prediction risks and demonstrate that the optimally sketched estimator exhibits stabler risk curves, eliminating the peaks of those for the full-sample estimator. To facilitate practical implementation, we propose an empirical procedure to determine the optimal sketching size. Finally, we extend our analysis to cover central limit theorems and misspecified models. Numerical studies strongly support our theory.
Biophysical aspects of neurocognitive modeling with long-term sustained temperature variations

—Long-term focused attention with visualization and breathing exercises is at the core of various Eastern traditions. Neurocognitive and psychosomatic phenomena demonstrated dur- ing such exercises were instrumentally explored with EEG and other sensors. Neurocognitive modeling in the form of meditative visualization produced persistent temperature effects in the body long after the exercise ﬁnished; this raises the question about their psychosomatic or biophysical origin. The work explores this question by comparing experiments with focusing attention inside and outside the body. EEG, temperature, heart and breathing sensors monitor internal body conditions, high resolution differen- tial calorimetric sensors are used to detect thermal effects outside the body. Experiments with 159 attempts (2427 operator-sensor sessions) were carried over ﬁve months, control measurements run in the same conditions in parallel to experimental series. Increase of body temperature up to moderate fever zone 38.5 ◦ C and intentional control of up and down trend of core temperature by 1.6 ◦ C are demonstrated. Persistent temperature variations last > 60 min. Experiments also demonstrated induced thermal ﬂuctuations at 10 − 3 ◦ C level in external calorimetric systems with 15 ml of water for 60-90 min. Repeatability of these attempts is over 90%, statistical Chi-square and Mann-Whitney tests reject the null hypotheses about random character of outcomes. Thus, the obtained data conﬁrm the persistent thermal effects reported in previous publications and indicate their biophysical dimension. To explain these results we refer to a new model in neuroscience that involves spin phenomena in biochemical and physical systems. These experiments demonstrate complex biophysical mechanisms of altered states of consciousness; their function in the body’s neurohumoral regulation and non-classical brain functions is discussed.
MonoFlow: Rethinking Divergence GANs via the Perspective of Differential Equations

The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs were developed following this paradigm, the current theoretical understanding of GANs and their practical algorithms are inconsistent. In this paper, we leverage Wasserstein gradient ﬂows which characterize the evolution of particles in the sample space, to gain theoretical insights and algorithmic inspiration of GANs. We introduce a uniﬁed generative modeling framework – MonoFlow: the particle evolution is rescaled via a monotonically increasing mapping of the log density ratio. Under our framework, adversarial training can be viewed as a procedure ﬁrst obtaining MonoFlow’s vector ﬁeld via training the discriminator and the generator learns to draw the particle ﬂow deﬁned by the corresponding vector ﬁeld. We also reveal the fundamental difference between variational divergence minimization and adversarial training. This analysis helps us to identify what types of generator loss functions can lead to the successful training of GANs and suggest that GANs may have more loss designs beyond the literature (e.g., non-saturated loss), as long as they realize MonoFlow. Consistent empirical studies are included to validate the effectiveness of our framework.
Adjusting for Incomplete Baseline Covariates in Randomized Controlled Trials: A Cross-World Imputation Framework

In randomized controlled trials, adjusting for baseline covariates is often applied to improve the precision of treatment effect estimation. However, missingness in covariates is common. Recently, Zhao&Ding (2022) studied two simple strategies, the single imputation method and missingness indicator method (MIM), to deal with missing covariates, and showed that both methods can provide efficiency gain. To better understand and compare these two strategies, we propose and investigate a novel imputation framework termed cross-world imputation (CWI), which includes single imputation and MIM as special cases. Through the lens of CWI, we show that MIM implicitly searches for the optimal CWI values and thus achieves optimal efficiency. We also derive conditions under which the single imputation method, by searching for the optimal single imputation values, can achieve the same efficiency as the MIM.
Conditions implying the normality of $\ast$-paranormal operators in the closure of $\mathcal{AN}$-operators

In this article, we ﬁrst prove the existence of an invariant subspace for a norm attaining ∗ -paranormal operator. Then give a representation for ∗ -paranormal operators in the closure of absolutely norm attaining operators and further study a few suﬃcient conditions for the normality of such operators. Finally, we discuss Toeplitz and Hankel ∗ -paranormal operators in the closure of absolutely norm attaining operators on the Hardy space.
Double Permutation Equivariance for Knowledge Graph Completion

This work provides a formalization of Knowledge Graphs (KGs) as a new class of graphs that we denote doubly exchangeable attributed graphs , where node and pairwise (joint 2-node) representations must be equivariant to permutations of both node ids and edge (& node) attributes (relations & node features). Double-permutation equivariant KG representations open a new research direction in KGs. We show that this equiv-ariance imposes a structural representation of relations that allows neural networks to perform complex logical reasoning tasks in KGs. Finally, we introduce a general blueprint for such equivariant representations and test a simple GNN-based double-permutation equivariant neural architecture that achieve state-of-the-art Hits@10 test accuracy in the WN18RR, FB237 and NELL995 inductive KG completion tasks, and can accurately perform logical reasoning tasks that no existing methods can perform, to the best of our knowledge.
Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values

The influential claim that most published results are false raised concerns about the trustworthiness and integrity of science. Since then, there have been numerous attempts to examine the rate of false-positive results that have failed to settle this question empirically. Here we propose a new way to estimate the false positive risk and apply the method to the results of (randomized) clinical trials in top medical journals. Contrary to claims that most published results are false, we find that the traditional significance criterion of α = . 05 produces a false positive risk of 13%. Adjusting α to.01 lowers the false positive risk to less than 5%. However, our method does provide clear evidence of publication bias that leads to inflated effect size estimates. These results provide a solid empirical foundation for evaluations of the trustworthiness of medical research.