Miracle-dz/UniFAR-FTUs huggingface dataset:
NON-HARMONIC DNA BENDING ELASTICITY IS REVEALED BY STATISTICS OF DNA MINICIRCLE SHAPES

Unusual DNA elasticity has been observed in a few recent experiments for DNA under tight bending conditions. Here we present new experimental evidence demonstrating unusual DNA bending elasticity by analyzing the atomic force microscope (AFM) images of DNA minicircles of sizes 189 and 378 base-pairs that are relaxed on 2D surface. The DNA shape distributions are found in disagreement with the canonical 50 nm persistence length based worm-like chain (WLC) model. Instead, they are found in agreement with the recently proposed flexible defect excitation model and linear subelastic chain (LSEC) model.
Determination of a time‐ and space‐dependent heat flux relaxation function by means of a restricted Dirichlet‐to‐Neumann operator

We consider an inverse problem to recover a space‐ and time‐dependent relaxation function of heat flux in a three‐dimensional body on the basis of the restriction of the Dirichlet‐to‐Neumann operator of the related equation of heat flow onto a set of Dirichlet data of the form of a product of a fixed time‐dependent coefficient and a free space‐dependent function. Uniqueness of the solution of this inverse problem is proved. Copyright © 2004 John Wiley & Sons, Ltd.
Contribution of higher-order nonlinearity to nonlinear ion-acoustic waves in a weakly relativistic warm plasma. Part 2. Non-isothermal case

The contribution of higher-order nonlinearity to nonlinear ion-acoustic waves in a weakly relativistic plasma consisting of a warm ion fluid and hot non- isothermal electrons is studied using reductive perturbation theory. At the lowest order of the perturbation theory a modified Korteweg–de Vries equation is obtained. At the next order a linear inhomogeneous equation is obtained. The stationary solution of the coupled equations is obtained using the renormalization method introduced by Kodama and Taniuti for reductive perturbation theory.
IDENTIFICATION OF COVARIANCE STRUCTURES

The issue of identification of covariance structures, which arises in a number of different contexts, has been so far linked to conditions on the true parameters to be estimated. In this paper, this limitation is removed. As done by Johansen (1995, Journal of Econometrics 69, 112–132) in the context of linear models, the present paper provides necessary and sufficient conditions for the identification of a covariance structure that depends only on the constraints and can therefore be checked independently of estimated parameters. A structure condition is developed, which only depends on the structure of the constraints. It is shown that this condition, if coupled with the familiar order condition, provides a sufficient condition for identification. In practice, because the structure condition holds if and only if a certain matrix, constructed from the constraint matrices, is invertible, automatic software checking for identification is feasible even for large-scale systems. Most of the paper focuses on structural vector autoregressions, but extensions to other statistical models are also briefly discussed.I thank all the participants at the meeting held in Pavia on June 11, 2004, in honor of Carlo Giannini for their comments; it goes without saying that Carlo himself provided not only acute observations on the day but also the main inspiration for this piece of work. Sadly, Carlo passed away on September 11, 2004, and this paper is dedicated to his memory. Pär Österholm spotted several mistakes in an earlier version and helped me clarify some implementation details. Thanks are also due to Gianni Amisano, Bruce Hansen, Giulio Palomba, Paolo Paruolo, and two anonymous referees. The usual disclaimer obviously applies.
Effort discrimination and curvature of contest technology in conflict networks

In a model of interconnected conflicts on a network, we compare the equilibrium effort profiles and payoffs under two scenarios: uniform effort (UE) in which each contestant is restricted to exert the same effort across all the battles she participates, and discriminatory effort (DE) in which such a restriction is lifted. When the contest technology in each battle is of Tullock form, a surprising neutrality result holds within the class of semi-symmetric conflict network structures: both the aggregate actions and equilibrium payoffs under two regimes are the same. We also show that, in some sense, the Tullock form is necessary for such a neutrality result. Moving beyond the Tullock family, we further demonstrate how the curvature of contest technology shapes the welfare and effort effects.
RetVec: Resilient and Efficient Text Vectorizer

This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec.
Temporal Interpolation is all You Need for Dynamic Neural Radiance Fields

Temporal interpolation often plays a crucial role to learn meaningful representations in dynamic scenes. In this paper, we propose a novel method to train spatiotemporal neural radiance fields of dynamic scenes based on temporal interpolation of feature vectors. Two feature interpolation methods are suggested depending on underlying representations, neural networks or grids. In the neural representation, we extract features from space-time inputs via multiple neural network modules and interpolate them based on time frames. The proposed multi-level feature interpolation network effectively captures features of both short-term and long-term time ranges. In the grid representation, space-time features are learned via four-dimensional hash grids, which remarkably reduces training time. The grid representation shows more than 100x faster training speed than the previous neural-net-based methods while maintaining the rendering quality. Concatenating static and dynamic features and adding a simple smoothness term further improve the performance of our proposed models. Despite the simplicity of the model architectures, our method achieved state-of-the-art performance both in rendering quality for the neural representation and in training speed for the grid representation.
Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics

We explore the efficacy of multimodal behavioral cues for explainable prediction of personality and interview-specific traits. We utilize elementary head-motion units named kinemes, atomic facial movements termed action units and speech features to estimate these human-centered traits. Empirical results confirm that kinemes and action units enable discovery of multiple trait-specific behaviors while also enabling explainability in support of the predictions. For fusing cues, we explore decision and feature-level fusion, and an additive attention-based fusion strategy which quantifies the relative importance of the three modalities for trait prediction. Examining various long-short term memory (LSTM) architectures for classification and regression on the MIT Interview and First Impressions Candidate Screening (FICS) datasets, we note that: (1) Multimodal approaches outperform unimodal counterparts; (2) Efficient trait predictions and plausible explanations are achieved with both unimodal and multimodal approaches, and (3) Following the thin-slice approach, effective trait prediction is achieved even from two-second behavioral snippets.
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization

Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance. The Sharpness-Aware Minimization (SAM) technique modifies the fundamental loss function that steers gradient descent methods toward flatter minima, which are believed to exhibit enhanced generalization prowess. Our study delves into a specific variant of SAM known as micro-batch SAM (mSAM). This variation involves aggregating updates derived from adversarial perturbations across multiple shards (micro-batches) of a mini-batch during training. We extend a recently developed and well-studied general framework for flatness analysis to theoretically show that SAM achieves flatter minima than SGD, and mSAM achieves even flatter minima than SAM. We provide a thorough empirical evaluation of various image classification and natural language processing tasks to substantiate this theoretical advancement. We also show that contrary to previous work, mSAM can be implemented in a flexible and parallelizable manner without significantly increasing computational costs. Our implementation of mSAM yields superior generalization performance across a wide range of tasks compared to SAM, further supporting our theoretical framework.
Learning Language Representations with Logical Inductive Bias

Transformer architectures have achieved great success in solving natural language tasks, which learn strong language representations from large-scale unlabeled texts. In this paper, we seek to go further beyond and explore a new logical inductive bias for better language representation learning. Logic reasoning is known as a formal methodology to reach answers from given knowledge and facts. Inspired by such a view, we develop a novel neural architecture named FOLNet (First-Order Logic Network), to encode this new inductive bias. We construct a set of neural logic operators as learnable Horn clauses, which are further forward-chained into a fully differentiable neural architecture (FOLNet). Interestingly, we find that the self-attention module in transformers can be composed by two of our neural logic operators, which probably explains their strong reasoning performance. Our proposed FOLNet has the same input and output interfaces as other pretrained models and thus could be pretrained/finetuned by using similar losses. It also allows FOLNet to be used in a plug-and-play manner when replacing other pretrained models. With our logical inductive bias, the same set of ``logic deduction skills'' learned through pretraining are expected to be equally capable of solving diverse downstream tasks. For this reason, FOLNet learns language representations that have much stronger transfer capabilities. Experimental results on several language understanding tasks show that our pretrained FOLNet model outperforms the existing strong transformer-based approaches.
Trends in electrospinning of natural nanofibers

Nanotechnology has become in recent years a topic of great interest to scientists and engineers, and is now established as prioritized research area in many countries. The reduction of the size to the nanometer range brings an array of new possibilities in terms of material properties, in particular with respect to achievable surface to volume ratios. Electrospinning of natural fibers is a novel process for producing superfine fibers by forcing a solution through a spinneret with an electric field. An experimental study on this technique has been made in this paper and the effect of systematic parameters on electrospun nanofibers were critically analyzed. Based on this study, the key process parameters have been identified according to our experimental observations. (© 2007 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)
Beamforming and Phase Shift Design for HR-IRS-aided Directional Modulation Network with a Malicious Attacker

In this paper, we propose to use hybrid relay-intelligent reflecting surface (HR-IRS) to improve the security performance of directional modulation (DM) system. In particular, the eavesdropper in this system works in full-duplex (FD) mode and he will eavesdrop on the confidential message (CM) as well as send malicious jamming. We aim to maximize the secrecy rate (SR) by jointly optimizing the receive beamforming, transmit beamforming and phase shift matrix (PSM) of HR-IRS. Since the optimization problem is un-convex and the variables are coupled to each other, we solve this problem by iteratively optimizing these variables. The receive beamforming and transmit beamforming are obtained based on generalized Rayleigh-Ritz theorem and Dinkelbach's Transform respectively. And for PSM, two methods, called separate optimization of PSM (SO-PSM) and joint optimization of PSM (JO-PSM) are proposed. Thus, two iterative algorithms are proposed accordingly, namely maximizing SR based on SO-PSM (Max-SR-SOP) and maximizing SR based on JO-PSM (Max-SR-JOP). The former has better performance and the latter has lower complexity. The simulation results show that when HR-IRS has sufficient power budget, the proposed Max-SR-SOP and Max-SR-JOP can enable HR-IRS-aided DM network to obtain higher SR than passive IRS-aided DM network.
New methods of observation and characterization of fractional quantum Hall states

In this work we study new ways to observe and characterize specific fractional quantum Hall (FQH) states. In the first chapter we investigate the possibility to realize specific FQH states in bilayer graphene (BLG). BLG is a novel material in which the electron-electron interaction can be tuned with the help of external parameters. This allows one to make one or another FQH state favourable. We develop a framework for theoretical investigation of the stability of FQH states in BLG. We apply our framework to investigate the stability of the Pfaffian state. We find that the region in which our framework allows for making reliable predictions is quite restricted because of Landau level mixing effects. However, within that region we find the conditions under which the Pfaffian is more stable than in the conventional "non-relativistic" systems. These conditions can, in principle, be realized experimentally. In the second chapter we focus on characterizing the FQH states with the help of measurements of the noise of the electric current tunnelling between two FQH edges. We develop a theoretical framework allowing for analysing such data, and test it by successfully applying it to describe the results of the experiment [Bid et al., Nature 466, 585 (2010)]. We further develop our framework and show that it is possible to determine the tunnelling quasiparticle scaling dimension from such measurements. We also investigate experimental conditions necessary for this.
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

Sparsely gated Mixture-of-Expert (MoE) has demonstrated its effectiveness in scaling up deep neural networks to an extreme scale. Despite that numerous efforts have been made to improve the performance of MoE from the model design or system optimization perspective, existing MoE dispatch patterns are still not able to fully exploit the underlying heterogeneous network environments. In this paper, we propose TA-MoE, a topology-aware routing strategy for large-scale MoE trainging, from a model-system co-design perspective, which can dynamically adjust the MoE dispatch pattern according to the network topology. Based on communication modeling, we abstract the dispatch problem into an optimization objective and obtain the approximate dispatch pattern under different topologies. On top of that, we design a topology-aware auxiliary loss, which can adaptively route the data to fit in the underlying topology without sacrificing the model accuracy. Experiments show that TA-MoE can substantially outperform its counterparts on various hardware and model configurations, with roughly 1.01x-1.61x, 1.01x-4.77x, 1.25x-1.54x improvements over the popular DeepSpeed-MoE, FastMoE and FasterMoE.
Reconstruction of Sequences Distorted by Two Insertions

Reconstruction codes are generalizations of error-correcting codes that can correct errors by a given number of noisy reads. The study of such codes was initiated by Levenshtein in 2001 and developed recently due to applications in modern storage devices such as racetrack memories and DNA storage. The central problem on this topic is to design codes with redundancy as small as possible for a given number <inline-formula> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> of noisy reads. In this paper, the minimum redundancy of such codes for binary channels with exactly two insertions is determined asymptotically for all values of <inline-formula> <tex-math notation="LaTeX">$N\ge 5$ </tex-math></inline-formula>. Previously, such codes were studied only for channels with single edit errors or two-deletion errors.
Orbital Operations for Phobos and Deimos Exploration

One of the deep-space human exploration activities proposed for the post-Shuttle era is a mission to one of the moons of Mars: Phobos or Deimos. There are several options available to the mission architect for operations around these bodies. These options include distant retrograde orbits (DROs), Lagrange-point orbits such as halos and Lyapunov orbits, and fixed-point station keeping or “hovering. ” These three orbit options are discussed in the context of the idealized circular restricted three body problem, full-dynamics propagations, and a concept of operations. The discussion is focused on Phobos, but all results hold for Deimos.
Structural Neural Additive Models: Enhanced Interpretable Machine Learning

Deep neural networks (DNNs) have shown exceptional performances in a wide range of tasks and have become the go-to method for problems requiring high-level predictive power. There has been extensive research on how DNNs arrive at their decisions, however, the inherently uninterpretable networks remain up to this day mostly unobservable"black boxes". In recent years, the field has seen a push towards interpretable neural networks, such as the visually interpretable Neural Additive Models (NAMs). We propose a further step into the direction of intelligibility beyond the mere visualization of feature effects and propose Structural Neural Additive Models (SNAMs). A modeling framework that combines classical and clearly interpretable statistical methods with the predictive power of neural applications. Our experiments validate the predictive performances of SNAMs. The proposed framework performs comparable to state-of-the-art fully connected DNNs and we show that SNAMs can even outperform NAMs while remaining inherently more interpretable.
Approximate Thompson Sampling via Epistemic Neural Networks

Thompson sampling (TS) is a popular heuristic for action selection, but it requires sampling from a posterior distribution. Unfortunately, this can become computationally intractable in complex environments, such as those modeled using neural networks. Approximate posterior samples can produce effective actions, but only if they reasonably approximate joint predictive distributions of outputs across inputs. Notably, accuracy of marginal predictive distributions does not suffice. Epistemic neural networks (ENNs) are designed to produce accurate joint predictive distributions. We compare a range of ENNs through computational experiments that assess their performance in approximating TS across bandit and reinforcement learning environments. The results indicate that ENNs serve this purpose well and illustrate how the quality of joint predictive distributions drives performance. Further, we demonstrate that the \textit{epinet} -- a small additive network that estimates uncertainty -- matches the performance of large ensembles at orders of magnitude lower computational cost. This enables effective application of TS with computation that scales gracefully to complex environments.
Guided Deep Kernel Learning

Combining Gaussian processes with the expressive power of deep neural networks is commonly done nowadays through deep kernel learning (DKL). Unfortunately, due to the kernel optimization process, this often results in losing their Bayesian benefits. In this study, we present a novel approach for learning deep kernels by utilizing infinite-width neural networks. We propose to use the Neural Network Gaussian Process (NNGP) model as a guide to the DKL model in the optimization process. Our approach harnesses the reliable uncertainty estimation of the NNGPs to adapt the DKL target confidence when it encounters novel data points. As a result, we get the best of both worlds, we leverage the Bayesian behavior of the NNGP, namely its robustness to overfitting, and accurate uncertainty estimation, while maintaining the generalization abilities, scalability, and flexibility of deep kernels. Empirically, we show on multiple benchmark datasets of varying sizes and dimensionality, that our method is robust to overfitting, has good predictive performance, and provides reliable uncertainty estimations.
Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE

This technical report briefly describes our JDExplore d-team's submission Vega v1 on the General Language Understanding Evaluation (GLUE) leaderboard, where GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference. [Method] We investigate several effective strategies and choose their best combination setting as the training recipes. As for model structure, we employ the vanilla Transformer with disentangled attention as the basic block encoder. For self-supervised training, we employ the representative denoising objective (i.e., replaced token detection) in phase 1 and combine the contrastive objective (i.e., sentence embedding contrastive learning) with it in phase 2. During fine-tuning, several advanced techniques such as transductive fine-tuning, self-calibrated fine-tuning, and adversarial fine-tuning are adopted. [Results] According to our submission record (Jan. 2022), with our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3. Encouragingly, our Vega v1 is the first to exceed powerful human performance on the two challenging tasks, i.e., SST-2 and WNLI. We believe our empirically successful recipe with a bag of tricks could shed new light on developing efficient discriminative large language models.
Dynamics of growing carbon nanotube interfaces probed by machine learning-enabled molecular simulations

Carbon nanotubes (CNTs) are currently considered a successor to silicon in future nanoelectronic devices. To realize this, controlled growth of defect-free nanotubes is required. Until now, the understanding of atomic-scale CNT growth mechanisms provided by molecular dynamics simulations has been hampered by their short timescales. Here, we develop an efficient and accurate machine learning force field, DeepCNT-22, to simulate the complete growth of defect-free single-walled CNTs (SWCNTs) on iron catalysts at near-microsecond timescales. We provide atomic-level insight into the nucleation and growth processes of SWCNTs, including the evolution of the tube-catalyst interface and the mechanisms underlying defect formation and healing. Our simulations highlight the maximization of SWCNT-edge configurational entropy during growth and how defect-free CNTs can grow ultralong if carbon supply and temperature are carefully controlled.