Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving Pixel-based MIM by Reducing Wasted Modeling Capability

There has been significant progress in Masked Image Modeling (MIM). Existing MIM methods can be broadly categorized into two groups based on the reconstruction target: pixel-based and tokenizer-based approaches. The former offers a simpler pipeline and lower computational cost, but it is known to be biased toward high-frequency details. In this paper, we provide a set of empirical studies to confirm this limitation of pixel-based MIM and propose a new method that explicitly utilizes low-level features from shallow layers to aid pixel reconstruction. By incorporating this design into our base method, MAE, we reduce the wasted modeling capability of pixel-based MIM, improving its convergence and achieving non-trivial improvements across various downstream tasks. To the best of our knowledge, we are the first to systematically investigate multilevel feature fusion for isotropic architectures like the standard Vision Transformer (ViT). Notably, when applied to a smaller model (e.g., ViT-S), our method yields significant performance gains, such as 1.2% on fine-tuning, 2.8% on linear probing, and 2.6% on semantic segmentation. Code and models are available in MMPretrain1.
Discourse-Aware Text Simplification: From Complex Sentences to Linked Propositions

Sentences that present a complex syntax act as a major stumbling block for downstream Natural Language Processing applications whose predictive quality deteriorates with sentence length and complexity. The task of Text Simplification (TS) may remedy this situation. It aims to modify sentences in order to make them easier to process, using a set of rewriting operations, such as reordering, deletion, or splitting. State-of-the-art syntactic TS approaches suffer from two major drawbacks: first, they follow a very conservative approach in that they tend to retain the input rather than transforming it, and second, they ignore the cohesive nature of texts, where context spread across clauses or sentences is needed to infer the true meaning of a statement. To address these problems, we present a discourse-aware TS approach that splits and rephrases complex English sentences within the semantic context in which they occur. Based on a linguistically grounded transformation stage that uses clausal and phrasal disembedding mechanisms, complex sentences are transformed into shorter utterances with a simple canonical structure that can be easily analyzed by downstream applications. With sentence splitting, we thus address a TS task that has hardly been explored so far. Moreover, we introduce the notion of minimality in this context, as we aim to decompose source sentences into a set of self-contained minimal semantic units. To avoid breaking down the input into a disjointed sequence of statements that is difficult to interpret because important contextual information is missing, we incorporate the semantic context between the split propositions in the form of hierarchical structures and semantic relationships. In that way, we generate a semantic hierarchy of minimal propositions that leads to a novel representation of complex assertions that puts a semantic layer on top of the simplified sentences.
Reinforcement Learning-based Non-Autoregressive Solver for Traveling Salesman Problems

The Traveling Salesman Problem (TSP) is a well-known combinatorial optimization problem with broad real-world applications. Recently, neural networks have gained popularity in this research area because they provide strong heuristic solutions to TSPs. Compared to autoregressive neural approaches, non-autoregressive (NAR) networks exploit the inference parallelism to elevate inference speed but suffer from comparatively low solution quality. In this paper, we propose a novel NAR model named NAR4TSP, which incorporates a specially designed architecture and an enhanced reinforcement learning strategy. To the best of our knowledge, NAR4TSP is the first TSP solver that successfully combines RL and NAR networks. The key lies in the incorporation of NAR network output decoding into the training process. NAR4TSP efficiently represents TSP encoded information as rewards and seamlessly integrates it into reinforcement learning strategies, while maintaining consistent TSP sequence constraints during both training and testing phases. Experimental results on both synthetic and real-world TSP instances demonstrate that NAR4TSP outperforms four state-of-the-art models in terms of solution quality, inference speed, and generalization to unseen scenarios.
Collective behavior of self-steering active particles with velocity alignment and visual perception

The formation and dynamics of swarms is wide spread in living systems, from bacterial bio-films to schools of fish and flocks of birds. We study this emergent collective behavior in a model of active Brownian particles with visual-perception-induced steering and alignment interactions through agent-based simulations. The dynamics, shape, and internal structure of the emergent aggregates, clusters, and swarms of these intelligent active Brownian particles (iABPs) is determined by the maneuverabilities $\Omega_v$ and $\Omega_a$, quantifying the steering based on the visual signal and polar alignment, respectively, the propulsion velocity, characterized by the P{\'e}clet number $Pe$, the vision angle $\theta$, and the orientational noise. Various non-equilibrium dynamical aggregates -- like motile worm-like swarms and millings, and close-packed or dispersed clusters -- are obtained. Small vision angles imply the formation of small clusters, while large vision angles lead to more complex clusters. In particular, a strong polar-alignment maneuverability $\Omega_a$ favors elongated worm-like swarms, which display super-diffusive motion over a much longer time range than individual ABPs, whereas a strong vision-based maneuverability $\Omega_v$ favors compact, nearly immobile aggregates. Swarm trajectories show long persistent directed motion, interrupted by sharp turns. Milling rings, where a worm-like swarm bites its own tail, emerge for an intermediate regime of $Pe$ and vision angles. Our results offer new insights into the behavior of animal swarms, and provide design criteria for swarming microbots.
On the transport of CO$_2$ through humidified facilitated transport membranes

Membrane-based CO$_2$ removal from exhaust streams has recently gained much attention as a means of reducing emissions and limiting climate change. Novel membranes for CO$_2$ removal include so called facilitated transport membranes (FTMs), which offer very high selectivities for CO$_2$ while maintaining decent permeabilities. Recently, these FTMs have been scaled up from laboratory level to plant-sized pilot modules with promising results. However, the molecular details of CO$_2$ transport in these has not yet been fully unraveled. In this work, experimental studies were combined with quantum-mechanical ab initio molecular dynamics simulations to gain insight into the underlying molecular mechanism of CO$_2$ permeation through FTMs. Various compositions of polyvinyl alcohol (PVA) as the membrane matrix with polyvinyl amine (PVAm), monoethanolamine (MEA), or 4-amino-1-butanol (BA) as carrier molecules were experimentally tested. Our experiments revealed that water was essential for the CO$_2$ transport and a transport superposition was achieved with a mixed composition of PVAm and MEA in PVA. Furthermore, sorption measurements with PVA were conducted with humidified N$_2$ and CO$_2$ to quantify water sorption-induced swelling and its contribution to the gas uptake. As the carbonic acid--amine interaction is assumed to cause transport facilitation, electronic structure-based ab initio molecular dynamics simulations were conducted to study the transport of CO$_2$ in the form of carbonic acid along PVAm polymer chains. In particular, the necessity of local water for transport facilitation was studied at different water contents. The simulations show that transport is fastest in the system with low water content and does not happen in the absence of water.
Efficient Multi-Change Point Analysis to Decode Economic Crisis Information from the S&P500 Mean Market Correlation

Identifying macroeconomic events that are responsible for dramatic changes of economy is of particular relevance to understanding the overall economic dynamics. We introduce an open-source available efficient Python implementation of a Bayesian multi-trend change point analysis, which solves significant memory and computing time limitations to extract crisis information from a correlation metric. Therefore, we focus on the recently investigated S&P500 mean market correlation in a period of roughly 20 years that includes the dot-com bubble, the global financial crisis, and the Euro crisis. The analysis is performed two-fold: first, in retrospect on the whole dataset and second, in an online adaptive manner in pre-crisis segments. The online sensitivity horizon is roughly determined to be 80 up to 100 trading days after a crisis onset. A detailed comparison to global economic events supports the interpretation of the mean market correlation as an informative macroeconomic measure by a rather good agreement of change point distributions and major crisis events. Furthermore, the results hint at the importance of the U.S. housing bubble as a trigger of the global financial crisis, provide new evidence for the general reasoning of locally (meta)stable economic states, and could work as a comparative impact rating of specific economic events.
Programmable numerical function generators based on quadratic approximation: architecture and synthesis method

This paper presents architecture and a synthesis method for programmable numerical function generators (NFGs) for trigonometric, logarithmic, square root, and reciprocal functions. Our NFG partitions a given domain of the function into nonuniform segments using a LUT cascade, and approximates the given function by a quadratic polynomial for each segment. Thus, we can implement fast and compact NFGs for a wide range of functions. Implementation results on an FPGA show that: 1) our NFGs require only 4% of the memory needed by NFGs based on the linear approximation with nonuniform segmentation; and 2) our NFGs require only 22% of the memory needed by NFGs based on the 5th-order approximation with uniform segmentation. Our automatic synthesis system generates such compact NFGs quickly
Acoustics of Bubble Arrays: Role Played by the Dipole Response of Bubbles

A model for acoustic transmission through a 2D square crystal of R-radius bubbles with a lattice constant L was previously proposed. Assuming a purely monopole response of the bubbles, this model offers a simple analytical expression of the transmission. However, it is not applicable when the bubbles are too close to each other ( L / R < 5). This article proposes an extension of the model by including the dipole response of the bubbles. Comparisons with numerical and experimental results show that the new expression gives a good estimate of the concentration at which the monopole model is no longer valid, but fails at properly predicting the transmission.
A many-sorted epistemic logic for chromatic hypergraphs

We propose a many-sorted modal logic for reasoning about knowledge in multi-agent systems. Our logic introduces a clear distinction between participating agents and the environment. This allows to express local properties of agents and global properties of worlds in a uniform way, as well as to talk about the presence or absence of agents in a world. The logic subsumes the standard epistemic logic and is a conservative extension of it. The semantics is given in chromatic hypergraphs, a generalization of chromatic simplicial complexes, which were recently used to model knowledge in distributed systems. We show that the logic is sound and complete with respect to the intended semantics. We also show a further connection of chromatic hypergraphs with neighborhood frames.
Isospectral Reductions of Non-negative Matrices

Isospectral reduction is an important tool for network/matrix analysis as it reduces the dimension of a matrix/network while preserving all its eigenvalues and eigenvectors. The main contribution of this manuscript is a proposed algorithmic scheme to approximate the stationary measure of a stochastic matrix based on isospectral reduction. This scheme can be advantageous when there is more than one eigenvalue near 1, precisely the case where iterative methods perform poorly. In addition we give a partial explanation why this scheme should work well, showing that in some situations isospectral reduction improves the spectral gap.
Dispersive Estimates for Maxwell's Equations in the Exterior of a Sphere

The goal of this article is to establish general principles for high frequency dispersive estimates for the $p$-form Laplacian with relative boundary conditions on co-closed forms. In dimension $3$ for the case $p=1$, we show that the propagator corresponding to Maxwell's equations on compactly supported co-closed forms satisfies the same dispersive estimates as in $\mathbb{R}^3$ for the corresponding wave equation in the exterior of a ball -- but only for certain polarizations. In particular we show that some, but not all, polarizations of electromagnetic waves scatter at the same rate as the usual wave equation and this rate is not expected to hold in general. The Dirichlet Laplacian wave equation $L^1-L^{\infty}$ scattering rate does not hold true for the $1-$form Laplacian with relative boundary conditions by itself in the exterior of a sphere, or any smooth obstacle for that matter. We also do not expect it to hold in general for Maxwell's equations because of the presence of $L^2$ harmonic $1$ forms.
Semi-Supervised Laplacian Learning on Stiefel Manifolds

Motivated by the need to address the degeneracy of canonical Laplace learning algorithms in low label rates, we propose to reformulate graph-based semi-supervised learning as a nonconvex generalization of a \emph{Trust-Region Subproblem} (TRS). This reformulation is motivated by the well-posedness of Laplacian eigenvectors in the limit of infinite unlabeled data. To solve this problem, we first show that a first-order condition implies the solution of a manifold alignment problem and that solutions to the classical \emph{Orthogonal Procrustes} problem can be used to efficiently find good classifiers that are amenable to further refinement. Next, we address the criticality of selecting supervised samples at low-label rates. We characterize informative samples with a novel measure of centrality derived from the principal eigenvectors of a certain submatrix of the graph Laplacian. We demonstrate that our framework achieves lower classification error compared to recent state-of-the-art and classical semi-supervised learning methods at extremely low, medium, and high label rates. Our code is available on github\footnote{anonymized for submission}.
Data Augmentation of Bridging the Delay Gap for DL-based Massive MIMO CSI Feedback

In massive multiple-input multiple-output (MIMO) systems under the frequency division duplexing (FDD) mode, the user equipment (UE) needs to feed channel state information (CSI) back to the base station (BS). Though deep learning approaches have made a hit in the CSI feedback problem, whether they can remain excellent in actual environments needs to be further investigated. In this letter, we point out that the real-time dataset in application often has the domain gap from the training dataset caused by the time delay. To bridge the gap, we propose bubble-shift (B-S) data augmentation, which attempts to offset performance degradation by changing the delay and remaining the channel information as much as possible. Moreover, random-generation (R-G) data augmentation is especially proposed for outdoor scenarios due to the complex distribution of its channels. It generalizes the characteristics of the channel matrix and alleviates the over-fitting problem. Simulation results show that the proposed data augmentation boosts the robustness of networks in both indoor and outdoor environments. The open source codes are available at https://github.com/zhanghy23/CRNet-Aug.
Wakey-Wakey: Animate Text by Mimicking Characters in a GIF

With appealing visual effects, kinetic typography (animated text) has prevailed in movies, advertisements, and social media. However, it remains challenging and time-consuming to craft its animation scheme. We propose an automatic framework to transfer the animation scheme of a rigid body on a given meme GIF to text in vector format. First, the trajectories of key points on the GIF anchor are extracted and mapped to the text’s control points based on local affine transformation. Then the temporal positions of the control points are optimized to maintain the text topology. We also develop an authoring tool that allows intuitive human control in the generation process. A questionnaire study provides evidence that the output results are aesthetically pleasing and well preserve the animation patterns in the original GIF, where participants were impressed by a similar emotional semantics of the original GIF. In addition, we evaluate the utility and effectiveness of our approach through a workshop with general users and designers.
Multiscale estimates for the condition number of non-harmonic Fourier matrices

This paper studies the extreme singular values of non-harmonic Fourier matrices. Such a matrix of size $m\times s$ can be written as $\Phi=[ e^{-2\pi i j x_k}]_{j=0,1,\dots,m-1, k=1,2,\dots,s}$ for some set $\mathcal{X}=\{x_k\}_{k=1}^s$. The main results provide explicit lower bounds for the smallest singular value of $\Phi$ under the assumption $m\geq 6s$ and without any restrictions on $\mathcal{X}$. They show that for an appropriate scale $\tau$ determined by a density criteria, interactions between elements in $\mathcal{X}$ at scales smaller than $\tau$ are most significant and depends on the multiscale structure of $\mathcal{X}$ at fine scales, while distances larger than $\tau$ are less important and only depend on the local sparsity of the far away points. Theoretical and numerical comparisons show that the main results significantly improve upon classical bounds and achieve the same rate that was previously discovered for more restrictive settings.
Tuning the Electronic and Mechanical Properties of Two-Dimensional Diamond through N and B Doping

In the present paper, we investigate the structural, thermodynamic, dynamic, elastic, and electronic properties of doped 2D diamond C$_4$X$_2$ (X = B or N) nanosheets in both AA$'$A$''$ and ABC stacking configurations, by first-principles calculations. Those systems are composed of 3 diamond-like graphene sheets, with an undoped graphene layer between two 50% doped ones. Our results, based on the analysis of ab-initio molecular dynamics simulations, phonon dispersion spectra, and Born's criteria for mechanical stability, revealed that all four structures are stable. Additionally, their standard enthalpy of formation values are similar to the one of pristine 2D diamond, recently synthesized by compressing three graphene layers. The C$_4$X$_2$ (X = B or N) systems exhibit high elastic constant values and stiffness comparable to the diamond. The C$_4$N$_2$ nanosheets present wide indirect band gaps that could be advantageous for applications similar to the ones of the hexagonal boron nitride (h-BN), such as a substrate for high-mobility 2D devices. On the other hand, the C$_4$B$_2$ systems are semiconductors with direct band gaps, in the 1.6 - 2.0 eV range, and small effective masses, which are characteristics that may be favorable to high carrier mobility and optoelectronics applications.
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation

Asynchronous Federated Learning with Buffered Aggregation (FedBuff) is a state-of-the-art algorithm known for its efficiency and high scalability. However, it has a high communication cost, which has not been examined with quantized communications. To tackle this problem, we present a new algorithm (QAFeL), with a quantization scheme that establishes a shared"hidden"state between the server and clients to avoid the error propagation caused by direct quantization. This approach allows for high precision while significantly reducing the data transmitted during client-server interactions. We provide theoretical convergence guarantees for QAFeL and corroborate our analysis with experiments on a standard benchmark.
Lee-Yang zeros at $O(3)$ and deconfined quantum critical points

Lee-Yang theory, based on the study of zeros of the partition function, is widely regarded as a powerful and complimentary approach to the study of critical phenomena and forms a foundational part of the theory of phase transitions. Its widespread use, however, is complicated by the fact that it requires introducing complex-valued fields that create an obstacle for many numerical methods, especially in the quantum case where very limited studies exist beyond one dimension. Here we present a simple and statistically exact method to compute partition function zeros with general complex-valued external fields in the context of large-scale quantum Monte Carlo simulations. We demonstrate the power of this approach by extracting critical exponents from the leading Lee-Yang zeros of 2D quantum antiferromagnets with a complex staggered field, focusing on the Heisenberg bilayer and square-lattice $J$-$Q$ models. The method also allows us to introduce a complex field that couples to valence bond solid order, where we observe extended rings of zeros in the $J$-$Q$ model with purely imaginary staggered and valence bond solid fields.
What's Logs Got to do With it: On the Perils of log Dependent Variables and Difference-in-Differences

The log transformation of the dependent variable is not innocuous when using a difference-in-differences (DD) model. With a dependent variable in logs, the DD term captures an approximation of the proportional difference in growth rates across groups. As I show with both simulations and two empirical examples, if the baseline outcome distributions are sufficiently different across groups, the DD parameter for a log-specification can be different in sign to that of a levels-specification. I provide a condition, based on (i) the aggregate time effect, and (ii) the difference in relative baseline outcome means, for when the sign-switch will occur.
Hessian-Aware Bayesian Optimization for Decision Making Systems

Many approaches for optimizing decision making systems rely on gradient based methods requiring informative feedback from the environment. However, in the case where such feedback is sparse or uninformative, such approaches may result in poor performance. Derivative-free approaches such as Bayesian Optimization mitigate the dependency on the quality of gradient feedback, but are known to scale poorly in the high-dimension setting of complex decision making systems. This problem is exacerbated if the system requires interactions between several actors cooperating to accomplish a shared goal. To address the dimensionality challenge, we propose a compact multi-layered architecture modeling the dynamics of actor interactions through the concept of role. We introduce Hessian-aware Bayesian Optimization to efficiently optimize the multi-layered architecture parameterized by a large number of parameters, and give the first improved regret bound in additive high-dimensional Bayesian Optimization since Mutny&Krause (2018). Our approach shows strong empirical results under malformed or sparse reward.