Miracle-dz/UniFAR-FTUs huggingface dataset:
Understanding Implicit Regularization in Over-Parameterized Single Index Model

Abstract In this article, we leverage over-parameterization to design regularization-free algorithms for the high-dimensional single index model and provide theoretical guarantees for the induced implicit regularization phenomenon. Specifically, we study both vector and matrix single index models where the link function is nonlinear and unknown, the signal parameter is either a sparse vector or a low-rank symmetric matrix, and the response variable can be heavy-tailed. To gain a better understanding of the role played by implicit regularization without excess technicality, we assume that the distribution of the covariates is known a priori. For both the vector and matrix settings, we construct an over-parameterized least-squares loss function by employing the score function transform and a robust truncation step designed specifically for heavy-tailed data. We propose to estimate the true parameter by applying regularization-free gradient descent to the loss function. When the initialization is close to the origin and the stepsize is sufficiently small, we prove that the obtained solution achieves minimax optimal statistical rates of convergence in both the vector and matrix cases. In addition, our experimental results support our theoretical findings and also demonstrate that our methods empirically outperform classical methods with explicit regularization in terms of both l2-statistical rate and variable selection consistency. Supplementary materials for this article are available online.
Faster-Than-Nyquist Signaling for MIMO Communications

Faster-than-Nyquist (FTN) signaling is a non-orthogonal transmission technique, which has the potential to provide significant spectral efficiency improvement. This paper studies the capacity of FTN signaling for both frequency flat and for frequency selective (FS) multiple-input multiple-output (MIMO) channels. As FTN is another reason of frequency selectivity, we find that precoding in time (or equivalently spectrum shaping in frequency) and waterfilling in spatial domain is capacity achieving for frequency flat MIMO channels with FTN. Meanwhile, waterfilling in both spatial domain and spectrum domain, followed by spectrum shaping, is capacity achieving for FS MIMO channels with FTN.
Multi-View Motion Synthesis via Applying Rotated Dual-Pixel Blur Kernels

Portrait mode is widely available on smartphone cameras to provide an enhanced photographic experience. One of the primary effects applied to images captured in portrait mode is a synthetic shallow depth of field (DoF). The synthetic DoF (or bokeh effect) selectively blurs regions in the image to emulate the effect of using a large lens with a wide aperture. In addition, many applications now incorporate a new image motion attribute (NIMAT) to emulate background motion, where the motion is correlated with estimated depth at each pixel. In this work, we follow the trend of rendering the NIMAT effect by introducing a modification on the blur synthesis procedure in portrait mode. In particular, our modification enables a high-quality synthesis of multi-view bokeh from a single image by applying rotated blurring kernels. Given the synthesized multiple views, we can generate aesthetically realistic image motion similar to the NIMAT effect. We validate our approach qualitatively compared to the original NIMAT effect and other similar image motions, like Facebook 3D image. Our image motion demonstrates a smooth image view transition with fewer artifacts around the object boundary.
Finite Rate QLDPC-GKP Coding Scheme that Surpasses the CSS Hamming Bound

Quantum error correction has recently been shown to benefit greatly from specific physical encodings of the code qubits. In particular, several researchers have considered the individual code qubits being encoded with the continuous variable GottesmanKitaev-Preskill (GKP) code, and then imposed an outer discrete-variable code such as the surface code on these GKP qubits. Under such a concatenation scheme, the analog information from the inner GKP error correction improves the noise threshold of the outer code. However, the surface code has vanishing rate and demands a lot of resources with growing distance. In this work, we concatenate the GKP code with generic quantum low-density parity-check (QLDPC) codes and demonstrate a natural way to exploit the GKP analog information in iterative decoding algorithms. We first show the noise thresholds for two lifted product QLDPC code families, and then show the improvements of noise thresholds when the iterative decoder – a hardware-friendly min-sum algorithm (MSA) – utilizes the GKP analog information. We also show that, when the GKP analog information is combined with a sequential update schedule for MSA, the scheme surpasses the well-known CSS Hamming bound for these code families. Furthermore, we observe that the GKP analog information helps the iterative decoder in escaping harmful trapping sets in the Tanner graph of the QLDPC code, thereby eliminating or significantly lowering the error floor of the logical error rate curves. Finally, we discuss new fundamental and practical questions that arise from this work on channel capacity under GKP analog information, and on improving decoder design and analysis.
Travelling wave analysis of substrate-mediated cellular invasion

We consider a continuum mathematical model of biological tissue formation inspired by recent experiments describing thin tissue growth in 3D–printed bioscaffolds. The continuum model involves a partial differential equation describing the density of tissue, û(x̂, t̂), that is coupled to the concentration of an immobile extracellular substrate, ŝ(x̂, t̂). Cell migration is modelled with a nonlinear diffusion term, where the diffusive flux is proportional to ŝ, while a logistic growth term models cell proliferation. The extracellular substrate ŝ is produced by cells, and undergoes linear decay. Preliminary numerical simulations show that this mathematical model, which we call the substrate model, is able to recapitulate key features of recent tissue growth experiments, including the formation of sharp fronts. To provide a deeper understanding of the model we then analyse travelling wave solutions of the substrate model, showing that the model supports both sharp–fronted travelling wave solutions that move with a minimum wave speed, c = cmin, as well as smooth–fronted travelling wave solutions that move with a faster travelling wave speed, c > cmin. We provide a geometric interpretation that explains the difference between smooth– and sharp–fronted travelling wave solutions that is based on a slow manifold reduction of the desingularised three–dimensional phase space. In addition to exploring the nature of the smooth– and sharp–fronted travelling waves, we also develop and test a series of useful approximations that describe the shape of the travelling wave solutions in various limits. These approximations apply to both the sharp–fronted travelling wave solutions, and the smooth–fronted travelling wave solutions. Software to implement all calculations is available at GitHub.
Improved Decoding of Expander Codes

We study the classical expander codes, introduced by Sipser and Spielman, (1996). Given any constants <inline-formula> <tex-math notation="LaTeX">$0 < \alpha, \varepsilon < 1/2$ </tex-math></inline-formula>, and an arbitrary bipartite graph with <inline-formula> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> vertices on the left, <inline-formula> <tex-math notation="LaTeX">$M < N$ </tex-math></inline-formula> vertices on the right, and left degree <inline-formula> <tex-math notation="LaTeX">$D$ </tex-math></inline-formula> such that any left subset <inline-formula> <tex-math notation="LaTeX">$S$ </tex-math></inline-formula> of size at most <inline-formula> <tex-math notation="LaTeX">$\alpha N$ </tex-math></inline-formula> has at least <inline-formula> <tex-math notation="LaTeX">$(1- \varepsilon)|S|D$ </tex-math></inline-formula> neighbors, we show that the corresponding linear code given by parity checks on the right has distance at least roughly <inline-formula> <tex-math notation="LaTeX">$\frac {\alpha N}{2 \varepsilon }$ </tex-math></inline-formula>. This is strictly better than the best known previous result of <inline-formula> <tex-math notation="LaTeX">$2(1- \varepsilon) \alpha N$ </tex-math></inline-formula> Sudan, (2000), Viderman, (2013) whenever <inline-formula> <tex-math notation="LaTeX">$\varepsilon < 1/2$ </tex-math></inline-formula>, and improves the previous result significantly when <inline-formula> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula> is small. Furthermore, we show that this distance is tight in general, thus providing a complete characterization of the distance of general expander codes. Next, we provide several efficient decoding algorithms, which vastly improve previous results in terms of the fraction of errors corrected, whenever <inline-formula> <tex-math notation="LaTeX">$\varepsilon < \frac {1}{4}$ </tex-math></inline-formula>. Finally, we also give a bound on the list-decoding radius of general expander codes, which beats the classical Johnson bound in certain situations (e.g., when the graph is almost regular and the code has a high rate). Our techniques exploit novel combinatorial properties of bipartite expander graphs. In particular, we establish a new size-expansion tradeoff, which may be of independent interests.
Renormalization group for open quantum systems using environment temperature as flow parameter

We present the T-flow renormalization group method, which computes the memory kernel for the density-operator evolution of an open quantum system by lowering the physical temperature T of its environment. This has the key advantage that it can be formulated directly in real time, making it particularly suitable for transient dynamics, while automatically accumulating the full temperature dependence of transport quantities. We solve the T-flow equations numerically for the example of the single impurity Anderson model. We benchmark in the stationary limit, readily accessible in real-time for voltages on the order of the coupling or larger using results obtained by the functional renormalization group, density-matrix renormalization group and the quantum Monte Carlo method. Here we find quantitative agreement even in the worst case of strong interactions and low temperatures, indicating the reliability of the method. For transient charge currents we find good agreement with results obtained by the 2PI Green's function approach. Furthermore, we analytically show that the short-time dynamics of both local and non-local observables follow a "universal" temperature-independent behaviour when the metallic reservoirs have a flat wide band.
NICER view on holographic QCD

The holographic models for dense QCD matter work surprisingly well. A general implication seems that the deconfinement phase transition dictates the maximum mass of neutron stars. The nuclear matter phase turns out to be rather stiff which, if continuously merged with nuclear matter models based on effective field theories, leads to the conclusion that neutron stars do not have quark matter cores in the light of all current astrophysical data. We comment that as the perturbative QCD results are in stark contrast with strong coupling results, any future simulations of neutron star mergers incorporating corrections beyond ideal fluid should proceed cautiously. For this purpose, we provide a model which treats nuclear and quark matter phases in a unified framework at strong coupling.
A Finite-Sampling, Operational Domain Specific, and Provably Unbiased Connected and Automated Vehicle Safety Metric

A connected and automated vehicle safety metric determines the performance of a subject vehicle (SV) by analyzing the data involving the interactions among the SV and other dynamic road users and environmental features. When the data set contains only a finite set of samples collected from the naturalistic mixed multi-modal traffic driving environment, a metric is expected to generalize the safety assessment outcome from the observed finite samples to the unobserved cases by specifying in what domain the SV is expected to be safe and how safe the SV is, statistically, in that domain. However, to the best of our knowledge, none of the existing safety metrics is able to justify the above properties with an operational domain specific, guaranteed complete, and provably unbiased safety evaluation outcome. In this paper, we propose a novel safety metric that involves the $\alpha $ -shape and the $\epsilon $ -almost robustly forward invariant set to characterize the SV’s almost safe operable domain and the probability for the SV to remain inside the safe domain indefinitely, respectively. The empirical performance of the proposed method is demonstrated in several different operational design domains through a series of cases covering a variety of fidelity levels (real-world and simulators), driving environments (highway, urban, and intersections), road users (car, truck, and pedestrian), and SV driving behaviors (human driver and self driving algorithms).
Learning to Evolve on Dynamic Graphs

Representation learning in dynamic graphs is a challenging problem because the topology of graph and node features vary at different time. This requires the model to be able to effectively capture both graph topology information and temporal information. Most existing works are built on recurrent neural networks (RNNs), which are used to exact temporal information of dynamic graphs, and thus they inherit the same drawbacks of RNNs. In this paper, we propose Learning to Evolve on Dynamic Graphs (LEDG) - a novel algorithm that jointly learns graph information and time information. Specifically, our approach utilizes gradient-based meta-learning to learn updating strategies that have better generalization ability than RNN on snapshots. It is model-agnostic and thus can train any message passing based graph neural network (GNN) on dynamic graphs. To enhance the representation power, we disentangle the embeddings into time embeddings and graph intrinsic embeddings. We conduct experiments on various datasets and down-stream tasks, and the experimental results validate the effectiveness of our method.
Electrons flow like falling cats: Deformations and emergent gravity in quantum transport

The effective low-energy excitations in a metallic or semimetallic crystalline system (i.e. electronic quasiparticles) always have a finite spatial extent. It is self-evident but virtually unexplored how the associated internal degrees of freedom manifest themselves in the quasiparticle response. Here, we investigate this question by illuminating an intimate connection between the theory of nonlinear response and the equations of motion of classical deformable bodies. This connection establishes that nth-order response in an external perturbation corresponds to nth-order derivatives of the quasiparticle motion, where the resulting motion is anomalous at every order due to the internal degrees of freedom. This new point of view predicts that quasiparticles necessarily move in an emergent curved spacetime, even in a homogeneous and defectless lattice. We underscore these concepts using recent results on the second order electrical conductivity, elucidating the associated anomalous acceleration that the quasiparticle exhibits. Based on our observations, we predict the existence of an infinite series of anomalous components of the quasiparticle motion, and propose a new mapping between response theory in flat space and a gravitational theory for the center-of-mass coordinate.
Universal voltage-mode filter with four inputs and one output using two CCII s

A novel universal voltage-mode second-order filter with four inputs and one output employing only two plus-type second-generation current conveyors (CCII+s), two capacitors and three resistors is presented. The proposed circuit offers the advantageous features: realization of lowpass, bandpass, highpass, notch and allpass signals from the same configuration, orthogonal control of ω0 and Q and low active and passive sensitivities. The use of only CCII+s simplifies the configuration except for class-A operation.
iBOT: Image BERT Pre-Training with Online Tokenizer

The success of language Transformers is primarily attributed to the pretext task of masked language modeling (MLM), where texts are first tokenized into semantically meaningful pieces. In this work, we study masked image modeling (MIM) and indicate the advantages and challenges of using a semantically meaningful visual tokenizer. We present a self-supervised framework iBOT that can perform masked prediction with an online tokenizer. Specifically, we perform self-distillation on masked patch tokens and take the teacher network as the online tokenizer, along with self-distillation on the class token to acquire visual semantics. The online tokenizer is jointly learnable with the MIM objective and dispenses with a multi-stage training pipeline where the tokenizer needs to be pre-trained beforehand. We show the prominence of iBOT by achieving an 82.3% linear probing accuracy and an 87.8% fine-tuning accuracy evaluated on ImageNet-1K. Beyond the state-of-the-art image classification results, we underline emerging local semantic patterns, which helps the models to obtain strong robustness against common corruptions and achieve leading results on dense downstream tasks, eg. , object detection, instance segmentation, and semantic segmentation.
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement in the coded aperture snapshot spectral imaging (CASSI) system. The HSI representations are highly similar and correlated across the spectral dimension. Modeling the inter-spectra interactions is beneficial for HSI reconstruction. However, existing CNN-based methods show limitations in capturing spectral-wise similarity and long-range dependencies. Besides, the HSI information is modulated by a coded aperture (physical mask) in CASSI. Nonetheless, current algorithms have not fully explored the guidance effect of the mask for HSI restoration. In this paper, we propose a novel framework, Mask-guided Spectral-wise Transformer (MST), for HSI reconstruction. Specifically, we present a Spectral-wise Multi-head Self-Attention (S-MSA) that treats each spectral feature as a token and calculates self-attention along the spectral dimension. In addition, we customize a Mask-guided Mechanism (MM) that directs S- MSA to pay attention to spatial regions with high-fidelity spectral representations. Extensive experiments show that our MST significantly outperforms state-of-the-art (SOTA) methods on simulation and real HSI datasets while requiring dramatically cheaper computational and memory costs. https://github.com/caiyuanhao1998/MST/
Energy-optimal Design and Control of Electric Powertrains under Motor Thermal Constraints

This paper presents a modeling and optimization framework to minimize the energy consumption of a fully electric powertrain by optimizing its design and control strategies whilst explicitly accounting for the thermal behavior of the Electric Motor (EM). Specifically, we first derive convex models of the powertrain components, including the battery, the EM, the transmission and a Lumped Parameter Thermal Network (LPTN) capturing the thermal dynamics of the EM. Second, we frame the optimal control problem in time domain, and devise a two-step algorithm to accelerate convergence and efficiently solve the resulting convex problem via nonlinear programming. Subsequently, we present a case study for a compact family car, optimize its transmission design and operation jointly with the regenerative braking and EM cooling control strategies for a finite number of motors and transmission technologies. We validate our proposed models using the high-fidelity simulation software Motor-CAD, showing that the LPTN quite accurately captures the thermal dynamics of the EM, and that the permanent magnets' temperature is the limiting factor during extended driving. Furthermore, our results reveal that powertrains equipped with a continuously variable transmission (CVT) result into a lower energy consumption than with a fixed-gear transmission (FGT), as a CVT can lower the EM losses, resulting in lower EM temperatures. Finally, our results emphasize the significance of considering the thermal behavior when designing an EM and the potential offered by CVTs in terms of downsizing.
Symmetric states and dynamics of three quantum bits

The unitary group acting on the Hilbert space ${\cal H}:=(C^2)^{\otimes 3}$ of three quantum bits admits a Lie subgroup, $U^{S_3}(8)$, of elements which permute with the symmetric group of permutations of three objects. Under the action of such a Lie subgroup, the Hilbert space ${\cal H}$ splits into three invariant subspaces of dimensions $4$, $2$ and $2$ respectively, each corresponding to an irreducible representation of $su(2)$. The subspace of dimension $4$ is uniquely determined and corresponds to states that are themselves invariant under the action of the symmetric group. This is the so called {\it symmetric sector. } The subspaces of dimension two are not uniquely determined and we parametrize them all. We provide an analysis of pure states that are in the subspaces invariant under $U^{S_3}(8)$. This concerns their entanglement properties, separability criteria and dynamics under the Lie subgroup $U^{S_3}(8)$. As a physical motivation for the states and dynamics we study, we propose a physical set-up which consists of a symmetric network of three spin $\frac{1}{2}$ particles under a common driving electro-magnetic field. {For such system, we solve the control theoretic problem of driving a separable state to a state with maximal distributed entanglement.
Fast adjoint differentiation of chaos

We devise a fast algorithm for the gradient of the long-timeaverage statistics of chaotic systems, with cost almost independent of the number of parameters. It runs on one sample orbit; its cost is linear to the unstable dimension. The algorithm is based on two theoretical results in this paper: the adjoint shadowing lemma for the shadowing contribution and the fast adjoint formula for the unstable divergence in the linear response.
Fast radio burst distributions consistent with the first CHIME/FRB catalog

Currently, fast radio bursts (FRBs) have become a very active field in astronomy and cosmology. However, the origin of FRBs is still unknown to date. The studies on the intrinsic FRB distributions might help us to reveal the possible origins of FRBs, and improve the simulations for FRB cosmology. Recently, the first CHIME/FRB catalog of 536 events was released. Such a large uniform sample of FRBs detected by a single telescope is very valuable to test the FRB distributions. Later, it has been claimed that the FRB distribution model tracking the cosmic star formation history (SFH) was rejected by the first CHIME/FRB catalog. In the present work, we consider some empirical FRB distribution models, and find that many of them can be fully consistent with the CHIME/FRB observational data for some suitable model parameters. Notice that a suppressed evolution with respect to SFH is commonly found for FRBs. In particular, we independently confirm that the FRB distribution model tracking SFH can be rejected at very high confidence. On the other hand, all the "successful" models effectively require a certain degree of "delay" with respect to SFH. These results might shed light on the origin of FRBs and FRB cosmology.