Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Minimax Rate of the Gaussian Sequence Model Under Bounded Convex Constraints

We determine the exact minimax rate of a Gaussian sequence model under bounded convex constraints, purely in terms of the local geometry of the given constraint set <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>. Our main result shows that the minimax risk<inline-formula> <tex-math notation="LaTeX">$\vphantom {_{\int _{\int }}}$ </tex-math></inline-formula> (up to constant factors) under the squared <inline-formula> <tex-math notation="LaTeX">$\ell _{2}$ </tex-math></inline-formula> loss is given by <inline-formula> <tex-math notation="LaTeX">$\varepsilon ^{*2} \wedge \mathrm {diam} (K)^{2}$ </tex-math></inline-formula> with <inline-formula> <tex-math notation="LaTeX">$\varepsilon ^{*} = \sup \bigg \{\varepsilon: ({\varepsilon ^{2}}/{\sigma ^{2}}) \leq \log M^{\mathrm {loc}}(\varepsilon)\bigg \}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$\log M^{\mathrm {loc}}(\varepsilon)$ </tex-math></inline-formula> denotes the local entropy of the set <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">$\sigma ^{2}$ </tex-math></inline-formula> is the variance of the noise. We utilize our abstract result to re-derive known minimax rates for some special sets <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> such as hyperrectangles, ellipses, and more generally quadratically convex orthosymmetric sets. Finally, we extend our results to the unbounded case with known <inline-formula> <tex-math notation="LaTeX">$\sigma ^{2}$ </tex-math></inline-formula> to show that the minimax rate in that case is <inline-formula> <tex-math notation="LaTeX">$\varepsilon ^{*2}$ </tex-math></inline-formula>.
Self-Supervised Deep Blind Video Super-Resolution

Existing deep learning-based video super-resolution (SR) methods usually depend on the supervised learning approach, where the training data is usually generated by the blurring operation with known or predefined kernels (e.g., Bicubic kernel) followed by a decimation operation. However, this does not hold for real applications as the degradation process is complex and cannot be approximated by these idea cases well. Moreover, obtaining high-resolution (HR) videos and the corresponding low-resolution (LR) ones in real-world scenarios is difficult. To overcome these problems, we propose a self-supervised learning method to solve the blind video SR problem, which simultaneously estimates blur kernels and HR videos from the LR videos. As directly using LR videos as supervision usually leads to trivial solutions, we develop a simple and effective method to generate auxiliary paired data from original LR videos according to the image formation of video SR, so that the networks can be better constrained by the generated paired data for both blur kernel estimation and latent HR video restoration. In addition, we introduce an optical flow estimation module to exploit the information from adjacent frames for HR video restoration. Experiments show that our method performs favorably against state-of-the-art ones on benchmarks and real-world videos.
Interpretable Single-Cell Set Classification with Kernel Mean Embeddings

Modern single-cell ﬂow and mass cytometry technologies measure the expression of several proteins of the individual cells within a blood or tissue sample. Each proﬁled biological sample is thus represented by a set of hundreds of thousands of multidimensional cell feature vectors, which incurs a high computational cost to predict each biological sample’s associated phenotype with machine learning models. Such a large set cardinality also limits the interpretability of machine learning models due to the diﬃculty in tracking how each individual cell inﬂuences the ultimate prediction. Using Kernel Mean Embedding to encode the cellular landscape of each proﬁled biological sample, we can train a simple linear classiﬁer and achieve state-of-the-art classiﬁcation accuracy on 3 ﬂow and mass cytometry datasets. Our model contains few parameters but still performs similarly to deep learning models with millions of parameters. In contrast with deep learning approaches, the linearity and sub-selection step of our model make it easy to interpret classiﬁcation results. Clustering analysis further shows that our method admits rich biological interpretability for linking cellular heterogeneity to clinical phenotype. Our codes are publicly available at https://github.com/shansiliu95/CKME .
Aspects of Muchnik's paradox in restricted betting

Muchnik's paradox says that enumerable betting strategies are not always reducible to enumerable strategies whose bets are restricted to either even rounds or odd rounds. In other words, there are outcome sequences x where an effectively enumerable strategy succeeds, but no such parity-restricted effectively enumerable strategy does. We characterize the effective Hausdorff dimension of such $x$, showing that it can be as low as 1/2 but not less. We also show that such reals that are random with respect to parity-restricted effectively enumerable strategies with packing dimension as low as $\log\sqrt3$. Finally we exhibit Muchnik's paradox in the case of computable integer-valued strategies.
Knowledge Tracing: A Survey

Humans’ ability to transfer knowledge through teaching is one of the essential aspects for human intelligence. A human teacher can track the knowledge of students to customize the teaching on students’ needs. With the rise of online education platforms, there is a similar need for machines to track the knowledge of students and tailor their learning experience. This is known as the Knowledge Tracing (KT) problem in the literature. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials’ recommendation. Moreover, from a more general viewpoint, a student may represent any kind of intelligent agents including both human and artificial agents. Thus, the potential of KT can be extended to any machine teaching application scenarios which seek for customizing the learning experience for a student agent (i.e., a machine learning model). In this paper, we provide a comprehensive survey for the KT literature. We cover a broad range of methods starting from the early attempts to the recent state-of-the-art methods using deep learning, while highlighting the theoretical aspects of models and the characteristics of benchmark datasets. Besides these, we shed light on key modelling differences between closely related methods and summarize them in an easy-to-understand format. Finally, we discuss current research gaps in the KT literature and possible future research and application directions.
Landau-type sudden transitions of quantum correlations

Sudden changes of quantum correlations in the Bell-diagonal states are well-known effects. They occur when the set of optimal parameters that determine the quantum correlation consists of isolated points and optimal parameters during the evolution of the system jump from one such point to another (e.g., the optimal measurement angle of the quantum discord changes discontinuously from zero to π/2 or vice versa). However, when considering more general X quantum states, we found that quantum discord and one-way quantum work deficit can experience sudden changes of other kinds. Namely, the optimal measurement angle may suddenly start to shift continuously from its stationary value 0 or π/2 to an intermediate optimal measurement angle ϑ ∈ (0, π/2). This leads to a new behavior of quantum correlations, which is mathematically described by the Landau phenomenological theory of second-order phase transitions. In addition, for the one-way quantum work deficit, we found cases where the optimal measurement angle jumps from zero to a nonzero step less than π/2, and then continuously changes its value. This behavior of quantum correlation is similar to a first-order phase transition in Landau’s theory. Dependencies of quantum discord and one-way quantum work deficit near the boundaries, which separate regions with state-dependent (variable) and state-independent (stationary, constant) optimal measurement angles, are examined in detail on an example of the XXZ spin model in an external field at thermal equilibrium.
Time Series Generation with Masked Autoencoder

This paper shows that masked autoencoder with extrapolator (ExtraMAE) is a scalable self-supervised model for time series generation. ExtraMAE randomly masks some patches of the original time series and learns temporal dynamics by recovering the masked patches. Our approach has two core designs. First, ExtraMAE is self-supervised. Supervision allows ExtraMAE to effectively and efficiently capture the temporal dynamics of the original time series. Second, ExtraMAE proposes an extrapolator to disentangle two jobs of the decoder: recovering latent representations and mapping them back into the feature space. These unique designs enable ExtraMAE to consistently and significantly outperform state-of-the-art (SoTA) benchmarks in time series generation. The lightweight architecture also makes ExtraMAE fast and scalable. ExtraMAE shows outstanding behavior in various downstream tasks such as time series classification, prediction, and imputation. As a self-supervised generative model, ExtraMAE allows explicit management of the synthetic data. We hope this paper will usher in a new era of time series generation with self-supervised models.
Prospective Learning: Back to the Future

Research on both natural intelligence (NI) and artiﬁcial intelligence (AI) generally assumes that the future resembles the past: intelligent agents or systems (what we call ‘intelligence’) observe and act on the world, then use this experience to act on future experiences of the same kind. We call this ‘retrospective learning’. For example, an intelligence may see a set of pictures of objects, along with their names, and learn to name them. A retrospective learning intelligence would merely be able to name more pictures of the same objects. We argue that this is not what true intelligence is about. In many real world problems, both NIs and AIs will have to learn for an uncertain future. Both must update their internal models to be useful for future tasks, such as naming fundamentally new objects and using these objects effectively in a new context or to achieve previously unencountered goals. This ability to learn for the future we call ‘prospective learning’. We articulate four relevant factors that jointly deﬁne prospective learning. Continual learning enables intelligences to remember those aspects of the past which it believes will be most useful in the future. Prospective constraints (including biases and priors) facilitate the intelligence ﬁnding general solutions that will be applicable to future problems. Curiosity motivates taking actions that inform future decision making, including in previously unmet situations. Causal estimation enables learning the structure of relations that guide choosing actions for speciﬁc outcomes, even when the speciﬁc action-outcome contingencies have never been observed before. We argue that a paradigm shift from retrospective to prospective learning will enable the communities that study intelligence to unite and overcome existing bottlenecks to more effectively explain, augment, and engineer intelligences.
Hiding Data in Colors: Secure and Lossless Deep Image Steganography via Conditional Invertible Neural Networks

Deep image steganography is a data hiding technology that conceal data in digital images via deep neural networks. However, existing deep image steganography methods only consider the visual similarity of container images to host images, and neglect the statistical security (stealthiness) of container images. Besides, they usually hides data limited to image type and thus relax the constraint of lossless extraction. In this paper, we address the above issues in a unified manner, and propose deep image steganography that can embed data with arbitrary types into images for secure data hiding and lossless data revealing. First, we formulate the data hiding as an image colorization problem, in which the data is binarized and further mapped into the color information for a gray-scale host image. Second, we design a conditional invertible neural network which uses gray-scale image as prior to guide the color generation and perform data hiding in a secure way. Finally, to achieve lossless data revealing, we present a multi-stage training scheme to manage the data loss due to rounding errors between hiding and revealing processes. Extensive experiments demonstrate that the proposed method can perform secure data hiding by generating realism color images and successfully resisting the detection of steganalysis. Moreover, we can achieve 100% revealing accuracy in different scenarios, indicating the practical utility of our steganography in the real-world.
Creation of turbulence in polyatomic gas flow via an intermolecular potential

A bstract . We develop a tractable interaction model for a polyatomic gas, whose kinetic equation combines a Vlasov-type mean ﬁeld forcing due to an intermolecular potential, and a Boltzmann-type collision integral due to rotational interactions. We construct a velocity moment hierarchy for the new kinetic equation, and ﬁnd that, under the high Reynolds number condition, the pressure equation becomes decoupled from the angular momentum and stress. For the heat ﬂux, we propose a novel closure by prescribing the speciﬁc heat capacity of the gas ﬂow. Setting the speciﬁc heat capacity to that of a constant-pressure process leads to the system of equations for a balanced ﬂow, where the momentum transport equation contains the mean ﬁeld forcing, which is an averaged effect of the intermolecular potential. Remarkably, the balanced ﬂow equations do not contain any information about internal thermodynamic properties of the gas, and are thereby applicable to a broad range of different gases. We conduct numerical simulations for an air-like gas at normal conditions in the inertial ﬂow regime, where the pressure is constant throughout the domain. We ﬁnd that the presence of the intermolecular potential produces a distinctly turbulent ﬂow, whose time-averaged Fourier spectra of the kinetic energy and temperature exhibit Kolmogorov’s power decay.
The Euclidean $\phi^4_2$ theory as a limit of an interacting Bose gas

We prove that the complex Euclidean field theory with local quartic self-interaction in two dimensions arises as a limit of an interacting Bose gas at positive temperature, when the density of the gas becomes large and the range of the interaction becomes small. The field theory is supported on distributions of negative regularity, which requires a renormalization by divergent mass and energy counterterms. We obtain convergence of the relative partition function and uniform convergence of the renormalized reduced density matrices. The proof is based on three main ingredients: (a) a quantitative analysis of the infinite-dimensional saddle point argument for the functional integral introduced in [29], (b) a Nelson-type estimate for a general nonlocal field theory in two dimensions, and (c) repeated Gaussian integration by parts in field space to obtain uniform control on the renormalized correlation functions.
Bose-Einstein Condensation, the Lambda Transition, and Superfluidity for Interacting Bosons

Bose-Einstein condensation and the λ-transition are described in molecular detail for bosons interacting with a pair potential. New phenomena are identified that are absent in the usual ideal gas treatment. Monte Carlo simulations of Lennard-Jones helium-4 neglecting ground momentum state bosons give a diverging heat capacity approaching the transition. Pure permutation loops give continuous growth in the occupancy of the ground momentum state. Mixed ground and excited momentum state permutation loops give a discontinuous transition to the condensed phase. The consequent latent heat for the λ-transition is 3% of the total energy. The predicted critical velocity for superfluid flow is within a factor of three of the measured values over three orders of magnitude in pore diameter.
Improved Receivers for Optical Wireless OFDM: An Information Theoretic Perspective

We consider performance enhancement of asymmetrically-clipped optical orthogonal frequency division multiplexing (ACO-OFDM) and related optical OFDM schemes, which are variations of OFDM in intensity-modulated optical wireless communications. Unlike most existing studies on specific designs of improved receivers, this paper investigates information theoretic limits of all possible receivers. For independent and identically distributed (IID) complex Gaussian inputs, we obtain an exact characterization of information rate of ACO-OFDM with improved receivers for all SNRs. It is proved that the high-SNR gain of improved receivers asymptotically achieve 1/4 bits per channel use, which is equivalent to 3 dB in electrical SNR or 1.5 dB in optical SNR; as the SNR decreases, the maximum achievable SNR gain of improved receivers decreases monotonically to a non-zero low-SNR limit, corresponding to an information rate gain of 36.3%. For practically used constellations, we derive an upper bound on the gain of improved receivers. Numerical results demonstrate that the upper bound can be approached to within 1 dB in optical SNR by combining existing improved receivers and coded modulation. We also show that our information theoretic analyses can be extended to Flip-OFDM and PAM-DMT. Our results imply that, for the considered schemes, improved receivers may reduce the gap to channel capacity significantly at low-to-moderate SNR.
On Lieb-Robinson Bounds for the Double Bracket Flow

We consider the possibility of developing a Lieb-Robinson bound for the double bracket flow[3, 4]. This is a differential equation m ( ) = [[+, ( )], ( )] which may be used to diagonalize Hamiltonians. Here, + is fixed and (0) = . We argue (but do not prove) that ( ) need not converge to a limit for nonzero real in the infinite volume limit, even assuming several conditions on (0). However, we prove Lieb-Robinson bounds for all for the double-bracket flow for free fermion systems, but the range increases exponentially with the control parameter . 1.1 Some Cases Where Lieb-Robinson Bounds Can and Cannot be Proven Lieb-Robinson bounds describe how an operator “spreads out" over space while undergoing the Heisenberg equation of motion: mC$ (C) = 8 [ ,$ (C)]. for some Hamiltonian on some quantum lattice system. Crucially, one can bound this spread in a way that is uniform in the size of the system if obeys suitable locality properties, so that the spread depends only on the strength and range of terms in . Indeed, while the first proof[10] also depended on the local Hilbert space dimension, later it was possible to derive bounds uniform in this dimension, as in [7]. For local Hamiltonians, the spread is at most linear in time, governed by a LiebRobinson velocity. However, for more general Hamiltonians with polynomially-decaying long-range interactions, one can derive in some cases a weaker result[5,6,8,11] where the spread is superlinear in time, but one can still find bounds uniform in system size. However, all these proofs use crucially the fact that the evolution considered is unitary. If we instead consider an imaginary time version of the Heisenberg equation:
High-level synthesis performance prediction using GNNs: benchmarking, modeling, and advancing

Agile hardware development requires fast and accurate circuit quality evaluation from early design stages. Existing work of high-level synthesis (HLS) performance prediction usually requires extensive feature engineering after the synthesis process. To expedite circuit evaluation from as early design stage as possible, we propose rapid and accurate performance prediction methods, which exploit the representation power of graph neural networks (GNNs) by representing C/C++ programs as graphs. The contribution of this work is three-fold. (1) Benchmarking. We build a standard benchmark suite with 40k C programs, which includes synthetic programs and three sets of real-world HLS benchmarks. Each program is synthesized and implemented on FPGA to obtain post place-and-route performance metrics as the ground truth. (2) Modeling. We formally formulate the HLS performance prediction problem on graphs and propose multiple modeling strategies with GNNs that leverage different trade-offs between prediction timeliness (early/late prediction) and accuracy. (3) Advancing. We further propose a novel hierarchical GNN that does not sacrifice timeliness but largely improves prediction accuracy, significantly outperforming HLS tools. We apply extensive evaluations for both synthetic and unseen real-case programs; our proposed predictor largely outperforms HLS by up to 40X and excels existing predictors by 2X to 5X in terms of resource usage and timing prediction. The benchmark and explored GNN models are publicly available at https://github.com/lydiawunan/HLS-Perf-Prediction-with-GNNs.
Laser Cooling Beyond Rate Equations: Approaches from Quantum Thermodynamics

Solids can be cooled by driving impurity ions with lasers, allowing them to transfer heat from the lattice phonons to the electromagnetic surroundings. This exemplifies a quantum thermal machine, which uses a quantum system as a working medium to transfer heat between reservoirs. We review the derivation of the Bloch-Redfield equation for a quantum system coupled to a reservoir, and its extension, using counting fields, to calculate heat currents. We use the full form of this equation, which makes only the weak-coupling and Markovian approximations, to calculate the cooling power for a simple model of laser cooling. We compare its predictions with two other time-local master equations: the secular approximation to the full Bloch-Redfield equation, and the Lindblad form expected for phonon transitions in the absence of driving. We conclude that the full Bloch-Redfield equation provides accurate results for the heat current in both the weak- and strong- driving regimes, whereas the other forms have more limited applicability. Our results support the use of Bloch-Redfield equations in quantum thermal machines, despite their potential to give unphysical results.
A 334uW 0.158mm2 Saber Learning with Rounding based Post-Quantum Crypto Accelerator

The arrival of large-scale quantum computers will break the security assurances of our current public-key cryptography. National Institute of Standard & Technology (NIST) is currently running a multi-year-long standardization procedure to select quantum-safe or postquantum cryptographic schemes to be used in the future. Energy efficiency is an important criterion in the selection process. This paper presents the first Silicon verified ASIC implementation for Saber (LWR algorithm as proposed in [1], [2]), a NIST PQC Round 3 finalist candidate in the key-encapsulation mechanism (KEM) category. Fig. 1 briefly describes the learning with rounding (LWR) problem, which is hard to solve even in the presence of large quantum computers due to the noise generated from rounding. IC features are tabulated in Fig. 1. which also shows a simplified version of the Saber KEM scheme to establish a secret key between two communicating parties Alice and Bob. Due to learning with rounding, secret $s$ is hard to guess based on publicly available data as shown in Fig. 1.