Miracle-dz/UniFAR-FTUs huggingface dataset:
Simulation of IBM's kicked Ising experiment with Projected Entangled Pair Operator

We perform classical simulations of the 127-qubit kicked Ising model, which was recently emulated using a quantum circuit with error mitigation [Nature 618, 500 (2023)]. Our approach is based on the projected entangled pair operator (PEPO) in the Heisenberg picture. Its main feature is the ability to automatically identify the underlying low-rank and low-entanglement structures in the quantum circuit involving Clifford and near-Clifford gates. We assess our approach using the quantum circuit with 5+1 trotter steps which was previously considered beyond classical verification. We develop a Clifford expansion theory to compute exact expectation values and use them to evaluate algorithms. The results indicate that PEPO significantly outperforms existing methods, including the tensor network with belief propagation, the matrix product operator, and the Clifford perturbation theory, in both efficiency and accuracy. In particular, PEPO with bond dimension $\chi=2$ already gives similar accuracy to the CPT with $K=10$ and MPO with bond dimension $\chi=1024$. And PEPO with $\chi=184$ provides exact results in $3$ seconds using a single CPU. Furthermore, we apply our method to the circuit with 20 Trotter steps. We observe the monotonic and consistent convergence of the results with $\chi$, allowing us to estimate the outcome with $\chi\to\infty$ through extrapolations. We then compare the extrapolated results to those achieved in quantum hardware and with existing tensor network methods. Additionally, we discuss the potential usefulness of our approach in simulating quantum circuits, especially in scenarios involving near-Clifford circuits and quantum approximate optimization algorithms. Our approach is the first use of PEPO in solving the time evolution problem, and our results suggest it could be a powerful tool for exploring the dynamical properties of quantum many-body systems.
Friedrichs' systems discretized with the Discontinuous Galerkin method: domain decomposable model order reduction and Graph Neural Networks approximating vanishing viscosity solutions

Friedrichs' systems (FS) are symmetric positive linear systems of first-order partial differential equations (PDEs), which provide a unified framework for describing various elliptic, parabolic and hyperbolic semi-linear PDEs such as the linearized Euler equations of gas dynamics, the equations of compressible linear elasticity and the Dirac-Klein-Gordon system. FS were studied to approximate PDEs of mixed elliptic and hyperbolic type in the same domain. For this and other reasons, the versatility of the discontinuous Galerkin method (DGM) represents the best approximation space for FS. We implement a distributed memory solver for stationary FS in deal.II. Our focus is model order reduction. Since FS model hyperbolic PDEs, they often suffer from a slow Kolmogorov n-width decay. We develop two approaches to tackle this problem. The first is domain decomposable reduced-order models (DD-ROMs). We will show that the DGM offers a natural formulation of DD-ROMs, in particular regarding interface penalties, compared to the continuous finite element method. We also develop new repartitioning strategies to obtain more efficient local approximations of the solution manifold. The second approach involves graph neural networks used to infer the limit of a succession of projection-based linear ROMs corresponding to lower viscosity constants: the heuristic behind is to develop a multi-fidelity super-resolution paradigm to mimic the mathematical convergence to vanishing viscosity solutions while exploiting to the most interpretable and certified projection-based ROMs.
AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation

Segmentation is a fundamental problem in surgical scene analysis using artificial intelligence. However, the inherent data scarcity in this domain makes it challenging to adapt traditional segmentation techniques for this task. To tackle this issue, current research employs pretrained models and finetunes them on the given data. Even so, these require training deep networks with millions of parameters every time new data becomes available. A recently published foundation model, Segment-Anything (SAM), generalizes well to a large variety of natural images, hence tackling this challenge to a reasonable extent. However, SAM does not generalize well to the medical domain as is without utilizing a large amount of compute resources for fine-tuning and using task-specific prompts. Moreover, these prompts are in the form of bounding-boxes or foreground/background points that need to be annotated explicitly for every image, making this solution increasingly tedious with higher data size. In this work, we propose AdaptiveSAM - an adaptive modification of SAM that can adjust to new datasets quickly and efficiently, while enabling text-prompted segmentation. For finetuning AdaptiveSAM, we propose an approach called bias-tuning that requires a significantly smaller number of trainable parameters than SAM (less than 2\%). At the same time, AdaptiveSAM requires negligible expert intervention since it uses free-form text as prompt and can segment the object of interest with just the label name as prompt. Our experiments show that AdaptiveSAM outperforms current state-of-the-art methods on various medical imaging datasets including surgery, ultrasound and X-ray. Code is available at https://github.com/JayParanjape/biastuning
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning

In this work, we investigate the dynamics of stochastic gradient descent (SGD) when training a single-neuron autoencoder with linear or ReLU activation on orthogonal data. We show that for this non-convex problem, randomly initialized SGD with a constant step size successfully finds a global minimum for any batch size choice. However, the particular global minimum found depends upon the batch size. In the full-batch setting, we show that the solution is dense (i.e., not sparse) and is highly aligned with its initialized direction, showing that relatively little feature learning occurs. On the other hand, for any batch size strictly smaller than the number of samples, SGD finds a global minimum which is sparse and nearly orthogonal to its initialization, showing that the randomness of stochastic gradients induces a qualitatively different type of"feature selection"in this setting. Moreover, if we measure the sharpness of the minimum by the trace of the Hessian, the minima found with full batch gradient descent are flatter than those found with strictly smaller batch sizes, in contrast to previous works which suggest that large batches lead to sharper minima. To prove convergence of SGD with a constant step size, we introduce a powerful tool from the theory of non-homogeneous random walks which may be of independent interest.
Toward More Human-Like AI Communication: A Review of Emergent Communication Research

In the recent shift towards human-centric AI, the need for machines to accurately use natural language has become increasingly important. While a common approach to achieve this is to train large language models, this method presents a form of learning misalignment where the model may not capture the underlying structure and reasoning humans employ in using natural language, potentially leading to unexpected or unreliable behavior. Emergent communication (EmCom) is a field of research that has seen a growing number of publications in recent years, aiming to develop artificial agents capable of using natural language in a way that goes beyond simple discriminative tasks and can effectively communicate and learn new concepts. In this review, we present EmCom under two aspects. Firstly, we delineate all the common proprieties we find across the literature and how they relate to human interactions. Secondly, we identify two subcategories and highlight their characteristics and open challenges. We encourage researchers to work together by demonstrating that different methods can be viewed as diverse solutions to a common problem and emphasize the importance of including diverse perspectives and expertise in the field. We believe a deeper understanding of human communication and human-AI trust dynamics are crucial to develop machines that can accurately use natural language in human-machine interactions.
Rearrangement of Granular Surfaces on Asteroids due to Thermal Cycling

In granular systems, thermal cycling causes compaction, creep, penetration of dense objects, and ratcheting of grains against each other. On asteroid surfaces, thermal cycling is high amplitude and can happen billions of times in a few million years. We use a 1-dimensional thermophysical conductivity model to estimate the relative displacement of grains in proximity to one another, caused by variations in thermal conductivity or shadows. We find that grains would experience relative displacements of order a few microns during each thermal cycle. If thermal cycling causes diffusive transport, then the asteroid's few centimeters deep thermal skin depth could flow a few centimeters in a million years. Thermal cycling could cause long-distance flows on sloped surfaces, allowing fine materials to collect in depressions.
Provably Efficient Learning in Partially Observable Contextual Bandit

In this paper, we investigate transfer learning in partially observable contextual bandits, where agents have limited knowledge from other agents and partial information about hidden confounders. We first convert the problem to identifying or partially identifying causal effects between actions and rewards through optimization problems. To solve these optimization problems, we discretize the original functional constraints of unknown distributions into linear constraints, and sample compatible causal models via sequentially solving linear programmings to obtain causal bounds with the consideration of estimation error. Our sampling algorithms provide desirable convergence results for suitable sampling distributions. We then show how causal bounds can be applied to improving classical bandit algorithms and affect the regrets with respect to the size of action sets and function spaces. Notably, in the task with function approximation which allows us to handle general context distributions, our method improves the order dependence on function space size compared with previous literatures. We formally prove that our causally enhanced algorithms outperform classical bandit algorithms and achieve orders of magnitude faster convergence rates. Finally, we perform simulations that demonstrate the efficiency of our strategy compared to the current state-of-the-art methods. This research has the potential to enhance the performance of contextual bandit agents in real-world applications where data is scarce and costly to obtain.
No reduction in motor‐evoked potential amplitude during the rubber hand illusion

In the rubber hand illusion (RHI), touches are applied to a fake hand at the same time as touches are applied to a participant's real hand that is hidden in a congruent position. Synchronous (but not asynchronous) tactile stimulation of the two hands may induce the sensation that the fake hand is the participant's own. As such, the illusion is commonly used to examine the sense of body ownership. Some studies indicate that in addition to the subjective experience of limb ownership reported by participants, the RHI can also reduce corticospinal excitability (e.g., as reflected in motor‐evoked potential [MEP] amplitude) and alter parietal‐motor cortical connectivity in passive participants. These findings have been taken to support a link between motor cortical processing and the subjective experience of body ownership.
Adaptively Placed Multi-Grid Scene Representation Networks for Large-Scale Data Visualization

Scene representation networks (SRNs) have been recently proposed for compression and visualization of scientific data. However, state-of-the-art SRNs do not adapt the allocation of available network parameters to the complex features found in scientific data, leading to a loss in reconstruction quality. We address this shortcoming with an adaptively placed multi-grid SRN (APMGSRN) and propose a domain decomposition training and inference technique for accelerated parallel training on multi-GPU systems. We also release an open-source neural volume rendering application that allows plug-and-play rendering with any PyTorch-based SRN. Our proposed APMGSRN architecture uses multiple spatially adaptive feature grids that learn where to be placed within the domain to dynamically allocate more neural network resources where error is high in the volume, improving state-of-the-art reconstruction accuracy of SRNs for scientific data without requiring expensive octree refining, pruning, and traversal like previous adaptive models. In our domain decomposition approach for representing large-scale data, we train an set of APMGSRNs in parallel on separate bricks of the volume to reduce training time while avoiding overhead necessary for an out-of-core solution for volumes too large to fit in GPU memory. After training, the lightweight SRNs are used for realtime neural volume rendering in our open-source renderer, where arbitrary view angles and transfer functions can be explored. A copy of this paper, all code, all models used in our experiments, and all supplemental materials and videos are available at https://github.com/skywolf829/APMGSRN.
Learning to Schedule in Non-Stationary Wireless Networks With Unknown Statistics

The emergence of large-scale wireless networks with partially-observable and time-varying dynamics has imposed new challenges on the design of optimal control policies. This paper studies efficient scheduling algorithms for wireless networks subject to generalized interference constraint, where mean arrival and mean service rates are unknown and non-stationary. This model exemplifies realistic edge devices' characteristics of wireless communication in modern networks. We propose a novel algorithm termed MW-UCB for generalized wireless network scheduling, which is based on the Max-Weight policy and leverages the Sliding-Window Upper-Confidence Bound to learn the channels' statistics under non-stationarity. MW-UCB is provably throughput-optimal under mild assumptions on the variability of mean service rates. Specifically, as long as the total variation in mean service rates over any time period grows sub-linearly in time, we show that MW-UCB can achieve the stability region arbitrarily close to the stability region of the class of policies with full knowledge of the channel statistics. Extensive simulations validate our theoretical results and demonstrate the favorable performance of MW-UCB.
Semilinear elliptic eigenvalue problem: Parametric analyticity and the uncertainty quantification

In this paper, to the best of our knowledge, we make the first attempt at studying the parametric semilinear elliptic eigenvalue problems with the parametric coefficient and some power-type nonlinearities. The parametric coefficient is assumed to have an affine dependence on the countably many parameters with an appropriate class of sequences of functions. In this paper, we obtain the upper bound estimation for the mixed derivatives of the ground eigenpairs that has the same form obtained recently for the linear eigenvalue problem. The three most essential ingredients for this estimation are the parametric analyticity of the ground eigenpairs, the uniform boundedness of the ground eigenpairs, and the uniform positive differences between ground eigenvalues of linear operators. All these three ingredients need new techniques and a careful investigation of the nonlinear eigenvalue problem that will be presented in this paper. As an application, considering each parameter as a uniformly distributed random variable, we estimate the expectation of the eigenpairs using a randomly shifted quasi-Monte Carlo lattice rule and show the dimension-independent error bound.
Scaling may be all you need for achieving human-level object recognition capacity with human-like visual experience

This paper asks whether current self-supervised learning methods, if sufficiently scaled up, would be able to reach human-level visual object recognition capabilities with the same type and amount of visual experience humans learn from. Previous work on this question only considered the scaling of data size. Here, we consider the simultaneous scaling of data size, model size, and image resolution. We perform a scaling experiment with vision transformers up to 633M parameters in size (ViT-H/14) trained with up to 5K hours of human-like video data (long, continuous, mostly egocentric videos) with image resolutions of up to 476x476 pixels. The efficiency of masked autoencoders (MAEs) as a self-supervised learning algorithm makes it possible to run this scaling experiment on an unassuming academic budget. We find that it is feasible to reach human-level object recognition capacity at sub-human scales of model size, data size, and image size, if these factors are scaled up simultaneously. To give a concrete example, we estimate that a 2.5B parameter ViT model trained with 20K hours (2.3 years) of human-like video data with a spatial resolution of 952x952 pixels should be able to reach roughly human-level accuracy on ImageNet. Human-level competence is thus achievable for a fundamental perceptual capability from human-like perceptual experience (human-like in both amount and type) with extremely generic learning algorithms and architectures and without any substantive inductive biases.
Observations of Oxidation Products above a Forest Observations of Oxidation Products above a Forest Imply Biogenic Emissions of Very Reactive Compounds Observations of Oxidation Products above a Forest

Measurements of volatile organic compounds in a pine forest (Central California, 38.90 • N, 120.63 • W, 1315 m) reveal large quantities of previously unreported oxidation products of short lived biogenic precursors. The emission of biogenic precursors must be in the range of 13–66 µmol m −2 h −1 to produce the observed oxidation prod-5 ucts. That is 6–30 times the emissions of total monoterpenes observed above the forest canopy on a molar basis. These reactive precursors constitute the largest fraction of biogenic emissions at this site, and are not included in current emission inventories. When oxidized by ozone they should efficiently produce secondary aerosol and hydroxyl radicals.
Seizure reduction in TSC2‐mutant mouse model by an mTOR catalytic inhibitor

Tuberous sclerosis complex (TSC) is a neurodevelopmental disorder caused by autosomal‐dominant pathogenic variants in either the TSC1 or TSC2 gene, and it is characterized by hamartomas in multiple organs, such as skin, kidney, lung, and brain. These changes can result in epilepsy, learning disabilities, and behavioral complications, among others. The mechanistic link between TSC and the mechanistic target of the rapamycin (mTOR) pathway is well established, thus mTOR inhibitors can potentially be used to treat the clinical manifestations of the disorder, including epilepsy.
Mondrian: Prompt Abstraction Attack Against Large Language Models for Cheaper API Pricing

The Machine Learning as a Service (MLaaS) market is rapidly expanding and becoming more mature. For example, OpenAI's ChatGPT is an advanced large language model (LLM) that generates responses for various queries with associated fees. Although these models can deliver satisfactory performance, they are far from perfect. Researchers have long studied the vulnerabilities and limitations of LLMs, such as adversarial attacks and model toxicity. Inevitably, commercial ML models are also not exempt from such issues, which can be problematic as MLaaS continues to grow. In this paper, we discover a new attack strategy against LLM APIs, namely the prompt abstraction attack. Specifically, we propose Mondrian, a simple and straightforward method that abstracts sentences, which can lower the cost of using LLM APIs. In this approach, the adversary first creates a pseudo API (with a lower established price) to serve as the proxy of the target API (with a higher established price). Next, the pseudo API leverages Mondrian to modify the user query, obtain the abstracted response from the target API, and forward it back to the end user. Our results show that Mondrian successfully reduces user queries' token length ranging from 13% to 23% across various tasks, including text classification, generation, and question answering. Meanwhile, these abstracted queries do not significantly affect the utility of task-specific and general language models like ChatGPT. Mondrian also reduces instruction prompts' token length by at least 11% without compromising output quality. As a result, the prompt abstraction attack enables the adversary to profit without bearing the cost of API development and deployment.
Systematic compactification of the (multi) two-channel Kondo model. III. Extended field-theoretic renormalization group analysis

We carry out a field-theoretical renormalization group procedure based on the Callan-Symanzik equation to calculate the detailed flow for the (multi) two-channel Kondo model and its compactified versions. In doing so, we go beyond the universal terms in the beta function we obtained using poor man's scaling (see companion paper II) and culminate our analysis of how the compactified versions of the model fare against the original one. Among other results, we explore the large-channel-number limit and also extend our considerations to the finite temperature crossover region. Moreover, we gain insights into the contradistinction between the consistent vs. conventional bosonization-debosonization formalisms, thereby advancing our understanding on multiple fronts. In particular, we make use of renormalization-flow arguments to further justify the consistent refermionization of the parallel Kondo interaction we presented earlier (see companion paper I).
REED: Chiplet-Based Scalable Hardware Accelerator for Fully Homomorphic Encryption

Fully Homomorphic Encryption (FHE) has emerged as a promising technology for processing encrypted data without the need for decryption. Despite its potential, its practical implementation has faced challenges due to substantial computational overhead. To address this issue, we propose the $first$ chiplet-based FHE accelerator design `REED', which enables scalability and offers high throughput, thereby enhancing homomorphic encryption deployment in real-world scenarios. It incorporates well-known wafer yield issues during fabrication which significantly impacts production costs. In contrast to state-of-the-art approaches, we also address data exchange overhead by proposing a non-blocking inter-chiplet communication strategy. We incorporate novel pipelined Number Theoretic Transform and automorphism techniques, leveraging parallelism and providing high throughput. Experimental results demonstrate that REED 2.5D integrated circuit consumes 177 mm$^2$ chip area, 82.5 W average power in 7nm technology, and achieves an impressive speedup of up to 5,982$\times$ compared to a CPU (24-core 2$\times$Intel X5690), and 2$\times$ better energy efficiency and 50\% lower development cost than state-of-the-art ASIC accelerator. To evaluate its practical impact, we are the $first$ to benchmark an encrypted deep neural network training. Overall, this work successfully enhances the practicality and deployability of fully homomorphic encryption in real-world scenarios.