Miracle-dz/UniFAR-FTUs huggingface dataset:
On the Convergence of Differentially-Private Fine-tuning: To Linearly Probe or to Fully Fine-tune?

Differentially private (DP) machine learning pipelines typically involve a two-phase process: non-private pre-training on a public dataset, followed by fine-tuning on private data using DP optimization techniques. In the DP setting, it has been observed that full fine-tuning may not always yield the best test accuracy, even for in-distribution data. This paper (1) analyzes the training dynamics of DP linear probing (LP) and full fine-tuning (FT), and (2) explores the phenomenon of sequential fine-tuning, starting with linear probing and transitioning to full fine-tuning (LP-FT), and its impact on test loss. We provide theoretical insights into the convergence of DP fine-tuning within an overparameterized neural network and establish a utility curve that determines the allocation of privacy budget between linear probing and full fine-tuning. The theoretical results are supported by empirical evaluations on various benchmarks and models. The findings reveal the complex nature of DP fine-tuning methods. These results contribute to a deeper understanding of DP machine learning and highlight the importance of considering the allocation of privacy budget in the fine-tuning process.
Towards the boundary of the fine curve graph

The fine curve graph was introduced as a geometric tool to study the homeomorphisms of surfaces. In this paper we study the Gromov boundary of this space and the local topology near points associated with minimal measurable foliations. We then give several applications including finding explicit elements with positive stable commutator length, and proving a Tits alternative for subgroups of $\textrm{Homeo}(S)$ containing a pseudo-Anosov map, generalizing a result of Hurtado-Xue.
Beyond Language Models: Byte Models are Digital World Simulators

Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. Inspired by the success of next token prediction in natural language processing, we introduce bGPT, a model with next byte prediction to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new possibilities for predicting, simulating, and diagnosing algorithm or hardware behaviour. It has almost flawlessly replicated the process of converting symbolic music data, achieving a low error rate of 0.0011 bits per byte in converting ABC notation to MIDI format. In addition, bGPT demonstrates exceptional capabilities in simulating CPU behaviour, with an accuracy exceeding 99.99% in executing various operations. Leveraging next byte prediction, models like bGPT can directly learn from vast binary data, effectively simulating the intricate patterns of the digital world.
Boundary Treatment for Variational Quantum Simulations of Partial Differential Equations on Quantum Computers

The paper presents a variational quantum algorithm to solve initial-boundary value problems described by second-order partial differential equations. The approach uses hybrid classical/quantum hardware that is well suited for quantum computers of the current noisy intermediate-scale quantum era. The partial differential equation is initially translated into an optimal control problem with a modular control-to-state operator (ansatz). The objective function and its derivatives required by the optimizer can efficiently be evaluated on a quantum computer by measuring an ancilla qubit, while the optimization procedure employs classical hardware. The focal aspect of the study is the treatment of boundary conditions, which is tailored to the properties of the quantum hardware using a correction technique. For this purpose, the boundary conditions and the discretized terms of the partial differential equation are decomposed into a sequence of unitary operations and subsequently compiled into quantum gates. The accuracy and gate complexity of the approach are assessed for second-order partial differential equations by classically emulating the quantum hardware. The examples include steady and unsteady diffusive transport equations for a scalar property in combination with various Dirichlet, Neumann, or Robin conditions. The results of this flexible approach display a robust behavior and a strong predictive accuracy in combination with a remarkable polylog complexity scaling in the number of qubits of the involved quantum circuits. Remaining challenges refer to adaptive ansatz strategies that speed up the optimization procedure.
High multiplicity of positive solutions in a superlinear problem of Moore-Nehari type

In this paper we consider a superlinear one-dimensional elliptic boundary value problem that generalizes the one studied by Moore and Nehari in [43]. Specifically, we deal with piecewise-constant weight functions in front of the nonlinearity with an arbitrary number $\kappa\geq 1$ of vanishing regions. We study, from an analytic and numerical point of view, the number of positive solutions, depending on the value of a parameter $\lambda$ and on $\kappa$. Our main results are twofold. On the one hand, we study analytically the behavior of the solutions, as $\lambda\downarrow-\infty$, in the regions where the weight vanishes. Our result leads us to conjecture the existence of $2^{\kappa+1}-1$ solutions for sufficiently negative $\lambda$. On the other hand, we support such a conjecture with the results of numerical simulations which also shed light on the structure of the global bifurcation diagrams in $\lambda$ and the profiles of positive solutions. Finally, we give additional numerical results suggesting that the same high multiplicity result holds true for a much larger class of weights, also arbitrarily close to situations where there is uniqueness of positive solutions.
Viscoelastic response and anisotropic hydrodynamics in Weyl semimetals

We study viscoelastic response in Weyl semimetals with broken time-reversal symmetry. Topology and anisotropy of the Fermi surface are manifested in the viscoelasticity tensor of the electron fluid. In the dynamic (inter-band) part of this tensor, the anisotropy leads to a qualitatively different, compared to isotropic models, scaling with frequency and the Fermi energy. While components of the viscosity tensor determined by the Fermi surface properties agree in the Kubo and kinetic formalisms, the latter misses the anomalous Hall viscosity determined by filled states below the Fermi surface. The anisotropy of the dispersion relation is also manifested in the acceleration and relaxation terms of the hydrodynamic equations.
WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Concepts

Recent advancements in neural networks have showcased their remarkable capabilities across various domains. Despite these successes, the"black box"problem still remains. Addressing this, we propose a novel framework, WWW, that offers the 'what', 'where', and 'why' of the neural network decisions in human-understandable terms. Specifically, WWW utilizes adaptive selection for concept discovery, employing adaptive cosine similarity and thresholding techniques to effectively explain 'what'. To address the 'where' and 'why', we proposed a novel combination of neuron activation maps (NAMs) with Shapley values, generating localized concept maps and heatmaps for individual inputs. Furthermore, WWW introduces a method for predicting uncertainty, leveraging heatmap similarities to estimate 'how' reliable the prediction is. Experimental evaluations of WWW demonstrate superior performance in both quantitative and qualitative metrics, outperforming existing methods in interpretability. WWW provides a unified solution for explaining 'what', 'where', and 'why', introducing a method for localized explanations from global interpretations and offering a plug-and-play solution adaptable to various architectures.
Primordial Rotating Disk Composed of $\geq$15 Dense Star-Forming Clumps at Cosmic Dawn

Early galaxy formation, initiated by the dark matter and gas assembly, evolves through frequent mergers and feedback processes into dynamically hot, chaotic structures. In contrast, dynamically cold, smooth rotating disks have been observed in massive evolved galaxies merely 1.4 billion years after the Big Bang, suggesting rapid morphological and dynamical evolution in the early Universe. Probing this evolution mechanism necessitates studies of young galaxies, yet efforts have been hindered by observational limitations in both sensitivity and spatial resolution. Here we report high-resolution observations of a strongly lensed and quintuply imaged, low-luminosity, young galaxy at $z=6.072$ (dubbed the Cosmic Grapes), 930 million years after the Big Bang. Magnified by gravitational lensing, the galaxy is resolved into at least 15 individual star-forming clumps with effective radii of $r_{\rm e}\simeq$ 10--60 parsec (pc), which dominate $\simeq$ 70\% of the galaxy's total flux. The cool gas emission unveils a smooth, underlying rotating disk characterized by a high rotational-to-random motion ratio and a gravitationally unstable state (Toomre $Q \simeq$ 0.2--0.3), with high surface gas densities comparable to local dusty starbursts with $\simeq10^{3-5}$ $M_{\odot}$/pc$^{2}$. These gas properties suggest that the numerous star-forming clumps are formed through disk instabilities with weak feedback effects. The clumpiness of the Cosmic Grapes significantly exceeds that of galaxies at later epochs and the predictions from current simulations for early galaxies. Our findings shed new light on internal galaxy substructures and their relation to the underlying dynamics and feedback mechanisms at play during their early formation phases, potentially explaining the high abundance of bright galaxies observed in the early Universe and the dark matter core-cusp problem.
A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D

The development of generative models that create 3D content from a text prompt has made considerable strides thanks to the use of the score distillation sampling (SDS) method on pre-trained diffusion models for image generation. However, the SDS method is also the source of several artifacts, such as the Janus problem, the misalignment between the text prompt and the generated 3D model, and 3D model inaccuracies. While existing methods heavily rely on the qualitative assessment of these artifacts through visual inspection of a limited set of samples, in this work we propose more objective quantitative evaluation metrics, which we cross-validate via human ratings, and show analysis of the failure cases of the SDS technique. We demonstrate the effectiveness of this analysis by designing a novel computationally efficient baseline model that achieves state-of-the-art performance on the proposed metrics while addressing all the above-mentioned artifacts.
Tsunami inundation from heterogeneous earthquake slip distributions: Evaluation of synthetic source models

This study investigates whether eight different synthetic finite fault models (SFFM) can simulate stochastic earthquake‐tsunami with similar statistical properties to “real” earthquake‐tsunami events, where the latter are represented using heterogeneous slip distributions from 66 Finite Fault Inversions (FFI) for oceanic subduction interface earthquakes. A new method is derived to estimate SFFM parameters from FFI, and predictive relations between the earthquake moment magnitude and the SFFM corner wave numbers are developed to support model applications. SFFM with more capacity to spatially localize slip are better able to simulate higher slip regions on the FFI, and this strongly influences their associated tsunami inundation, which is computed in two dimensions over idealized topography. The best performing SFFM generates tsunami inundation which envelopes the FFI inundation in 81% of cases using 10 synthetic events (close to the ideal value of 82%), while the other SFFM show greater tendencies to underestimate inundation. These differences are related to the capacity of each SFFM to produce spatially localized slip distributions. None of the SFFM showed a tendency to overpredict inundation. The results highlight that SFFM cannot be assumed to reliably quantify uncertainties in the tsunami inundation of real earthquakes, and the use of untested SFFM could create nonconservative bias in tsunami hazard assessments. However, the most successful model used here performs quite well, although it may still underestimate inundation more often than an optimal model.
DiffuseRAW: End-to-End Generative RAW Image Processing for Low-Light Images

Imaging under extremely low-light conditions presents a significant challenge and is an ill-posed problem due to the low signal-to-noise ratio (SNR) caused by minimal photon capture. Previously, diffusion models have been used for multiple kinds of generative tasks and image-to-image tasks, however, these models work as a post-processing step. These diffusion models are trained on processed images and learn on processed images. However, such approaches are often not well-suited for extremely low-light tasks. Unlike the task of low-light image enhancement or image-to-image enhancement, we tackle the task of learning the entire image-processing pipeline, from the RAW image to a processed image. For this task, a traditional image processing pipeline often consists of multiple specialized parts that are overly reliant on the downstream tasks. Unlike these, we develop a new generative ISP that relies on fine-tuning latent diffusion models on RAW images and generating processed long-exposure images which allows for the apt use of the priors from large text-to-image generation models. We evaluate our approach on popular end-to-end low-light datasets for which we see promising results and set a new SoTA on the See-in-Dark (SID) dataset. Furthermore, with this work, we hope to pave the way for more generative and diffusion-based image processing and other problems on RAW data.
Teaching Large Language Models an Unseen Language on the Fly

Existing large language models struggle to support numerous low-resource languages, particularly the extremely low-resource ones where there is minimal training data available for effective parameter updating. We thus investigate whether LLMs can learn a new language on the fly solely through prompting. To study this question, we collect a research suite for Zhuang, a language supported by no LLMs currently. We introduce \textsc{DiPMT++}, a framework for adapting LLMs to unseen languages by in-context learning. Using a dictionary and only 5K parallel sentences, \textsc{DiPMT++} significantly enhances the performance of GPT-4 from 0 to 16 BLEU for Chinese-to-Zhuang translation and achieves 32 BLEU for Zhuang-to-Chinese translation. Furthermore, we demonstrate the practical utility of this framework in aiding humans to translate completely unseen languages, which could contribute to the preservation of linguistic diversity.
Functionally-Complete Boolean Logic in Real DRAM Chips: Experimental Characterization and Analysis

Processing-using-DRAM (PuD) is an emerging paradigm that leverages the analog operational properties of DRAM circuitry to enable massively parallel in-DRAM computation. PuD has the potential to significantly reduce or eliminate costly data movement between processing elements and main memory. Prior works experimentally demonstrate three-input MAJ (i.e., MAJ3) and two-input AND and OR operations in commercial off-the-shelf (COTS) DRAM chips. Yet, demonstrations on COTS DRAM chips do not provide a functionally complete set of operations (e.g., NAND or AND and NOT). We experimentally demonstrate that COTS DRAM chips are capable of performing 1) functionally-complete Boolean operations: NOT, NAND, and NOR and 2) many-input (i.e., more than two-input) AND and OR operations. We present an extensive characterization of new bulk bitwise operations in 256 off-the-shelf modern DDR4 DRAM chips. We evaluate the reliability of these operations using a metric called success rate: the fraction of correctly performed bitwise operations. Among our 19 new observations, we highlight four major results. First, we can perform the NOT operation on COTS DRAM chips with a 98.37% success rate on average. Second, we can perform up to 16-input NAND, NOR, AND, and OR operations on COTS DRAM chips with high reliability (e.g., 16-input NAND, NOR, AND, and OR with an average success rate of 94.94%, 95.87%, 94.94%, and 95.85%, respectively). Third, data pattern only slightly affects bitwise operations. Our results show that executing NAND, NOR, AND, and OR operations with random data patterns decreases the success rate compared to all logic-1/logic-0 patterns by 1.39%, 1.97%, 1.43%, and 1.98%, respectively. Fourth, bitwise operations are highly resilient to temperature changes, with small success rate fluctuations of at most 1.66% among all the tested operations when the temperature is increased from 50C to 95C.
Think Fast, Think Slow, Think Critical: Designing an Automated Propaganda Detection Tool

In today's digital age, characterized by rapid news consumption and increasing vulnerability to propaganda, fostering citizens' critical thinking is crucial for stable democracies. This paper introduces the design of ClarifAI, a novel automated propaganda detection tool designed to nudge readers towards more critical news consumption by activating the analytical mode of thinking, following Kahneman's dual-system theory of cognition. Using Large Language Models, ClarifAI detects propaganda in news articles and provides context-rich explanations, enhancing users' understanding and critical thinking. Our contribution is threefold: first, we propose the design of ClarifAI; second, in an online experiment, we demonstrate that this design effectively encourages news readers to engage in more critical reading; and third, we emphasize the value of explanations for fostering critical thinking. The study thus offers both a practical tool and useful design knowledge for mitigating propaganda in digital news.
Electrically defined quantum dots for bosonic excitons

Quantum dots are semiconductor nano-structures where particle motion is confined in all three spatial dimensions. Since their first experimental realization, nanocrystals confining the quanta of polarization waves, termed excitons, have found numerous applications in fields ranging from single photon sources for quantum information processing to commercial displays. A major limitation to further extending the range of potential applications has been the large inhomogeneity in, and lack-of tunability of, exciton energy that is generic to quantum dot materials. Here, we address this challenge by demonstrating electrically-defined quantum dots for excitons in monolayer semiconductors where the discrete exciton energies can be tuned using applied gate voltages. Resonance fluorescence measurements show strong spectral jumps and blinking of these resonances, verifying their zero-dimensional nature. Our work paves the way for realizing quantum confined bosonic modes where nonlinear response would arise exclusively from exciton--exciton interactions.
Automatic Radar Signal Detection and FFT Estimation using Deep Learning

This paper addresses a critical preliminary step in radar signal processing: detecting the presence of a radar signal and robustly estimating its bandwidth. Existing methods which are largely statistical feature-based approaches face challenges in electronic warfare (EW) settings where prior information about signals is lacking. While alternate deep learning based methods focus on more challenging environments, they primarily formulate this as a binary classification problem. In this research, we propose a novel methodology that not only detects the presence of a signal, but also localises it in the time domain and estimates its operating frequency band at that point in time. To achieve robust estimation, we introduce a compound loss function that leverages complementary information from both time-domain and frequency-domain representations. By integrating these approaches, we aim to improve the efficiency and accuracy of radar signal detection and parameter estimation, reducing both unnecessary resource consumption and human effort in downstream tasks.
Theoretical Foundations of Deep Selective State-Space Models

Structured state-space models (SSMs) such as S4, stemming from the seminal work of Gu et al., are gaining popularity as effective approaches for modeling sequential data. Deep SSMs demonstrate outstanding performance across a diverse set of domains, at a reduced training and inference cost compared to attention-based transformers. Recent developments show that if the linear recurrence powering SSMs allows for multiplicative interactions between inputs and hidden states (e.g. GateLoop, Mamba, GLA), then the resulting architecture can surpass in both in accuracy and efficiency attention-powered foundation models trained on text, at scales of billion parameters. In this paper, we give theoretical grounding to this recent finding using tools from Rough Path Theory: we show that when random linear recurrences are equipped with simple input-controlled transitions (selectivity mechanism), then the hidden state is provably a low-dimensional projection of a powerful mathematical object called the signature of the input -- capturing non-linear interactions between tokens at distinct timescales. Our theory not only motivates the success of modern selective state-space models such as Mamba but also provides a solid framework to understand the expressive power of future SSM variants.
Overweight in preschool children behavioral problems and early risk factors

El sobrepes en la infancia s'ha convertit en un problema de salut publica ja que esta relacionat amb importants comorbiditats mediques i cada cop nens mes petits tenen sobrepes. Objectius: Examinar de manera transversal i longitudinal les associacions entre problemes de conducta i el sobrepes, i identificar els factors de risc pel sobrepes. Metode: Una mostra comunitaria de n=622 nens de 3 anys va ser seguida fins als 5 anys, registrant anualment l'altura, el pes i la psicopatologia. Els seguents instruments van ser administrats als pares: el Questionari de Fortaleses i Dificultats, l'Entrevista Diagnostica per Nens i Adolescents i l'Entrevista de Factors de Risc. L'analisi del pes i l'altura va considerar l'index de massa corporal (IMC) i el sobrepes definit segons els estandards de l'Organitzacio Mundial de la Salut. Els analisis estadistics van incloure els models generals lineals i regressions binaries logistiques. Resultats: Els nens que tenien sobrepes i un IMC mes elevat, tenien mes risc de tenir problemes amb els companys i simptomes del trastorn de deficit d'atencio amb hiperactivitat (TDAH). L'analisi longitudinal va mostrar que un major IMC als 3 anys era predictiu de puntuacions mes elevades en la dimensio problemes amb els companys als 4 i 5 anys i majors puntuacions en la dimensio d'hiperactivitat i TDAH als 4 anys. Addicionalment, els resultats van mostrar dos factors de risc pel sobrepes i l'IMC en l'etapa preescolar: nivell socioeconomic baix i major pes al neixer. Conclusions: Aquest estudi es el primer, que usant un instrument psicologic de diagnostic, mostra una relacio entre el IMC/sobrepes i el TDAH en preescolars. Els nens que tenen sobrepes, ja des de ben petits, podrien beneficiar-se del cribratge per problemes de conducta i problemes de relacio amb els companys. A mes, la identificacio en una etapa primerenca dels nens amb un major risc de tenir sobrepes ofereix el potencial per l'elaboracio de programes de prevencio de sobrepes.