Miracle-dz/UniFAR-FTUs huggingface dataset:
Steady-state analysis of reflected Brownian motions: characterization, numerical methods and queueing applications

This dissertation is concerned with multidimensional diffusion processes that arise as approximate models of queueing networks. To be specific, we consider two classes of semimartingale reflected Brownian motions (SRBM’s), each with polyhedral state space. For one class the state space is a two-dimensional rectangle, and for the other class it is the general d-dimensional non-negative orthant R+. SRBM in a rectangle has been identified as an approximate model of a two-station queueing network with finite storage space at each station. Until now, however, the foundational characteristics of the process have not been rigorously established. Building on previous work by Varadhan and Williams [53] and by Taylor and Williams [50], we show how SRBM in a rectangle can be constructed by means of localization. The process is shown to be unique in law, and therefore to be a Feller continuous strong Markov process. Taylor and Williams [50] have proved the analogous foundational result for SRBM in the non-negative orthant R+, which arises as an approximate model of a d-station open queueing network with infinite storage space at every station. Motivated by the applications in queueing theory, our focus is on steady-state analysis of SRBM, which involves three tasks: (a) determining when a stationary distribution exists; (b) developing an analytical characterization of the stationary distribution; and (c) computing the stationary distribution from that characterization. With regard to (a), we give a sufficient condition for the existence of a stationary distribution in terms of Liapunov functions. With regard to (b), for a special class of SRBM’s in an orthant, Harrison and Williams [26] showed that the stationary distribution must satisfy a weak form of an adjoint linear elliptic partial differential equation with oblique derivative boundary conditions, which they called the basic adjoint relationship (BAR). They further conjectured that (BAR) characterizes the stationary distribution. We give two proofs of their conjecture. For an SRBM in a rectangle, using Echeverria’s Theorem [10], we give a direct proof of their
2D and Slab Turbulent Cascade Rates in the Inner Heliosphere

We present a theoretical and observational study of 2D and slab turbulence cascade (or heating) rates of transverse total turbulence energies, transverse cross helicity, transverse outward and inward Elsässer energy, transverse fluctuating magnetic energy density, and transverse fluctuating kinetic energy from the perihelion of the first Parker Solar Probe (PSP) orbit at ∼36.6 R ⊙ to Solar Orbiter (SolO) at ∼177 R ⊙. We use the Adhikari et al. (2021a) approach to calculate the observed transverse turbulence heating rate, and the nearly incompressible magnetohydrodynamic (NI MHD) turbulence transport theory to calculate the theoretical turbulence cascade rate. We find from the 1 day long PSP measurements at 66.5 R ⊙, and the SolO measurements at 176.3 R ⊙ that various transverse turbulent cascade rates increase with increasing angle, from 10° to 98°, between the mean solar wind speed and mean magnetic field (θ UB), indicating that the 2D heating rate is largest in the inner heliosphere. Similarly, we find from the theoretical and observed results that the 2D heating rate is larger than the slab heating rate as a function of heliocentric distance. We present a comparison between the theoretical and observed 2D and slab turbulence cascade rates as a function of heliocentric distance.
Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions

Policy Optimization (PO) algorithms have been proven particularly suited to handle the high-dimensionality of real-world continuous control tasks. In this context, Trust Region Policy Optimization methods represent a popular approach to stabilize the policy updates. These usually rely on the Kullback-Leibler (KL) divergence to limit the change in the policy. The Wasserstein distance represents a natural alternative, in place of the KL divergence, to define trust regions or to regularize the objective function. However, state-of-the-art works either resort to its approximations or do not provide an algorithm for continuous state-action spaces, reducing the applicability of the method. In this paper, we explore optimal transport discrepancies (which include the Wasserstein distance) to define trust regions, and we propose a novel algorithm - Optimal Transport Trust Region Policy Optimization (OT-TRPO) - for continuous state-action spaces. We circumvent the infinite-dimensional optimization problem for PO by providing a one-dimensional dual reformulation for which strong duality holds. We then analytically derive the optimal policy update given the solution of the dual problem. This way, we bypass the computation of optimal transport costs and of optimal transport maps, which we implicitly characterize by solving the dual formulation. Finally, we provide an experimental evaluation of our approach across various control tasks. Our results show that optimal transport discrepancies can offer an advantage over state-of-the-art approaches.
Topology Optimization via Machine Learning and Deep Learning: A Review

Topology optimization (TO) is a method of deriving an optimal design that satisfies a given load and boundary conditions within a design domain. This method enables effective design without initial design, but has been limited in use due to high computational costs. At the same time, machine learning (ML) methodology including deep learning has made great progress in the 21st century, and accordingly, many studies have been conducted to enable effective and rapid optimization by applying ML to TO. Therefore, this study reviews and analyzes previous research on ML-based TO (MLTO). Two different perspectives of MLTO are used to review studies: (1) TO and (2) ML perspectives. The TO perspective addresses “why” to use ML for TO, while the ML perspective addresses “how” to apply ML to TO. In addition, the limitations of current MLTO research and future research directions are examined.
Decomposing cubic graphs into isomorphic linear forests

A common problem in graph colouring seeks to decompose the edge set of a given graph into few similar and simple subgraphs, under certain divisibility conditions. In 1987 Wormald conjectured that the edges of every cubic graph on $4n$ vertices can be partitioned into two isomorphic linear forests. We prove this conjecture for large connected cubic graphs. Our proof uses a wide range of probabilistic tools in conjunction with intricate structural analysis, and introduces a variety of local recolouring techniques.
Robustcaps: a transformation-robust capsule network for image classification

Geometric transformations of the training data as well as the test data present challenges to the use of deep neural networks to vision-based learning tasks. In order to address this issue, we present a deep neural network model that exhibits the desirable property of transformation-robustness. Our model, termed RobustCaps, uses group-equivariant convolutions in an improved capsule network model. RobustCaps uses a global context-normalised procedure in its routing algorithm to learn transformation-invariant part-whole relationships within image data. This learning of such relationships allows our model to outperform both capsule and convolutional neural network baselines on transformation-robust classification tasks. Specifically, RobustCaps achieves state-of-the-art accuracies on CIFAR-10, FashionMNIST, and CIFAR-100 when the images in these datasets are subjected to train and test-time rotations and translations.
Ultrafast behavior of induced and intrinsic magnetic moments in CoFeB/Pt bilayers probed by element-specific measurements in the extreme ultraviolet spectral range

The ultrafast and element-specific response of magnetic systems containing ferromagnetic 3d transition metals and 4d/5d heavy metals is of interest both from a fundamental as well as an applied research perspective. However, to date no consensus about the main microscopic processes describing the interplay between intrinsic 3d and induced 4d/5d magnetic moments upon femtosecond laser excitation exist. In this work, we study the ultrafast response of CoFeB/Pt bilayers by probing element-specific, core-to-valence band transitions in the extreme ultraviolet spectral range using high harmonic radiation. We show that the combination of magnetic scattering simulations and analysis of the energy- and time-dependent magnetic asymmetries allows to accurately disentangle the element-specific response in spite of overlapping Co and Fe M$_{2,3}$ as well as Pt O$_{2,3}$ and N$_7$ resonances. We find a considerably smaller demagnetization time constant as well as much larger demagnetization amplitudes of the induced moment of Pt compared to the intrinsic moment of CoFeB. Our results are in agreement with enhanced spin-flip probabilities due to the high spin-orbit coupling localized at the heavy metal Pt, as well as with the recently formulated hypothesis that a laser generated, incoherent magnon population within the ferromagnetic film leads to an overproportional reduction of the induced magnetic moment of Pt.
Cluster and Aggregate: Face Recognition with Large Probe Set

Feature fusion plays a crucial role in unconstrained face recognition where inputs (probes) comprise of a set of $N$ low quality images whose individual qualities vary. Advances in attention and recurrent modules have led to feature fusion that can model the relationship among the images in the input set. However, attention mechanisms cannot scale to large $N$ due to their quadratic complexity and recurrent modules suffer from input order sensitivity. We propose a two-stage feature fusion paradigm, Cluster and Aggregate, that can both scale to large $N$ and maintain the ability to perform sequential inference with order invariance. Specifically, Cluster stage is a linear assignment of $N$ inputs to $M$ global cluster centers, and Aggregation stage is a fusion over $M$ clustered features. The clustered features play an integral role when the inputs are sequential as they can serve as a summarization of past features. By leveraging the order-invariance of incremental averaging operation, we design an update rule that achieves batch-order invariance, which guarantees that the contributions of early image in the sequence do not diminish as time steps increase. Experiments on IJB-B and IJB-S benchmark datasets show the superiority of the proposed two-stage paradigm in unconstrained face recognition. Code and pretrained models are available in https://github.com/mk-minchul/caface
Klt Varieties With Conjecturally Minimal Volume

We construct klt projective varieties with ample canonical class and the smallest known volume. We also find exceptional klt Fano varieties with the smallest known anti-canonical volume. We conjecture that our examples have the smallest volume in every dimension, and we give low-dimensional evidence for that. In order to improve on earlier examples, we are forced to consider weighted hypersurfaces that are not quasi-smooth. We show that our Fano varieties are exceptional by computing their global log canonical threshold (or $\alpha $-invariant) exactly; it is extremely large, roughly $2^{2^n}$ in dimension $n$. These examples give improved lower bounds in Birkar’s theorem on boundedness of complements for Fano varieties.
Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation

Simultaneous machine translation (SiMT) outputs the translation while receiving the source inputs, and hence needs to balance the received source information and translated target information to make a reasonable decision between waiting for inputs or outputting translation. Previous methods always balance source and target information at the token level, either directly waiting for a fixed number of tokens or adjusting the waiting based on the current token. In this paper, we propose a Wait-info Policy to balance source and target at the information level. We first quantify the amount of information contained in each token, named info. Then during simultaneous translation, the decision of waiting or outputting is made based on the comparison results between the total info of previous target outputs and received source inputs. Experiments show that our method outperforms strong baselines under and achieves better balance via the proposed info.
Global Convergence of SGD On Two Layer Neural Nets

In this note we demonstrate provable convergence of SGD to the global minima of appropriately regularized $\ell_2-$empirical risk of depth $2$ nets -- for arbitrary data and with any number of gates, if they are using adequately smooth and bounded activations like sigmoid and tanh. We build on the results in [1] and leverage a constant amount of Frobenius norm regularization on the weights, along with sampling of the initial weights from an appropriate distribution. We also give a continuous time SGD convergence result that also applies to smooth unbounded activations like SoftPlus. Our key idea is to show the existence loss functions on constant sized neural nets which are"Villani Functions". [1] Bin Shi, Weijie J. Su, and Michael I. Jordan. On learning rates and schr\"odinger operators, 2020. arXiv:2004.06977
Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models

Prompt learning is a new learning paradigm which reformulates downstream tasks as similar pretraining tasks on pretrained models by leveraging textual prompts. Recent works have demonstrated that prompt learning is particularly useful for few-shot learning, where there is limited training data. Depending on the granularity of prompts, those methods can be roughly divided into task-level prompting and instance-level prompting. Task-level prompting methods learn one universal prompt for all input samples, which is efficient but ineffective to capture subtle differences among different classes. Instance-level prompting methods learn a specific prompt for each input, though effective but inefficient. In this work, we develop a novel prototype-based prompt learning method to overcome the above limitations. In particular, we focus on few-shot image recognition tasks on pretrained vision-language models (PVLMs) and develop a method of prompting through prototype (PTP), where we define $K$ image prototypes and $K$ prompt prototypes. In PTP, the image prototype represents a centroid of a certain image cluster in the latent space and a prompt prototype is defined as a soft prompt in the continuous space. The similarity between a query image and an image prototype determines how much this prediction relies on the corresponding prompt prototype. Hence, in PTP, similar images will utilize similar prompting ways. Through extensive experiments on seven real-world benchmarks, we show that PTP is an effective method to leverage the latent knowledge and adaptive to various PVLMs. Moreover, through detailed analysis, we discuss pros and cons for prompt learning and parameter-efficient fine-tuning under the context of few-shot learning.
Competitive Equilibrium for Dynamic Multi-Agent Systems: Social Shaping and Price Trajectories

In this paper, we consider dynamic multi-agent systems (MAS) for decentralized resource allocation. The MAS operates at a competitive equilibrium to ensure supply and demand are balanced. First, we investigate the MAS over a finite horizon. The utility functions of agents are parameterized to incorporate individual preferences. We shape individual preferences through a set of utility functions to guarantee the resource price at a competitive equilibrium remains socially acceptable, i.e., the price is upper-bounded by an affordability threshold. We show this problem is solvable at the conceptual level. Next, we consider quadratic MAS and formulate the associated social shaping problem as a multi-agent linear quadratic regulator (LQR) problem which enables us to propose explicit utility sets using quadratic programming and dynamic programming. Then, a numerical algorithm is presented for calculating a tight range of the preference function parameters which guarantees a socially accepted price. We investigate the properties of a competitive equilibrium over an infinite horizon. Considering general utility functions, we show that under feasibility assumptions, any competitive equilibrium maximizes the social welfare. Then, we prove that for sufficiently small initial conditions, the social welfare maximization solution constitutes a competitive equilibrium with zero price. We also prove for general feasible initial conditions, there exists a time instant after which the optimal price, corresponding to a competitive equilibrium, becomes zero. Finally, we specifically focus on quadratic MAS and propose explicit results.
Diverse Sample Generation: Pushing the Limit of Generative Data-Free Quantization

Generative data-free quantization emerges as a practical compression approach that quantizes deep neural networks to low bit-width without accessing the real data. This approach generates data utilizing batch normalization (BN) statistics of the full-precision networks to quantize the networks. However, it always faces the serious challenges of accuracy degradation in practice. We first give a theoretical analysis that the diversity of synthetic samples is crucial for the data-free quantization, while in existing approaches, the synthetic data completely constrained by BN statistics experimentally exhibit severe homogenization at distribution and sample levels. This paper presents a generic Diverse Sample Generation (DSG) scheme for the generative data-free quantization, to mitigate detrimental homogenization. We first slack the statistics alignment for features in the BN layer to relax the distribution constraint. Then, we strengthen the loss impact of the specific BN layers for different samples and inhibit the correlation among samples in the generation process, to diversify samples from the statistical and spatial perspectives, respectively. Comprehensive experiments show that for large-scale image classification tasks, our DSG can consistently quantization performance on different neural architectures, especially under ultra-low bit-width. And data diversification caused by our DSG brings a general gain to various quantization-aware training and post-training quantization approaches, demonstrating its generality and effectiveness.
Singlet Mesons in Dark $Sp(4)$ Theories

We explore some aspects of $Sp(4)$ gauge theory with two fundamental fermions in the context of composite Goldstone Dark Matter. We present preliminary lattice results for the mass of the pseudoscalar iso-singlet meson $\eta'$ using unimproved Wilson fermions and the standard plaquette action. We find that the $\eta'$ is slightly heavier than the pseudoscalar non-singlets $\pi$ and lighter than the vector mesons $\rho$ for multiple ensembles with $m_\pi/m_\rho \geq 0.77$. This pattern is potentially relevant for Beyond the Standard Model physics model building. Furthermore, we show that for $N_f=1+1$ flavours the disconnected contributions to the unflavoured pseudoscalar $\pi^0$ are small. We supplement this measurement by a calculation of the $\pi\pi$ scattering length $a_0$ which shows that the ensemble studied might be phenomenologically relevant for Dark Matter models such as the Strongly Interacting Massive Particles paradigm.
Neutrinoless double-beta decay at colliders: interference between Majorana states

Heavy neutral leptons (HNLs) are hypothetical particles able to explain several puzzles of fundamental physics, first and foremost - neutrino oscillations. Being sterile with respect to Standard Model interactions, these particles admit Majorana masses, allowing for violation of the total lepton number. Lepton number violating (LNV) processes thus become a key signature of HNLs, pursued by many experiments. In this work, we demonstrate that if HNLs are the sole origin of neutrino masses, destructive interference between Majorana states suppresses the same-sign di-lepton signal. In the phenomenologically interesting case of large HNL couplings, such suppression is akin to the cancellation of HNLs' contributions to neutrino masses. Nevertheless, the signal can be much larger than coming from the Weinberg operator alone. We identify regions of the parameter space of such realistic HNL models where the LNV signal is maximised at the LHC and future FCC-hh. Our results are obtained within the effective W approximation which allows for analytic treatment and gives a clear dependence on the model parameters. Although approximate, they are argued to be correct within a factor of a few.
Optimal time for sensing in open quantum systems

We study the time-dependent quantum Fisher information (QFI) in an open quantum system satisfying the Gorini-Kossakowski-Sudarshan-Lindblad master equation. We also study the dynamics of the system from an effective non-Hermitian dynamics standpoint and use it to understand the scaling of the QFI when multiple probes are used. A focus of our work is how the QFI is maximized at certain times suggesting that the best precision in parameter estimation can be achieved by focusing on these times. The propagation of errors analysis allows us to confirm and better understand this idea. We also propose a parameter estimation procedure involving relatively low resource consuming measurements followed by higher resource consuming measurements and demonstrate it in simulation.
Scaling Instruction-Finetuned Language Models

Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.