Miracle-dz/UniFAR-FTUs huggingface dataset:
A 3-D Finite Element Computation of Free Convection From A Cubical Structure Buried in A Fluid Saturated Porous Enclosure on Anu-Cluster

Abstract Free convection from a hot cubical structure buried inside a fluid saturated porous media has been studied numerically by finite element method. In view of the large size of the linear systems encountered in the 3-D finite element computations, a numerical scheme for a segregated variable analysis under the paradigm of domain decomposition method has been proposed. Suitable sparse data structures and algorithms are designed for efficient implementation on a cluster of PCs called ANUCLUSTER under ANULIB parallel processing environment. The parallel numerical simulations are carried out for various values of Ra and for different sizes of isothermal hot cubical objects.
Structured Voronoi Sampling

Gradient-based sampling algorithms have demonstrated their effectiveness in text generation, especially in the context of controlled text generation. However, there exists a lack of theoretically grounded and principled approaches for this task. In this paper, we take an important step toward building a principled approach for sampling from language models with gradient-based methods. We use discrete distributions given by language models to define densities and develop an algorithm based on Hamiltonian Monte Carlo to sample from them. We name our gradient-based technique Structured Voronoi Sampling (SVS). In an experimental setup where the reference distribution is known, we show that the empirical distribution of SVS samples is closer to the reference distribution compared to alternative sampling schemes. Furthermore, in a controlled generation task, SVS is able to generate fluent and diverse samples while following the control targets significantly better than other methods.
Learning to Stabilize Online Reinforcement Learning in Unbounded State Spaces

In many reinforcement learning (RL) applications, we want policies that reach desired states and then keep the controlled system within an acceptable region around the desired states over an indefinite period of time. This latter objective is called stability and is especially important when the state space is unbounded, such that the states can be arbitrarily far from each other and the agent can drift far away from the desired states. For example, in stochastic queuing networks, where queues of waiting jobs can grow without bound, the desired state is all-zero queue lengths. Here, a stable policy ensures queue lengths are finite while an optimal policy minimizes queue lengths. Since an optimal policy is also stable, one would expect that RL algorithms would implicitly give us stable policies. However, in this work, we find that deep RL algorithms that directly minimize the distance to the desired state during online training often result in unstable policies, i.e., policies that drift far away from the desired state. We attribute this instability to poor credit-assignment for destabilizing actions. We then introduce an approach based on two ideas: 1) a Lyapunov-based cost-shaping technique and 2) state transformations to the unbounded state space. We conduct an empirical study on various queueing networks and traffic signal control problems and find that our approach performs competitively against strong baselines with knowledge of the transition dynamics.
Using Cosmic Ray Muons to Assess Geological Characteristics in the Subsurface

Cosmic rays are energetic nuclei and elementary particles that originate from stars and intergalactic events. The interaction of these particles with the upper atmosphere produces a range of secondary particles that reach the surface of the earth, of which muons are the most prominent. With enough energy, muons can travel up to a few kilometers beneath the surface of the earth before being stopped completely. The terrestrial muon flux profile and associated zenith angle can be utilized to determine geological characteristics of a location without having to use conventional methods. This work intends to use a low-power plastic scintillator-based muon detection system for this non-destructive geological assay methodology. 4 custom designed plastic scintillation panels are used to realize two orthogonal detection planes. Simultaneous triggers between detectors from two planes indicate a coincidence event which is recorded using a data acquisition system from FNAL. In order to quantify the systematic uncertainties associated with the detector, such as energy depositions and angular resolution of the detector design, a Monte Carlo simulation using Geant4 is being developed. Simulated and experimental data will drive the development and validation of a reconstruction algorithm that, upon completion, is expected to predict average overburden and rock density. Extended detector exposure to muons can be used as a means to understand changes in the surrounding environment like rock porosity. On the experimental front, the measured flux data will be used to benchmark independent and established models. Successful proof-of-concept demonstration of this technology can open doors for long term non-invasive geological monitoring. The detector design, and experimental methodology are detailed in this work.
Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardware

Existing methods to recover model accuracy on analog-digital hardware in the presence of quantization and analog noise include noise-injection training. However, it can be slow in practice, incurring high computational costs, even when starting from pretrained models. We introduce the Sensitivity-Aware Finetuning (SAFT) approach that identifies noise sensitive layers in a model, and uses the information to freeze specific layers for noise-injection training. Our results show that SAFT achieves comparable accuracy to noise-injection training and is 2x to 8x faster.
Efficient Approximation Algorithms for Scheduling Coflows With Total Weighted Completion Time in Identical Parallel Networks

This article addresses the scheduling problem of coflows in identical parallel networks, a well-known <inline-formula><tex-math notation="LaTeX">$\mathcal {NP}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">NP</mml:mi></mml:math><inline-graphic xlink:href="chen-ieq1-3340729.gif"/></alternatives></inline-formula>-hard problem. We consider both flow-level scheduling and coflow-level scheduling problems. In the flow-level scheduling problem, flows within a coflow can be transmitted through different network cores, while in the coflow-level scheduling problem, flows within a coflow must be transmitted through the same network core. The key difference between these two problems lies in their scheduling granularity. Previous approaches relied on linear programming to solve the scheduling order. In this article, we enhance the efficiency of solving by utilizing the primal-dual method. For the flow-level scheduling problem, we propose an approximation algorithm that achieves approximation ratios of <inline-formula><tex-math notation="LaTeX">$6-\frac{2}{m}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>-</mml:mo><mml:mfrac><mml:mn>2</mml:mn><mml:mi>m</mml:mi></mml:mfrac></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq2-3340729.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">$5-\frac{2}{m}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>5</mml:mn><mml:mo>-</mml:mo><mml:mfrac><mml:mn>2</mml:mn><mml:mi>m</mml:mi></mml:mfrac></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq3-3340729.gif"/></alternatives></inline-formula> for arbitrary and zero release times, respectively, where <inline-formula><tex-math notation="LaTeX">$m$</tex-math><alternatives><mml:math><mml:mi>m</mml:mi></mml:math><inline-graphic xlink:href="chen-ieq4-3340729.gif"/></alternatives></inline-formula> represents the number of network cores. Additionally, for the coflow-level scheduling problem, we introduce an approximation algorithm that achieves approximation ratios of <inline-formula><tex-math notation="LaTeX">$4m+1$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>4</mml:mn><mml:mi>m</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq5-3340729.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\text{4}m$</tex-math><alternatives><mml:math><mml:mrow><mml:mtext>4</mml:mtext><mml:mi>m</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq6-3340729.gif"/></alternatives></inline-formula> for arbitrary and zero release times, respectively. The algorithm presented in this article has practical applications in data centers, such as those operated by Google or Facebook. The simulated results demonstrate the superior performance of our algorithms compared to previous approach, emphasizing their practical utility.
Unsupervised Low Light Image Enhancement Using SNR-Aware Swin Transformer

Image captured under low-light conditions presents unpleasing artifacts, which debilitate the performance of feature extraction for many upstream visual tasks. Low-light image enhancement aims at improving brightness and contrast, and further reducing noise that corrupts the visual quality. Recently, many image restoration methods based on Swin Transformer have been proposed and achieve impressive performance. However, on one hand, trivially employing Swin Transformer for low-light image enhancement would expose some artifacts, including over-exposure, brightness imbalance and noise corruption, etc. On the other hand, it is impractical to capture image pairs of low-light images and corresponding ground-truth, i.e. well-exposed image in same visual scene. In this paper, we propose a dual-branch network based on Swin Transformer, guided by a signal-to-noise ratio prior map which provides the spatial-varying information for low-light image enhancement. Moreover, we leverage unsupervised learning to construct the optimization objective based on Retinex model, to guide the training of proposed network. Experimental results demonstrate that the proposed model is competitive with the baseline models.
Learning in Domain Randomization via Continuous Time Non-Stochastic Control

Domain randomization is a popular method for robustly training agents to adapt to diverse environments and real-world tasks. In this paper, we examine how to train an agent in domain randomization environments from a nonstochastic control perspective. We first theoretically study online control of continuous-time linear systems under nonstochastic noises. We present a novel two-level online algorithm, by integrating a higher-level learning strategy and a lower-level feedback control strategy. This method offers a practical solution, and for the first time achieves sublinear regret in continuous-time nonstochastic systems. Compared to standard online learning algorithms, our algorithm features a stack and skip procedure. By applying stack and skip to the SAC (Soft Actor-Critic) algorithm, we achieved improved results in multiple reinforcement learning tasks within domain randomization environments. Our work provides new insights into nonasymptotic analyses of controlling continuous-time systems. Further, our work justifies the importance of stacked and skipped in controller learning under nonstochastic environments.
Learning Similarity among Users for Personalized Session-Based Recommendation from hierarchical structure of User-Session-Item

The task of the session-based recommendation is to predict the next interaction of the user based on the anonymized user's behavior pattern. And personalized version of this system is a promising research field due to its availability to deal with user information. However, there's a problem that the user's preferences and historical sessions were not considered in the typical session-based recommendation since it concentrates only on user-item interaction. In addition, the existing personalized session-based recommendation model has a limited capability in that it only considers the preference of the current user without considering those of similar users. It means there can be the loss of information included within the hierarchical data structure of the user-session-item. To tackle with this problem, we propose USP-SBR(abbr. of User Similarity Powered - Session Based Recommender). To model global historical sessions of users, we propose UserGraph that has two types of nodes - ItemNode and UserNode. We then connect the nodes with three types of edges. The first type of edges connects ItemNode as chronological order, and the second connects ItemNode to UserNode, and the last connects UserNode to ItemNode. With these user embeddings, we propose additional contrastive loss, that makes users with similar intention be close to each other in the vector space. we apply graph neural network on these UserGraph and update nodes. Experimental results on two real-world datasets demonstrate that our method outperforms some state-of-the-art approaches.
Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs

Even after fine-tuning and reinforcement learning, large language models (LLMs) can be difficult, if not impossible, to control reliably with prompts alone. We propose a new inference-time approach to enforcing syntactic and semantic constraints on the outputs of LLMs, called sequential Monte Carlo (SMC) steering. The key idea is to specify language generation tasks as posterior inference problems in a class of discrete probabilistic sequence models, and replace standard decoding with sequential Monte Carlo inference. For a computational cost similar to that of beam search, SMC can steer LLMs to solve diverse tasks, including infilling, generation under syntactic constraints, and prompt intersection. To facilitate experimentation with SMC steering, we present a probabilistic programming library, LLaMPPL (https://github.com/probcomp/hfppl), for concisely specifying new generation tasks as language model probabilistic programs, and automating steering of LLaMA-family Transformers.
On Emergence of Clean-Priority Learning in Early Stopped Neural Networks

When random label noise is added to a training dataset, the prediction error of a neural network on a label-noise-free test dataset initially improves during early training but eventually deteriorates, following a U-shaped dependence on training time. This behaviour is believed to be a result of neural networks learning the pattern of clean data first and fitting the noise later in the training, a phenomenon that we refer to as clean-priority learning. In this study, we aim to explore the learning dynamics underlying this phenomenon. We theoretically demonstrate that, in the early stage of training, the update direction of gradient descent is determined by the clean subset of training data, leaving the noisy subset has minimal to no impact, resulting in a prioritization of clean learning. Moreover, we show both theoretically and experimentally, as the clean-priority learning goes on, the dominance of the gradients of clean samples over those of noisy samples diminishes, and finally results in a termination of the clean-priority learning and fitting of the noisy samples.
Tight Regret Bounds for Single-pass Streaming Multi-armed Bandits

Regret minimization in streaming multi-armed bandits (MABs) has been studied extensively in recent years. In the single-pass setting with $K$ arms and $T$ trials, a regret lower bound of $\Omega(T^{2/3})$ has been proved for any algorithm with $o(K)$ memory (Maiti et al. [NeurIPS'21]; Agarwal at al. [COLT'22]). On the other hand, however, the previous best regret upper bound is still $O(K^{1/3} T^{2/3}\log^{1/3}(T))$, which is achieved by the streaming implementation of the simple uniform exploration. The $O(K^{1/3}\log^{1/3}(T))$ gap leaves the open question of the tight regret bound in the single-pass MABs with sublinear arm memory. In this paper, we answer this open problem and complete the picture of regret minimization in single-pass streaming MABs. We first improve the regret lower bound to $\Omega(K^{1/3}T^{2/3})$ for algorithms with $o(K)$ memory, which matches the uniform exploration regret up to a logarithm factor in $T$. We then show that the $\log^{1/3}(T)$ factor is not necessary, and we can achieve $O(K^{1/3}T^{2/3})$ regret by finding an $\varepsilon$-best arm and committing to it in the rest of the trials. For regret minimization with high constant probability, we can apply the single-memory $\varepsilon$-best arm algorithms in Jin et al. [ICML'21] to obtain the optimal bound. Furthermore, for the expected regret minimization, we design an algorithm with a single-arm memory that achieves $O(K^{1/3} T^{2/3}\log(K))$ regret, and an algorithm with $O(\log^{*}(n))$-memory with the optimal $O(K^{1/3} T^{2/3})$ regret following the $\varepsilon$-best arm algorithm in Assadi and Wang [STOC'20]. We further tested the empirical performances of our algorithms. The simulation results show that the proposed algorithms consistently outperform the benchmark uniform exploration algorithm by a large margin, and on occasion, reduce the regret by up to 70%.
Citizen Perspectives on Necessary Safeguards to the Use of AI by Law Enforcement Agencies

In the light of modern technological advances, Artificial Intelligence (AI) is relied upon to enhance performance, increase efficiency, and maximize gains. For Law Enforcement Agencies (LEAs), it can prove valuable in optimizing evidence analysis and establishing proactive prevention measures. Nevertheless, citizens raise legitimate concerns around privacy invasions, biases, inequalities, and inaccurate decisions. This study explores the views of 111 citizens towards AI use by police through interviews, and integrates societal concerns along with propositions of safeguards from negative effects of AI use by LEAs in the context of cybercrime and terrorism.
Influence of Lossy Speech Codecs on Hearing-aid, Binaural Sound Source Localisation using DNNs

Hearing aids are typically equipped with multiple microphones to exploit spatial information for source localisation and speech enhancement. Especially for hearing aids, a good source localisation is important: it not only guides source separation methods but can also be used to enhance spatial cues, increasing user-awareness of important events in their surroundings. We use a state-of-the-art deep neural network (DNN) to perform binaural direction-of-arrival (DoA) estimation, where the DNN uses information from all microphones at both ears. However, hearing aids have limited bandwidth to exchange this data. Bluetooth low-energy (BLE) is emerging as an attractive option to facilitate such data exchange, with the LC3plus codec offering several bitrate and latency trade-off possibilities. In this paper, we investigate the effect of such lossy codecs on localisation accuracy. Specifically, we consider two conditions: processing at one ear vs processing at a central point, which influences the number of channels that need to be encoded. Performance is benchmarked against a baseline that allows full audio-exchange - yielding valuable insights into the usage of DNNs under lossy encoding. We also extend the Pyroomacoustics library to include hearing-device and head-related transfer functions (HD-HRTFs) to suitably train the networks. This can also benefit other researchers in the field.
Communication-Efficient Federated Learning With Data and Client Heterogeneity

Federated Learning (FL) enables large-scale distributed training of machine learning models, while still allowing individual nodes to maintain data locally. However, executing FL at scale comes with inherent practical challenges: 1) heterogeneity of the local node data distributions, 2) heterogeneity of node computational speeds (asynchrony), but also 3) constraints in the amount of communication between the clients and the server. In this work, we present the first variant of the classic federated averaging (FedAvg) algorithm which, at the same time, supports data heterogeneity, partial client asynchrony, and communication compression. Our algorithm comes with a rigorous analysis showing that, in spite of these system relaxations, it can provide similar convergence to FedAvg in interesting parameter regimes. Experimental results in the rigorous LEAF benchmark on setups of up to $300$ nodes show that our algorithm ensures fast convergence for standard federated tasks, improving upon prior quantized and asynchronous approaches.
Gibbs Sampling the Posterior of Neural Networks

In this paper, we study sampling from a posterior derived from a neural network. We propose a new probabilistic model consisting of adding noise at every pre- and post-activation in the network, arguing that the resulting posterior can be sampled using an efficient Gibbs sampler. For small models, the Gibbs sampler attains similar performances as the state-of-the-art Monte Carlo Markov chain methods, such as the Hamiltonian Monte Carlo or the Metropolis adjusted Langevin algorithm, both on real and synthetic data. By framing our analysis in the teacher-student setting, we introduce a thermalization criterion that allows us to detect when an algorithm, when run on data with synthetic labels, fails to sample from the posterior. The criterion is based on the fact that in the teacher-student setting we can initialize an algorithm directly at equilibrium.
Role of three-body dynamics in nucleon-deuteron correlation functions

Correlation functions of hadrons can be accessed in high-energy collisions of atomic nuclei, revealing information about the underlying interaction. This work complements experimental efforts to study nucleon-deuteron $Nd$ -- with $N=p$ (proton) or $N=n$ (neutron) -- correlations with theory evaluations using different techniques. The correlation functions $C_{nd}$ and $C_{pd}$ are calculated based on scattering wave function, extending previous benchmarks for the $Nd$ scattering matrix to this new observable. We use hyperspherical harmonics and Faddeev techniques with one of the widely used nucleon-nucleon ($NN$) interactions, the Argonne $v_{18}$ potential. Moreover, in the low-energy region we perform additional calculations in the framework of pionless effective field theory. The $pd$ correlation function is computed in the large-energy region to make contact with a recent measurement by the ALICE Collaboration. We show that the scattering wave function has the proper dynamical input to describe an initial rise and subsequent oscillations of $C_{pd}$ as a function of the energy. Effects on the observables using different $NN$ and three-nucleon potentials are evaluated with the conclusion that variations of around $2\%$ are observed. Although these effects are small, future measurements can go beyond this accuracy allowing for new detailed studies of strong interaction in light nuclear systems. The present study supports the current efforts devoted to the measurement of correlation functions in systems dominated by the strong interactions, such as $pd$, $ppp$, $\Lambda d$ and $pp\Lambda$.