Miracle-dz/UniFAR-FTUs huggingface dataset:
Almost Cost-Free Communication in Federated Best Arm Identification

We study the problem of best arm identification in a federated learning multi-armed bandit setup with a central server and multiple clients. Each client is associated with a multi-armed bandit in which each arm yields i.i. d. rewards following a Gaussian distribution with an unknown mean and known variance. The set of arms is assumed to be the same at all the clients. We define two notions of best arm local and global. The local best arm at a client is the arm with the largest mean among the arms local to the client, whereas the global best arm is the arm with the largest average mean across all the clients. We assume that each client can only observe the rewards from its local arms and thereby estimate its local best arm. The clients communicate with a central server on uplinks that entail a cost of C>=0 units per usage per uplink. The global best arm is estimated at the server. The goal is to identify the local best arms and the global best arm with minimal total cost, defined as the sum of the total number of arm selections at all the clients and the total communication cost, subject to an upper bound on the error probability. We propose a novel algorithm FedElim that is based on successive elimination and communicates only in exponential time steps and obtain a high probability instance-dependent upper bound on its total cost. The key takeaway from our paper is that for any C>=0 and error probabilities sufficiently small, the total number of arm selections (resp. the total cost) under FedElim is at most 2 (resp. 3) times the maximum total number of arm selections under its variant that communicates in every time step. Additionally, we show that the latter is optimal in expectation up to a constant factor, thereby demonstrating that communication is almost cost-free in FedElim. We numerically validate the efficacy of FedElim on two synthetic datasets and the MovieLens dataset.
Mouse Dynamics Behavioral Biometrics: A Survey

Utilization of internet in everyday life has made us vulnerable in terms of privacy and security of our data and systems. Therefore, there are pressing needs to protect our data and systems by improving authentication mechanisms, which needs to be low cost, unobtrusive, and ideally ubiquitous in nature. Behavioral biometrics modalities such as mouse dynamics (i.e., mouse behaviors on a graphical user interface-GUI) and widget interactions (i.e., another modality closely related to mouse dynamics that also considers the target (widget) of a GUI interaction, such as links, button, and combo-box) can bolster the security of existing authentication systems because of their ability to distinguish an individual based on their unique features. As a result, it can be difficult for an imposter to impersonate these behavioral biometrics, making them suitable for authentication. In this paper, we survey the literature on mouse dynamics and widget interactions dated from 1897 to 2021. We begin our survey with an account of the psychological perspectives on behavioral biometrics. We then analyze the literature along the following dimensions: tasks and experimental settings for data collection, taxonomy of raw attributes, feature extractions and mathematical definitions, publicly available datasets, algorithms (statistical, machine learning, and deep learning), fusion and performance, and limitations. Lastly, we end the paper with presenting more opportunities for future research
DAFT: Distilling Adversarially Fine-tuned Models for Better OOD Generalization

We consider the problem of OOD generalization, where the goal is to train a model that performs well on test distributions that are different from the training distribution. Deep learning models are known to be fragile to such shifts and can suffer large accuracy drops even for slightly different test distributions. We propose a new method - DAFT - based on the intuition that adversarially robust combination of a large number of rich features should provide OOD robustness. Our method carefully distills the knowledge from a powerful teacher that learns several discriminative features using standard training while combining them using adversarial training. The standard adversarial training procedure is modified to produce teachers which can guide the student better. We evaluate DAFT on standard benchmarks in the DomainBed framework, and demonstrate that DAFT achieves significant improvements over the current state-of-the-art OOD generalization methods. DAFT consistently out-performs well-tuned ERM and distillation baselines by up to 6%, with more pronounced gains for smaller networks.
ZRH22: An improved analysis of the pion-nucleon measurements at low energy

Carried out in this study is an improved analysis of the measurements of the three pion-nucleon (πN) reactions, which can be subjected to experimental investigation at low energy (pion laboratory kinetic energy T ≤ 100 MeV), i.e., of the two elasticscattering (ES) processes πp → πp and of the πp charge-exchange (CX) reaction πp → π0n. After the application of the four steps of the first phase of the new analysis procedure, 52 entries of the 1150 degrees of freedom of the new initial database (DB) of this project were identified as outliers and were removed from the input. In this manner, three ‘clean’ (devoid of outliers) DBs were obtained, which were submitted to further analysis using the ETH model. The phase-shift analysis (PSA) of the ES measurements comprises the results of one hundred joint fits to the data, each fit being carried out at one value (randomly generated, in normal distribution, according to a recent result) of the model parameter mσ, which is associated with the effective range of the scalar-isoscalar part of the πN interaction. The result of the PSA of the ES data for the (square of the) charged-pion coupling constant fc is: f 2 c = 0.0764(15), a value which is in good agreement with the results obtained in the analyses of the SAID group over time. An exclusive fit of the ETH model to the low-energy πp CX DB was also performed, for the first time in the recent years, after suppressing the contributions to the partial-wave amplitudes of the ETH model which originate from the scalar-isoscalar t-channel Feynman diagram. The predictions, obtained from these two types of fits (PSA of the ES data and exclusive fit of the ETH model to the πp CX DB) for the (popular) symmetrised relative difference R2 were compared in the entire low-energy region, separately for the s wave, as well as for the no-spin-flip and spin-flip p-wave parts of the πN scattering amplitude. Sizeable effects were observed, decreasing with increasing energy, in the former two cases; less significant effects were obtained in the spin-flip p-wave part. Assuming the validity of the absolute normalisation of the bulk of the low-energy πN measurements and the smallness of any residual contributions to the electromagnetic corrections which are applied to the data, these findings agree with all results and conclusions of the former analyses, which had been carried out within the ETH πN project since 1997 and suggested the violation of the isospin invariance in the low-energy πN interaction beyond the expectations of Chiral-Perturbation Theory. PACS: 13.75. Gx; 25.80. Dj; 25.80. Gn; 11.30. -j
Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching

Graph Neural Networks (GNNs) are receiving a spotlight as a powerful tool that can effectively serve various inference tasks on graph structured data. As the size of real-world graphs continues to scale, the GNN training system faces a scalability challenge. Distributed training is a popular approach to address this challenge by scaling out CPU nodes. However, not much attention has been paid to disk-based GNN training, which can scale up the single-node system in a more cost-effective manner by leveraging high-performance storage devices like NVMe SSDs. We observe that the data movement between the main memory and the disk is the primary bottleneck in the SSD-based training system, and that the conventional GNN training pipeline is sub-optimal without taking this overhead into account. Thus, we propose Ginex, the first SSD-based GNN training system that can process billion-scale graph datasets on a single machine. Inspired by the inspector-executor execution model in compiler optimization, Ginex restructures the GNN training pipeline by separating sample and gather stages. This separation enables Ginex to realize a provably optimal replacement algorithm, known as Belady's algorithm , for caching feature vectors in memory, which account for the dominant portion of I/O accesses. According to our evaluation with four billion-scale graph datasets and two GNN models, Ginex achieves 2.11X higher training throughput on average (2.67X at maximum) than the SSD-extended PyTorch Geometric.
Cross-Domain Evaluation of a Deep Learning-Based Type Inference System

Optional type annotations allow for enriching dynamic programming languages with static typing features like better Integrated Development Environment (IDE) support, more precise program analysis, and early detection and prevention of type-related runtime errors. Machine learning-based type inference promises interesting results for automating this task. However, the practical usage of such systems depends on their ability to generalize across different domains, as they are often applied outside their training domain.In this work, we investigate Type4Py as a representative of state-of-the-art deep learning-based type inference systems, by conducting extensive cross-domain experiments. Thereby, we address the following problems: class imbalances, out-of-vocabulary words, dataset shifts, and unknown classes.To perform such experiments, we use the datasets Many-Types4Py and CrossDomainTypes4Py. The latter we introduce in this paper. Our dataset enables the evaluation of type inference systems in different domains of software projects and has over 1,000,000 type annotations mined on the platforms GitHub and Libraries. It consists of data from the two domains web development and scientific calculation.Through our experiments, we detect that the shifts in the dataset and the long-tailed distribution with many rare and unknown data types decrease the performance of the deep learning-based type inference system drastically. In this context, we test unsupervised domain adaptation methods and fine-tuning to overcome these issues. Moreover, we investigate the impact of out-of-vocabulary words.
Learn to Detect and Detect to Learn: Structure Learning and Decision Feedback for MIMO-OFDM Receive Processing

One of the major open challenges in MIMO-OFDM receive processing is how to efﬁciently and effectively utilize the extremely limited over-the-air pilot symbols to detect the transmitted data symbols. Recent advances have been devoted to investigating effective ways to utilize the limited pilots. However, we notice that besides exploiting the pilots, one can take advantage of the data symbols to improve the detection performance. Thus, this paper introduces an online subframe-based approach, namely RC-StructNet, that can efﬁciently learn from the precious pilot symbols and be dynamically updated with the detected payload data using the decision feedback (DF) approach. The network consists of a reservoir computing (RC) module in the time domain and a neural network StructNet in the frequency domain. The unique design of the network allows it to be dynamically updated with the changes of the channel by learning from the detected data symbols. Experiments demonstrate the effectiveness of RC-StructNet in detection under dynamic transmission modes and in reducing the training overhead requirement when taking the DF approach. data symbols can be used as the training data to further reduce the training overhead requirement and dynamically update the network weights to track the channel changes. Experiments show that by utilizing StructNet, RC-StructNet outperforms RC-Struct in different scenarios under the 3GPP 3D channel and it is more robust to label errors. Moreover, evaluation results also demonstrate the effectiveness of utilizing the DF in detection and in lessening the training overhead. Our
Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

Standard diffusion models involve an image transform -- adding Gaussian noise -- and an image restoration operator that inverts this degradation. We observe that the generative behavior of diffusion models is not strongly dependent on the choice of image degradation, and in fact an entire family of generative models can be constructed by varying this choice. Even when using completely deterministic degradations (e.g., blur, masking, and more), the training and test-time update rules that underlie diffusion models can be easily generalized to create generative models. The success of these fully deterministic models calls into question the community's understanding of diffusion models, which relies on noise in either gradient Langevin dynamics or variational inference, and paves the way for generalized diffusion models that invert arbitrary processes. Our code is available at https://github.com/arpitbansal297/Cold-Diffusion-Models
Computation of masses of quarkonium bound states using heavy quark potentials from lattice QCD

We compute masses of bottomonium and charmonium bound states using a Schr\"odinger equation with a heavy quark-antiquark potential including $1/m$ and $1/m^2$ corrections previously derived in potential Non-Relativistic QCD and computed with lattice QCD. This is a preparatory step for a future project, where we plan to take into account similar corrections to study quarkonium resonances and tetraquarks above the lowest meson-meson thresholds.
Synchrotron Polarization of Gamma-Ray Burst Afterglow Shocks with Hydrodynamic-scale Turbulent Magnetic Field

Afterglows of gamma-ray bursts (GRBs) are emitted from expanding forward shocks, which are expected to have magnetic fields much stronger than the interstellar field, although the origin of the field is a long-standing problem. Two field amplification mechanisms, plasma kinetic instabilities and magnetohydrodynamic instabilities, have been discussed so far. The coherence-length scales of the fields amplified by these two processes are different by 7–10 orders of magnitude, and the polarimetric observations may distinguish them. We construct a semi-analytic model of the forward-shock afterglow polarization under the assumption of hydrodynamic-scale turbulent magnetic field. We perform numerical calculations of synchrotron polarization for the isotropic turbulence and the zero viewing angle. We find that the polarization degrees are ∼1% when the field coherence-length scale in the fluid co-moving frame is of the order of the thickness of the shocked regions. This range of polarization degree is comparable to that of the observed late-phase optical afterglows. Our model also shows that the radio polarization degrees are comparable to the optical ones on average but can be higher than the optical ones at some time intervals. The polarization angles are shown to vary randomly and continuously. These polarimetric properties are clearly different from the case of plasma kinetic instability. Simultaneous polarimetric observations of GRB afterglows at the radio and optical bands have recently started, which will help us constrain the magnetic field amplification mechanism.
User Experience of Symptom Checkers: A Systematic Review

This review reports the user experience of symptom checkers, aiming to characterize users studied in the existing literature, identify the aspects of user experience of symptom checkers that have been studied, and offer design suggestions. Our literature search resulted in 31 publications. We found that (1) most symptom checker users are relatively young; (2) eight relevant aspects of user experience have been explored, including motivation, trust, acceptability, satisfaction, accuracy, usability, safety/security, and functionality; (3) future symptom checkers should improve their accuracy, safety, and usability. Although many facets of user experience have been explored, methodological challenges exist and some important aspects of user experience remain understudied. Further research should be conducted to explore users' needs and the context of use. More qualitative and mixed-method studies are needed to understand actual users' experiences in the future.
How to Fine-tune Models with Few Samples: Update, Data Augmentation, and Test-time Augmentation

Most of the recent few-shot learning (FSL) algorithms are based on transfer learning, where a model is pre-trained using a large amount of source data, and the pre-trained model is fine-tuned using a small amount of target data. In transfer learning-based FSL, sophisticated pre-training methods have been widely studied for universal representation. Therefore, it has become more important to utilize the universal representation for downstream tasks, but there are few studies on fine-tuning in FSL. In this paper, we focus on how to transfer pre-trained models to few-shot downstream tasks from the three perspectives: update, data augmentation, and test-time augmentation. First, we compare the two popular update methods, full fine-tuning (i.e., updating the entire network, FT) and linear probing (i.e., updating only a linear classifier, LP). We find that LP is better than FT with extremely few samples, whereas FT outperforms LP as training samples increase. Next, we show that data augmentation cannot guarantee few-shot performance improvement and investigate the effectiveness of data augmentation based on the intensity of augmentation. Finally, we adopt augmentation to both a support set for update (i.e., data augmentation) as well as a query set for prediction (i.e., test-time augmentation), considering support-query distribution shifts, and improve few-shot performance. The code is available at https://github.com/kimyuji/updating_FSL.
Suboptimal Performance of the Bayes Optimal Algorithm in Frequentist Best Arm Identification

We consider the fixed-budget best-arm identification problem with Normal reward distributions. In this problem, the forecaster is given $K$ arms (or treatments) and $T$ time steps. The forecaster attempts to find the best arm, defined by the largest mean, via an adaptive experiment conducted using an algorithm. The algorithm's performance is measured by the simple regret, that is, the quality of the estimated best arm. The frequentist simple regret can be exponentially small to $T$, whereas the Bayesian simple regret is polynomially small to $T$. This paper demonstrates that Bayes optimal algorithm, which minimizes the Bayesian simple regret, does not produce an exponential simple regret for some parameters, a finding that contrasts with the many results indicating the asymptotic equivalence of Bayesian and frequentist algorithms in the context of fixed sampling regimes. While the Bayes optimal algorithm is described in terms of a recursive equation that is virtually impossible to compute exactly, we establish the foundations for further analysis by introducing a key quantity that we call the expected Bellman improvement.
Time-dependent shocks and line emission in Herbig-Haro jets

Herbig-Haro jets cool radiatively in a very effective way. Nevertheless, high resolution observations show that temperature along the jet remains above 5,000 K and ionization above 10%, showing a tendency to increase in some regions. Therefore one needs to examine whether there are mechanisms that can successfully balance radiative losses. In this paper we assume that the fraction of the jet that emits the observed spectral lines is small, filling factor ∼1%, and consider shock dissipation as possible excitation mechanism. We carry out time-dependent numerical simulations of planar perturbations that evolve into shocks and compare the results with observations.
The DYF-5 RCK and CDKL-1 CDKL5 kinases contribute differentially to shape distinct sensory neuron cilia morphologies in C. elegans

The conserved CCRK, RCK, and CDKL5 kinases regulate cilia length in diverse organisms. In C. elegans , DYF-18 CCRK regulates DYF-5 RCK to shape both simple and complex cilia morphologies. The CDKL5 ortholog CDKL-1 has also been suggested to act downstream of DYF-18 but independently of DYF-5 to regulate lengths of simple rod-like cilia. Here we show that CDKL-1 is largely dispensable for regulation of complex cilia structures. Using genetic epistasis experiments, we confirm that CDKL-1 and DYF-5 act independently to control cilia architecture. Our results indicate that multiple kinases act via distinct pathways to regulate unique cilia ultrastructures.
Analysis of the distribution of vagal afferent projections from different peripheral organs to the nucleus of the solitary tract in rats

Anatomical tracing studies examining the vagal system can conflate details of sensory afferent and motor efferent neurons. Here, we used a serotype of adeno‐associated virus that transports retrogradely and exhibits selective tropism for vagal afferents, to map their soma location and central termination sites within the nucleus of the solitary tract (NTS). We examined the vagal sensory afferents innervating the trachea, duodenum, stomach, or heart, and in some animals, from two organs concurrently. We observed no obvious somatotopy in the somata distribution within the nodose ganglion. The central termination patterns of afferents from different organs within the NTS overlap substantially. Convergence of vagal afferent inputs from different organs onto single NTS neurons is observed. Abdominal and thoracic afferents terminate throughout the NTS, including in the rostral NTS, where the 7th cranial nerve inputs are known to synapse. To address whether the axonal labeling produced by viral transduction is so widespread because it fills axons traveling to their targets, and not just terminal fields, we labeled pre and postsynaptic elements of vagal afferents in the NTS . Vagal afferents form multiple putative synapses as they course through the NTS, with each vagal afferent neuron distributing sensory signals to multiple second‐order NTS neurons. We observe little selectivity between vagal afferents from different visceral targets and NTS neurons with common neurochemical phenotypes, with afferents from different organs making close appositions with the same NTS neuron. We conclude that specific viscerosensory information is distributed widely within the NTS and that the coding of this input is probably determined by the intrinsic properties and projections of the second‐order neuron.