Miracle-dz/UniFAR-FTUs huggingface dataset:
Scalable Model-Based Gaussian Process Clustering

Gaussian process is an indispensable tool in clustering functional data, owing to it's flexibility and inherent uncertainty quantification. However, when the functional data is observed over a large grid (say, of length $p$), Gaussian process clustering quickly renders itself infeasible, incurring $O(p^2)$ space complexity and $O(p^3)$ time complexity per iteration; and thus prohibiting it's natural adaptation to large environmental applications. To ensure scalability of Gaussian process clustering in such applications, we propose to embed the popular Vecchia approximation for Gaussian processes at the heart of the clustering task, provide crucial theoretical insights towards algorithmic design, and finally develop a computationally efficient expectation maximization (EM) algorithm. Empirical evidence of the utility of our proposal is provided via simulations and analysis of polar temperature anomaly (\href{https://www.ncei.noaa.gov/access/monitoring/climate-at-a-glance/global/time-series}{noaa.gov}) data-sets.
Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?

Recently sequential recommendations and next-item prediction task has become increasingly popular in the field of recommender systems. Currently, two state-of-the-art baselines are Transformer-based models SASRec and BERT4Rec. Over the past few years, there have been quite a few publications comparing these two algorithms and proposing new state-of-the-art models. In most of the publications, BERT4Rec achieves better performance than SASRec. But BERT4Rec uses cross-entropy over softmax for all items, while SASRec uses negative sampling and calculates binary cross-entropy loss for one positive and one negative item. In our work, we show that if both models are trained with the same loss, which is used by BERT4Rec, then SASRec will significantly outperform BERT4Rec both in terms of quality and training speed. In addition, we show that SASRec could be effectively trained with negative sampling and still outperform BERT4Rec, but the number of negative examples should be much larger than one.
In-Contextual Gender Bias Suppression for Large Language Models

Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have been reported to encode worrying-levels of gender biases. Prior work has proposed debiasing methods that require human labelled examples, data augmentation and fine-tuning of LLMs, which are computationally costly. Moreover, one might not even have access to the model parameters for performing debiasing such as in the case of closed LLMs such as GPT-4. To address this challenge, we propose bias suppression that prevents biased generations of LLMs by simply providing textual preambles constructed from manually designed templates and real-world statistics, without accessing to model parameters. We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2. Moreover, we find that gender-neutral descriptions of gender-biased objects can also suppress their gender biases. Moreover, we show that bias suppression has acceptable adverse effect on downstream task performance with HellaSwag and COPA.
Inhomogeneous order 1 iterative functional equations with applications to combinatorics

We show that if a Laurent series $f\in\mathbb{C}((t))$ satisfies a particular kind of linear iterative equation, then $f$ is either a rational function or it is differentially transcendental over $\mathbb{C}(t)$. This condition is more precisely stated as follows: We consider $R,b\in \mathbb{C}(t)$ with $R(0)=0$, such that $f(R(t))=f(t)+b(t)$. If either $R'(0)=0$ or $R'(0)$ is a root of unity, then either $f$ is a rational function, or $f$ does not satisfy a polynomial differential equation. More generally a solution of a functional equation of the form $f(R(t))=a(t)f(t)+b(t)$ will be either differentially trascendental or the solution of an inhomogeneous linear differential equation of order $1$ with rational coefficients. We illustrate how to apply these results to deduce the differential transcendence of combinatorial generating functions by considering three examples: the ordinary generating function for a family of complete trees; the Green function for excursions on the Sierpinski graph; and a series related to the enumeration of permutations avoiding the consecutive pattern 1423. The proof strategy is inspired by the Galois theory of functional equations and relies on the property of the dynamics of $R$.
Scattering for the Wave Equation on de Sitter Space in All Even Spatial Dimensions

For any $n\geq4$ even, we establish a complete scattering theory for the linear wave equation on the $(n+1)$-dimensional de Sitter space. We prove the existence and uniqueness of scattering states, and asymptotic completeness. Moreover, we construct the scattering map taking asymptotic data at past infinity $\mathscr{I}^-$ to asymptotic data at future infinity $\mathscr{I}^+$. Identifying $\mathscr{I}^-$ and $\mathscr{I}^+$ with $S^n,$ we prove that the scattering map is a Banach space isomorphism on $H^{s+n}(S^n)\times H^{s}(S^n),$ for any $s\geq1. $ The main analysis is carried out at the level of the model equation obtained by differentiating the linear wave equation $\frac{n}{2}$ times in the time variable. The main result of the paper follows from proving a scattering theory for this equation. In particular, for the model equation we construct a scattering isomorphism from asymptotic data in $H^{s+\frac{1}{2}}(S^n)\times H^s(S^n)\times H^s(S^n)$ to Cauchy initial data in $H^{s+\frac{1}{2}}(S^n)\times H^{s+\frac{1}{2}}(S^n)\times H^{s-\frac{1}{2}}(S^n)$.
Improved Distributed Algorithms for Random Colorings

Markov Chain Monte Carlo (MCMC) algorithms are a widely-used algorithmic tool for sampling from high-dimensional distributions, a notable example is the equilibirum distribution of graphical models. The Glauber dynamics, also known as the Gibbs sampler, is the simplest example of an MCMC algorithm; the transitions of the chain update the configuration at a randomly chosen coordinate at each step. Several works have studied distributed versions of the Glauber dynamics and we extend these efforts to a more general family of Markov chains. An important combinatorial problem in the study of MCMC algorithms is random colorings. Given a graph $G$ of maximum degree $\Delta$ and an integer $k\geq\Delta+1$, the goal is to generate a random proper vertex $k$-coloring of $G$. Jerrum (1995) proved that the Glauber dynamics has $O(n\log{n})$ mixing time when $k>2\Delta$. Fischer and Ghaffari (2018), and independently Feng, Hayes, and Yin (2018), presented a parallel and distributed version of the Glauber dynamics which converges in $O(\log{n})$ rounds for $k>(2+\varepsilon)\Delta$ for any $\varepsilon>0$. We improve this result to $k>(11/6-\delta)\Delta$ for a fixed $\delta>0$. This matches the state of the art for randomly sampling colorings of general graphs in the sequential setting. Whereas previous works focused on distributed variants of the Glauber dynamics, our work presents a parallel and distributed version of the more general flip dynamics presented by Vigoda (2000) (and refined by Chen, Delcourt, Moitra, Perarnau, and Postle (2019)), which recolors local maximal two-colored components in each step.
Hierarchical Metadata Information Constrained Self-Supervised Learning for Anomalous Sound Detection Under Domain Shift

Self-supervised learning methods have achieved promising performance for anomalous sound detection (ASD) under domain shift, where the type of domain shift is considered in feature learning by incorporating section IDs. However, the attributes accompanying audio files under each section, such as machine operating conditions and noise types, have not been considered, although they are also crucial for characterizing domain shifts. In this paper, we present a hierarchical metadata information constrained self-supervised (HMIC) ASD method, where the hierarchical relation between section IDs and attributes is constructed, and used as constraints to obtain finer feature representation. In addition, we propose an attribute-group-center (AGC)-based method for calculating the anomaly score under the domain shift condition. Experiments are performed to demonstrate its improved performance over the state-of-the-art self-supervised methods in DCASE 2022 challenge Task 2.
Refractive lensing of scintillating FRBs by sub-parsec cloudlets in the multi-phase CGM

We consider the refractive lensing effects of ionized cool ($T \sim 10^4\,{\rm K}$) gas cloudlets in the circumgalactic medium (CGM) of galaxies. In particular, we discuss the combined effects of lensing from these cloudlets and scintillation from plasma screens in the Milky Way interstellar medium (ISM). We show that, if the CGM comprises a mist of sub-parsec cloudlets with column densities of order $10^{17}\,{\rm cm}^{-2}$ (as predicted by McCourt et al. (2018)), then FRBs whose sightlines pass within a virial radius of a CGM halo will generically be lensed into tens of refractive images with a $\sim 10\,{\rm ms}$ scattering timescale. These images will be resolved by scintillating screens in the Milky Way ISM, and therefore are expected to suppress scintillation. From this, we argue that positive detections of FRB scintillation can constrain the properties of these cool-gas cloudlets, with current scintillation observation disfavouring the cloudlet model. We propose that sheet-like geometries for the cool gas in the CGM can reconcile quasar absorption measurements (from which we infer the presence of the cool gas with structure on sub-parsec scales) and the unexpected lack of lensing signals from this gas thus far observed.
Identifying the Group-Theoretic Structure of Machine-Learned Symmetries

Deep learning was recently successfully used in deriving symmetry transformations that preserve important physics quantities. Being completely agnostic, these techniques postpone the identification of the discovered symmetries to a later stage. In this letter we propose methods for examining and identifying the group-theoretic structure of such machine-learned symmetries. We design loss functions which probe the subalgebra structure either during the deep learning stage of symmetry discovery or in a subsequent post-processing stage. We illustrate the new methods with examples from the U(n) Lie group family, obtaining the respective subalgebra decompositions. As an application to particle physics, we demonstrate the identification of the residual symmetries after the spontaneous breaking of non-Abelian gauge symmetries like SU(3) and SU(5) which are commonly used in model building.
Dataset Size Dependence of Rate-Distortion Curve and Threshold of Posterior Collapse in Linear VAE

In the Variational Autoencoder (VAE), the variational posterior often aligns closely with the prior, which is known as posterior collapse and hinders the quality of representation learning. To mitigate this problem, an adjustable hyperparameter beta has been introduced in the VAE. This paper presents a closed-form expression to assess the relationship between the beta in VAE, the dataset size, the posterior collapse, and the rate-distortion curve by analyzing a minimal VAE in a high-dimensional limit. These results clarify that a long plateau in the generalization error emerges with a relatively larger beta. As the beta increases, the length of the plateau extends and then becomes infinite beyond a certain beta threshold. This implies that the choice of beta, unlike the usual regularization parameters, can induce posterior collapse regardless of the dataset size. Thus, beta is a risky parameter that requires careful tuning. Furthermore, considering the dataset-size dependence on the rate-distortion curve, a relatively large dataset is required to obtain a rate-distortion curve with high rates. Extensive numerical experiments support our analysis.
Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions

Perceiving and manipulating 3D articulated objects in diverse environments is essential for home-assistant robots. Recent studies have shown that point-level affordance provides actionable priors for downstream manipulation tasks. However, existing works primarily focus on single-object scenarios with homogeneous agents, overlooking the realistic constraints imposed by the environment and the agent's morphology, e.g., occlusions and physical limitations. In this paper, we propose an environment-aware affordance framework that incorporates both object-level actionable priors and environment constraints. Unlike object-centric affordance approaches, learning environment-aware affordance faces the challenge of combinatorial explosion due to the complexity of various occlusions, characterized by their quantities, geometries, positions and poses. To address this and enhance data efficiency, we introduce a novel contrastive affordance learning framework capable of training on scenes containing a single occluder and generalizing to scenes with complex occluder combinations. Experiments demonstrate the effectiveness of our proposed approach in learning affordance considering environment constraints. Project page at https://chengkaiacademycity.github. io/EnvAwareAfford/
The effect of data augmentation and 3D-CNN depth on Alzheimer's Disease detection

Machine Learning (ML) has emerged as a promising approach in healthcare, outperforming traditional statistical techniques. However, to establish ML as a reliable tool in clinical practice, adherence to best practices regarding data handling, experimental design, and model evaluation is crucial. This work summarizes and strictly observes such practices to ensure reproducible and reliable ML. Specifically, we focus on Alzheimer's Disease (AD) detection, which serves as a paradigmatic example of challenging problem in healthcare. We investigate the impact of different data augmentation techniques and model complexity on the overall performance. We consider MRI data from ADNI dataset to address a classification problem employing 3D Convolutional Neural Network (CNN). The experiments are designed to compensate for data scarcity and initial random parameters by utilizing cross-validation and multiple training trials. Within this framework, we train 15 predictive models, considering three different data augmentation strategies and five distinct 3D CNN architectures, each varying in the number of convolutional layers. Specifically, the augmentation strategies are based on affine transformations, such as zoom, shift, and rotation, applied concurrently or separately. The combined effect of data augmentation and model complexity leads to a variation in prediction performance up to 10% of accuracy. When affine transformation are applied separately, the model is more accurate, independently from the adopted architecture. For all strategies, the model accuracy followed a concave behavior at increasing number of convolutional layers, peaking at an intermediate value of layers. The best model (8 CL, (B)) is the most stable across cross-validation folds and training trials, reaching excellent performance both on the testing set and on an external test set.
Improving Swim Accessibility through Vibro-Tactile Sensory Substitution: A Preliminary Study

Practicing sports and in particular swimming has several positive effects, among which is improving individuals’ perception of the self and, therefore, their control over their own body. Nevertheless, swimming accessibility has several barriers for people with visual impairments. This paper investigates the challenges that blind and visually-impaired (BVI) athletes encounter when practicing swim. It then illustrates a human-centered process that involved two Paralympic swimmers and that, thanks to the gathered insights, led to the design of VIBS, a vibro-tactile swimming cap that aims to make BVI swimmers feel safe and independent.
Power Loss Function in Neural Networks for Predicting Click-Through Rate

Loss functions guide machine learning models towards concentrating on the error most important to improve upon. We introduce power loss functions for neural networks and apply them on imbalanced click-through rate datasets. Power loss functions decrease the loss for confident predictions and increase the loss for error-prone predictions. They improve both AUC and F1 and produce better calibrated results. We obtain improvements in the results on four different classifiers and on two different datasets. We obtain significant improvements in AUC that reach 0.44% for DeepFM on the Avazu dataset.
Effectiveness of music-based interventions to address well-being in people with a vision impairment: a scoping review

Objectives The objectives of this review were to identify the types of music-based interventions and associated accessibility challenges for people who have visual impairment (VI) and their reported effects on psychological, physiological and social well-being. Design A scoping review was developed according to the Joanna Briggs Institute methodology and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews checklist and guidelines. A narrative synthesis was conducted to map out the types of music-based interventions undertaken and to compare the therapeutic outcomes. The studies were evaluated according to the music reporting checklist. Results In total 5082 records were identified, 69 full-text articles were screened and 13 studies were included. Eleven studies included younger children and teenagers, two focused on adults with acquired VI. Ten studies involved active music therapy strategies and three used passive music listening. Eleven of the studies focused on social outcomes and two reported mental health. Although the studies reported that music-based intervention strategies improved psychosocial well-being in people with a VI, conclusions could not be drawn as robust outcome measures were not generally used and only four of the studies included any statistical analysis. Conclusions Although potential was evident, details of intervention protocols and training requirements were not sufficiently reported and further, high-quality evidence-based studies are required.
PROGrasp: Pragmatic Human-Robot Communication for Object Grasping

Interactive Object Grasping (IOG) is the task of identifying and grasping the desired object via human-robot natural language interaction. Current IOG systems assume that a human user initially specifies the target object's category (e.g., bottle). Inspired by pragmatics, where humans often convey their intentions by relying on context to achieve goals, we introduce a new IOG task, Pragmatic-IOG, and the corresponding dataset, Intention-oriented Multi-modal Dialogue (IM-Dial). In our proposed task scenario, an intention-oriented utterance (e.g.,"I am thirsty") is initially given to the robot. The robot should then identify the target object by interacting with a human user. Based on the task setup, we propose a new robotic system that can interpret the user's intention and pick up the target object, Pragmatic Object Grasping (PROGrasp). PROGrasp performs Pragmatic-IOG by incorporating modules for visual grounding, question asking, object grasping, and most importantly, answer interpretation for pragmatic inference. Experimental results show that PROGrasp is effective in offline (i.e., target object discovery) and online (i.e., IOG with a physical robot arm) settings. Code and data are available at https://github.com/gicheonkang/prograsp.
On geometric-type approximations with applications to risk and related processes

We explore two aspects of geometric approximation via a coupling approach to Stein's method. Firstly, we refine precision and increase scope for applications by convoluting the approximating geometric distribution with a simple translation selected based on the problem at hand. Secondly, we apply our geometric and geometric-type approximations to risk processes and related areas, including the approximation of Poisson processes with a random time horizon and Markov chain hitting times. Particular attention is given to geometric approximation of random sums, for which explicit bounds are established. These are applied to give simple approximations, including error bounds, for the infinite-horizon ruin probability in the compound binomial risk process.
PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts

The meanings of words and phrases depend not only on where they are used (contexts) but also on who use them (writers). Pretrained language models (PLMs) are powerful tools for capturing context, but they are typically pretrained and fine-tuned for universal use across different writers. This study aims to improve the accuracy of text understanding tasks by personalizing the fine-tuning of PLMs for specific writers. We focus on a general setting where only the plain text from target writers are available for personalization. To avoid the cost of fine-tuning and storing multiple copies of PLMs for different users, we exhaustively explore using writer-specific prompts to personalize a unified PLM. Since the design and evaluation of these prompts is an underdeveloped area, we introduce and compare different types of prompts that are possible in our setting. To maximize the potential of prompt-based personalized fine-tuning, we propose a personalized intermediate learning based on masked language modeling to extract task-independent traits of writers' text. Our experiments, using multiple tasks, datasets, and PLMs, reveal the nature of different prompts and the effectiveness of our intermediate learning approach.
ESRO: Experience Assisted Service Reliability against Outages

Modern cloud services are prone to failures due to their complex architecture, making diagnosis a critical process. Site Reliability Engineers (SREs) spend hours leveraging multiple sources of data, including the alerts, error logs, and domain expertise through past experiences to locate the root cause(s). These experiences are documented as natural language text in outage reports for previous outages. However, utilizing the raw yet rich semi-structured information in the reports systematically is time-consuming. Structured information, on the other hand, such as alerts that are often used during fault diagnosis, is voluminous and requires expert knowledge to discern. Several strategies have been proposed to use each source of data separately for root cause analysis. In this work, we build a diagnostic service called ESRO that recommends root causes and remediation for failures by utilizing structured as well as semi-structured sources of data systematically. ESRO constructs a causal graph using alerts and a knowledge graph using outage reports, and merges them in a novel way to form a unified graph during training. A retrieval based mechanism is then used to search the unified graph and rank the likely root causes and remediation techniques based on the alerts fired during an outage at inference time. Not only the individual alerts, but their respective importance in predicting an outage group is taken into account during recommendation. We evaluated our model on several cloud service outages of a large SaaS enterprise over the course of ~2 years, and obtained an average improvement of 27% in rouge scores after comparing the likely root causes against the ground truth over state-of-the-art baselines. We further establish the effectiveness of ESRO through qualitative analysis on multiple real outage examples.