Miracle-dz/UniFAR-FTUs huggingface dataset:
Resolution-independent generative models based on operator learning for physics-constrained Bayesian inverse problems

The Bayesian inference approach is widely used to tackle inverse problems due to its versatile and natural ability to handle ill-posedness. However, it often faces challenges when dealing with situations involving continuous fields or large-resolution discrete representations (high-dimensional). Moreover, the prior distribution of unknown parameters is commonly difficult to be determined. In this study, an Operator Learning-based Generative Adversarial Network (OL-GAN) is proposed and integrated into the Bayesian inference framework to handle these issues. Unlike most Bayesian approaches, the distinctive characteristic of the proposed method is to learn the joint distribution of parameters and responses. By leveraging the trained generative model, the posteriors of the unknown parameters can theoretically be approximated by any sampling algorithm (e.g., Markov Chain Monte Carlo, MCMC) in a low-dimensional latent space shared by the components of the joint distribution. The latent space is typically a simple and easy-to-sample distribution (e.g., Gaussian, uniform), which significantly reduces the computational cost associated with the Bayesian inference while avoiding prior selection concerns. Furthermore, incorporating operator learning enables resolution-independent in the generator. Predictions can be obtained at desired coordinates, and inversions can be performed even if the observation data are misaligned with the training data. Finally, the effectiveness of the proposed method is validated through several numerical experiments.
MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classification

The eXtreme Multi-label text Classification(XMC) refers to training a classifier that assigns a text sample with relevant labels from an extremely large-scale label set (e.g., millions of labels). We propose MatchXML, an efficient text-label matching framework for XMC. We observe that the label embeddings generated from the sparse Term Frequency-Inverse Document Frequency(TF-IDF) features have several limitations. We thus propose label2vec to effectively train the semantic dense label embeddings by the Skip-gram model. The dense label embeddings are then used to build a Hierarchical Label Tree by clustering. In fine-tuning the pre-trained encoder Transformer, we formulate the multi-label text classification as a text-label matching problem in a bipartite graph. We then extract the dense text representations from the fine-tuned Transformer. Besides the fine-tuned dense text embeddings, we also extract the static dense sentence embeddings from a pre-trained Sentence Transformer. Finally, a linear ranker is trained by utilizing the sparse TF-IDF features, the fine-tuned dense text representations and static dense sentence features. Experimental results demonstrate that MatchXML achieves state-of-the-art accuracy on five out of six datasets. As for the speed, MatchXML outperforms the competing methods on all the six datasets. Our source code is publicly available at https://github.com/huiyegit/MatchXML.
Generalized cardinal invariants for an inaccessible $\kappa$ with compactness at $\kappa^{++}$

We study the relationship between non-trivial values of generalized cardinal invariants at an inaccessible cardinal $\kappa$ and compactness principles at $\kappa^{++}$. We show that if the existence of a supercompact cardinal $\kappa$ with a weakly compact cardinal $\lambda$ above $\kappa$ is consistent, then the following are consistent as well (where $\mathfrak{t}(\kappa)$ and $\mathfrak{u}(\kappa)$ are the tower number and the ultrafilter number, respectively): (i) There is an inaccessible cardinal $\kappa$ such that $\kappa^+<\mathfrak{t}(\kappa)= \mathfrak{u}(\kappa)<2^\kappa$ and stationary reflection and the disjoint stationary sequence property at $\kappa^{++}$ hold. (ii) There is an inaccessible cardinal $\kappa$ such that $\kappa^+ = \mathfrak{t}(\kappa)<\mathfrak{u}(\kappa)<2^\kappa$ and the principles in (i) hold plus the tree property at $\kappa^{++}$ and the negation of the weak Kurepa Hypothesis at $\kappa^+$. The cardinals $\mathfrak{u}(\kappa)$ and $2^\kappa$ can have any reasonable values in these models. We compute several other values of cardinal invariants, such $\mathfrak{a}(\kappa)$ and $\mathfrak{b}(\kappa)$, and the invariants of the meager ideal (they are all equal to $\mathfrak{u}(\kappa)$). In (ii), we compute $\mathfrak{p}(\kappa) = \mathfrak{t}(\kappa) = \kappa^+$ by observing that the $\kappa^+$-distributive quotient of the Mitchell forcing adds a tower of size $\kappa^+$. As a corollary of the construction, we obtain that (i) and (ii) are also true for $\kappa = \omega$ (starting with a weakly compact cardinal in the ground model).
Enhancement of gravitational waves at Q-ball decay including non-linear density perturbations

The existence of a stochastic gravitational wave background is indicated by the recent pulsar timing array (PTA) experiments. We study the enhanced production of second-order gravitational waves from the scalar perturbations when the universe experiences a transition from the early matter-dominated era to the radiation-dominated era due to Q-ball decay. We extend the analysis in previous work by including the frequency range where density perturbations go non-linear and find that the resultant gravitational wave spectrum can be consistent with that favored by the recent PTA experiment results.
Using Visual and Vehicular Sensors for Driver Behavior Analysis: A Survey

Risky drivers account for 70% of fatal accidents in the United States. With recent advances in sensors and intelligent vehicular systems, there has been significant research on assessing driver behavior to improve driving experiences and road safety. This paper examines the various techniques used to analyze driver behavior using visual and vehicular data, providing an overview of the latest research in this field. The paper also discusses the challenges and open problems in the field and offers potential recommendations for future research. The survey concludes that integrating vision and vehicular information can significantly enhance the accuracy and effectiveness of driver behavior analysis, leading to improved safety measures and reduced traffic accidents.
Neural networks in feedback for flow analysis, sensor placement and control

This work presents a novel methodology for analysis and control of nonlinear fluid systems using neural networks. The approach is demonstrated on four different study cases being the Lorenz system, a modified version of the Kuramoto-Sivashinsky equation, a streamwise-periodic 2D channel flow, and a confined cylinder flow. Neural networks are trained as models to capture the complex system dynamics and estimate equilibrium points through a Newton method, enabled by backpropagation. These neural network surrogate models (NNSMs) are leveraged to train a second neural network, which is designed to act as a stabilizing closed-loop controller. The training process employs a recurrent approach, whereby the NNSM and the neural network controller (NNC) are chained in closed loop along a finite time horizon. By cycling through phases of combined random open-loop actuation and closed-loop control, an iterative training process is introduced to overcome the lack of data near equilibrium points. This approach improves the accuracy of the models in the most critical region for achieving stabilization. Through the use of L1 regularization within loss functions, the NNSMs can also guide optimal sensor placement, reducing the number of sensors from an initial candidate set. The datasets produced during the iterative training process are also leveraged for conducting a linear stability analysis through a modified dynamic mode decomposition approach. The results demonstrate the effectiveness of computationally inexpensive neural networks in modeling, controlling, and enabling stability analysis of nonlinear systems, providing insights into the system behaviour and offering potential for stabilization of complex fluid systems.
Tracing the Origins of Mass Segregation in M35: Evidence for Primordially Segregated Binaries

M35 is a young open cluster and home to an extensive binary population. Using Gaia Data Release 3, Pan-STARRS, and Two Micron All Sky Survey photometry with the Bayesian statistical software, BASE-9, we derive precise cluster parameters, identify single and binary cluster members, and extract their masses. We identify 571 binaries down to Gaia G = 20.3 and a lower limit on the binary frequency of f b = 0.41 ± 0.02. We extend the binary demographics by many magnitudes faint-ward of previous (radial-velocity) studies of this cluster and further away from the cluster center (1.°78, roughly 10 core radii). We find the binary stars to be more centrally concentrated than the single stars in the cluster. Furthermore, we find strong evidence for mass segregation within the binary population itself, with progressively more-massive binary samples becoming more and more centrally concentrated. For the single stars, we find weaker evidence for mass segregation; only the most massive single stars (>2.5M ⊙) appear more centrally concentrated. Given the cluster age of ∼200 Myr, and our derived half-mass relaxation time for the cluster of 230 ± 84 Myr, we estimate ∼47% of the binary stars and ∼12% of single stars in the cluster have had time to become dynamically mass segregated. Importantly, when we investigate only stars with mass segregation timescales greater than the cluster age, we still find the binaries to be more centrally concentrated than the singles, suggesting the binaries may have formed with a primordially different spatial distribution from that of the single stars.
A Large-Scale Study of IoT Security Weaknesses and Vulnerabilities in the Wild

Internet of Things (IoT) is defined as the connection between places and physical objects (i.e., things) over the internet/network via smart computing devices. We observed that IoT software developers share solutions to programming questions as code examples on three Stack Exchange Q&A sites: Stack Overflow (SO), Arduino, and Raspberry Pi. Previous research studies found vulnerabilities/weaknesses in C/C++ code examples shared in Stack Overflow. However, the studies did not investigate C/C++ code examples related to IoT. The studies investigated SO code examples only. In this paper, we conduct a large-scale empirical study of all IoT C/C++ code examples shared in the three Stack Exchange sites, i.e., SO, Arduino, and Raspberry Pi. From the 11,329 obtained code snippets from the three sites, we identify 29 distinct CWE (Common Weakness Enumeration) types in 609 snippets. These CWE types can be categorized into 8 general weakness categories, and we observe that evaluation, memory, and initialization related weaknesses are the most common to be introduced by users when posting programming solutions. Furthermore, we find that 39.58% of the vulnerable code snippets contain instances of CWE types that can be mapped to real-world occurrences of those CWE types (i.e. CVE instances). The most number vulnerable IoT code examples was found in Arduino, followed by SO, and Raspberry Pi. Memory type vulnerabilities are on the rise in the sites. For example, from the 3595 mapped CVE instances, we find that 28.99% result in Denial of Service (DoS) errors, which is particularly harmful for network reliant IoT devices such as smart cars. Our study results can guide various IoT stakeholders to be aware of such vulnerable IoT code examples and to inform IoT researchers during their development of tools that can help prevent developers the sharing of such vulnerable code examples in the sites. [Abridged].
Reconstructing the potential configuration in a high-mobility semiconductor heterostructure with scanning gate microscopy

The weak disorder potential seen by the electrons of a two-dimensional electron gas in high-mobility semiconductor heterostructures leads to fluctuations in the physical properties and can be an issue for nanodevices. In this paper, we show that a scanning gate microscopy (SGM) image contains information about the disorder potential, and that a machine learning approach based on SGM data can be used to determine the disorder. We reconstruct the electric potential of a sample from its experimental SGM data and validate the result through an estimate of its accuracy.
Learning Compact Neural Networks with Deep Overparameterised Multitask Learning

Compact neural network offers many benefits for real-world applications. However, it is usually challenging to train the compact neural networks with small parameter sizes and low computational costs to achieve the same or better model performance compared to more complex and powerful architecture. This is particularly true for multitask learning, with different tasks competing for resources. We present a simple, efficient and effective multitask learning overparameterisation neural network design by overparameterising the model architecture in training and sharing the overparameterised model parameters more effectively across tasks, for better optimisation and generalisation. Experiments on two challenging multitask datasets (NYUv2 and COCO) demonstrate the effectiveness of the proposed method across various convolutional networks and parameter sizes.
A lower bound on the multicolor size-Ramsey numbers of paths in hypergraphs

The $r$-color size-Ramsey number of a $k$-uniform hypergraph $H$, denoted by $\hat{R}_r(H)$, is the minimum number of edges in a $k$-uniform hypergraph $G$ such that for every $r$-coloring of the edges of $G$ there exists a monochromatic copy of $H$. In the case of $2$-uniform paths $P_n$, it is known that $\Omega(r^2n)=\hat{R}_r(P_n)=O((r^2\log r)n)$ with the best bounds essentially due to Krivelevich. In a recent breakthrough result, Letzter, Pokrovskiy, and Yepremyan gave a linear upper bound on the $r$-color size-Ramsey number of the $k$-uniform tight path $P_{n}^{(k)}$; i.e. $\hat{R}_r(P_{n}^{(k)})=O_{r,k}(n)$. Winter gave the first non-trivial lower bounds on the 2-color size-Ramsey number of $P_{n}^{(k)}$ for $k\geq 3$; i.e. $\hat{R}_2(P_{n}^{(3)})\geq \frac{8}{3}n-O(1)$ and $\hat{R}_2(P_{n}^{(k)})\geq \lceil\log_2(k+1)\rceil n-O_k(1)$ for $k\geq 4$. We consider the problem of giving a lower bound on the $r$-color size-Ramsey number of $P_{n}^{(k)}$ (for fixed $k$ and growing $r$). Our main result is that $\hat{R}_r(P_n^{(k)})=\Omega_k(r^kn)$ which generalizes the best known lower bound for graphs mentioned above. One of the key elements of our proof is a determination of the correct order of magnitude of the $r$-color size-Ramsey number of every sufficiently short tight path; i.e. $\hat{R}_r(P_{k+m}^{(k)})=\Theta_k(r^m)$ for all $1\leq m\leq k$. All of our results generalize to $\ell$-overlapping $k$-uniform paths $P_{n}^{(k, \ell)}$. In particular we note that when $1\leq \ell\leq \frac{k}{2}$, we have $\Omega_k(r^{2}n)=\hat{R}_r(P_{n}^{(k, \ell)})=O((r^2\log r)n)$ which essentially matches the best known bounds for graphs mentioned above. Additionally, in the case $k=3$, $\ell=2$, and $r=2$, we give a more precise estimate which implies $\hat{R}_2(P^{(3)}_{n})\geq \frac{28}{9}n-O(1)$, improving on the above-mentioned lower bound of Winter in the case $k=3$.
Open Gaze: Open Source eye tracker for smartphone devices using Deep Learning

Eye tracking has been a pivotal tool in diverse fields such as vision research, language analysis, and usability assessment. The majority of prior investigations, however, have concentrated on expansive desktop displays employing specialized, costly eye tracking hardware that lacks scalability. Remarkably little insight exists into ocular movement patterns on smartphones, despite their widespread adoption and significant usage. In this manuscript, we present an open-source implementation of a smartphone-based gaze tracker that emulates the methodology proposed by a GooglePaper (whose source code remains proprietary). Our focus is on attaining accuracy comparable to that attained through the GooglePaper's methodology, without the necessity for supplementary hardware. Through the integration of machine learning techniques, we unveil an accurate eye tracking solution that is native to smartphones. Our approach demonstrates precision akin to the state-of-the-art mobile eye trackers, which are characterized by a cost that is two orders of magnitude higher. Leveraging the vast MIT GazeCapture dataset, which is available through registration on the dataset's website, we successfully replicate crucial findings from previous studies concerning ocular motion behavior in oculomotor tasks and saliency analyses during natural image observation. Furthermore, we emphasize the applicability of smartphone-based gaze tracking in discerning reading comprehension challenges. Our findings exhibit the inherent potential to amplify eye movement research by significant proportions, accommodating participation from thousands of subjects with explicit consent. This scalability not only fosters advancements in vision research, but also extends its benefits to domains such as accessibility enhancement and healthcare applications.
RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network

Cross-domain few-shot segmentation (CD-FSS) aims to achieve semantic segmentation in previously unseen domains with a limited number of annotated samples. Although existing CD-FSS models focus on cross-domain feature transformation, relying exclusively on inter-domain knowledge transfer may lead to the loss of critical intra-domain information. To this end, we propose a novel residual transformation network (RestNet) that facilitates knowledge transfer while retaining the intra-domain support-query feature information. Specifically, we propose a Semantic Enhanced Anchor Transform (SEAT) module that maps features to a stable domain-agnostic space using advanced semantics. Additionally, an Intra-domain Residual Enhancement (IRE) module is designed to maintain the intra-domain representation of the original discriminant space in the new space. We also propose a mask prediction strategy based on prototype fusion to help the model gradually learn how to segment. Our RestNet can transfer cross-domain knowledge from both inter-domain and intra-domain without requiring additional fine-tuning. Extensive experiments on ISIC, Chest X-ray, and FSS-1000 show that our RestNet achieves state-of-the-art performance. Our code will be available soon.
Approximation Algorithms to Enhance Social Sharing of Fresh Point-of-Interest Information

In location-based social networks (LBSNs), such as Gowalla and Waze, users sense urban point-of-interest (PoI) information (e.g., restaurants' queue length and real-time traffic conditions) in the vicinity and share such information with friends in online social networks. Given each user's social connections and the severe lags in disseminating fresh PoI to all users, major LBSNs aim to enhance users' social PoI sharing by selecting a subset $k$ out of all $m$ users as hotspots and broadcasting their PoI information to the entire user community. This motivates us to study a new combinatorial optimization problem by integrating two urban sensing and online social networks. We prove that this problem is NP-hard and also renders existing approximation solutions not viable. Through analyzing the interplay effects between the sensing and social networks, we successfully transform the involved PoI-sharing process across two networks to matrix computations for deriving a closed-form objective and present a polynomial-time algorithm to ensure ($1-\frac{m-2}{m}(\frac{k-1}{k})^k$) approximation of the optimum. Furthermore, we allow each selected user to move around and sense more PoI information to share. To this end, we propose an augmentation-adaptive algorithm, which benefits from a resource-augmented technique and achieves bounded approximation, ranging from $\frac{1}{k}(1-\frac{1}{e})$ to $1-\frac{1}{e}>0.632$ by adjusting our augmentation factors. %Particularly when all sensing nodes are associated with users, we devise, by leveraging our augmentation-adaptive algorithm as a subroutine, an algorithm that eliminates the need for augmentation while still ensuring a satisfactory approximation $1-\frac{m-2}{m}(\frac{k-1}{k})^k$. Finally, our theoretical results are corroborated by our simulation findings using both synthetic and real-world datasets.
Bayesian low-rank adaptation for large language models

Low-rank adaptation (LoRA) has emerged as a new paradigm for cost-efficient fine-tuning of large language models (LLMs). However, fine-tuned LLMs often become overconfident especially when fine-tuned on small datasets. Bayesian methods, with their inherent ability to estimate uncertainty, serve as potent tools to mitigate overconfidence and enhance calibration. In this work, we introduce Laplace-LoRA, which applies a Bayesian approach to the LoRA parameters. Specifically, Laplace-LoRA applies a Laplace approximation to the posterior over the LoRA parameters, considerably improving the calibration of fine-tuned LLMs.
Age of Information Diffusion on Social Networks: Optimizing Multi-Stage Seeding Strategies

To promote viral marketing, major social platforms (e.g., Facebook Marketplace and Pinduoduo) repeatedly select and invite different users (as seeds) in online social networks to share fresh information about a product or service with their friends. Thereby, we are motivated to optimize a multi-stage seeding process of viral marketing in social networks, and adopt the recent notions of the peak and the average age of information (AoI) to measure the timeliness of promotion information received by network users. Our problem is different from the literature on information diffusion in social networks, which limits to one-time seeding and overlooks AoI dynamics or information replacement over time. As a critical step, we manage to develop closed-form expressions that characterize and trace AoI dynamics over any social network. For the peak AoI problem, we first prove the NP-hardness of our multi-stage seeding problem by a highly non-straightforward reduction from the dominating set problem, and then present a new polynomial-time algorithm that achieves good approximation guarantees (e.g., less than 2 for linear network topology). For minimizing the average AoI, we also prove that our problem is NP-hard by properly reducing it from the set cover problem. Benefiting from our two-side bound analysis on the average AoI objective, we build up a new framework for approximation analysis and link our problem to a much simplified sum-distance minimization problem. This intriguing connection inspires us to develop another polynomial-time algorithm that achieves a good approximation guarantee. Additionally, our theoretical results are well corroborated by experiments on a real social network.