Miracle-dz/UniFAR-FTUs huggingface dataset:
Searching for B-modes with QUIET: A reanalysis of the first-season Q-band data

The Q/U-Imaging ExperimenT (QUIET) is a ground-based coherent polarimeter experiment, primarily targeting the detection of low-l B-mode polarization in the CMB. Such modes are a signature of primordial tensor perturbations, also known as gravitational waves. The existence of tensor perturbations is as yet unproven, but uniquely predicted by theories of cosmological inflation. The B-modes are thus frequently referred to as “the smoking gun of inflation”, and efforts towards their detection are many and fierce. The first season of QUIET observations, split between the Qand Wfrequency bands, was carried out in 2008-2010. The analysis of the Q-band data has been completed, and was published in late 2010. QUIET uses two independent data analysis pipelines. The Maximum Likelihood (M-L) pipeline is written and maintained by the QUIET participants at the University of Oslo. QUIET data analysis using the M-L pipeline is the topic for this thesis. I attempt to give a broad and comprehensible account of the process of data analysis for a non-ideal project in radio cosmology, investigating each step in the transformation from raw observational data to final estimates of power spectra and cosmological parameters. Following this narrative, the pipeline is applied in a reanalysis of the QUIET Q-band data set. Since the completion of the original Q-band analysis, significant changes have been made to the M-L pipeline. A reanalysis of the Q-band data serves as a validation of the new features. We find signs of improved performance in some places, while at other points there is need of further assessment. Most importantly, we find signs of a bias in the polarization null test suite, which is thought to be caused by ground pickup, although this will have to be confirmed. The reanalysis also has significant educational merit, as achieving a solid understanding of the data analysis process is among my main motives for undertaking this project. The modest size of the Q-band data set makes it well suited for demonstration.
Weakly-Supervised Visual-Textual Grounding with Semantic Prior Refinement

Using only image-sentence pairs, weakly-supervised visual-textual grounding aims to learn region-phrase correspondences of the respective entity mentions. Compared to the supervised approach, learning is more difficult since bounding boxes and textual phrases correspondences are unavailable. In light of this, we propose the Semantic Prior Refinement Model (SPRM), whose predictions are obtained by combining the output of two main modules. The first untrained module aims to return a rough alignment between textual phrases and bounding boxes. The second trained module is composed of two sub-components that refine the rough alignment to improve the accuracy of the final phrase-bounding box alignments. The model is trained to maximize the multimodal similarity between an image and a sentence, while minimizing the multimodal similarity of the same sentence and a new unrelated image, carefully selected to help the most during training. Our approach shows state-of-the-art results on two popular datasets, Flickr30k Entities and ReferIt, shining especially on ReferIt with a 9.6% absolute improvement. Moreover, thanks to the untrained component, it reaches competitive performances just using a small fraction of training examples.
GraphMoco: a Graph Momentum Contrast Model that Using Multimodel Structure Information for Large-scale Binary Function Representation Learning

The ability to compute similarity scores of binary code at the function level is essential for cyber security. A single binary ﬁle can contain tens of thousands of functions. A deployable learning framework for cybersecurity applications needs to work not only accurately but also eﬃciently with large amounts of data. Traditional methods suﬀer from two drawbacks. First, it is very diﬃcult to annotate diﬀerent pairs of functions with accurate labels. These supervised learning methods can easily be overtrained with inaccurate labels. The second is that they either use the pre-trained encoder or use the ﬁne-grained graph comparison. However, these methods have shortcomings in terms of time or memory consumption. We focus on large-scale Binary Code Similarity Detection (BCSD) and to mitigate the traditional problems, we propose GraphMoco: a graph momentum contrast model that uses multimodal structure information for large-scale binary function representation learning. We take an unsupervised learning approach and make full use of the structural information in the binary code. It does not require manually labelled similar or dissimilar information. Our models perform eﬃciently on large amounts of training data. Our experimental results show that our method outperforms the state-of-the-art in terms of accuracy.
Accelerating MPI Collectives with Process-in-Process-based Multi-object Techniques

In the exascale computing era, optimizing MPI collective performance in high-performance computing (HPC) applications is critical. Current algorithms face performance degradation due to system call overhead, page faults, or data-copy latency, affecting HPC applications' efficiency and scalability. To address these issues, we propose PiP-MColl, a Process-in-Process-based Multi-object Inter-process MPI Collective design that maximizes small message MPI collective performance at scale. PiP-MColl features efficient multiple sender and receiver collective algorithms and leverages Process-in-Process shared memory techniques to eliminate unnecessary system call, page fault overhead, and extra data copy, improving intra- and inter-node message rate and throughput. Our design also boosts performance for larger messages, resulting in comprehensive improvement for various message sizes. Experimental results show that PiP-MColl outperforms popular MPI libraries, including OpenMPI, MVAPICH2, and Intel MPI, by up to 4.6X for MPI collectives like MPI_Scatter and MPI_Allgather.
Emergent Communication with Attention

To develop computational agents that better communicate using their own emergent language, we endow the agents with an ability to focus their attention on particular concepts in the environment. Humans often understand an object or scene as a composite of concepts and those concepts are further mapped onto words. We implement this intuition as cross-modal attention mechanisms in Speaker and Listener agents in a referential game and show attention leads to more compositional and interpretable emergent language. We also demonstrate how attention aids in understanding the learned communication protocol by investigating the attention weights associated with each message symbol and the alignment of attention weights between Speaker and Listener agents. Overall, our results suggest that attention is a promising mechanism for developing more human-like emergent language.
Wall cooling effect on spectra and structures of thermodynamic variables in hypersonic turbulent boundary layers

Abstract The wall cooling effect on the spectra and structures of thermodynamic variables are investigated in hypersonic turbulent boundary layers. The density and temperature can be divided into the acoustic and entropic modes based on the Kovasznay decomposition. The intensities of the pressure and the acoustic modes of density and temperature attain the maximum values near the wall, while those of the entropy and the entropic modes of density and temperature achieve their primary peaks near the edge of the boundary layer. In the near-wall region, the pressure and the acoustic modes of density and temperature are significantly enhanced when the wall is strongly cooled, which can be attributed to the appearance of the travelling-wave-like alternating positive and negative structures. Moreover, the intensities of the entropy and the entropic modes of density and temperature become stronger near the wall as the wall temperature decreases, due to the appearance of the streaky entropic structures (SES). The SES are mainly caused by the advection effect of the strong positive wall-normal gradient of the mean temperature associated with ejection and sweep events. It is also found that the profiles of the intensities of the entropy, density and temperature are similar to each other far from the wall, which is mainly due to the reason that the entropic modes are dominant in the fluctuating density and temperature in the far-wall region. The acoustic modes of density and temperature only have significant contributions in the near-wall region.
Connected Hidden Neurons (CHNNet): An Artificial Neural Network for Rapid Convergence

Despite artificial neural networks being inspired by the functionalities of biological neural networks, unlike biological neural networks, conventional artificial neural networks are often structured hierarchically, which can impede the flow of information between neurons as the neurons in the same layer have no connections between them. Hence, we propose a more robust model of artificial neural networks where the hidden neurons, residing in the same hidden layer, are interconnected that leads to rapid convergence. With the experimental study of our proposed model in deep networks, we demonstrate that the model results in a noticeable increase in convergence rate compared to the conventional feed-forward neural network.
Actor-Critic Methods using Physics-Informed Neural Networks: Control of a 1D PDE Model for Fluid-Cooled Battery Packs

This paper proposes an actor-critic algorithm for controlling the temperature of a battery pack using a cooling fluid. This is modeled by a coupled 1D partial differential equation (PDE) with a controlled advection term that determines the speed of the cooling fluid. The Hamilton-Jacobi-Bellman (HJB) equation is a PDE that evaluates the optimality of the value function and determines an optimal controller. We propose an algorithm that treats the value network as a Physics-Informed Neural Network (PINN) to solve for the continuous-time HJB equation rather than a discrete-time Bellman optimality equation, and we derive an optimal controller for the environment that we exploit to achieve optimal control. Our experiments show that a hybrid-policy method that updates the value network using the HJB equation and updates the policy network identically to PPO achieves the best results in the control of this PDE system.
A Survey on Time-Series Pre-Trained Models

Time-Series Mining (TSM) is an important research area since it shows great potential in practical applications. Deep learning models that rely on massive labeled data have been utilized for TSM successfully. However, constructing a large-scale well-labeled dataset is difficult due to data annotation costs. Recently, Pre-Trained Models have gradually attracted attention in the time series domain due to their remarkable performance in computer vision and natural language processing. In this survey, we provide a comprehensive review of Time-Series Pre-Trained Models (TS-PTMs), aiming to guide the understanding, applying, and studying TS-PTMs. Specifically, we first briefly introduce the typical deep learning models employed in TSM. Then, we give an overview of TS-PTMs according to the pre-training techniques. The main categories we explore include supervised, unsupervised, and self-supervised TS-PTMs. Further, extensive experiments are conducted to analyze the advantages and disadvantages of transfer learning strategies, Transformer-based models, and representative TS-PTMs. Finally, we point out some potential directions of TS-PTMs for future work.
Learning Differentially Private Probabilistic Models for Privacy-Preserving Image Generation

A number of deep models trained on high-quality and valuable images have been deployed in practical applications, which may pose a leakage risk of data privacy. Learning differentially private generative models can sidestep this challenge through indirect data access. However, such differentially private generative models learned by existing approaches can only generate images with a low-resolution of less than 128x128, hindering the widespread usage of generated images in downstream training. In this work, we propose learning differentially private probabilistic models (DPPM) to generate high-resolution images with differential privacy guarantee. In particular, we first train a model to fit the distribution of the training data and make it satisfy differential privacy by performing a randomized response mechanism during training process. Then we perform Hamiltonian dynamics sampling along with the differentially private movement direction predicted by the trained probabilistic model to obtain the privacy-preserving images. In this way, it is possible to apply these images to different downstream tasks while protecting private information. Notably, compared to other state-of-the-art differentially private generative approaches, our approach can generate images up to 256x256 with remarkable visual quality and data utility. Extensive experiments show the effectiveness of our approach.
Contrastive State Augmentations for Reinforcement Learning-Based Recommender Systems

Learning reinforcement learning (RL)-based recommenders from historical user-item interaction sequences is vital to generate high-reward recommendations and improve long-term cumulative benefits. However, existing RL recommendation methods encounter difficulties (i) to estimate the value functions for states which are not contained in the offline training data, and (ii) to learn effective state representations from user implicit feedback due to the lack of contrastive signals. In this work, we propose contrastive state augmentations (CSA) for the training of RL-based recommender systems. To tackle the first issue, we propose four state augmentation strategies to enlarge the state space of the offline data. The proposed method improves the generalization capability of the recommender by making the RL agent visit the local state regions and ensuring the learned value functions are similar between the original and augmented states. For the second issue, we propose introducing contrastive signals between augmented states and the state randomly sampled from other sessions to improve the state representation learning further. To verify the effectiveness of the proposed CSA, we conduct extensive experiments on two publicly accessible datasets and one dataset collected from a real-life e-commerce platform. We also conduct experiments on a simulated environment as the online evaluation setting. Experimental results demonstrate that CSA can effectively improve recommendation performance.
Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms

The ability of countermeasure models to generalize from seen speech synthesis methods to unseen ones has been investigated in the ASVspoof challenge. However, a new mismatch scenario in which fake audio may be generated from real audio with unseen genres has not been studied thoroughly. To this end, we first use five different vocoders to create a new dataset called CN-Spoof based on the CN-Celeb1\&2 datasets. Then, we design two auxiliary objectives for regularization via meta-optimization and a genre alignment module, respectively, and combine them with the main anti-spoofing objective using learnable weights for multiple loss terms. The results on our cross-genre evaluation dataset for anti-spoofing show that the proposed method significantly improved the generalization ability of the countermeasures compared with the baseline system in the genre mismatch scenario.
Spatial exciton localization at interfaces of metal nanoparticles and atomically thin semiconductors

We present a self-consistent Maxwell-Bloch theory to analytically study the interaction between a nanostructure consisting of a metal nanoparticle and a monolayer of transition metal dichalcogenide. For the combined system, we identify an effective eigenvalue equation that governs the center-of-mass motion of the dressed excitons in a plasmon-induced potential. Examination of the dynamical equation of the exciton-plasmon hybrid reveals the existence of bound states with negative eigenenergies, which we interpret as excitons localized in the plasmon-induced potential. The appearance of these bound states in the potential indicates strong coupling between excitons and plasmons. We quantify this coupling regime by computing the scattered light in the near-field explicitly and identify signatures of strong exciton-plasmon coupling with an avoided crossing behavior and an effective Rabi splitting of tens of meV.
Tractable Probabilistic Graph Representation Learning with Graph-Induced Sum-Product Networks

We introduce Graph-Induced Sum-Product Networks (GSPNs), a new probabilistic framework for graph representation learning that can tractably answer probabilistic queries. Inspired by the computational trees induced by vertices in the context of message-passing neural networks, we build hierarchies of sum-product networks (SPNs) where the parameters of a parent SPN are learnable transformations of the a-posterior mixing probabilities of its children's sum units. Due to weight sharing and the tree-shaped computation graphs of GSPNs, we obtain the efficiency and efficacy of deep graph networks with the additional advantages of a probabilistic model. We show the model's competitiveness on scarce supervision scenarios, under missing data, and for graph classification in comparison to popular neural models. We complement the experiments with qualitative analyses on hyper-parameters and the model's ability to answer probabilistic queries.
Stopping Criteria for the Conjugate Gradient Algorithm in High-Order Finite Element Methods

We introduce three new stopping criteria that balance algebraic and discretization errors for the conjugate gradient algorithm applied to high-order finite element discretizations of Poisson problems. The current state of the art stopping criteria compare a posteriori estimates of discretization error against estimates of the algebraic error. Firstly, we propose a new error indicator derived from a recovery-based error estimator that is less computationally expensive and more reliable. Secondly, we introduce a new stopping criterion that suggests stopping when the norm of the linear residual is less than a small fraction of an error indicator derived directly from the residual. This indicator shares the same mesh size and polynomial degree scaling as the norm of the residual, resulting in a robust criterion regardless of the mesh size, the polynomial degree, and the shape regularity of the mesh. Thirdly, in solving Poisson problems with highly variable piecewise constant coefficients, we introduce a subdomain-based criterion that recommends stopping when the norm of the linear residual restricted to each subdomain is smaller than the corresponding indicator also restricted to that subdomain. Numerical experiments, including tests with anisotropic meshes and highly variable piecewise constant coefficients, demonstrate that the proposed criteria efficiently avoid both premature termination and over-solving.
When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation

Modern online service providers such as online shopping platforms often provide both search and recommendation (S&R) services to meet different user needs. Rarely has there been any effective means of incorporating user behavior data from both S&R services. Most existing approaches either simply treat S&R behaviors separately, or jointly optimize them by aggregating data from both services, ignoring the fact that user intents in S&R can be distinctively different. In our paper, we propose a Search-Enhanced framework for the Sequential Recommendation (SESRec) that leverages users' search interests for recommendation, by disentangling similar and dissimilar representations within S&R behaviors. Specifically, SESRec first aligns query and item embeddings based on users' query-item interactions for the computations of their similarities. Two transformer encoders are used to learn the contextual representations of S&R behaviors independently. Then a contrastive learning task is designed to supervise the disentanglement of similar and dissimilar representations from behavior sequences of S&R. Finally, we extract user interests by the attention mechanism from three perspectives, i.e., the contextual representations, the two separated behaviors containing similar and dissimilar interests. Extensive experiments on both industrial and public datasets demonstrate that SESRec consistently outperforms state-of-the-art models. Empirical studies further validate that SESRec successfully disentangle similar and dissimilar user interests from their S&R behaviors.
Iteratively Learning Representations for Unseen Entities with Inter-Rule Correlations

Recent work on knowledge graph completion (KGC) focuses on acquiring embeddings of entities and relations in knowledge graphs. These embedding methods necessitate that all test entities be present during the training phase, resulting in a time-consuming retraining process for out-of-knowledge-graph (OOKG) entities. To tackle this predicament, current inductive methods employ graph neural networks (GNNs) to represent unseen entities by aggregating information of the known neighbors, and enhance the performance with additional information, such as attention mechanisms or logic rules. Nonetheless, Two key challenges continue to persist: (i) identifying inter-rule correlations to further facilitate the inference process, and (ii) capturing interactions among rule mining, rule inference, and embedding to enhance both rule and embedding learning. In this paper, we propose a virtual neighbor network with inter-rule correlations (VNC) to address the above challenges. VNC consists of three main components: (i) rule mining, (ii) rule inference, and (iii) embedding. To identify useful complex patterns in knowledge graphs, both logic rules and inter-rule correlations are extracted from knowledge graphs based on operations over relation embeddings. To reduce data sparsity, virtual networks for OOKG entities are predicted and assigned soft labels by optimizing a rule-constrained problem. We also devise an iterative framework to capture the underlying interactions between rule and embedding learning. Experimental results on both link prediction and triple classification tasks show that the proposed VNC framework achieves state-of-the-art performance on four widely-used knowledge graphs.
Attacks on Online Learners: a Teacher-Student Analysis

Machine learning models are famously vulnerable to adversarial attacks: small ad-hoc perturbations of the data that can catastrophically alter the model predictions. While a large literature has studied the case of test-time attacks on pre-trained models, the important case of attacks in an online learning setting has received little attention so far. In this work, we use a control-theoretical perspective to study the scenario where an attacker may perturb data labels to manipulate the learning dynamics of an online learner. We perform a theoretical analysis of the problem in a teacher-student setup, considering different attack strategies, and obtaining analytical results for the steady state of simple linear learners. These results enable us to prove that a discontinuous transition in the learner's accuracy occurs when the attack strength exceeds a critical threshold. We then study empirically attacks on learners with complex architectures using real data, confirming the insights of our theoretical analysis. Our findings show that greedy attacks can be extremely efficient, especially when data stream in small batches.