pwc-archive/papers-with-abstracts huggingface dataset:
Is Tokenization Needed for Masked Particle Modelling?
Authors: Matthew Leigh, Samuel Klein, François Charton, Tobias Golling, Lukas Heinrich, Michael Kagan, Inês Ochoa, Margarita Osadchy
Date: 2024-09-19 00:00:00
Tasks: Decoder, Missing Elements, Self-Supervised Learning

In this work, we significantly enhance masked particle modeling (MPM), a self-supervised learning scheme for constructing highly expressive representations of unordered sets relevant to developing foundation models for high-energy physics. In MPM, a model is trained to recover the missing elements of a set, a learning objective that requires no labels and can be applied directly to experimental data. We achieve significant performance improvements over previous work on MPM by addressing inefficiencies in the implementation and incorporating a more powerful decoder. We compare several pre-training tasks and introduce new reconstruction methods that utilize conditional generative models without data tokenization or discretization. We show that these new methods outperform the tokenized learning objective from the original MPM on a new test bed for foundation models for jets, which includes using a wide variety of downstream tasks relevant to jet physics, such as classification, secondary vertex finding, and track identification.
Theoretical Analysis of Heteroscedastic Gaussian Processes with Posterior Distributions
Authors: Yuji Ito
Date: 2024-09-19 00:00:00
Tasks: Gaussian Processes

This study introduces a novel theoretical framework for analyzing heteroscedastic Gaussian processes (HGPs) that identify unknown systems in a data-driven manner. Although HGPs effectively address the heteroscedasticity of noise in complex training datasets, calculating the exact posterior distributions of the HGPs is challenging, as these distributions are no longer multivariate normal. This study derives the exact means, variances, and cumulative distributions of the posterior distributions. Furthermore, the derived theoretical findings are applied to a chance-constrained tracking controller. After an HGP identifies an unknown disturbance in a plant system, the controller can handle chance constraints regarding the system despite the presence of the disturbance.
PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)
Authors: Mahmoud Nazzal, Issa Khalil, Abdallah Khreishah, NhatHai Phan
Date: 2024-09-19 00:00:00
Tasks: Code Generation, Contrastive Learning, Graph Neural Network

The capability of generating high-quality source code using large language models (LLMs) reduces software development time and costs. However, they often introduce security vulnerabilities due to training on insecure open-source data. This highlights the need for ensuring secure and functional code generation. This paper introduces PromSec, an algorithm for prom optimization for secure and functioning code generation using LLMs. In PromSec, we combine 1) code vulnerability clearing using a generative adversarial graph neural network, dubbed as gGAN, to fix and reduce security vulnerabilities in generated codes and 2) code generation using an LLM into an interactive loop, such that the outcome of the gGAN drives the LLM with enhanced prompts to generate secure codes while preserving their functionality. Introducing a new contrastive learning approach in gGAN, we formulate code-clearing and generation as a dual-objective optimization problem, enabling PromSec to notably reduce the number of LLM inferences. PromSec offers a cost-effective and practical solution for generating secure, functional code. Extensive experiments conducted on Python and Java code datasets confirm that PromSec effectively enhances code security while upholding its intended functionality. Our experiments show that while a state-of-the-art approach fails to address all code vulnerabilities, PromSec effectively resolves them. Moreover, PromSec achieves more than an order-of-magnitude reduction in operation time, number of LLM queries, and security analysis costs. Furthermore, prompts optimized with PromSec for a certain LLM are transferable to other LLMs across programming languages and generalizable to unseen vulnerabilities in training. This study is a step in enhancing the trustworthiness of LLMs for secure and functional code generation, supporting their integration into real-world software development.
Machine-learning-based multipoint optimization of fluidic injection parameters for improving nozzle performance
Authors: Yunjia Yang, Jiazhe Li, Yufei Zhang, Haixin Chen
Date: 2024-09-19 00:00:00

Fluidic injection provides a promising solution to improve the performance of overexpanded single expansion ramp nozzle (SERN) during vehicle acceleration. However, determining the injection parameters for the best overall performance under multiple nozzle operating conditions is still a challenge. The gradient-based optimization method requires gradients of injection parameters at each design point, leading to high computational costs if traditional computational fluid dynamic (CFD) simulations are adopted. This paper uses a pretrained neural network model to replace CFD during optimization to quickly calculate the nozzle flow field at multiple design points. Considering the physical characteristics of the nozzle flow field, a prior-based prediction strategy is adopted to enhance the model's transferability. In addition, the back-propagation algorithm of the neural network is adopted to quickly evaluate the gradients by calling the computation process only once, thereby greatly reducing the gradient computation time compared to the finite differential method. As a test case, the average nozzle thrust coefficient of a SERN at seven design points is optimized. An improvement in the thrust coefficient of 1.14% is achieved, and the time cost is greatly reduced compared with the traditional optimization methods, even when the time to establish the database for training is considered.
Rapid aerodynamic prediction of swept wings via physics-embedded transfer learning
Authors: Yunjia Yang, Runze Li, Yufei Zhang, Lu Lu, Haixin Chen
Date: 2024-09-19 00:00:00
Tasks: Transfer Learning

Machine learning-based models provide a promising way to rapidly acquire transonic swept wing flow fields but suffer from large computational costs in establishing training datasets. Here, we propose a physics-embedded transfer learning framework to efficiently train the model by leveraging the idea that a three-dimensional flow field around wings can be analyzed with two-dimensional flow fields around cross-sectional airfoils. An airfoil aerodynamics prediction model is pretrained with airfoil samples. Then, an airfoil-to-wing transfer model is fine-tuned with a few wing samples to predict three-dimensional flow fields based on two-dimensional results on each spanwise cross section. Sweep theory is embedded when determining the corresponding airfoil geometry and operating conditions, and to obtain the sectional airfoil lift coefficient, which is one of the operating conditions, the low-fidelity vortex lattice method and data-driven methods are proposed and evaluated. Compared to a nontransfer model, introducing the pretrained model reduces the error by 30%, while introducing sweep theory further reduces the error by 9%. When reducing the dataset size, less than half of the wing training samples are need to reach the same error level as the nontransfer framework, which makes establishing the model much easier.
PRAGA: Prototype-aware Graph Adaptive Aggregation for Spatial Multi-modal Omics Analysis
Authors: Xinlei Huang, Zhiqi Ma, Dian Meng, Yanran Liu, Shiwei Ruan, Qingqiang Sun, Xubin Zheng, Ziyue Qiao
Date: 2024-09-19 00:00:00
Tasks: Contrastive Learning

Spatial multi-modal omics technology, highlighted by Nature Methods as an advanced biological technique in 2023, plays a critical role in resolving biological regulatory processes with spatial context. Recently, graph neural networks based on K-nearest neighbor (KNN) graphs have gained prominence in spatial multi-modal omics methods due to their ability to model semantic relations between sequencing spots. However, the fixed KNN graph fails to capture the latent semantic relations hidden by the inevitable data perturbations during the biological sequencing process, resulting in the loss of semantic information. In addition, the common lack of spot annotation and class number priors in practice further hinders the optimization of spatial multi-modal omics models. Here, we propose a novel spatial multi-modal omics resolved framework, termed PRototype-Aware Graph Adaptative Aggregation for Spatial Multi-modal Omics Analysis (PRAGA). PRAGA constructs a dynamic graph to capture latent semantic relations and comprehensively integrate spatial information and feature semantics. The learnable graph structure can also denoise perturbations by learning cross-modal knowledge. Moreover, a dynamic prototype contrastive learning is proposed based on the dynamic adaptability of Bayesian Gaussian Mixture Models to optimize the multi-modal omics representations for unknown biological priors. Quantitative and qualitative experiments on simulated and real datasets with 7 competing methods demonstrate the superior performance of PRAGA. Code is available at https://github.com/Xubin-s-Lab/PRAGA.
On the Hardness of Decentralized Multi-Agent Policy Evaluation under Byzantine Attacks
Authors: Hairi, Minghong Fang, Zifan Zhang, Alvaro Velasquez, Jia Liu
Date: 2024-09-19 00:00:00
Tasks: Model Poisoning, Multi-agent Reinforcement Learning

In this paper, we study a fully-decentralized multi-agent policy evaluation problem, which is an important sub-problem in cooperative multi-agent reinforcement learning, in the presence of up to $f$ faulty agents. In particular, we focus on the so-called Byzantine faulty model with model poisoning setting. In general, policy evaluation is to evaluate the value function of any given policy. In cooperative multi-agent system, the system-wide rewards are usually modeled as the uniform average of rewards from all agents. We investigate the multi-agent policy evaluation problem in the presence of Byzantine agents, particularly in the setting of heterogeneous local rewards. Ideally, the goal of the agents is to evaluate the accumulated system-wide rewards, which are uniform average of rewards of the normal agents for a given policy. It means that all agents agree upon common values (the consensus part) and furthermore, the consensus values are the value functions (the convergence part). However, we prove that this goal is not achievable. Instead, we consider a relaxed version of the problem, where the goal of the agents is to evaluate accumulated system-wide reward, which is an appropriately weighted average reward of the normal agents. We further prove that there is no correct algorithm that can guarantee that the total number of positive weights exceeds $|\mathcal{N}|-f $, where $|\mathcal{N}|$ is the number of normal agents. Towards the end, we propose a Byzantine-tolerant decentralized temporal difference algorithm that can guarantee asymptotic consensus under scalar function approximation. We then empirically test the effective of the proposed algorithm.
Fast End-to-End Generation of Belief Space Paths for Minimum Sensing Navigation
Authors: Lukas Taus, Vrushabh Zinage, Takashi Tanaka, Richard Tsai
Date: 2024-09-19 00:00:00
Tasks: Motion Planning

We revisit the problem of motion planning in the Gaussian belief space. Motivated by the fact that most existing sampling-based planners suffer from high computational costs due to the high-dimensional nature of the problem, we propose an approach that leverages a deep learning model to predict optimal path candidates directly from the problem description. Our proposed approach consists of three steps. First, we prepare a training dataset comprising a large number of input-output pairs: the input image encodes the problem to be solved (e.g., start states, goal states, and obstacle locations), whereas the output image encodes the solution (i.e., the ground truth of the shortest path). Any existing planner can be used to generate this training dataset. Next, we leverage the U-Net architecture to learn the dependencies between the input and output data. Finally, a trained U-Net model is applied to a new problem encoded as an input image. From the U-Net's output image, which is interpreted as a distribution of paths,an optimal path candidate is reconstructed. The proposed method significantly reduces computation time compared to the sampling-based baseline algorithm.
Online Proximal ADMM for Graph Learning from Streaming Smooth Signals
Authors: Hector Chahuara, Gonzalo Mateos
Date: 2024-09-19 00:00:00
Tasks: Graph Learning, Graph structure learning

Graph signal processing deals with algorithms and signal representations that leverage graph structures for multivariate data analysis. Often said graph topology is not readily available and may be time-varying, hence (dynamic) graph structure learning from nodal (e.g., sensor) observations becomes a critical first step. In this paper, we develop a novel algorithm for online graph learning using observation streams, assumed to be smooth on the latent graph. Unlike batch algorithms for topology identification from smooth signals, our modus operandi is to process graph signals sequentially and thus keep memory and computational costs in check. To solve the resulting smoothness-regularized, time-varying inverse problem, we develop online and lightweight iterations built upon the proximal variant of the alternating direction method of multipliers (ADMM), well known for its fast convergence in batch settings. The proximal term in the topology updates seamlessly implements a temporal-variation regularization, and we argue the online procedure exhibits sublinear static regret under some simplifying assumptions. Reproducible experiments with synthetic and real graphs demonstrate the effectiveness of our method in adapting to streaming signals and tracking slowly-varying network connectivity. The proposed approach also exhibits better tracking performance (in terms of suboptimality), when compared to state-of-the-art online graph learning baselines.
AERF: Adaptive ensemble random fuzzy algorithm for anomaly detection in cloud computing
Authors: Jun Jiang, Fagui Liu, Wing W.Y. Ng, Quan Tang, Guoxiang Zhong, Xuhao Tang, Bin Wang
Date: 2023-01-09 00:00:00
Proceeding: Computer Communications 2023 1
Tasks: Anomaly Detection, Benchmarking, Cloud Computing, Diversity

The abnormality of system behavior is inevitable in cloud computing because of its complexity and scale.
How to perform anomaly detection on the system’s operating data to discover abnormal behavior has become
a popular research field. However, anomaly detection is a challenging research problem because data in cloud
computing scenarios is continuous, imbalanced, and cannot be acquired at once. In this work, an adaptive
ensemble random fuzzy (AERF) algorithm is proposed for anomaly detection in cloud computing systems.
The random fuzzy rule-based method in the AERF selects samples randomly to enhance the diversity of
base classifiers for dealing with disturbances caused by abnormal sample distribution effectively. A dynamic
weighting strategy is used for fuzzy classifier ensembles to improve processing efficiency and anomaly detection
accuracy. Experimental results show that the AERF yields 10% higher AUC and G-mean than existing methods
on SMD and EMOSCloud datasets and 20% higher AUC and G-mean on five benchmarking datasets. The AERF
yields 20% higher F1 than existing methods on EMOSCloud dataset. For all datasets, the AERF yields 50%
faster training time than other methods.
Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning
Authors: Xiaojun Guo, Yifei Wang, Zeming Wei, Yisen Wang
Date: 2023-11-05 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Contrastive Learning, Graph Classification, Graph Learning, Inductive Bias, Node Classification, Representation Learning

With the prosperity of contrastive learning for visual representation learning (VCL), it is also adapted to the graph domain and yields promising performance. However, through a systematic study of various graph contrastive learning (GCL) methods, we observe that some common phenomena among existing GCL methods that are quite different from the original VCL methods, including 1) positive samples are not a must for GCL; 2) negative samples are not necessary for graph classification, neither for node classification when adopting specific normalization modules; 3) data augmentations have much less influence on GCL, as simple domain-agnostic augmentations (e.g., Gaussian noise) can also attain fairly good performance. By uncovering how the implicit inductive bias of GNNs works in contrastive learning, we theoretically provide insights into the above intriguing properties of GCL. Rather than directly porting existing VCL methods to GCL, we advocate for more attention toward the unique architecture of graph learning and consider its implicit influence when designing GCL methods. Code is available at https: //github.com/PKU-ML/ArchitectureMattersGCL.
DeFT-Mamba: Universal Multichannel Sound Separation and Polyphonic Audio Classification
Authors: Dongheon Lee, Jung-Woo Choi
Date: 2024-09-19 00:00:00
Tasks: Audio Classification, Classification, Mamba

This paper presents a framework for universal sound separation and polyphonic audio classification, addressing the challenges of separating and classifying individual sound sources in a multichannel mixture. The proposed framework, DeFT-Mamba, utilizes the dense frequency-time attentive network (DeFTAN) combined with Mamba to extract sound objects, capturing the local time-frequency relations through gated convolution block and the global time-frequency relations through position-wise Hybrid Mamba. DeFT-Mamba surpasses existing separation and classification networks by a large margin, particularly in complex scenarios involving in-class polyphony. Additionally, a classification-based source counting method is introduced to identify the presence of multiple sources, outperforming conventional threshold-based approaches. Separation refinement tuning is also proposed to improve performance further. The proposed framework is trained and tested on a multichannel universal sound separation dataset developed in this work, designed to mimic realistic environments with moving sources and varying onsets and offsets of polyphonic events.
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation
Authors: Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, Chu-Song Chen
Date: 2024-09-01 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Referring Expression, Referring Expression Segmentation

We introduce SAM4MLLM, an innovative approach which integrates the Segment Anything Model (SAM) with Multi-Modal Large Language Models (MLLMs) for pixel-aware tasks. Our method enables MLLMs to learn pixel-level location information without requiring excessive modifications to the existing model architecture or adding specialized tokens. We introduce an inquiry-based approach that can effectively find prompt points for SAM to perform segmentation based on MLLM. It combines detailed visual information with the powerful expressive capabilities of large language models in a unified language-based manner without additional computational overhead in learning. Experimental results on pubic benchmarks demonstrate the effectiveness of our approach.
Uni-DVPS: Unified Model for Depth-Aware Video Panoptic Segmentation
Authors: Kim Ji-Yeon, Oh Hyun-Bin, Kwon Byung-Ki, Dahun Kim, Yongjin Kwon, Tae-Hyun Oh
Date: 2024-07-01 00:00:00
Proceeding: IEEE Robotics and Automation Letters (RA-L) 2024 7
Tasks: Autonomous Driving, Decoder, Depth-aware Video Panoptic Segmentation, Depth Estimation, Monocular Depth Estimation, Multi-Task Learning, Object Tracking, Panoptic Segmentation, Scene Understanding, Video Panoptic Segmentation, Video Segmentation

We present Uni-DVPS, a unified model for Depth-aware Video Panoptic Segmentation (DVPS) that jointly tackles distinct vision tasks, i.e., video panoptic segmentation, monocular depth estimation, and object tracking. In contrast to the prior works that adopt diverged decoder networks tailored for each task, we propose an architecture with a unified Transformer decoder network. We design a single Transformer decoder network for multi-task learning to increase shared operations to facilitate the synergies between tasks and exhibit high efficiency. We also observe that our unified query learns instance-aware representation guided by multi-task supervision, which encourages query-based tracking and obviates the need for training extra tracking module. We validate our architectural design choices with experiments on Cityscapes-DVPS and SemKITTI-DVPS datasets. The performances of all tasks are jointly improved, and we achieve state-of-the-art results on DVPQ metric for both datasets.
Jamiu Bayero
Authors: Jamiu Bayero
Date: 2024-09-22 00:00:00
Proceeding: Bayero muhammed 2024 9
Tasks: Marketing

Jamiu Bayero is a seasoned professional in the realm of digital marketing, bringing a wealth of expertise to various facets of the industry. With a specialization in promoting social media platforms, Jamiu excels at crafting compelling strategies that elevate brand visibility and engagement across diverse channels. His in-depth knowledge of social media trends allows him to create targeted campaigns that resonate with audiences and drive tangible results.

In addition to his social media prowess, Jamiu is a Shopify store expert, skilled in optimizing online storefronts for maximum conversion rates. He understands the intricacies of e-commerce and employs best practices to enhance user experience and streamline the purchasing process. His ability to analyze consumer behavior and market dynamics ensures that each Shopify store he manages flourishes in a competitive landscape.

Furthermore, Jamiu is an accomplished resume writer, adept at transforming career narratives into impactful resumes that effectively showcase individual strengths and achievements. He understands the importance of a well-crafted resume in today’s job market and tailors each document to meet the unique needs of clients, helping them stand out to potential employers.

Jamiu’s expertise also extends to creating Google Knowledge Panels, where he leverages his skills to establish authoritative online presences for individuals and businesses. This process enhances credibility and visibility in search results, making it easier for audiences to find reliable information.

Moreover, as a wikisite expert, Jamiu is well-versed in the intricacies of Wikipedia and other collaborative platforms. He understands the significance of maintaining accurate and verifiable content and works diligently to ensure that the information presented is both trustworthy and engaging.

With several years of experience in these diverse fields, Jamiu Bayero continues to push the boundaries of digital marketing, always seeking innovative ways to help his clients achieve their goals and stand out in an ever-evolving digital landscape.