pwc-archive/papers-with-abstracts huggingface dataset:
Multi-objective Representation for Numbers in Clinical Narratives: A CamemBERT-Bio-Based Alternative to Large-Scale LLMs
Authors: Boammani Aser Lompo, Thanh-Dung Le
Date: 2024-05-28 00:00:00

The processing of numerical values is a rapidly developing area in the field of Language Models (LLMs). Despite numerous advancements achieved by previous research, significant challenges persist, particularly within the healthcare domain. This paper investigates the limitations of Transformer models in understanding numerical values. \textit{Objective:} this research aims to categorize numerical values extracted from medical documents into eight specific physiological categories using CamemBERT-bio. \textit{Methods:} In a context where scalable methods and Large Language Models (LLMs) are emphasized, we explore lifting the limitations of transformer-based models. We examine two strategies: fine-tuning CamemBERT-bio on a small medical dataset, integrating Label Embedding for Self-Attention (LESA), and combining LESA with additional enhancement techniques such as Xval. Given that CamemBERT-bio is already pre-trained on a large medical dataset, the first approach aims to update its encoder with the newly added label embeddings technique. In contrast, the second approach seeks to develop multiple representations of numbers (contextual and magnitude-based) to achieve more robust number embeddings. \textit{Results:} As anticipated, fine-tuning the standard CamemBERT-bio on our small medical dataset did not improve F1 scores. However, significant improvements were observed with CamemBERT-bio + LESA, resulting in an over 13\% increase. Similar enhancements were noted when combining LESA with Xval, outperforming conventional methods and giving comparable results to GPT-4 \textit{Conclusions and Novelty:} This study introduces two innovative techniques for handling numerical data, which are also applicable to other modalities. We illustrate how these techniques can improve the performance of Transformer-based models, achieving more reliable classification results even with small datasets.
New perspectives on the optimal placement of detectors for suicide bombers using metaheuristics
Authors: Carlos Cotta, José E. Gallardo
Date: 2024-05-29 00:00:00

We consider an operational model of suicide bombing attacks -- an increasingly prevalent form of terrorism -- against specific targets, and the use of protective countermeasures based on the deployment of detectors over the area under threat. These detectors have to be carefully located in order to minimize the expected number of casualties or the economic damage suffered, resulting in a hard optimization problem for which different metaheuristics have been proposed. Rather than assuming random decisions by the attacker, the problem is approached by considering different models of the latter, whereby he takes informed decisions on which objective must be targeted and through which path it has to be reached based on knowledge on the importance or value of the objectives or on the defensive strategy of the defender (a scenario that can be regarded as an adversarial game). We consider four different algorithms, namely a greedy heuristic, a hill climber, tabu search and an evolutionary algorithm, and study their performance on a broad collection of problem instances trying to resemble different realistic settings such as a coastal area, a modern urban area, and the historic core of an old town. It is shown that the adversarial scenario is harder for all techniques, and that the evolutionary algorithm seems to adapt better to the complexity of the resulting search landscape.
Learning Mixture-of-Experts for General-Purpose Black-Box Discrete Optimization
Authors: Shengcai Liu, Zhiyuan Wang, Yew-Soon Ong, Xin Yao, Ke Tang
Date: 2024-05-29 00:00:00
Tasks: Mixture-of-Experts

Real-world applications involve various discrete optimization problems. Designing a specialized optimizer for each of these problems is challenging, typically requiring significant domain knowledge and human efforts. Hence, developing general-purpose optimizers as an off-the-shelf tool for a wide range of problems has been a long-standing research target. This article introduces MEGO, a novel general-purpose neural optimizer trained through a fully data-driven learning-to-optimize (L2O) approach. MEGO consists of a mixture-of-experts trained on experiences from solving training problems and can be viewed as a foundation model for optimization problems with binary decision variables. When presented with a problem to solve, MEGO actively selects relevant expert models to generate high-quality solutions. MEGO can be used as a standalone sample-efficient optimizer or in conjunction with existing search methods as an initial solution generator. The generality of MEGO is validated across six problem classes, including three classic problem classes and three problem classes arising from real-world applications in compilers, network analysis, and 3D reconstruction. Trained solely on classic problem classes, MEGO performs very well on all six problem classes, significantly surpassing widely used general-purpose optimizers in both solution quality and efficiency. In some cases, MEGO even surpasses specialized state-of-the-art optimizers. Additionally, MEGO provides a similarity measure between problems, yielding a new perspective for problem classification. In the pursuit of general-purpose optimizers through L2O, MEGO represents an initial yet significant step forward.
Evolving Reliable Differentiating Constraints for the Chance-constrained Maximum Coverage Problem
Authors: Saba Sadeghi Ahouei, Jacob de Nobel, Aneta Neumann, Thomas Bäck, Frank Neumann
Date: 2024-05-29 00:00:00

Chance-constrained problems involve stochastic components in the constraints which can be violated with a small probability. We investigate the impact of different types of chance constraints on the performance of iterative search algorithms and study the classical maximum coverage problem in graphs with chance constraints. Our goal is to evolve reliable chance constraint settings for a given graph where the performance of algorithms differs significantly not just in expectation but with high confidence. This allows to better learn and understand how different types of algorithms can deal with different types of constraint settings and supports automatic algorithm selection. We develop an evolutionary algorithm that provides sets of chance constraints that differentiate the performance of two stochastic search algorithms with high confidence. We initially use traditional approximation ratio as the fitness function of (1+1)~EA to evolve instances, which shows inadequacy to generate reliable instances. To address this issue, we introduce a new measure to calculate the performance difference for two algorithms, which considers variances of performance ratios. Our experiments show that our approach is highly successful in solving the instability issue of the performance ratios and leads to evolving reliable sets of chance constraints with significantly different performance for various types of algorithms.
Metaheuristic approaches to the placement of suicide bomber detectors
Authors: Carlos Cotta, José E. Gallardo
Date: 2024-05-28 00:00:00

Suicide bombing is an infamous form of terrorism that is becoming increasingly prevalent in the current era of global terror warfare. We consider the case of targeted attacks of this kind, and the use of detectors distributed over the area under threat as a protective countermeasure. Such detectors are non-fully reliable, and must be strategically placed in order to maximize the chances of detecting the attack, hence minimizing the expected number of casualties. To this end, different metaheuristic approaches based on local search and on population-based search are considered and benchmarked against a powerful greedy heuristic from the literature. We conduct an extensive empirical evaluation on synthetic instances featuring very diverse properties. Most metaheuristics outperform the greedy algorithm, and a hill-climber is shown to be superior to remaining approaches. This hill-climber is subsequently subject to a sensitivity analysis to determine which problem features make it stand above the greedy approach, and is finally deployed on a number of problem instances built after realistic scenarios, corroborating the good performance of the heuristic.
Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance
Authors: The Tien Mai
Date: 2024-05-29 00:00:00
Tasks: parameter estimation, regression

This paper investigates sparse high-dimensional linear regression, particularly examining the properties of the posterior under conditions of random design and unknown error variance. We provide consistency results for the posterior and analyze its concentration rates, demonstrating adaptiveness to the unknown sparsity level of the regression coefficient vector. Furthermore, we extend our investigation to establish concentration outcomes for parameter estimation using specific distance measures. These findings are in line with recent discoveries in frequentist studies. Additionally, by employing techniques to address model misspecification through a fractional posterior, we broaden our analysis through oracle inequalities to encompass the critical aspect of model misspecification for the regular posterior. Our novel findings are demonstrated using two different types of sparsity priors: a shrinkage prior and a spike-and-slab prior.
CHANI: Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration
Authors: Sophie Jaffard, Samuel Vaiter, Patricia Reynaud-Bouret
Date: 2024-05-29 00:00:00

The present work aims at proving mathematically that a neural network inspired by biology can learn a classification task thanks to local transformations only. In this purpose, we propose a spiking neural network named CHANI (Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration), whose neurons activity is modeled by Hawkes processes. Synaptic weights are updated thanks to an expert aggregation algorithm, providing a local and simple learning rule. We were able to prove that our network can learn on average and asymptotically. Moreover, we demonstrated that it automatically produces neuronal assemblies in the sense that the network can encode several classes and that a same neuron in the intermediate layers might be activated by more than one class, and we provided numerical simulations on synthetic dataset. This theoretical approach contrasts with the traditional empirical validation of biologically inspired networks and paves the way for understanding how local learning rules enable neurons to form assemblies able to represent complex concepts.
An engine not a camera: Measuring performative power of online search
Authors: Celestine Mendler-Dünner, Gabriele Carovano, Moritz Hardt
Date: 2024-05-29 00:00:00

The power of digital platforms is at the center of major ongoing policy and regulatory efforts. To advance existing debates, we designed and executed an experiment to measure the performative power of online search providers. Instantiated in our setting, performative power quantifies the ability of a search engine to steer web traffic by rearranging results. To operationalize this definition we developed a browser extension that performs unassuming randomized experiments in the background. These randomized experiments emulate updates to the search algorithm and identify the causal effect of different content arrangements on clicks. Analyzing tens of thousands of clicks, we discuss what our robust quantitative findings say about the power of online search engines, using the Google Shopping antitrust investigation as a case study. More broadly, we envision our work to serve as a blueprint for how the recent definition of performative power can help integrate quantitative insights from online experiments with future investigations into the economic power of digital platforms.
SynerGraph: An Integrated Graph Convolution Network for Multimodal Recommendation
Authors: Mert Burabak, Tevfik Aytekin
Date: 2024-05-29 00:00:00
Tasks: Multimodal Recommendation, Recommendation Systems

This article presents a novel approach to multimodal recommendation systems, focusing on integrating and purifying multimodal data. Our methodology starts by developing a filter to remove noise from various types of data, making the recommendations more reliable. We studied the impact of top-K sparsification on different datasets, finding optimal values that strike a balance between underfitting and overfitting concerns. The study emphasizes the significant role of textual information compared to visual data in providing a deep understanding of items. We conducted sensitivity analyses to understand how different modalities and the use of purifier circle loss affect the efficiency of the model. The findings indicate that systems that incorporate multiple modalities perform better than those relying on just one modality. Our approach highlights the importance of modality purifiers in filtering out irrelevant data, ensuring that user preferences remain relevant. Models without modality purifiers showed reduced performance, emphasizing the need for effective integration of pre-extracted features. The proposed model, which includes an novel self supervised auxiliary task, shows promise in accurately capturing user preferences. The main goal of the fusion technique is to enhance the modeling of user preferences by combining knowledge with item information, utilizing sophisticated language models. Extensive experiments show that our model produces better results than the existing state-of-the-art multimodal recommendation systems.
Mitigate Position Bias with Coupled Ranking Bias on CTR Prediction
Authors: Yao Zhao, Zhining Liu, Tianchi Cai, Haipeng Zhang, Chenyi Zhuang, Jinjie Gu
Date: 2024-05-29 00:00:00
Tasks: Click-Through Rate Prediction, Position, Recommendation Systems

Position bias, i.e., users' preference of an item is affected by its placing position, is well studied in the recommender system literature. However, most existing methods ignore the widely coupled ranking bias, which is also related to the placing position of the item. Using both synthetic and industrial datasets, we first show how this widely coexisted ranking bias deteriorates the performance of the existing position bias estimation methods. To mitigate the position bias with the presence of the ranking bias, we propose a novel position bias estimation method, namely gradient interpolation, which fuses two estimation methods using a fusing weight. We further propose an adaptive method to automatically determine the optimal fusing weight. Extensive experiments on both synthetic and industrial datasets demonstrate the superior performance of the proposed methods.
Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRF
Authors: Leheng Li, Qing Lian, Ying-Cong Chen
Date: 2023-09-04 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, Data Augmentation, NeRF, object-detection, Object Detection

Deep neural networks (DNNs) have been proven extremely susceptible to adversarial examples, which raises special safety-critical concerns for DNN-based autonomous driving stacks (i.e., 3D object detection). Although there are extensive works on image-level attacks, most are restricted to 2D pixel spaces, and such attacks are not always physically realistic in our 3D world. Here we present Adv3D, the first exploration of modeling adversarial examples as Neural Radiance Fields (NeRFs). Advances in NeRF provide photorealistic appearances and 3D accurate generation, yielding a more realistic and realizable adversarial example. We train our adversarial NeRF by minimizing the surrounding objects' confidence predicted by 3D detectors on the training set. Then we evaluate Adv3D on the unseen validation set and show that it can cause a large performance reduction when rendering NeRF in any sampled pose. To generate physically realizable adversarial examples, we propose primitive-aware sampling and semantic-guided regularization that enable 3D patch attacks with camouflage adversarial texture. Experimental results demonstrate that the trained adversarial NeRF generalizes well to different poses, scenes, and 3D detectors. Finally, we provide a defense method to our attacks that involves adversarial training through data augmentation. Project page: https://len-li.github.io/adv3d-web
Emergence and maintenance of modularity in neural networks with Hebbian and anti-Hebbian inhibitory STDP
Authors: Raphaël Bergoin, Alessandro Torcini, Gustavo Deco, Mathias Quoy, Gorka Zamora-López
Date: 2024-05-28 00:00:00

The modular and hierarchical organization of the brain is believed to support the coexistence of segregated (specialization) and integrated (binding) information processes. A relevant question is yet to understand how such architecture naturally emerges and is sustained over time, given the plastic nature of the brain's wiring. Following evidences that the sensory cortices organize into assemblies under selective stimuli, it has been shown that stable neuronal assemblies can emerge due to targeted stimulation, embedding various forms of synaptic plasticity in presence of homeostatic and/or control mechanisms. Here, we show that simple spike-timing-dependent plasticity (STDP) rules, based only on pre- and post-synaptic spike times, can also lead to the stable encoding of memories in the absence of any control mechanism. We develop a model of spiking neurons, trained by stimuli targeting different subpopulations. The model satisfies some biologically plausible features: (i) it contains excitatory and inhibitory neurons with Hebbian and anti-Hebbian STDP; (ii) neither the neuronal activity nor the synaptic weights are frozen after the learning phase. Instead, the neurons are allowed to fire spontaneously while synaptic plasticity remains active. We find that only the combination of two inhibitory STDP subpopulations allows for the formation of stable modules in the network, with each subpopulation playing a distinctive role. The Hebbian subpopulation controls for the firing activity, while the anti-Hebbian neurons promote pattern selectivity. After the learning phase, the network settles into an asynchronous irregular resting-state. This post-learning activity is associated with spontaneous memory recalls which turn out to be fundamental for the long-term consolidation of the learned memories.
CBAR‑UNet: A novel methodology for segmentation of cardiac magnetic resonance images using block attention‑based deep residual neural network
Authors: Rakesh Kumar, Meenu Gupta, Aman Agarwal, Anand Nayyar
Date: 2024-05-29 00:00:00
Proceeding: Multimedia Tools and Applications 2024 5
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Computer-aided Medical Image Segmentation (MIS) plays a leading role in diagnosing diseases automatically. MIS is used extensively in diagnosing medical ailments to obtain clinically relevant information of the shapes and volumes of the target organs and tissues. 
In order to facilitate accurate segmentation, a Block Attention Based Deep Residual Neural Network, i.e., CBAR-UNet model, is proposed to perform cardiac image segmentation on short axis Magnetic Resonance Images (MRI) stacks from top to bottom slice. The Automated Cardiac Diagnosis Challenge (ACDC) dataset is used in the proposed work, which comprise of a 3D MRI of 100 patients (i.e., 200 MRI stacks). Further, 3D MRIs are converted into 2D by slicing each image to train the model, and Contrast Limited Adaptive 
Histogram Equalization (CLAHE) is performed on the 2D dataset. Residual connections are introduced that aid in training deeper models. Convolutional Block Attention Module (CBAM) is also added in the proposed network, which consists of Spatial and Channel 
attention that enhances the spatial and channel-wise features in the image by providing information like ‘what’ and ‘where’ to pay attention respectively. To test and validate the proposed methodology, extensive experimentations were conducted and it was observed that proposed image segmentation model delivers better results by 2.47% when compared to both 2D and 3D State-Of-The-Art (SOTA) methods with a Dice Score of 0.9428
A Dense-Sparse Complementary Network for Human Action Recognition based on RGB and Skeleton Modalities
Authors: Qin Cheng，Jun Cheng，Zhen Liu，Ziliang Ren，Jianming Liu
Date: 2023-12-28 00:00:00
Proceeding: ExpertSystemsWithApplications 2023 12
Tasks: Action Recognition, Action Recognition In Videos, Skeleton Based Action Recognition, Temporal Action Localization

The vulnerability of RGB-based human action recognition in complex environment and variational scenes can be compensated by skeleton modality. Therefore, action recognition methods fusing RGB and skeleton modalities have received increasing attention. However, the recognition performance of the existing methods is still not satisfactory due to the insufficiently optimized sampling, modeling and fusion strategy, even the computational cost is heavy. In this paper, we propose a Dense-Sparse Complementary Network (DSCNet), which aims to leverage the complementary information of the RGB and skeleton modalities at light computational cost to obtain the competitive action recognition performance. Specifically, we first adopt dense and sparse sampling strategies according to the advantages of RGB and skeleton modalities, respectively. And then, we use the skeleton as guiding information to crop the key active region of the persons in the RGB frame, which largely eliminates the interference of the background. Moreover, a Short-Term Motion Extraction Module (STMEM) is proposed to compress the densely sampled RGB frames to fewer frames before feeding them into the backbone network, which avoids a surge in computational cost. And a Sparse Multi-Scale Spatial–Temporal convolutional neural Network (Sparse-MSSTNet) is designed to modeling sparse skeleton. Extensive experiments show that our method effectively combines complementary information of RGB and skeleton modalities to improve recognition accuracy. The DSCNet achieves competitive performance on NTU RGB+D 60, NTU RGB+D 120, PKU-MMD, UAV-human, IKEA ASM and Northwest-UCLA datasets with much less computational cost than exiting methods. The code is available at https://github.com/Maxchengqin/DSCNet.