pwc-archive/papers-with-abstracts huggingface dataset:
Cyberbullying Detection for Low-resource Languages and Dialects: Review of the State of the Art
Authors: Tanjim Mahmud, Michal Ptaszynski, Juuso Eronen, Fumito Masui
Date: 2023-08-30 00:00:00
Tasks: Abusive Language

The struggle of social media platforms to moderate content in a timely manner, encourages users to abuse such platforms to spread vulgar or abusive language, which, when performed repeatedly becomes cyberbullying a social problem taking place in virtual environments, yet with real-world consequences, such as depression, withdrawal, or even suicide attempts of its victims. Systems for the automatic detection and mitigation of cyberbullying have been developed but, unfortunately, the vast majority of them are for the English language, with only a handful available for low-resource languages. To estimate the present state of research and recognize the needs for further development, in this paper we present a comprehensive systematic survey of studies done so far for automatic cyberbullying detection in low-resource languages. We analyzed all studies on this topic that were available. We investigated more than seventy published studies on automatic detection of cyberbullying or related language in low-resource languages and dialects that were published between around 2017 and January 2023. There are 23 low-resource languages and dialects covered by this paper, including Bangla, Hindi, Dravidian languages and others. In the survey, we identify some of the research gaps of previous studies, which include the lack of reliable definitions of cyberbullying and its relevant subcategories, biases in the acquisition, and annotation of data. Based on recognizing those research gaps, we provide some suggestions for improving the general research conduct in cyberbullying detection, with a primary focus on low-resource languages. Based on those proposed suggestions, we collect and release a cyberbullying dataset in the Chittagonian dialect of Bangla and propose a number of initial ML solutions trained on that dataset. In addition, pre-trained transformer-based the BanglaBERT model was also attempted.
Quantifying and Analyzing Entity-level Memorization in Large Language Models
Authors: Zhenhong Zhou, Jiuyang Xiang, Chaomeng Chen, Sen Su
Date: 2023-08-30 00:00:00
Tasks: Language Modeling, Language Modelling, Memorization, Probing Language Models

Large language models (LLMs) have been proven capable of memorizing their training data, which can be extracted through specifically designed prompts. As the scale of datasets continues to grow, privacy risks arising from memorization have attracted increasing attention. Quantifying language model memorization helps evaluate potential privacy risks. However, prior works on quantifying memorization require access to the precise original data or incur substantial computational overhead, making it difficult for applications in real-world language models. To this end, we propose a fine-grained, entity-level definition to quantify memorization with conditions and metrics closer to real-world scenarios. In addition, we also present an approach for efficiently extracting sensitive entities from autoregressive language models. We conduct extensive experiments based on the proposed, probing language models' ability to reconstruct sensitive entities under different settings. We find that language models have strong memorization at the entity level and are able to reproduce the training data even with partial leakages. The results demonstrate that LLMs not only memorize their training data but also understand associations between entities. These findings necessitate that trainers of LLMs exercise greater prudence regarding model memorization, adopting memorization mitigation techniques to preclude privacy violations.
A Deep Dive into the Design Space of a Dynamically Reconfigurable Cryogenic Spiking Neuron
Authors: Md Mazharul Islam, Shamiul Alam, Catherine D Schuman, Md Shafayat Hossain, Ahmedullah Aziz
Date: 2023-08-30 00:00:00

Spiking neural network offers the most bio-realistic approach to mimic the parallelism and compactness of the human brain. A spiking neuron is the central component of an SNN which generates information-encoded spikes. We present a comprehensive design space analysis of the superconducting memristor (SM)-based electrically reconfigurable cryogenic neuron. A superconducting nanowire (SNW) connected in parallel with an SM function as a dual-frequency oscillator and two of these oscillators can be coupled to design a dynamically tunable spiking neuron. The same neuron topology was previously proposed where a fixed resistance was used in parallel with the SNW. Replacing the fixed resistance with the SM provides an additional tuning knob with four distinct combinations of SM resistances, which improves the reconfigurability by up to ~70%. Utilizing an external bias current (Ibias), the spike frequency can be modulated up to ~3.5 times. Two distinct spike amplitudes (~1V and ~1.8 V) are also achieved. Here, we perform a systematic sensitivity analysis and show that the reconfigurability can be further tuned by choosing a higher input current strength. By performing a 500-point Monte Carlo variation analysis, we find that the spike amplitude is more variation robust than spike frequency and the variation robustness can be further improved by choosing a higher Ibias. Our study provides valuable insights for further exploration of materials and circuit level modification of the neuron that will be useful for system-level incorporation of the neuron circuit
Temporal-spatial model via Trend Filtering
Authors: Carlos Misael Madrid Padilla, Oscar Hernan Madrid Padilla, Daren Wang
Date: 2023-08-30 00:00:00
Tasks: model

This research focuses on the estimation of a non-parametric regression function designed for data with simultaneous time and space dependencies. In such a context, we study the Trend Filtering, a nonparametric estimator introduced by \cite{mammen1997locally} and \cite{rudin1992nonlinear}. For univariate settings, the signals we consider are assumed to have a kth weak derivative with bounded total variation, allowing for a general degree of smoothness. In the multivariate scenario, we study a $K$-Nearest Neighbor fused lasso estimator as in \cite{padilla2018adaptive}, employing an ADMM algorithm, suitable for signals with bounded variation that adhere to a piecewise Lipschitz continuity criterion. By aligning with lower bounds, the minimax optimality of our estimators is validated. A unique phase transition phenomenon, previously uncharted in Trend Filtering studies, emerges through our analysis. Both Simulation studies and real data applications underscore the superior performance of our method when compared with established techniques in the existing literature.
Computational Lower Bounds for Graphon Estimation via Low-degree Polynomials
Authors: Yuetian Luo, Chao GAO
Date: 2023-08-30 00:00:00
Tasks: Community Detection, Graphon Estimation, Stochastic Block Model

Graphon estimation has been one of the most fundamental problems in network analysis and has received considerable attention in the past decade. From the statistical perspective, the minimax error rate of graphon estimation has been established by Gao et al (2015) for both stochastic block model and nonparametric graphon estimation. The statistical optimal estimators are based on constrained least squares and have computational complexity exponential in the dimension. From the computational perspective, the best-known polynomial-time estimator is based universal singular value thresholding, but it can only achieve a much slower estimation error rate than the minimax one. The computational optimality of the USVT or the existence of a computational barrier in graphon estimation has been a long-standing open problem. In this work, we provide rigorous evidence for the computational barrier in graphon estimation via low-degree polynomials. Specifically, in SBM graphon estimation, we show that for low-degree polynomial estimators, their estimation error rates cannot be significantly better than that of the USVT under a wide range of parameter regimes and in nonparametric graphon estimation, we show low-degree polynomial estimators achieve estimation error rates strictly slower than the minimax rate. Our results are proved based on the recent development of low-degree polynomials by Schramm and Wein (2022), while we overcome a few key challenges in applying it to the general graphon estimation problem. By leveraging our main results, we also provide a computational lower bound on the clustering error for community detection in SBM with a growing number of communities and this yields a new piece of evidence for the conjectured Kesten-Stigum threshold for efficient community recovery. Finally, we extend our computational lower bounds to sparse graphon estimation and biclustering.
Parametric quantile autoregressive conditional duration models with application to intraday value-at-risk
Authors: Helton Saulo, Suvra Pal, Rubens Souza, Roberto Vila, Alan Dasilva
Date: 2023-08-29 00:00:00
Tasks: Diagnostic, parameter estimation

The modeling of high-frequency data that qualify financial asset transactions has been an area of relevant interest among statisticians and econometricians -- above all, the analysis of time series of financial durations. Autoregressive conditional duration (ACD) models have been the main tool for modeling financial transaction data, where duration is usually defined as the time interval between two successive events. These models are usually specified in terms of a time-varying mean (or median) conditional duration. In this paper, a new extension of ACD models is proposed which is built on the basis of log-symmetric distributions reparametrized by their quantile. The proposed quantile log-symmetric conditional duration autoregressive model allows us to model different percentiles instead of the traditionally used conditional mean (or median) duration. We carry out an in-depth study of theoretical properties and practical issues, such as parameter estimation using maximum likelihood method and diagnostic analysis based on residuals. A detailed Monte Carlo simulation study is also carried out to evaluate the performance of the proposed models and estimation method in retrieving the true parameter values as well as to evaluate a form of residuals. Finally, the proposed class of models is applied to a price duration data set and then used to derive a semi-parametric intraday value-at-risk (IVaR) model.
Deep Learning and Bayesian inference for Inverse Problems
Authors: Ali Mohammad-Djafari, Ning Chu, Li Wang, Liang Yu
Date: 2023-08-28 00:00:00
Tasks: Bayesian Inference, Deep Learning

Inverse problems arise anywhere we have indirect measurement. As, in general they are ill-posed, to obtain satisfactory solutions for them needs prior knowledge. Classically, different regularization methods and Bayesian inference based methods have been proposed. As these methods need a great number of forward and backward computations, they become costly in computation, in particular, when the forward or generative models are complex and the evaluation of the likelihood becomes very costly. Using Deep Neural Network surrogate models and approximate computation can become very helpful. However, accounting for the uncertainties, we need first understand the Bayesian Deep Learning and then, we can see how we can use them for inverse problems. In this work, we focus on NN, DL and more specifically the Bayesian DL particularly adapted for inverse problems. We first give details of Bayesian DL approximate computations with exponential families, then we will see how we can use them for inverse problems. We consider two cases: First the case where the forward operator is known and used as physics constraint, the second more general data driven DL methods. keyword: Neural Network, Variational Bayesian inference, Bayesian Deep Learning (DL), Inverse problems, Physics based DL.
Adaptive Multi-Modalities Fusion in Sequential Recommendation Systems
Authors: Hengchang Hu, Wei Guo, Yong liu, Min-Yen Kan
Date: 2023-08-30 00:00:00
Tasks: Recommendation Systems, Sequential Recommendation

In sequential recommendation, multi-modal information (e.g., text or image) can provide a more comprehensive view of an item's profile. The optimal stage (early or late) to fuse modality features into item representations is still debated. We propose a graph-based approach (named MMSR) to fuse modality features in an adaptive order, enabling each modality to prioritize either its inherent sequential nature or its interplay with other modalities. MMSR represents each user's history as a graph, where the modality features of each item in a user's history sequence are denoted by cross-linked nodes. The edges between homogeneous nodes represent intra-modality sequential relationships, and the ones between heterogeneous nodes represent inter-modality interdependence relationships. During graph propagation, MMSR incorporates dual attention, differentiating homogeneous and heterogeneous neighbors. To adaptively assign nodes with distinct fusion orders, MMSR allows each node's representation to be asynchronously updated through an update gate. In scenarios where modalities exhibit stronger sequential relationships, the update gate prioritizes updates among homogeneous nodes. Conversely, when the interdependent relationships between modalities are more pronounced, the update gate prioritizes updates among heterogeneous nodes. Consequently, MMSR establishes a fusion order that spans a spectrum from early to late modality fusion. In experiments across six datasets, MMSR consistently outperforms state-of-the-art models, and our graph propagation methods surpass other graph neural networks. Additionally, MMSR naturally manages missing modalities.
Denoising Attention for Query-aware User Modeling in Personalized Search
Authors: Elias Bassani, Pranav Kasela, Gabriella Pasi
Date: 2023-08-30 00:00:00
Tasks: Denoising, Diversity, Information Retrieval, Retrieval

The personalization of search results has gained increasing attention in the past few years, thanks to the development of Neural Networks-based approaches for Information Retrieval and the importance of personalization in many search scenarios. Recent works have proposed to build user models at query time by leveraging the Attention mechanism, which allows weighing the contribution of the user-related information w.r.t. the current query. This approach allows taking into account the diversity of the user's interests by giving more importance to those related to the current search performed by the user. In this paper, we first discuss some shortcomings of the standard Attention formulation when employed for personalization. In particular, we focus on issues related to its normalization mechanism and its inability to entirely filter out noisy user-related information. Then, we introduce the Denoising Attention mechanism: an Attention variant that directly tackles the above shortcomings by adopting a robust normalization scheme and introducing a filtering mechanism. The reported experimental evaluation shows the benefits of the proposed approach over other Attention-based variants.
DRGame: Diversified Recommendation for Multi-category Video Games with Balanced Implicit Preferences
Authors: Kangzhe Liu, Jianghong Ma, Shanshan Feng, Haijun Zhang, Zhao Zhang
Date: 2023-08-30 00:00:00
Tasks: Representation Learning

The growing popularity of subscription services in video game consumption has emphasized the importance of offering diversified recommendations. Providing users with a diverse range of games is essential for ensuring continued engagement and fostering long-term subscriptions. However, existing recommendation models face challenges in effectively handling highly imbalanced implicit feedback in gaming interactions. Additionally, they struggle to take into account the distinctive characteristics of multiple categories and the latent user interests associated with these categories. In response to these challenges, we propose a novel framework, named DRGame, to obtain diversified recommendation. It is centered on multi-category video games, consisting of two {components}: Balance-driven Implicit Preferences Learning for data pre-processing and Clustering-based Diversified Recommendation {Module} for final prediction. The first module aims to achieve a balanced representation of implicit feedback in game time, thereby discovering a comprehensive view of player interests across different categories. The second module adopts category-aware representation learning to cluster and select players and games based on balanced implicit preferences, and then employs asymmetric neighbor aggregation to achieve diversified recommendations. Experimental results on a real-world dataset demonstrate the superiority of our proposed method over existing approaches in terms of game diversity recommendations.
A Survey on Multi-Behavior Sequential Recommendation
Authors: Xiaoqing Chen, Zhitao Li, Weike Pan, Zhong Ming
Date: 2023-08-30 00:00:00
Tasks: Information Retrieval, Recommendation Systems, Retrieval, Sequential Recommendation, Survey

Recommender systems is set up to address the issue of information overload in traditional information retrieval systems, which is focused on recommending information that is of most interest to users from massive information. Generally, there is a sequential nature and heterogeneity to the behavior of a person interacting with a system, leading to the proposal of multi-behavior sequential recommendation (MBSR). MBSR is a relatively new and worthy direction for in-depth research, which can achieve state-of-the-art recommendation through suitable modeling, and some related works have been proposed. This survey aims to shed light on the MBSR problem. Firstly, we introduce MBSR in detail, including its problem definition, application scenarios and challenges faced. Secondly, we detail the classification of MBSR, including neighborhood-based methods, matrix factorization-based methods and deep learning-based methods, where we further classify the deep learning-based methods into different learning architectures based on RNN, GNN, Transformer, and generic architectures as well as architectures that integrate hybrid techniques. In each method, we present related works based on the data perspective and the modeling perspective, as well as analyze the strengths, weaknesses and features of these works. Finally, we discuss some promising future research directions to address the challenges and improve the current status of MBSR.
Chunked Lists versus Extensible Arrays for Text Inversion
Authors: David Hawking, Bodo Billerbeck
Date: 2023-08-29 00:00:00
Tasks: Chunking

In our 2017 work on in-memory list-based text inversion [Hawking and Billerbeck. Efficient In-Memory, List-Based Text Inversion. ADCS 2017] we compared memory use and indexing speed of a considerable number of variants of chunked linked lists. In the present work we compare the best performing of those variants (FBB - dynamic Fibonacci chunking) with the extensible SQ array technique (SQA) presented in [Moffat and Mackenzie. Immediate-Access Indexing Using Space-Efficient Extensible Arrays. ADCS 2023].
Risk-aware Flexible Resource Utilization in an Unbalanced Three-Phase Distribution Network using SDP-based Distributionally Robust Optimal Power Flow
Authors: Zelong Lu, Jianxue Wang, Mohammad Shahidehpour, Linquan Bai, Zuyi Li, Lei Yan, Xianlong Chen
Date: 2023-08-30 00:00:00

The variability caused by the proliferation of distributed energy resources (DERs) and the significant growth in unbalanced three-phase loads pose unprecedented challenges to distribution network operations. This paper focuses on how a distribution system operator (DSO), taking over the distribution grid and market operations, would develop a risk-aware flexibility market to mitigate uncertainties in an unbalanced three-phase power distribution network. First, a distributionally robust chance constraint (DRCC) method is devised to solve the unbalanced three-phase optimal power flow using a semidefinite programming (SDP) model. The DSO can apply the proposed solution to jointly clear energy and flexibility markets. Then, the DRCC model accuracy is improved by an information-sharing mechanism characterized by spatially-correlated uncertainties in the distribution grid. Further, a novel system-wide response function is derived to make the DRCC model tractable. Using the duality theory, the paper further investigates the physical composition of the DSO's cleared flexibility prices to guide the unbalanced distribution network operation. Finally, the effectiveness of the risk-aware flexibility market is verified in a modified three-phase IEEE 34-node system. Results demonstrate that the flexibility market can quantify the impact of spatially correlated uncertainties and facilitate the utilization of flexible resources to mitigate uncertainties across the network.
Protecting World Leader Using Facial Speaking Pattern Against Deepfakes
Authors: Beilin Chu, Weike You, Zhen Yang, Linna Zhou, Renying Wang
Date: 2022-09-09 00:00:00
Proceeding: IEEE Signal Processing Letters 2022 9
Tasks: DeepFake Detection

Face forgery instances involving celebrities are on the rise, owing to the ease with which their large quantity of videos may be accessible on the Internet, world leaders particularly. While current face manipulation detectors have achieved impressive results on several open datasets, which incorporate persons with various identities, they show performance degradation on these high-quality ones targeting at celebrities. What is more, these online videos usually undergo compression processing, marking the detection task harder. Besides, more face manipulation techniques arise for celebrities other than face-swap, such as lip-synchronize and image-animation, with which most works have not been concerned. This paper proposes a dual stream learning facial and speaking patterns method to protect celebrities against deepfakes. We design an action unit module based on facial action coding system along with an Action Unit Transformer (AUT) to exploit facial expressions embeddings. Besides, our method's dual stream architecture utilizes a Temporal Convolutional Network (TCN) to extract lip motion pattern and learns the relatedness between facial and speaking patterns. Our method could protect the person of interest (POI) against deepfakes in an end-to-end manner. Extensive experiments show that our method achieves better performance and has a higher resistance to video compression than state-of-the-art detection models.