pwc-archive/papers-with-abstracts huggingface dataset:
Generalizing Hate Speech Detection Using Multi-Task Learning: A Case Study of Political Public Figures
Authors: Lanqin Yuan, Marian-Andrei Rizoiu
Date: 2022-08-22 00:00:00
Tasks: Hate Speech Detection, Multi-Task Learning

Automatic identification of hateful and abusive content is vital in combating the spread of harmful online content and its damaging effects. Most existing works evaluate models by examining the generalization error on train-test splits on hate speech datasets. These datasets often differ in their definitions and labeling criteria, leading to poor generalization performance when predicting across new domains and datasets. This work proposes a new Multi-task Learning (MTL) pipeline that trains simultaneously across multiple hate speech datasets to construct a more encompassing classification model. Using a dataset-level leave-one-out evaluation (designating a dataset for testing and jointly training on all others), we trial the MTL detection on new, previously unseen datasets. Our results consistently outperform a large sample of existing work. We show strong results when examining the generalization error in train-test splits and substantial improvements when predicting on previously unseen datasets. Furthermore, we assemble a novel dataset, dubbed PubFigs, focusing on the problematic speech of American Public Political Figures. We crowdsource-label using Amazon MTurk more than $20,000$ tweets and machine-label problematic speech in all the $305,235$ tweets in PubFigs. We find that the abusive and hate tweeting mainly originates from right-leaning figures and relates to six topics, including Islam, women, ethnicity, and immigrants. We show that MTL builds embeddings that can simultaneously separate abusive from hate speech, and identify its topics.
Enhanced Opposition Differential Evolution Algorithm for Multimodal Optimization
Authors: Shatendra Singh, Aruna Tiwari
Date: 2022-08-23 00:00:00
Tasks: Evolutionary Algorithms

Most of the real-world problems are multimodal in nature that consists of multiple optimum values. Multimodal optimization is defined as the process of finding multiple global and local optima (as opposed to a single solution) of a function. It enables a user to switch between different solutions as per the need while still maintaining the optimal system performance. Classical gradient-based methods fail for optimization problems in which the objective functions are either discontinuous or non-differentiable. Evolutionary Algorithms (EAs) are able to find multiple solutions within a population in a single algorithmic run as compared to classical optimization techniques that need multiple restarts and multiple runs to find different solutions. Hence, several EAs have been proposed to solve such kinds of problems. However, Differential Evolution (DE) algorithm is a population-based heuristic method that can solve such optimization problems, and it is simple to implement. The potential challenge in Multi-Modal Optimization Problems (MMOPs) is to search the function space efficiently to locate most of the peaks accurately. The optimization problem could be to minimize or maximize a given objective function and we aim to solve the maximization problems on multimodal functions in this study. Hence, we have proposed an algorithm known as Enhanced Opposition Differential Evolution (EODE) algorithm to solve the MMOPs. The proposed algorithm has been tested on IEEE Congress on Evolutionary Computation (CEC) 2013 benchmark functions, and it achieves competitive results compared to the existing state-of-the-art approaches.
A multiplicity-preserving crossover operator on graphs. Extended version
Authors: Henri Thölke, Jens Kosiol
Date: 2022-08-23 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms usually explore a search space of solutions by means of crossover and mutation. While a mutation consists of a small, local modification of a solution, crossover mixes the genetic information of two solutions to compute a new one. For model-driven optimization (MDO), where models directly serve as possible solutions (instead of first transforming them into another representation), only recently a generic crossover operator has been developed. Using graphs as a formal foundation for models, we further refine this operator in such a way that additional well-formedness constraints are preserved: We prove that, given two models that satisfy a given set of multiplicity constraints as input, our refined crossover operator computes two new models as output that also satisfy the set of constraints.
Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues
Authors: Nan Li, Lianbo Ma, Guo Yu, Bing Xue, Mengjie Zhang, Yaochu Jin
Date: 2022-08-23 00:00:00
Tasks: AutoML, Feature Engineering, Heuristic Search

Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This paper aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we firstly illuminate EDL from machine learning and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from feature engineering, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.
Saliency-Guided Local Full-Reference Image Quality Assessment
Authors: Domonkos Varga
Date: 2022-06-29 00:00:00
Proceeding: Signals 2022 6
Tasks: Full reference image quality assessment, Full-Reference Image Quality Assessment, Image Quality Assessment

Research and development of image quality assessment (IQA) algorithms have been in the focus of the computer vision and image processing community for decades. The intent of IQA methods is to estimate the perceptual quality of digital images correlating as high as possible with human judgements. Full-reference image quality assessment algorithms, which have full access to the distortion-free images, usually contain two phases: local image quality estimation and pooling. Previous works have utilized visual saliency in the final pooling stage. In addition to this, visual saliency was utilized as weights in the weighted averaging of local image quality scores, emphasizing image regions that are salient to human observers. In contrast to this common practice, visual saliency is applied in the computation of local image quality in this study, based on the observation that local image quality is determined both by local image degradation and visual saliency simultaneously. Experimental results on KADID-10k, TID2013, TID2008, and CSIQ have shown that the proposed method was able to improve the state-of-the-art’s performance at low computational costs.
Limits of Entrainment of Circadian Neuronal Networks
Authors: Yorgos M. Psarellis, Michail Kavousanakis, Michael A. Henson, Ioannis G. Kevrekidis
Date: 2022-08-23 00:00:00
Tasks: Rhythm

Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals, such as sleep, metabolism, homeostasis, mood changes and more. It has been shown that this rhythm arises from self-sustained biomolecular oscillations of a neuronal network located in the Suprachiasmatic Nucleus (SCN). Under normal circumstances, this network remains synchronized to the day-night cycle due to signaling from the retina. Misalignment of these neuronal oscillations with the external light signal can disrupt numerous physiological functions and take a long-lasting toll on health and well-being. In this work, we study a modern computational neuroscience model to determine the limits of circadian synchronization to external light signals of different frequency and duty cycle. We employ a matrix-free approach to locate periodic steady states of the high-dimensional model for various driving conditions. Our algorithmic pipeline enables numerical continuation and construction of bifurcation diagrams w.r.t. forcing parameters. We computationally explore the effect of heterogeneity in the circadian neuronal network, as well as the effect of corrective therapeutic interventions, such as that of the drug molecule Longdaysin. Lastly, we employ unsupervised learning to construct a data-driven embedding space for representing neuronal heterogeneity.
Convergence bounds for local least squares approximation
Authors: Philipp Trunschke
Date: 2022-08-23 00:00:00
Tasks: Tensor Networks

We consider the problem of approximating a function in a general nonlinear subset of $L^2$, when only a weighted Monte Carlo estimate of the $L^2$-norm can be computed. Of particular interest in this setting is the concept of sample complexity, the number of sample points that are necessary to achieve a prescribed error with high probability. Reasonable worst-case bounds for this quantity exist only for particular model classes, like linear spaces or sets of sparse vectors. For more general sets, like tensor networks or neural networks, the currently existing bounds are very pessimistic. By restricting the model class to a neighbourhood of the best approximation, we can derive improved worst-case bounds for the sample complexity. When the considered neighbourhood is a manifold with positive local reach, its sample complexity can be estimated by means of the sample complexities of the tangent and normal spaces and the manifold's curvature.
A flexible empirical Bayes approach to multiple linear regression and connections with penalized regression
Authors: Youngseok Kim, Wei Wang, Peter Carbonetto, Matthew Stephens
Date: 2022-08-23 00:00:00
Tasks: regression

We introduce a new empirical Bayes approach for large-scale multiple linear regression. Our approach combines two key ideas: (i) the use of flexible "adaptive shrinkage" priors, which approximate the nonparametric family of scale mixture of normal distributions by a finite mixture of normal distributions; and (ii) the use of variational approximations to efficiently estimate prior hyperparameters and compute approximate posteriors. Combining these two ideas results in fast and flexible methods, with computational speed comparable to fast penalized regression methods such as the Lasso, and with competitive prediction accuracy across a wide range of scenarios. Further, we provide new results that establish conceptual connections between our empirical Bayes methods and penalized methods. Specifically, we show that the posterior mean from our method solves a penalized regression problem, with the form of the penalty function being learned from the data by directly solving an optimization problem (rather than being tuned by cross-validation). Our methods are implemented in an R package, mr.ash.alpha, available from https://github.com/stephenslab/mr.ash.alpha.
pystacked: Stacking generalization and machine learning in Stata
Authors: Achim Ahrens, Christian B. Hansen, Mark E. Schaffer
Date: 2022-08-23 00:00:00
Tasks: Binary Classification, regression

pystacked implements stacked generalization (Wolpert, 1992) for regression and binary classification via Python's scikit-learn. Stacking combines multiple supervised machine learners -- the "base" or "level-0" learners -- into a single learner. The currently supported base learners include regularized regression, random forest, gradient boosted trees, support vector machines, and feed-forward neural nets (multi-layer perceptron). pystacked can also be used with as a `regular' machine learning program to fit a single base learner and, thus, provides an easy-to-use API for scikit-learn's machine learning algorithms.
Deriving time-averaged active inference from control principles
Authors: Eli Sennesh, Jordan Theriault, Jan-Willem van de Meent, Lisa Feldman Barrett, Karen Quigley
Date: 2022-08-22 00:00:00

Active inference offers a principled account of behavior as minimizing average sensory surprise over time. Applications of active inference to control problems have heretofore tended to focus on finite-horizon or discounted-surprise problems, despite deriving from the infinite-horizon, average-surprise imperative of the free-energy principle. Here we derive an infinite-horizon, average-surprise formulation of active inference from optimal control principles. Our formulation returns to the roots of active inference in neuroanatomy and neurophysiology, formally reconnecting active inference to optimal feedback control. Our formulation provides a unified objective functional for sensorimotor control and allows for reference states to vary over time.
Graph Embeddings via Tensor Products and Approximately Orthonormal Codes
Authors: Frank Qiu
Date: 2022-08-18 00:00:00

We propose a dynamic graph representation method, showcasing its rich representational capacity and establishing some of its theoretical properties. Our representation falls under the bind-and-sum approach in hyperdimensional computing (HDC), and we show that the tensor product is the most general binding operation that respects the superposition principle employed in HDC. We also establish some precise results characterizing the behavior of our method, including a memory vs. size analysis of how our representation's size must scale with the number of edges in order to retain accurate graph operations. True to its HDC roots, we also compare our graph representation to another typical HDC representation, the Hadamard-Rademacher scheme, showing that these two graph representations have the same memory-capacity scaling. We establish a link to adjacency matrices, showing that our method is a pseudo-orthogonal generalization of adjacency matrices. In light of this, we briefly discuss its applications toward a dynamic compressed representation of large sparse graphs.
VILT: Video Instructions Linking for Complex Tasks
Authors: Sophie Fischer, Carlos Gemmell, Iain Mackie, Jeffrey Dalton
Date: 2022-08-23 00:00:00
Tasks: Retrieval

This work addresses challenges in developing conversational assistants that support rich multimodal video interactions to accomplish real-world tasks interactively. We introduce the task of automatically linking instructional videos to task steps as "Video Instructions Linking for Complex Tasks" (VILT). Specifically, we focus on the domain of cooking and empowering users to cook meals interactively with a video-enabled Alexa skill. We create a reusable benchmark with 61 queries from recipe tasks and curate a collection of 2,133 instructional "How-To" cooking videos. Studying VILT with state-of-the-art retrieval methods, we find that dense retrieval with ANCE is the most effective, achieving an NDCG@3 of 0.566 and P@1 of 0.644. We also conduct a user study that measures the effect of incorporating videos in a real-world task setting, where 10 participants perform several cooking tasks with varying multimodal experimental conditions using a state-of-the-art Alexa TaskBot system. The users interacting with manually linked videos said they learned something new 64% of the time, which is a 9% increase compared to the automatically linked videos (55%), indicating that linked video relevance is important for task learning.
Enhancing User Behavior Sequence Modeling by Generative Tasks for Session Search
Authors: Haonan Chen, Zhicheng Dou, Yutao Zhu, Zhao Cao, Xiaohua Cheng, Ji-Rong Wen
Date: 2022-08-23 00:00:00
Tasks: Decoder, Document Ranking, Session Search

Users' search tasks have become increasingly complicated, requiring multiple queries and interactions with the results. Recent studies have demonstrated that modeling the historical user behaviors in a session can help understand the current search intent. Existing context-aware ranking models primarily encode the current session sequence (from the first behavior to the current query) and compute the ranking score using the high-level representations. However, there is usually some noise in the current session sequence (useless behaviors for inferring the search intent) that may affect the quality of the encoded representations. To help the encoding of the current user behavior sequence, we propose to use a decoder and the information of future sequences and a supplemental query. Specifically, we design three generative tasks that can help the encoder to infer the actual search intent: (1) predicting future queries, (2) predicting future clicked documents, and (3) predicting a supplemental query. We jointly learn the ranking task with these generative tasks using an encoder-decoder structured approach. Extensive experiments on two public search logs demonstrate that our model outperforms all existing baselines, and the designed generative tasks can actually help the ranking task. Besides, additional experiments also show that our approach can be easily applied to various Transformer-based encoder-decoder models and improve their performance.
Towards Communication Efficient and Fair Federated Personalized Sequential Recommendation
Authors: Sichun Luo, Yuanzhang Xiao, Yang Liu, Congduan Li, Linqi Song
Date: 2022-08-23 00:00:00
Tasks: Fairness, Federated Learning, Privacy Preserving, Recommendation Systems, Sequential Recommendation

Federated recommendations leverage the federated learning (FL) techniques to make privacy-preserving recommendations. Though recent success in the federated recommender system, several vital challenges remain to be addressed: (i) The majority of federated recommendation models only consider the model performance and the privacy-preserving ability, while ignoring the optimization of the communication process; (ii) Most of the federated recommenders are designed for heterogeneous systems, causing unfairness problems during the federation process; (iii) The personalization techniques have been less explored in many federated recommender systems. In this paper, we propose a Communication efficient and Fair personalized Federated personalized Sequential Recommendation algorithm (CF-FedSR) to tackle these challenges. CF-FedSR introduces a communication-efficient scheme that employs adaptive client selection and clustering-based sampling to accelerate the training process. A fairness-aware model aggregation algorithm that can adaptively capture the data and performance imbalance among different clients to address the unfairness problems is proposed. The personalization module assists clients in making personalized recommendations and boosts the recommendation performance via local fine-tuning and model adaption. Extensive experimental results show the effectiveness and efficiency of our proposed method.
No-Reference Quality Assessment of Authentically Distorted Images Based on Local and Global Features
Authors: Domonkos Varga
Date: 2022-06-19 00:00:00
Proceeding: Journal of Imaging 2022 6
Tasks: Image Quality Assessment, No-Reference Image Quality Assessment

With the development of digital imaging techniques, image quality assessment methods are receiving more attention in the literature. Since distortion-free versions of camera images in many practical, everyday applications are not available, the need for effective no-reference image quality assessment algorithms is growing. Therefore, this paper introduces a novel no-reference image quality assessment algorithm for the objective evaluation of authentically distorted images. Specifically, we apply a broad spectrum of local and global feature vectors to characterize the variety of authentic distortions. Among the employed local features, the statistics of popular local feature descriptors, such as SURF, FAST, BRISK, or KAZE, are proposed for NR-IQA; other features are also introduced to boost the performances of local features. The proposed method was compared to 12 other state-of-the-art algorithms on popular and accepted benchmark datasets containing RGB images with authentic distortions (CLIVE, KonIQ-10k, and SPAQ). The introduced algorithm significantly outperforms the state-of-the-art in terms of correlation with human perceptual quality ratings.
GenNet framework: interpretable deep learning for predicting phenotypes from genetic data
Authors: Arno van Hilten, Seven A. Kushner, Manfred Kayser, M. Arfan Ikram, Hieab H. H. Adams, Caroline C. W. Klaver, Wiro J. Niessen, Gennady V. Roshchupkin
Date: 2021-09-17 00:00:00
Proceeding: Nature Communications Biology 2021 9
Tasks: Deep Learning, Efficient Neural Network, Genetic Risk Prediction, Medical Genetics

Applying deep learning in population genomics is challenging because of computational issues and lack of interpretable models. Here, we propose GenNet, a novel open-source deep learning framework for predicting phenotypes from genetic variants. In this framework, interpretable and memory-efficient neural network architectures are constructed by embedding biologically knowledge from public databases, resulting in neural networks that contain only biologically plausible connections. 
We applied the framework to seventeen phenotypes and found well-replicated genes such as HERC2 and OCA2 for hair and eye color, and novel genes such as ZNF773 and PCNT for schizophrenia. Additionally, the framework identified ubiquitin mediated proteolysis, endocrine system and viral infectious diseases as most predictive biological pathways for schizophrenia. 
GenNet is a freely available, end-to-end deep learning framework that allows researchers to develop and use interpretable neural networks to obtain novel insights into the genetic architecture of complex traits and diseases.
Widely-Linear MMSE Estimation of Complex-Valued Graph Signals
Authors: Alon Amar, Tirza Routtenberg
Date: 2022-08-22 00:00:00
Tasks: State Estimation

In this paper, we consider the problem of recovering random graph signals with complex values. For general Bayesian estimation of complex-valued vectors, it is known that the widely-linear minimum mean-squared-error (WLMMSE) estimator can achieve a lower mean-squared-error (MSE) than that of the linear minimum MSE (LMMSE) estimator. Inspired by the WLMMSE estimator, in this paper we develop the graph signal processing (GSP)-WLMMSE estimator, which minimizes the MSE among estimators that are represented as a two-channel output of a graph filter, i.e. widely-linear GSP estimators. We discuss the properties of the proposed GSP-WLMMSE estimator. In particular, we show that the MSE of the GSP-WLMMSE estimator is always equal to or lower than the MSE of the GSP-LMMSE estimator. The GSP-WLMMSE estimator is based on diagonal covariance matrices in the graph frequency domain, and thus has reduced complexity compared with the WLMMSE estimator. This property is especially important when using the sample-mean versions of these estimators that are based on a training dataset. We then state conditions under which the low-complexity GSP-WLMMSE estimator coincides with the WLMMSE estimator. In the simulations, we investigate two synthetic estimation problems (with linear and nonlinear models) and the problem of state estimation in power systems. For these problems, it is shown that the GSP-WLMMSE estimator outperforms the GSP-LMMSE estimator and achieves similar performance to that of the WLMMSE estimator.