pwc-archive/papers-with-abstracts huggingface dataset:
Revolutionizing Genomics with Reinforcement Learning Techniques
Authors: M. Karami, K. Jahanian, R. Alizadehsani, A. Argha, I. Dehzangi, J. M. Gorriz, Y. Zhang, F. Hajati, M. Yang, H. Alinejad-Rokny
Date: 2023-02-26 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Survey

In recent years, Reinforcement Learning (RL) has emerged as a powerful tool for solving a wide range of problems, including decision-making and genomics. The exponential growth of raw genomic data over the past two decades has exceeded the capacity of manual analysis, leading to a growing interest in automatic data analysis and processing. RL algorithms are capable of learning from experience with minimal human supervision, making them well-suited for genomic data analysis and interpretation. One of the key benefits of using RL is the reduced cost associated with collecting labeled training data, which is required for supervised learning. While there have been numerous studies examining the applications of Machine Learning (ML) in genomics, this survey focuses exclusively on the use of RL in various genomics research fields, including gene regulatory networks (GRNs), genome assembly, and sequence alignment. We present a comprehensive technical overview of existing studies on the application of RL in genomics, highlighting the strengths and limitations of these approaches. We then discuss potential research directions that are worthy of future exploration, including the development of more sophisticated reward functions as RL heavily depends on the accuracy of the reward function, the integration of RL with other machine learning techniques, and the application of RL to new and emerging areas in genomics research. Finally, we present our findings and conclude by summarizing the current state of the field and the future outlook for RL in genomics.
Neuroadaptive Distributed Event-triggered Control of Networked Uncertain Pure-feedback Systems with Polluted Feedback
Authors: Libei Sun, Zhirong Zhang, Xinjian Huang, Xiucai Huang
Date: 2023-02-27 00:00:00
Tasks: Fault Detection

This paper investigates the distributed event-triggered control problem for a class of uncertain pure-feedback nonlinear multi-agent systems (MASs) with polluted feedback. Under the setting of event-triggered control, substantial challenges exist in both control design and stability analysis for systems in more general non-affine pure-feedback forms wherein all state variables are not directly and continuously available or even polluted due to sensor failures, and thus far very limited results are available in literature. In this work, a nominal control strategy under regular state feedback is firstly developed by combining neural network (NN) approximating with dynamic filtering technique, and then a NN-based distributed event-triggered control strategy is proposed by resorting to a novel replacement policy, making the non-differentiability issue arising from event-triggering setting completely circumvented. Besides, the sensor ineffectiveness is accommodated automatically without using fault detection and diagnosis unit or controller reconfiguration. It is shown that all the internal signals are semi-globally uniformly ultimately bounded (SGUUB) with the aid of several vital lemmas, while the outputs of all the subsystems reaching a consensus without infinitely fast execution. Finally, the efficiency of the developed algorithm are verified via numerical simulation.
Equilibrium Bandits: Learning Optimal Equilibria of Unknown Dynamics
Authors: Siddharth Chandak, Ilai Bistritz, Nicholas Bambos
Date: 2023-02-27 00:00:00

Consider a decision-maker that can pick one out of $K$ actions to control an unknown system, for $T$ turns. The actions are interpreted as different configurations or policies. Holding the same action fixed, the system asymptotically converges to a unique equilibrium, as a function of this action. The dynamics of the system are unknown to the decision-maker, which can only observe a noisy reward at the end of every turn. The decision-maker wants to maximize its accumulated reward over the $T$ turns. Learning what equilibria are better results in higher rewards, but waiting for the system to converge to equilibrium costs valuable time. Existing bandit algorithms, either stochastic or adversarial, achieve linear (trivial) regret for this problem. We present a novel algorithm, termed Upper Equilibrium Concentration Bound (UECB), that knows to switch an action quickly if it is not worth it to wait until the equilibrium is reached. This is enabled by employing convergence bounds to determine how far the system is from equilibrium. We prove that UECB achieves a regret of $\mathcal{O}(\log(T)+\tau_c\log(\tau_c)+\tau_c\log\log(T))$ for this equilibrium bandit problem where $\tau_c$ is the worst case approximate convergence time to equilibrium. We then show that both epidemic control and game control are special cases of equilibrium bandits, where $\tau_c\log \tau_c$ typically dominates the regret. We then test UECB numerically for both of these applications.
Can we avoid Double Descent in Deep Neural Networks?
Authors: Victor Quétu, Enzo Tartaglione
Date: 2023-02-26 00:00:00

Finding the optimal size of deep learning models is very actual and of broad impact, especially in energy-saving schemes. Very recently, an unexpected phenomenon, the ``double descent'', has caught the attention of the deep learning community. As the model's size grows, the performance gets first worse, and then goes back to improving. It raises serious questions about the optimal model's size to maintain high generalization: the model needs to be sufficiently over-parametrized, but adding too many parameters wastes training resources. Is it possible to find, in an efficient way, the best trade-off? Our work shows that the double descent phenomenon is potentially avoidable with proper conditioning of the learning problem, but a final answer is yet to be found. We empirically observe that there is hope to dodge the double descent in complex scenarios with proper regularization, as a simple $\ell_2$ regularization is already positively contributing to such a perspective.
Hierarchical Needs-driven Agent Learning Systems: From Deep Reinforcement Learning To Diverse Strategies
Authors: Qin Yang
Date: 2023-02-25 00:00:00
Tasks: Deep Reinforcement Learning

The needs describe the necessities for a system to survive and evolve, which arouses an agent to action toward a goal, giving purpose and direction to behavior. Based on Maslow hierarchy of needs, an agent needs to satisfy a certain amount of needs at the current level as a condition to arise at the next stage -- upgrade and evolution. Especially, Deep Reinforcement Learning (DAL) can help AI agents (like robots) organize and optimize their behaviors and strategies to develop diverse Strategies based on their current state and needs (expected utilities or rewards). This paper introduces the new hierarchical needs-driven Learning systems based on DAL and investigates the implementation in the single-robot with a novel approach termed Bayesian Soft Actor-Critic (BSAC). Then, we extend this topic to the Multi-Agent systems (MAS), discussing the potential research fields and directions.
Bayesian Networks for Named Entity Prediction in Programming Community Question Answering
Authors: Alexey Gorbatovski, Sergey Kovalchuk
Date: 2023-02-26 00:00:00
Tasks: Community Question Answering, Question Answering

Within this study, we propose a new approach for natural language processing using Bayesian networks to predict and analyze the context and how this approach can be applied to the Community Question Answering domain. We discuss how Bayesian networks can detect semantic relationships and dependencies between entities, and this is connected to different score-based approaches of structure-learning. We compared the Bayesian networks with different score metrics, such as the BIC, BDeu, K2 and Chow-Liu trees. Our proposed approach out-performs the baseline model at the precision metric. We also discuss the influence of penalty terms on the structure of Bayesian networks and how they can be used to analyze the relationships between entities. In addition, we examine the visualization of directed acyclic graphs to analyze semantic relationships. The article further identifies issues with detecting certain semantic classes that are separated in the structure of directed acyclic graphs. Finally, we evaluate potential improvements for the Bayesian network approach.
Q-Cogni: An Integrated Causal Reinforcement Learning Framework
Authors: Cris Cunha, Wei Liu, Tim French, Ajmal Mian
Date: 2023-02-26 00:00:00
Tasks: Causal Inference, Decision Making, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present Q-Cogni, an algorithmically integrated causal reinforcement learning framework that redesigns Q-Learning with an autonomous causal structure discovery method to improve the learning process with causal inference. Q-Cogni achieves optimal learning with a pre-learned structural causal model of the environment that can be queried during the learning process to infer cause-and-effect relationships embedded in a state-action space. We leverage on the sample efficient techniques of reinforcement learning, enable reasoning about a broader set of policies and bring higher degrees of interpretability to decisions made by the reinforcement learning agent. We apply Q-Cogni on the Vehicle Routing Problem (VRP) and compare against state-of-the-art reinforcement learning algorithms. We report results that demonstrate better policies, improved learning efficiency and superior interpretability of the agent's decision making. We also compare this approach with traditional shortest-path search algorithms and demonstrate the benefits of our causal reinforcement learning framework to high dimensional problems. Finally, we apply Q-Cogni to derive optimal routing decisions for taxis in New York City using the Taxi & Limousine Commission trip record data and compare with shortest-path search, reporting results that show 85% of the cases with an equal or better policy derived from Q-Cogni in a real-world domain.
Towards Tackling MaxSAT by Combining Nested Monte Carlo with Local Search
Authors: Hui Wang, Abdallah Saffidine, Tristan Cazenave
Date: 2023-02-26 00:00:00

Recent work proposed the UCTMAXSAT algorithm to address Maximum Satisfiability Problems (MaxSAT) and shown improved performance over pure Stochastic Local Search algorithms (SLS). UCTMAXSAT is based on Monte Carlo Tree Search but it uses SLS instead of purely random playouts. In this work, we introduce two algorithmic variations over UCTMAXSAT. We carry an empirical analysis on MaxSAT benchmarks from recent competitions and establish that both ideas lead to performance improvements. First, a nesting of the tree search inspired by the Nested Monte Carlo Search algorithm is effective on most instance types in the benchmark. Second, we observe that using a static flip limit in SLS, the ideal budget depends heavily on the instance size and we propose to set it dynamically. We show that it is a robust way to achieve comparable performance on a variety of instances without requiring additional tuning.
Beyond Discrete Selection: Continuous Embedding Space Optimization for Generative Feature Selection
Authors: Meng Xiao, Dongjie Wang, Min Wu, Pengfei Wang, Yuanchun Zhou, Yanjie Fu
Date: 2023-02-26 00:00:00
Tasks: Decoder, feature selection

The goal of Feature Selection - comprising filter, wrapper, and embedded approaches - is to find the optimal feature subset for designated downstream tasks. Nevertheless, current feature selection methods are limited by: 1) the selection criteria of these methods are varied for different domains, making them hard to generalize; 2) the selection performance of these approaches drops significantly when processing high-dimensional feature space coupled with small sample size. In light of these challenges, we pose the question: can selected feature subsets be more robust, accurate, and input dimensionality agnostic? In this paper, we reformulate the feature selection problem as a deep differentiable optimization task and propose a new research perspective: conceptualizing discrete feature subsetting as continuous embedding space optimization. We introduce a novel and principled framework that encompasses a sequential encoder, an accuracy evaluator, a sequential decoder, and a gradient ascent optimizer. This comprehensive framework includes four important steps: preparation of features-accuracy training data, deep feature subset embedding, gradient-optimized search, and feature subset reconstruction. Specifically, we utilize reinforcement feature selection learning to generate diverse and high-quality training data and enhance generalization. By optimizing reconstruction and accuracy losses, we embed feature selection knowledge into a continuous space using an encoder-evaluator-decoder model structure. We employ a gradient ascent search algorithm to find better embeddings in the learned embedding space. Furthermore, we reconstruct feature selection solutions using these embeddings and select the feature subset with the highest performance for downstream tasks as the optimal subset.
DeepCPG Policies for Robot Locomotion
Authors: Aditya M. Deshpande, Eric Hurd, Ali A. Minai, Manish Kumar
Date: 2023-02-25 00:00:00
Tasks: Deep Reinforcement Learning

Central Pattern Generators (CPGs) form the neural basis of the observed rhythmic behaviors for locomotion in legged animals. The CPG dynamics organized into networks allow the emergence of complex locomotor behaviors. In this work, we take this inspiration for developing walking behaviors in multi-legged robots. We present novel DeepCPG policies that embed CPGs as a layer in a larger neural network and facilitate end-to-end learning of locomotion behaviors in deep reinforcement learning (DRL) setup. We demonstrate the effectiveness of this approach on physics engine-based insectoid robots. We show that, compared to traditional approaches, DeepCPG policies allow sample-efficient end-to-end learning of effective locomotion strategies even in the case of high-dimensional sensor spaces (vision). We scale the DeepCPG policies using a modular robot configuration and multi-agent DRL. Our results suggest that gradual complexification with embedded priors of these policies in a modular fashion could achieve non-trivial sensor and motor integration on a robot platform. These results also indicate the efficacy of bootstrapping more complex intelligent systems from simpler ones based on biological principles. Finally, we present the experimental results for a proof-of-concept insectoid robot system for which DeepCPG learned policies initially using the simulation engine and these were afterwards transferred to real-world robots without any additional fine-tuning.
Vagueness in Predicates and Objects
Authors: Brandon Bennett, Lucía Gómez Álvarez
Date: 2023-02-25 00:00:00
Tasks: Diversity

Classical semantics assumes that one can model reference, predication and quantification with respect to a fixed domain of precise referent objects. Non-logical terms and quantification are then interpreted directly in terms of elements and subsets of this domain. We explore ways to generalise this classical picture of precise predicates and objects to account for variability of meaning due to factors such as vagueness, context and diversity of definitions or opinions. Both names and predicative expressions can be given either multiple semantic referents or be associated with semantic referents that incorporate some model of variability. We present a semantic framework, Variable Reference Semantics, that can accommodate several modes of variability in relation to both predicates and objects.
Tractable Diversity: Scalable Multiperspective Ontology Management via Standpoint EL
Authors: Lucía Gómez Álvarez, Sebastian Rudolph, Hannes Strass
Date: 2023-02-25 00:00:00
Tasks: Diversity, Management

The tractability of the lightweight description logic EL has allowed for the construction of large and widely used ontologies that support semantic interoperability. However, comprehensive domains with a broad user base are often at odds with strong axiomatisations otherwise useful for inferencing, since these are usually context-dependent and subject to diverging perspectives. In this paper we introduce Standpoint EL, a multi-modal extension of EL that allows for the integrated representation of domain knowledge relative to diverse, possibly conflicting standpoints (or contexts), which can be hierarchically organised and put in relation to each other. We establish that Standpoint EL still exhibits EL's favourable PTime standard reasoning, whereas introducing additional features like empty standpoints, rigid roles, and nominals makes standard reasoning tasks intractable.
Construction numbers: How to build a graph?
Authors: Paul C. Kainen
Date: 2023-02-25 00:00:00

A construction sequence for a graph is a listing of the elements of the graph (the set of vertices and edges) such that each edge follows both its endpoints. The construction number of the graph is the number of such sequences. We determine this number for various graph families.
Contrastive Video Question Answering via Video Graph Transformer
Authors: Junbin Xiao, Pan Zhou, Angela Yao, Yicong Li, Richang Hong, Shuicheng Yan, Tat-Seng Chua
Date: 2023-02-27 00:00:00
Tasks: Contrastive Learning, Question Answering, Video Question Answering

We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning. 2) It designs separate video and text transformers for contrastive learning between the video and text to perform QA, instead of multi-modal transformer for answer classification. Fine-grained video-text communication is done by additional cross-modal interaction modules. 3) It is optimized by the joint fully- and self-supervised contrastive objectives between the correct and incorrect answers, as well as the relevant and irrelevant questions respectively. With superior video encoding and QA solution, we show that CoVGT can achieve much better performances than previous arts on video reasoning tasks. Its performances even surpass those models that are pretrained with millions of external data. We further show that CoVGT can also benefit from cross-modal pretraining, yet with orders of magnitude smaller data. The results demonstrate the effectiveness and superiority of CoVGT, and additionally reveal its potential for more data-efficient pretraining. We hope our success can advance VideoQA beyond coarse recognition/description towards fine-grained relation reasoning of video contents. Our code is available at https://github.com/doc-doc/CoVGT.
Random projection tree similarity metric for SpectralNet
Authors: Mashaan Alshammari, John Stavrakakis, Adel F. Ahmed, Masahiro Takatsuka
Date: 2023-02-25 00:00:00
Tasks: Clustering, Graph Clustering, Graph Embedding, graph partitioning, Spectral Graph Clustering

SpectralNet is a graph clustering method that uses neural network to find an embedding that separates the data. So far it was only used with $k$-nn graphs, which are usually constructed using a distance metric (e.g., Euclidean distance). $k$-nn graphs restrict the points to have a fixed number of neighbors regardless of the local statistics around them. We proposed a new SpectralNet similarity metric based on random projection trees (rpTrees). Our experiments revealed that SpectralNet produces better clustering accuracy using rpTree similarity metric compared to $k$-nn graph with a distance metric. Also, we found out that rpTree parameters do not affect the clustering accuracy. These parameters include the leaf size and the selection of projection direction. It is computationally efficient to keep the leaf size in order of $\log(n)$, and project the points onto a random direction instead of trying to find the direction with the maximum dispersion.
A parameter-free graph reduction for spectral clustering and SpectralNet
Authors: Mashaan Alshammari, John Stavrakakis, Masahiro Takatsuka
Date: 2023-02-25 00:00:00
Tasks: Clustering, Graph Clustering, Graph Embedding, graph partitioning, Spectral Graph Clustering

Graph-based clustering methods like spectral clustering and SpectralNet are very efficient in detecting clusters of non-convex shapes. Unlike the popular $k$-means, graph-based clustering methods do not assume that each cluster has a single mean. However, these methods need a graph where vertices in the same cluster are connected by edges of large weights. To achieve this goal, many studies have proposed graph reduction methods with parameters. Unfortunately, these parameters have to be tuned for every dataset. We introduce a graph reduction method that does not require any parameters. First, the distances from every point $p$ to its neighbors are filtered using an adaptive threshold to only keep neighbors with similar surrounding density. Second, the similarities with close neighbors are computed and only high similarities are kept. The edges that survive these two filtering steps form the constructed graph that was passed to spectral clustering and SpectralNet. The experiments showed that our method provides a stable alternative, where other methods performance fluctuated according to the setting of their parameters.
Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
Authors: Tibor Krols, Yana Nikolova, Ninell Oldenburg
Date: 2023-02-26 00:00:00
Tasks: Emotion Recognition, Music Emotion Recognition, regression

This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify API, combined lyrics features including sentiment, TF-IDF, and Anew to predict valence and arousal (Russell, 1980) scores on the Deezer Mood Detection Dataset (DMDD) (Delbouys et al., 2018) with 4 different regression models. We find that out of the 11 high-level song features, mainly 5 contribute to the performance, multi-modal features do better than audio alone when predicting valence. We made our code publically available.
STACC: Code Comment Classification using SentenceTransformers
Authors: Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen
Date: 2023-02-25 00:00:00
Tasks: Classification

Code comments are a key resource for information about software artefacts. Depending on the use case, only some types of comments are useful. Thus, automatic approaches to classify these comments have been proposed. In this work, we address this need by proposing, STACC, a set of SentenceTransformers-based binary classifiers. These lightweight classifiers are trained and tested on the NLBSE Code Comment Classification tool competition dataset, and surpass the baseline by a significant margin, achieving an average F1 score of 0.74 against the baseline of 0.31, which is an improvement of 139%. A replication package, as well as the models themselves, are publicly available.
Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradient
Authors: Xiangyuan Zhang, Tamer Başar
Date: 2023-02-25 00:00:00

We revisit in this paper the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications. We provide a fine-grained sample complexity analysis for RHPG to learn a control policy that is both stabilizing and $\epsilon$-close to the optimal LQR solution, and our algorithm does not require knowing a stabilizing control policy for initialization. Combined with the recent application of RHPG in learning the Kalman filter, we demonstrate the general applicability of RHPG in linear control and estimation with streamlined analyses.