pwc-archive/papers-with-abstracts huggingface dataset:
System Integrated Information
Authors: William Marshall, Matteo Grasso, William GP Mayner, Alireza Zaeemzadeh, Leonardo S Barbosa, Erick Chastain, Graham Findlay, Shuntaro Sasai, Larissa Albantakis, Giulio Tononi
Date: 2022-12-30 00:00:00

Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of consciousness (called a complex), which are then used to formulate a mathematical framework for assessing both the quality and quantity of experience. The explanatory identity proposed by IIT is that an experience is identical to the cause-effect structure unfolded from a maximally irreducible substrate (a $\Phi$-structure). In this work we introduce a definition for the integrated information of a system ($\varphi_s$) that is based on the existence, intrinsicality, information, and integration postulates of IIT. We explore how notions of determinism, degeneracy, and fault lines in the connectivity impact system integrated information. We then demonstrate how the proposed measure identifies complexes as systems whose $\varphi_s$ is greater than the $\varphi_s$ of any overlapping candidate systems.
The Markovian and Memoryless Properties of Visual System: Evidence from Serial Face Processing
Authors: Jun-Ming Yu, Haojiang Ying
Date: 2022-12-29 00:00:00

The visual system can be viewed and studied as an information processing system. If so, then the visual system should follow specific fundamental properties: either a memory or a memoryless system. Previous studies in serial dependence in vision found that the perception of the current stimulus is positively determined by the previous one. However, we are not entirely sure whether this phenomenon is a Markov processing. In this study, participants were asked to rate the social characteristics (attractiveness, trustworthiness, and dominance) of a face, either followed by the same characteristic (the one-trait condition) or another one (the two-trait condition) in randomized orders. By doing so, we can directly test the contribution of the previous input and output to the current output and thus study the properties of the system. Using Derivative of Gaussian, Markov Chain and Linear Mixed effect modeling, convergent results suggested that the serial dependence was absent and the memoryless and Markovian properties were violated in the two-trait condition when testing both attractiveness and dominance, but not in the other conditions. Thus, different facets of (presumably) the same computational task may follow asymmetrical system properties. The study also develops serial dependence as an effective technique to reveal the relationships between different computation tasks.
Biophysical Sequence Analysis of Functional Differences of Piezo1 and Piezo2
Authors: J. C. Phillips
Date: 2022-12-27 00:00:00

Because of their large size and widespread mechanosensitive interactions the only recently discovered titled transmembrane proteins have attracted much attention. Here we present and discuss their hydropathic profiles using a new method of sequence analysis. We find large-scale similarities and differences not obtainable by conventional sequence or structural studies. These differences support the evolution-towards-criticality conjecture popular among physicists.
Spatial Structure Supports Diversity in Prebiotic Autocatalytic Chemical Ecosystems
Authors: Alex M. Plum, Christopher P. Kempes, Zhen Peng, David A. Baum
Date: 2022-12-29 00:00:00
Tasks: Diversity

Autocatalysis is thought to have played an important role in the earliest stages of the origin of life. An autocatalytic cycle (AC) is a set of reactions that results in stoichiometric increase in its constituent chemicals. When the reactions of multiple interacting ACs are active in a region of space, they can have interactions analogous to those between species in biological ecosystems. Prior studies of autocatalytic chemical ecosystems (ACEs) have suggested avenues for accumulating complexity, such as ecological succession, as well as obstacles such as competitive exclusion. We extend this ecological framework to investigate the effects of surface adsorption, desorption, and diffusion on ACE ecology. Simulating ACEs as particle-based stochastic reaction-diffusion systems in spatial environments-including open, two-dimensional reaction-diffusion systems and adsorptive mineral surfaces-we demonstrate that spatial structure can enhance ACE diversity by i) permitting otherwise mutually exclusive ACs to coexist and ii) subjecting new AC traits to selection.
Topological Data Analysis of Spatial Patterning in Heterogeneous Cell Populations: Clustering and Sorting with Varying Cell-Cell Adhesion
Authors: Dhananjay Bhaskar, William Y. Zhang, Alexandria Volkening, Björn Sandstede, Ian Y. Wong
Date: 2022-12-28 00:00:00
Tasks: Classification, Dimensionality Reduction, Diversity, Robust classification, Topological Data Analysis

Different cell types aggregate and sort into hierarchical architectures during the formation of animal tissues. The resulting spatial organization depends (in part) on the strength of adhesion of one cell type to itself relative to other cell types. However, automated and unsupervised classification of these multicellular spatial patterns remains challenging, particularly given their structural diversity and biological variability. Recent developments based on topological data analysis are intriguing to reveal similarities in tissue architecture, but these methods remain computationally expensive. In this article, we show that multicellular patterns organized from two interacting cell types can be efficiently represented through persistence images. Our optimized combination of dimensionality reduction via autoencoders, combined with hierarchical clustering, achieved high classification accuracy for simulations with constant cell numbers. We further demonstrate that persistence images can be normalized to improve classification for simulations with varying cell numbers due to proliferation. Finally, we systematically consider the importance of incorporating different topological features as well as information about each cell type to improve classification accuracy. We envision that topological machine learning based on persistence images will enable versatile and robust classification of complex tissue architectures that occur in development and disease.
eVAE: Evolutionary Variational Autoencoder
Authors: Zhangkai Wu, Longbing Cao, Lei Qi
Date: 2023-01-01 00:00:00
Tasks: Disentanglement, Image Generation, Text Generation

The surrogate loss of variational autoencoders (VAEs) poses various challenges to their training, inducing the imbalance between task fitting and representation inference. To avert this, the existing strategies for VAEs focus on adjusting the tradeoff by introducing hyperparameters, deriving a tighter bound under some mild assumptions, or decomposing the loss components per certain neural settings. VAEs still suffer from uncertain tradeoff learning.We propose a novel evolutionary variational autoencoder (eVAE) building on the variational information bottleneck (VIB) theory and integrative evolutionary neural learning. eVAE integrates a variational genetic algorithm into VAE with variational evolutionary operators including variational mutation, crossover, and evolution. Its inner-outer-joint training mechanism synergistically and dynamically generates and updates the uncertain tradeoff learning in the evidence lower bound (ELBO) without additional constraints. Apart from learning a lossy compression and representation of data under the VIB assumption, eVAE presents an evolutionary paradigm to tune critical factors of VAEs and deep neural networks and addresses the premature convergence and random search problem by integrating evolutionary optimization into deep learning. Experiments show that eVAE addresses the KL-vanishing problem for text generation with low reconstruction loss, generates all disentangled factors with sharp images, and improves the image generation quality,respectively. eVAE achieves better reconstruction loss, disentanglement, and generation-inference balance than its competitors.
Smooth Mathematical Function from Compact Neural Networks
Authors: I. K. Hong
Date: 2022-12-31 00:00:00
Tasks: regression

This is paper for the smooth function approximation by neural networks (NN). Mathematical or physical functions can be replaced by NN models through regression. In this study, we get NNs that generate highly accurate and highly smooth function, which only comprised of a few weight parameters, through discussing a few topics about regression. First, we reinterpret inside of NNs for regression; consequently, we propose a new activation function--integrated sigmoid linear unit (ISLU). Then special charateristics of metadata for regression, which is different from other data like image or sound, is discussed for improving the performance of neural networks. Finally, the one of a simple hierarchical NN that generate models substituting mathematical function is presented, and the new batch concept ``meta-batch" which improves the performance of NN several times more is introduced. The new activation function, meta-batch method, features of numerical data, meta-augmentation with metaparameters, and a structure of NN generating a compact multi-layer perceptron(MLP) are essential in this study.
Temporally Layered Architecture for Adaptive, Distributed and Continuous Control
Authors: Devdhar Patel, Joshua Russell, Francesca Walsh, Tauhidur Rahman, Terrence Sejnowski, Hava Siegelmann
Date: 2022-12-25 00:00:00
Tasks: continuous-control, Continuous Control

We present temporally layered architecture (TLA), a biologically inspired system for temporally adaptive distributed control. TLA layers a fast and a slow controller together to achieve temporal abstraction that allows each layer to focus on a different time-scale. Our design is biologically inspired and draws on the architecture of the human brain which executes actions at different timescales depending on the environment's demands. Such distributed control design is widespread across biological systems because it increases survivability and accuracy in certain and uncertain environments. We demonstrate that TLA can provide many advantages over existing approaches, including persistent exploration, adaptive control, explainable temporal behavior, compute efficiency and distributed control. We present two different algorithms for training TLA: (a) Closed-loop control, where the fast controller is trained over a pre-trained slow controller, allowing better exploration for the fast controller and closed-loop control where the fast controller decides whether to "act-or-not" at each timestep; and (b) Partially open loop control, where the slow controller is trained over a pre-trained fast controller, allowing for open loop-control where the slow controller picks a temporally extended action or defers the next n-actions to the fast controller. We evaluated our method on a suite of continuous control tasks and demonstrate the advantages of TLA over several strong baselines.
Genetic-tunneling driven energy optimizer for spin systems
Authors: Qichen Xu, Zhuanglin Shen, Manuel Pereiro, Pawel Herman, Olle Eriksson, Anna Delin
Date: 2022-12-31 00:00:00

A long-standing and difficult problem in, e.g., condensed matter physics is how to find the ground state of a complex many-body system where the potential energy surface has a large number of local minima. Spin systems containing complex and/or topological textures, for example spin spirals or magnetic skyrmions, are prime examples of such systems. We propose here a genetic-tunneling-driven variance-controlled optimization approach, and apply it to two-dimensional magnetic skyrmionic systems. The approach combines a local energy-minimizer backend and a metaheuristic global search frontend. The algorithm is naturally concurrent, resulting in short user execution time. We find that the method performs significantly better than simulated annealing (SA). Specifically, we demonstrate that for the Pd/Fe/Ir(111) system, our method correctly and efficiently identifies the experimentally observed spin spiral, skyrmion lattice and ferromagnetic ground states as a function of external magnetic field. To our knowledge, no other optimization method has until now succeeded in doing this. We envision that our findings will pave the way for evolutionary computing in mapping out phase diagrams for spin systems in general.
Dynamically Modular and Sparse General Continual Learning
Authors: Arnav Varma, Elahe Arani, Bahram Zonooz
Date: 2023-01-02 00:00:00
Tasks: Continual Learning

Real-world applications often require learning continuously from a stream of data under ever-changing conditions. When trying to learn from such non-stationary data, deep neural networks (DNNs) undergo catastrophic forgetting of previously learned information. Among the common approaches to avoid catastrophic forgetting, rehearsal-based methods have proven effective. However, they are still prone to forgetting due to task-interference as all parameters respond to all tasks. To counter this, we take inspiration from sparse coding in the brain and introduce dynamic modularity and sparsity (Dynamos) for rehearsal-based general continual learning. In this setup, the DNN learns to respond to stimuli by activating relevant subsets of neurons. We demonstrate the effectiveness of Dynamos on multiple datasets under challenging continual learning evaluation protocols. Finally, we show that our method learns representations that are modular and specialized, while maintaining reusability by activating subsets of neurons with overlaps corresponding to the similarity of stimuli.
Sentiment Analysis of COVID-19 Public Activity Restriction (PPKM) Impact using BERT Method
Authors: Fransiscus, A. S. Girsang
Date: 2022-12-31 00:00:00
Tasks: Sentiment Analysis

Covid-19 has grown rapidly in all parts of the world and is considered an international disaster because of its wide-reaching impact. The impact of Covid-19 has spread to Indonesia, especially in the slowdown in economic growth. This was influenced by the implementation of Community Activity Restrictions (PPKM) which limited community economic activities. This study aims to analyze the mapping of public sentiment towards PPKM policies in Indonesia during the pandemic based on Twitter data. Knowing the mapping of public sentiment regarding PPKM is expected to help stakeholders in the policy evaluation process for each region. The method used is BERT with IndoBERT specific model. The results showed the evaluation value of IndoBERT f-1 score reached 84%, precision 86%, and recall 84%. Meanwhile for the evaluation of the use of SVM f-1 score 70%, 72% precision, and 70% recall. Multinominal Na\"ive Bayes evaluation shows f-1 score 83%, precision 78%, and recall 80%. As a conclusion, BERT method with the IndoBERT model is proven to be higher than classical methods such as SVM and Multinominal Na\"ive Bayes.
RECOMED: A Comprehensive Pharmaceutical Recommendation System
Authors: Mariam Zomorodi, Ismail Ghodsollahee, Jennifer H. Martin, Nicholas J. Talley, Vahid Salari, Pawel Plawiak, Kazem Rahimi, U. Rajendra Acharya
Date: 2022-12-31 00:00:00
Tasks: Recommendation Systems, Sentiment Analysis

A comprehensive pharmaceutical recommendation system was designed based on the patients and drugs features extracted from Drugs.com and Druglib.com. First, data from these databases were combined, and a dataset of patients and drug information was built. Secondly, the patients and drugs were clustered, and then the recommendation was performed using different ratings provided by patients, and importantly by the knowledge obtained from patients and drug specifications, and considering drug interactions. To the best of our knowledge, we are the first group to consider patients conditions and history in the proposed approach for selecting a specific medicine appropriate for that particular user. Our approach applies artificial intelligence (AI) models for the implementation. Sentiment analysis using natural language processing approaches is employed in pre-processing along with neural network-based methods and recommender system algorithms for modeling the system. In our work, patients conditions and drugs features are used for making two models based on matrix factorization. Then we used drug interaction to filter drugs with severe or mild interactions with other drugs. We developed a deep learning model for recommending drugs by using data from 2304 patients as a training set, and then we used data from 660 patients as our validation set. After that, we used knowledge from critical information about drugs and combined the outcome of the model into a knowledge-based system with the rules obtained from constraints on taking medicine.
A Survey on Federated Recommendation Systems
Authors: Zehua Sun, Yonghui Xu, Yong liu, wei he, Lanju Kong, Fangzhao Wu, Yali Jiang, Lizhen Cui
Date: 2022-12-27 00:00:00
Tasks: Federated Learning, Recommendation Systems, Survey

Federated learning has recently been applied to recommendation systems to protect user privacy. In federated learning settings, recommendation systems can train recommendation models only collecting the intermediate parameters instead of the real user data, which greatly enhances the user privacy. Beside, federated recommendation systems enable to collaborate with other data platforms to improve recommended model performance while meeting the regulation and privacy constraints. However, federated recommendation systems faces many new challenges such as privacy, security, heterogeneity and communication costs. While significant research has been conducted in these areas, gaps in the surveying literature still exist. In this survey, we-(1) summarize some common privacy mechanisms used in federated recommendation systems and discuss the advantages and limitations of each mechanism; (2) review some robust aggregation strategies and several novel attacks against security; (3) summarize some approaches to address heterogeneity and communication costs problems; (4)introduce some open source platforms that can be used to build federated recommendation systems; (5) present some prospective research directions in the future. This survey can guide researchers and practitioners understand the research progress in these areas.
Ontology-based Context Aware Recommender System Application for Tourism
Authors: Vitor T. Camacho, José Cruz
Date: 2022-12-29 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

In this work a novel recommender system (RS) for Tourism is presented. The RS is context aware as is now the rule in the state-of-the-art for recommender systems and works on top of a tourism ontology which is used to group the different items being offered. The presented RS mixes different types of recommenders creating an ensemble which changes on the basis of the RS's maturity. Starting from simple content-based recommendations and iteratively adding popularity, demographic and collaborative filtering methods as rating density and user cardinality increases. The result is a RS that mutates during its lifetime and uses a tourism ontology and natural language processing (NLP) to correctly bin the items to specific item categories and meta categories in the ontology. This item classification facilitates the association between user preferences and items, as well as allowing to better classify and group the items being offered, which in turn is particularly useful for context-aware filtering.
Modified Query Expansion Through Generative Adversarial Networks for Information Extraction in E-Commerce
Authors: Altan Cakir, Mert Gurkan
Date: 2022-12-30 00:00:00
Tasks: Generative Adversarial Network, Semantic Similarity, Semantic Textual Similarity

This work addresses an alternative approach for query expansion (QE) using a generative adversarial network (GAN) to enhance the effectiveness of information search in e-commerce. We propose a modified QE conditional GAN (mQE-CGAN) framework, which resolves keywords by expanding the query with a synthetically generated query that proposes semantic information from text input. We train a sequence-to-sequence transformer model as the generator to produce keywords and use a recurrent neural network model as the discriminator to classify an adversarial output with the generator. With the modified CGAN framework, various forms of semantic insights gathered from the query document corpus are introduced to the generation process. We leverage these insights as conditions for the generator model and discuss their effectiveness for the query expansion task. Our experiments demonstrate that the utilization of condition structures within the mQE-CGAN framework can increase the semantic similarity between generated sequences and reference documents up to nearly 10% compared to baseline models
A Concept Knowledge Graph for User Next Intent Prediction at Alipay
Authors: Yacheng He, Qianghuai Jia, Lin Yuan, Ruopeng Li, Yixin Ou, Ningyu Zhang
Date: 2023-01-02 00:00:00

This paper illustrates the technologies of user next intent prediction with a concept knowledge graph. The system has been deployed on the Web at Alipay, serving more than 100 million daily active users. To explicitly characterize user intent, we propose AlipayKG, which is an offline concept knowledge graph in the Life-Service domain modeling the historical behaviors of users, the rich content interacted by users and the relations between them. We further introduce a Transformer-based model which integrates expert rules from the knowledge graph to infer the online user's next intent. Experimental results demonstrate that the proposed system can effectively enhance the performance of the downstream tasks while retaining explainability.
Robust Consensus Clustering and its Applications for Advertising Forecasting
Authors: Deguang Kong, Miao Lu, Konstantin Shmakov, Jian Yang
Date: 2022-12-27 00:00:00
Tasks: Clustering

Consensus clustering aggregates partitions in order to find a better fit by reconciling clustering results from different sources/executions. In practice, there exist noise and outliers in clustering task, which, however, may significantly degrade the performance. To address this issue, we propose a novel algorithm -- robust consensus clustering that can find common ground truth among experts' opinions, which tends to be minimally affected by the bias caused by the outliers. In particular, we formalize the robust consensus clustering problem as a constraint optimization problem, and then derive an effective algorithm upon alternating direction method of multipliers (ADMM) with rigorous convergence guarantee. Our method outperforms the baselines on benchmarks. We apply the proposed method to the real-world advertising campaign segmentation and forecasting tasks using the proposed consensus clustering results based on the similarity computed via Kolmogorov-Smirnov Statistics. The accurate clustering result is helpful for building the advertiser profiles so as to perform the forecasting.