pwc-archive/papers-with-abstracts huggingface dataset:
A reversible system based on hybrid toggle radius-4 cellular automata and its application as a block cipher
Authors: Everton R. Lira, Heverton B. de Macêdo, Danielli A. Lima, Leonardo Alt, Gina M. B. Oliveira
Date: 2021-06-09 00:00:00

The dynamical system described herein uses a hybrid cellular automata (CA) mechanism to attain reversibility, and this approach is adapted to create a novel block cipher algorithm called HCA. CA are widely used for modeling complex systems and employ an inherently parallel model. Therefore, applications derived from CA have a tendency to fit very well in the current computational paradigm where scalability and multi-threading potential are quite desirable characteristics. HCA model has recently received a patent by the Brazilian agency INPI. Several evaluations and analyses performed on the model are presented here, such as theoretical discussions related to its reversibility and an analysis based on graph theory, which reduces HCA security to the well-known Hamiltonian cycle problem that belongs to the NP-complete class. Finally, the cryptographic robustness of HCA is empirically evaluated through several tests, including avalanche property compliance and the NIST randomness suite.
Probabilistic Neural Network to Quantify Uncertainty of Wind Power Estimation
Authors: Farzad Karami, Nasser Kehtarnavaz, Mario Rotea
Date: 2021-06-04 00:00:00

Each year a growing number of wind farms are being added to power grids to generate electricity. The power curve of a wind turbine, which exhibits the relationship between generated power and wind speed, plays a major role in assessing the performance of a wind farm. Neural networks have been used for power curve estimation. However, they do not produce a confidence measure for their output, unless computationally prohibitive Bayesian methods are used. In this paper, a probabilistic neural network with Monte Carlo dropout is considered to quantify the model (epistemic) uncertainty of the power curve estimation. This approach offers a minimal increase in computational complexity over deterministic approaches. Furthermore, by incorporating a probabilistic loss function, the noise or aleatoric uncertainty in the data is estimated. The developed network captures both model and noise uncertainty which is found to be useful tools in assessing performance. Also, the developed network is compared with existing ones across a public domain dataset showing superior performance in terms of prediction accuracy.
Information theoretic analysis of computational models as a tool to understand the neural basis of behaviors
Authors: Madhavun Candadai
Date: 2021-06-02 00:00:00

One of the greatest research challenges of this century is to understand the neural basis for how behavior emerges in brain-body-environment systems. To this end, research has flourished along several directions but have predominantly focused on the brain. While there is in an increasing acceptance and focus on including the body and environment in studying the neural basis of behavior, animal researchers are often limited by technology or tools. Computational models provide an alternative framework within which one can study model systems where ground-truth can be measured and interfered with. These models act as a hypothesis generation framework that would in turn guide experimentation. Furthermore, the ability to intervene as we please, allows us to conduct in-depth analysis of these models in a way that cannot be performed in natural systems. For this purpose, information theory is emerging as a powerful tool that can provide insights into the operation of these brain-body-environment models. In this work, I provide an introduction, a review and discussion to make a case for how information theoretic analysis of computational models is a potent research methodology to help us better understand the neural basis of behavior.
Condition Integration Memory Network: An Interpretation of the Meaning of the Neuronal Design
Authors: Cheng Qian
Date: 2021-05-21 00:00:00

Understanding the basic operational logics of the nervous system is essential to advancing neuroscientific research. However, theoretical efforts to tackle this fundamental problem are lacking, despite the abundant empirical data about the brain that has been collected in the past few decades. To address this shortcoming, this document introduces a hypothetical framework for the functional nature of primitive neural networks. It analyzes the idea that the activity of neurons and synapses can symbolically reenact the dynamic changes in the world and thus enable an adaptive system of behavior. More significantly, the network achieves this without participating in an algorithmic structure. When a neuron's activation represents some symbolic element in the environment, each of its synapses can indicate a potential change to the element and its future state. The efficacy of a synaptic connection further specifies the element's particular probability for, or contribution to, such a change. As it fires, a neuron's activation is transformed to its postsynaptic targets, resulting in a chronological shift of the represented elements. As the inherent function of summation in a neuron integrates the various presynaptic contributions, the neural network mimics the collective causal relationship of events in the observed environment.
On the Use of Minimum Penalties in Statistical Learning
Authors: Ben Sherwood, Bradley S. Price
Date: 2021-06-09 00:00:00
Tasks: Model Selection, regression

Modern multivariate machine learning and statistical methodologies estimate parameters of interest while leveraging prior knowledge of the association between outcome variables. The methods that do allow for estimation of relationships do so typically through an error covariance matrix in multivariate regression which does not scale to other types of models. In this article we proposed the MinPEN framework to simultaneously estimate regression coefficients associated with the multivariate regression model and the relationships between outcome variables using mild assumptions. The MinPen framework utilizes a novel penalty based on the minimum function to exploit detected relationships between responses. An iterative algorithm that generalizes current state of the art methods is proposed as a solution to the non-convex optimization that is required to obtain estimates. Theoretical results such as high dimensional convergence rates, model selection consistency, and a framework for post selection inference are provided. We extend the proposed MinPen framework to other exponential family loss functions, with a specific focus on multiple binomial responses. Tuning parameter selection is also addressed. Finally, simulations and two data examples are presented to show the finite sample properties of this framework.
Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints
Authors: Shosei Sakaguchi
Date: 2021-06-09 00:00:00

Many policies involve dynamics in their treatment assignments, where individuals receive sequential interventions over multiple stages. We study estimation of an optimal dynamic treatment regime that guides the optimal treatment assignment for each individual at each stage based on their history. We propose an empirical welfare maximization approach in this dynamic framework, which estimates the optimal dynamic treatment regime using data from an experimental or quasi-experimental study while satisfying exogenous constraints on policies. The paper proposes two estimation methods: one solves the treatment assignment problem sequentially through backward induction, and the other solves the entire problem simultaneously across all stages. We establish finite-sample upper bounds on worst-case average welfare regrets for these methods and show their optimal $n^{-1/2}$ convergence rates. We also modify the simultaneous estimation method to accommodate intertemporal budget/capacity constraints.
Automatically Differentiable Random Coefficient Logistic Demand Estimation
Authors: Andrew Chia
Date: 2021-06-08 00:00:00

We show how the random coefficient logistic demand (BLP) model can be phrased as an automatically differentiable moment function, including the incorporation of numerical safeguards proposed in the literature. This allows gradient-based frequentist and quasi-Bayesian estimation using the Continuously Updating Estimator (CUE). Drawing from the machine learning literature, we outline hitherto under-utilized best practices in both frequentist and Bayesian estimation techniques. Our Monte Carlo experiments compare the performance of CUE, 2S-GMM, and LTE estimation. Preliminary findings indicate that the CUE estimated using LTE and frequentist optimization has a lower bias but higher MAE compared to the traditional 2-Stage GMM (2S-GMM) approach. We also find that using credible intervals from MCMC sampling for the non-linear parameters together with frequentist analytical standard errors for the concentrated out linear parameters provides empirical coverage closest to the nominal level. The accompanying admest Python package provides a platform for replication and extensibility.
Sirius: Visualization of Mixed Features as a Mutual Information Network Graph
Authors: Jane L. Adams, Todd F. Deluca, Christopher M. Danforth, Peter S. Dodds, Yuhang Zheng, Konstantinos Anastasakis, Boyoon Choi, Allison Min, Michael M. Bessey
Date: 2021-06-09 00:00:00
Tasks: Dimensionality Reduction, feature selection, Graph Mining

Data scientists across disciplines are increasingly in need of exploratory analysis tools for data sets with a high volume of features of mixed data type (quantitative continuous and discrete categorical). We introduce Sirius, a novel visualization package for researchers to explore feature relationships among mixed data types using mutual information. The visualization of feature relationships aids data scientists in finding meaningful dependence among features prior to the development of predictive modeling pipelines, which can inform downstream analysis such as feature selection, feature extraction, and early detection of potential proxy variables. Using an information theoretic approach, Sirius supports network visualization of heterogeneous data sets (consisting of continuous and discrete data types), and provides a user interface for exploring feature pairs with locally significant mutual information scores. Mutual information algorithm and bivariate chart types are assigned on a data type pairing basis (continuous-continuous, discrete-discrete, and discrete-continuous). We show how this tool can be used for tasks such as hypothesis confirmation, identification of predictive features, suggestions for feature extraction, or early warning of data abnormalities. The accompanying website for this paper can be accessed at https://sirius.universalities.com/. All code and supplemental materials can be accessed at https://osf.io/pdm9r/.
Helping results assessment by adding explainable elements to the deep relevance matching model
Authors: Ioannis Chios, Suzan Verberne
Date: 2021-06-09 00:00:00

In this paper we address the explainability of web search engines. We propose two explainable elements on the search engine result page: a visualization of query term weights and a visualization of passage relevance. The idea is that search engines that indicate to the user why results are retrieved are valued higher by users and gain user trust. We deduce the query term weights from the term gating network in the Deep Relevance Matching Model (DRMM) and visualize them as a doughnut chart. In addition, we train a passage-level ranker with DRMM that selects the most relevant passage from each document and shows it as snippet on the result page. Next to the snippet we show a document thumbnail with this passage highlighted. We evaluate the proposed interface in an online user study, asking users to judge the explainability and assessability of the interface. We found that users judge our proposed interface significantly more explainable and easier to assess than a regular search engine result page. However, they are not significantly better in selecting the relevant documents from the top-5. This indicates that the explainability of the search engine result page leads to a better user experience. Thus, we conclude that the proposed explainable elements are promising as visualization for search engine users.
Global Context Enhanced Graph Neural Networks for Session-based Recommendation
Authors: Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xian-Ling Mao, Minghui Qiu
Date: 2021-06-09 00:00:00
Tasks: Representation Learning, Session-Based Recommendations

Session-based recommendation (SBR) is a challenging task, which aims at recommending items based on anonymous behavior sequences. Almost all the existing solutions for SBR model user preference only based on the current session without exploiting the other sessions, which may contain both relevant and irrelevant item-transitions to the current session. This paper proposes a novel approach, called Global Context Enhanced Graph Neural Networks (GCE-GNN) to exploit item transitions over all sessions in a more subtle manner for better inferring the user preference of the current session. Specifically, GCE-GNN learns two levels of item embeddings from session graph and global graph, respectively: (i) Session graph, which is to learn the session-level item embedding by modeling pairwise item-transitions within the current session; and (ii) Global graph, which is to learn the global-level item embedding by modeling pairwise item-transitions over all sessions. In GCE-GNN, we propose a novel global-level item representation learning layer, which employs a session-aware attention mechanism to recursively incorporate the neighbors' embeddings of each node on the global graph. We also design a session-level item representation learning layer, which employs a GNN on the session graph to learn session-level item embeddings within the current session. Moreover, GCE-GNN aggregates the learnt item representations in the two levels with a soft attention mechanism. Experiments on three benchmark datasets demonstrate that GCE-GNN outperforms the state-of-the-art methods consistently.
AutoFT: Automatic Fine-Tune for Parameters Transfer Learning in Click-Through Rate Prediction
Authors: Xiangli Yang, Qing Liu, Rong Su, Ruiming Tang, Zhirong Liu, Xiuqiang He
Date: 2021-06-09 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems, Transfer Learning

Recommender systems are often asked to serve multiple recommendation scenarios or domains. Fine-tuning a pre-trained CTR model from source domains and adapting it to a target domain allows knowledge transferring. However, optimizing all the parameters of the pre-trained network may result in over-fitting if the target dataset is small and the number of parameters is large. This leads us to think of directly reusing parameters in the pre-trained model which represent more general features learned from multiple domains. However, the design of freezing or fine-tuning layers of parameters requires much manual effort since the decision highly depends on the pre-trained model and target instances. In this work, we propose an end-to-end transfer learning framework, called Automatic Fine-Tuning (AutoFT), for CTR prediction. AutoFT consists of a field-wise transfer policy and a layer-wise transfer policy. The field-wise transfer policy decides how the pre-trained embedding representations are frozen or fine-tuned based on the given instance from the target domain. The layer-wise transfer policy decides how the high?order feature representations are transferred layer by layer. Extensive experiments on two public benchmark datasets and one private industrial dataset demonstrate that AutoFT can significantly improve the performance of CTR prediction compared with state-of-the-art transferring approaches.
Combining ResNet and Transformer for Chinese Grammatical Error Diagnosis
Authors: Shaolei Wang, Baoxin Wang, Jiefu Gong, Zhongyuan Wang, Xiao Hu, Xingyi Duan, Zizhuo Shen, Gang Yue, Ruiji Fu, Dayong Wu, Wanxiang Che, Shijin Wang, Guoping Hu, Ting Liu
Date: 2020-10-01 00:00:00
Tasks: Position

Grammatical error diagnosis is an important task in natural language processing. This paper introduces our system at NLPTEA-2020 Task: Chinese Grammatical Error Diagnosis (CGED). CGED aims to diagnose four types of grammatical errors which are missing words (M), redundant words (R), bad word selection (S) and disordered words (W). Our system is built on the model of multi-layer bidirectional transformer encoder and ResNet is integrated into the encoder to improve the performance. We also explore two ensemble strategies including weighted averaging and stepwise ensemble selection from libraries of models to improve the performance of single model. In official evaluation, our system obtains the highest F1 scores at identification level and position level. We also recommend error corrections for specific error types and achieve the second highest F1 score at correction level.
An ordinal CNN approach for the assessment of neurological damage in Parkinson's disease patients
Authors: Javier Barbero-Gómez, Pedro-Antonio Gutiérrez, Víctor-Manuel Vargas, Juan-Antonio Vallejo-Casas, César Hervás-Martínez
Date: 2021-05-31 00:00:00
Tasks: Data Augmentation

3D image scans are an assessment tool for neurological damage in Parkinson's disease (PD) patients. This diagnosis process can be automatized to help medical staff through Decision Support Systems (DSSs), and Convolutional Neural Networks (CNNs) are good candidates, because they are effective when applied to spatial data. This paper proposes a 3D CNN ordinal model for assessing the level or neurological damage in PD patients. Given that CNNs need large datasets to achieve acceptable performance, a data augmentation method is adapted to work with spatial data. We consider the Ordinal Graph-based Oversampling via Shortest Paths (OGO-SP) method, which applies a gamma probability distribution for inter-class data generation. A modification of OGO-SP is proposed, the OGO-SP-$\beta$ algorithm, which applies the beta distribution for generating synthetic samples in the inter-class region, a better suited distribution when compared to gamma. The evaluation of the different methods is based on a novel 3D image dataset provided by the Hospital Universitario 'Reina Sof\'ia' (C\'ordoba, Spain). We show how the ordinal methodology improves the performance with respect to the nominal one, and how OGO-SP-$\beta$ yields better performance than OGO-SP.
The NNI Query-by-Example System for MediaEval 2014
Authors: Peng Yang, HaiHua Xu, Xiong Xiao, Lei Xie, Cheung-Chi Leung, Hongjie Chen, JIA YU, Hang Lv, Lei Wang, Su Jun Leow, Bin Ma, Eng Siong Chng, Haizhou Li
Date: 2014-10-16 00:00:00
Tasks: Dynamic Time Warping, Keyword Spotting, Template Matching

In this paper we describe the system proposed by NNI (NWPU-NTU-I2R) team for the QUESST task within the Mediaeval 2014 evaluation. To solve the problem, we used both dynamic time warping (DTW) and symbolic search (SS) based approaches. The DTW system performs template matching using subsequence DTW algorithm and posterior representations. The symbolic search is performed on phone sequences generated by phone recognizers. For both symbolic and DTW search, partial sequence matching is performed to reduce missing rate, especially for query type 2 and 3. After fusing 9 DTW systems, 7 symbolic systems, and query length side information, we obtained 0.6023 actual normalized cross entropy (actCnxe) for all queries combined. For type 3 complex queries, we achieved 0.7252 actCnxe.
ELiRF at MediaEval 2014: Query by Example Search on Speech Task (QUESST)
Authors: Marcos Calvo, Mayte Giménez, Lluís-F. Hurtado, Emilio Sanchis, Jon A. Gómez
Date: 2014-10-16 00:00:00
Tasks: Dynamic Time Warping, Keyword Spotting

In this paper, we present the systems that the Natural Language Engineering and Pattern Recognition group (ELiRF) has submitted to the MediaEval 2014 Query by Example Search on Speech task. All of them are based on a Subsequence Dynamic Time Warping algorithm and do not use any other information from outside the task (zero-resources systems).
DPER: Efficient Parameter Estimation for Randomly Missing Data
Authors: Thu Nguyen, Khoi Minh Nguyen-Duy, Duy Ho Minh Nguyen, Binh T. Nguyen, Bruce Alan Wade
Date: 2021-06-06 00:00:00
Tasks: Imputation, parameter estimation

The missing data problem has been broadly studied in the last few decades and has various applications in different areas such as statistics or bioinformatics. Even though many methods have been developed to tackle this challenge, most of those are imputation techniques that require multiple iterations through the data before yielding convergence. In addition, such approaches may introduce extra biases and noises to the estimated parameters. In this work, we propose novel algorithms to find the maximum likelihood estimates (MLEs) for a one-class/multiple-class randomly missing data set under some mild assumptions. As the computation is direct without any imputation, our algorithms do not require multiple iterations through the data, thus promising to be less time-consuming than other methods while maintaining superior estimation performance. We validate these claims by empirical results on various data sets of different sizes and release all codes in a GitHub repository to contribute to the research community related to this problem.
Fair Machine Learning under Limited Demographically Labeled Data
Authors: Mustafa Safa Ozdayi, Murat Kantarcioglu, Rishabh Iyer
Date: 2021-06-03 00:00:00
Tasks: BIG-bench Machine Learning, Bilevel Optimization, Fairness

Research has shown that, machine learning models might inherit and propagate undesired social biases encoded in the data. To address this problem, fair training algorithms are developed. However, most algorithms assume we know demographic/sensitive data features such as gender and race. This assumption falls short in scenarios where collecting demographic information is not feasible due to privacy concerns, and data protection policies. A recent line of work develops fair training methods that can function without any demographic feature on the data, that are collectively referred as Rawlsian methods. Yet, we show in experiments that, Rawlsian methods tend to exhibit relatively high bias. Given this, we look at the middle ground between the previous approaches, and consider a setting where we know the demographic attributes for only a small subset of our data. In such a setting, we design fair training algorithms which exhibit both good utility, and low bias. In particular, we show that our techniques can train models to significantly outperform Rawlsian approaches even when 0.1% of demographic attributes are available in the training data. Furthermore, our main algorithm can accommodate multiple training objectives easily. We expand our main algorithm to achieve robustness to label noise in addition to fairness in the limited demographics setting to highlight that property as well.
Self-Adaptive Swarm System (SASS)
Authors: Qin Yang
Date: 2021-05-25 00:00:00
Tasks: Decision Making

Distributed artificial intelligence (DAI) studies artificial intelligence entities working together to reason, plan, solve problems, organize behaviors and strategies, make collective decisions and learn. This Ph.D. research proposes a principled Multi-Agent Systems (MAS) cooperation framework -- Self-Adaptive Swarm System (SASS) -- to bridge the fourth level automation gap between perception, communication, planning, execution, decision-making, and learning.