pwc-archive/papers-with-abstracts huggingface dataset:
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions
Authors: Arnulf Jentzen, Adrian Riekert
Date: 2021-08-10 00:00:00

Gradient descent (GD) type optimization methods are the standard instrument to train artificial neural networks (ANNs) with rectified linear unit (ReLU) activation. Despite the great success of GD type optimization methods in numerical simulations for the training of ANNs with ReLU activation, it remains - even in the simplest situation of the plain vanilla GD optimization method with random initializations and ANNs with one hidden layer - an open problem to prove (or disprove) the conjecture that the risk of the GD optimization method converges in the training of such ANNs to zero as the width of the ANNs, the number of independent random initializations, and the number of GD steps increase to infinity. In this article we prove this conjecture in the situation where the probability distribution of the input data is equivalent to the continuous uniform distribution on a compact interval, where the probability distributions for the random initializations of the ANN parameters are standard normal distributions, and where the target function under consideration is continuous and piecewise affine linear. Roughly speaking, the key ingredients in our mathematical convergence analysis are (i) to prove that suitable sets of global minima of the risk functions are \emph{twice continuously differentiable submanifolds of the ANN parameter spaces}, (ii) to prove that the Hessians of the risk functions on these sets of global minima satisfy an appropriate \emph{maximal rank condition}, and, thereafter, (iii) to apply the machinery in [Fehrman, B., Gess, B., Jentzen, A., Convergence rates for the stochastic gradient descent method for non-convex objective functions. J. Mach. Learn. Res. 21(136): 1--48, 2020] to establish convergence of the GD optimization method with random initializations.
Empirical Analysis on Effectiveness of NLP Methods for Predicting Code Smell
Authors: Himanshu Gupta, Abhiram Anand Gulanikar, Lov Kumar, Lalita Bhanu Murthy Neti
Date: 2021-08-08 00:00:00
Tasks: Feature Engineering

A code smell is a surface indicator of an inherent problem in the system, most often due to deviation from standard coding practices on the developers part during the development phase. Studies observe that code smells made the code more susceptible to call for modifications and corrections than code that did not contain code smells. Restructuring the code at the early stage of development saves the exponentially increasing amount of effort it would require to address the issues stemming from the presence of these code smells. Instead of using traditional features to detect code smells, we use user comments to manually construct features to predict code smells. We use three Extreme learning machine kernels over 629 packages to identify eight code smells by leveraging feature engineering aspects and using sampling techniques. Our findings indicate that the radial basis functional kernel performs best out of the three kernel methods with a mean accuracy of 98.52.
An Empirical Study on Predictability of Software Code Smell Using Deep Learning Models
Authors: Himanshu Gupta, Tanmay G. Kulkarni, Lov Kumar, Lalita Bhanu Murthy Neti, Aneesh Krishna
Date: 2021-08-08 00:00:00
Tasks: Deep Learning, feature selection

Code Smell, similar to a bad smell, is a surface indication of something tainted but in terms of software writing practices. This metric is an indication of a deeper problem lies within the code and is associated with an issue which is prominent to experienced software developers with acceptable coding practices. Recent studies have often observed that codes having code smells are often prone to a higher probability of change in the software development cycle. In this paper, we developed code smell prediction models with the help of features extracted from source code to predict eight types of code smell. Our work also presents the application of data sampling techniques to handle class imbalance problem and feature selection techniques to find relevant feature sets. Previous studies had made use of techniques such as Naive - Bayes and Random forest but had not explored deep learning methods to predict code smell. A total of 576 distinct Deep Learning models were trained using the features and datasets mentioned above. The study concluded that the deep learning models which used data from Synthetic Minority Oversampling Technique gave better results in terms of accuracy, AUC with the accuracy of some models improving from 88.47 to 96.84.
ChemiRise: a data-driven retrosynthesis engine
Authors: Xiangyan Sun, Ke Liu, Yuquan Lin, Lingjie Wu, Haoming Xing, Minghong Gao, Ji Liu, Suocheng Tan, Zekun Ni, Qi Han, Junqiu Wu, Jie Fan
Date: 2021-08-09 00:00:00
Tasks: Retrosynthesis

We have developed an end-to-end, retrosynthesis system, named ChemiRise, that can propose complete retrosynthesis routes for organic compounds rapidly and reliably. The system was trained on a processed patent database of over 3 million organic reactions. Experimental reactions were atom-mapped, clustered, and extracted into reaction templates. We then trained a graph convolutional neural network-based one-step reaction proposer using template embeddings and developed a guiding algorithm on the directed acyclic graph (DAG) of chemical compounds to find the best candidate to explore. The atom-mapping algorithm and the one-step reaction proposer were benchmarked against previous studies and showed better results. The final product was demonstrated by retrosynthesis routes reviewed and rated by human experts, showing satisfying functionality and a potential productivity boost in real-life use cases.
Crowdsourced Databases and Sui Generis Rights
Authors: Gonçalo Simões de Almeida, Gonçalo Faria Abreu
Date: 2021-08-10 00:00:00

In this study we propose a new concept of databases (crowdsourced databases), adding a new conceptual approach to the debate on legal protection of databases in Europe. We also summarise the current legal framework and current indexing and web scraping practices - it would not be prudent to suggest a new theory without contextualising it in the legal and practical context in which it is developed.
Spectral Robustness for Correlation Clustering Reconstruction in Semi-Adversarial Models
Authors: Flavio Chierichetti, Alessandro Panconesi, Giuseppe Re, Luca Trevisan
Date: 2021-08-10 00:00:00
Tasks: Clustering

Correlation Clustering is an important clustering problem with many applications. We study the reconstruction version of this problem in which one is seeking to reconstruct a latent clustering that has been corrupted by random noise and adversarial modifications. Concerning the latter, there is a standard "post-adversarial" model in the literature, in which adversarial modifications come after the noise. Here, we introduce and analyse a "pre-adversarial" model in which adversarial modifications come before the noise. Given an input coming from such a semi-adversarial generative model, the goal is to reconstruct almost perfectly and with high probability the latent clustering. We focus on the case where the hidden clusters have nearly equal size and show the following. In the pre-adversarial setting, spectral algorithms are optimal, in the sense that they reconstruct all the way to the information-theoretic threshold beyond which no reconstruction is possible. This is in contrast to the post-adversarial setting, in which their ability to restore the hidden clusters stops before the threshold, but the gap is optimally filled by SDP-based algorithms. These results highlight a heretofore unknown robustness of spectral algorithms, showing them less brittle than previously thought.
Multi-Factors Aware Dual-Attentional Knowledge Tracing
Authors: Moyu Zhang, Xinning Zhu, Chunhong Zhang, Yang Ji, Feng Pan, Changchuan Yin
Date: 2021-08-10 00:00:00
Tasks: Knowledge Tracing

With the increasing demands of personalized learning, knowledge tracing has become important which traces students' knowledge states based on their historical practices. Factor analysis methods mainly use two kinds of factors which are separately related to students and questions to model students' knowledge states. These methods use the total number of attempts of students to model students' learning progress and hardly highlight the impact of the most recent relevant practices. Besides, current factor analysis methods ignore rich information contained in questions. In this paper, we propose Multi-Factors Aware Dual-Attentional model (MF-DAKT) which enriches question representations and utilizes multiple factors to model students' learning progress based on a dual-attentional mechanism. More specifically, we propose a novel student-related factor which records the most recent attempts on relevant concepts of students to highlight the impact of recent exercises. To enrich questions representations, we use a pre-training method to incorporate two kinds of question information including questions' relation and difficulty level. We also add a regularization term about questions' difficulty level to restrict pre-trained question representations to fine-tuning during the process of predicting students' performance. Moreover, we apply a dual-attentional mechanism to differentiate contributions of factors and factor interactions to final prediction in different practice records. At last, we conduct experiments on several real-world datasets and results show that MF-DAKT can outperform existing knowledge tracing methods. We also conduct several studies to validate the effects of each component of MF-DAKT.
Spiderweb nanomechanical resonators via Bayesian optimization: inspired by nature and guided by machine learning
Authors: Dongil Shin, Andrea Cupertino, Matthijs H. J. de Jong, Peter G. Steeneken, Miguel A. Bessa, Richard A. Norte
Date: 2021-08-10 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning

From ultra-sensitive detectors of fundamental forces to quantum networks and sensors, mechanical resonators are enabling next-generation technologies to operate in room temperature environments. Currently, silicon nitride nanoresonators stand as a leading microchip platform in these advances by allowing for mechanical resonators whose motion is remarkably isolated from ambient thermal noise. However, to date, human intuition has remained the driving force behind design processes. Here, inspired by nature and guided by machine learning, a spiderweb nanomechanical resonator is developed that exhibits vibration modes which are isolated from ambient thermal environments via a novel "torsional soft-clamping" mechanism discovered by the data-driven optimization algorithm. This bio-inspired resonator is then fabricated; experimentally confirming a new paradigm in mechanics with quality factors above 1 billion in room temperature environments. In contrast to other state-of-the-art resonators, this milestone is achieved with a compact design which does not require sub-micron lithographic features or complex phononic bandgaps, making it significantly easier and cheaper to manufacture at large scales. Here we demonstrate the ability of machine learning to work in tandem with human intuition to augment creative possibilities and uncover new strategies in computing and nanotechnology.
Extending LIME for Business Process Automation
Authors: Sohini Upadhyay, Vatche Isahagian, Vinod Muthusamy, Yara Rizk
Date: 2021-08-09 00:00:00

AI business process applications automate high-stakes business decisions where there is an increasing demand to justify or explain the rationale behind algorithmic decisions. Business process applications have ordering or constraints on tasks and feature values that cause lightweight, model-agnostic, existing explanation methods like LIME to fail. In response, we propose a local explanation framework extending LIME for explaining AI business process applications. Empirical evaluation of our extension underscores the advantage of our approach in the business process setting.
Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation
Authors: Shangshang Yang, Ye Tian, Xiaoshu Xiang, Shichen Peng, Xingyi Zhang
Date: 2021-08-10 00:00:00
Tasks: GPU, Neural Architecture Search

Evolutionary neural architecture search (ENAS) has recently received increasing attention by effectively finding high-quality neural architectures, which however consumes high computational cost by training the architecture encoded by each individual for complete epochs in individual evaluation. Numerous ENAS approaches have been developed to reduce the evaluation cost, but it is often difficult for most of these approaches to achieve high evaluation accuracy. To address this issue, in this paper we propose an accelerated ENAS via multifidelity evaluation termed MFENAS, where the individual evaluation cost is significantly reduced by training the architecture encoded by each individual for only a small number of epochs. The balance between evaluation cost and evaluation accuracy is well maintained by suggesting a multi-fidelity evaluation, which identifies the potentially good individuals that cannot survive from previous generations by integrating multiple evaluations under different numbers of training epochs. For high diversity of neural architectures, a population initialization strategy is devised to produce different neural architectures varying from ResNet-like architectures to Inception-like ones. Experimental results on CIFAR-10 show that the architecture obtained by the proposed MFENAS achieves a 2.39% test error rate at the cost of only 0.6 GPU days on one NVIDIA 2080TI GPU, demonstrating the superiority of the proposed MFENAS over state-of-the-art NAS approaches in terms of both computational cost and architecture quality. The architecture obtained by the proposed MFENAS is then transferred to CIFAR-100 and ImageNet, which also exhibits competitive performance to the architectures obtained by existing NAS approaches. The source code of the proposed MFENAS is available at https://github.com/DevilYangS/MFENAS/.
Deep Joint Learning of Pathological Region Localization and Alzheimer's Disease Diagnosis
Authors: Changhyun Park, Heung-Il Suk
Date: 2021-08-10 00:00:00
Tasks: Decision Making, Position

The identification of Alzheimer's disease (AD) and its early stages using structural magnetic resonance imaging (MRI) has been attracting the attention of researchers. Various data-driven approaches have been introduced to capture subtle and local morphological changes of the brain accompanied by the disease progression. One of the typical approaches for capturing subtle changes is patch-level feature representation. However, the predetermined regions to extract patches can limit classification performance by interrupting the exploration of potential biomarkers. In addition, the existing patch-level analyses have difficulty explaining their decision-making. To address these problems, we propose the BrainBagNet with a position-based gate (PG-BrainBagNet), a framework for jointly learning pathological region localization and AD diagnosis in an end-to-end manner. In advance, as all scans are aligned to a template in image processing, the position of brain images can be represented through the 3D Cartesian space shared by the overall MRI scans. The proposed method represents the patch-level response from whole-brain MRI scans and discriminative brain-region from position information. Based on the outcomes, the patch-level class evidence is calculated, and then the image-level prediction is inferred by a transparent aggregation. The proposed models were evaluated on the ADNI datasets. In five-fold cross-validation, the classification performance of the proposed method outperformed that of the state-of-the-art methods in both AD diagnosis (AD vs. normal control) and mild cognitive impairment (MCI) conversion prediction (progressive MCI vs. stable MCI) tasks. In addition, changes in the identified discriminant regions and patch-level class evidence according to the patch size used for model training are presented and analyzed.
A Novel Markovian Framework for Integrating Absolute and Relative Ordinal Emotion Information
Authors: Jingyao Wu, Ting Dang, Vidhyasaharan Sethu, Eliathamby Ambikairajah
Date: 2021-08-10 00:00:00
Tasks: Prediction

There is growing interest in affective computing for the representation and prediction of emotions along ordinal scales. However, the term ordinal emotion label has been used to refer to both absolute notions such as low or high arousal, as well as relation notions such as arousal is higher at one instance compared to another. In this paper, we introduce the terminology absolute and relative ordinal labels to make this distinction clear and investigate both with a view to integrate them and exploit their complementary nature. We propose a Markovian framework referred to as Dynamic Ordinal Markov Model (DOMM) that makes use of both absolute and relative ordinal information, to improve speech based ordinal emotion prediction. Finally, the proposed framework is validated on two speech corpora commonly used in affective computing, the RECOLA and the IEMOCAP databases, across a range of system configurations. The results consistently indicate that integrating relative ordinal information improves absolute ordinal emotion prediction.
Logical Information Cells I
Authors: Jean-Claude Belfiore, Daniel Bennequin, Xavier Giraud
Date: 2021-08-10 00:00:00

In this study we explore the spontaneous apparition of visible intelligible reasoning in simple artificial networks, and we connect this experimental observation with a notion of semantic information. We start with the reproduction of a DNN model of natural neurons in monkeys, studied by Neromyliotis and Moschovakis in 2017 and 2018, to explain how "motor equivalent neurons", coding only for the action of pointing, are supplemented by other neurons for specifying the actor of the action, the eye E, the hand H, or the eye and the hand together EH. There appear inner neurons performing a logical work, making intermediary proposition, for instance E V EH. Then, we remarked that adding a second hidden layer and choosing a symmetric metric for learning, the activities of the neurons become almost quantized and more informative. Using the work of Carnap and Bar-Hillel 1952, we define a measure of the logical value for collections of such cells. The logical score growths with the depth of the layer, i.e. the information on the output decision increases, which confirms a kind of bottleneck principle. Then we study a bit more complex tasks, a priori involving predicate logic. We compare the logic and the measured weights. This shows, for groups of neurons, a neat correlation between the logical score and the size of the weights. It exhibits a form of sparsity between the layers. The most spectacular result concerns the triples which can conclude for all conditions: when applying their weight matrices to their logical matrix, we recover the classification. This shows that weights precisely perform the proofs.
Multi-Valued Cognitive Maps: Calculations with Linguistic Variables without Using Numbers
Authors: Dmitry Maximov
Date: 2021-08-10 00:00:00

A concept of multi-valued cognitive maps is introduced in this paper. The concept expands the fuzzy one. However, all variables and weights are not linearly ordered in the concept, but are only partially-ordered. Such an ap- proach allows us to operate in cognitive maps with partially-ordered linguis- tic variables directly, without vague fuzzification/defuzzification methods. Hence, we may consider more subtle differences in degrees of experts' uncer- tainty, than in the fuzzy case. We prove the convergence of such cognitive maps and give a simple computational example which demonstrates using such a partially-ordered uncertainty degree scale.
On the Foundations of Grounding in Answer Set Programming
Authors: Roland Kaminski, Torsten Schaub
Date: 2021-08-10 00:00:00

We provide a comprehensive elaboration of the theoretical foundations of variable instantiation, or grounding, in Answer Set Programming (ASP). Building on the semantics of ASP's modeling language, we introduce a formal characterization of grounding algorithms in terms of (fixed point) operators. A major role is played by dedicated well-founded operators whose associated models provide semantic guidance for delineating the result of grounding along with on-the-fly simplifications. We address an expressive class of logic programs that incorporates recursive aggregates and thus amounts to the scope of existing ASP modeling languages. This is accompanied with a plain algorithmic framework detailing the grounding of recursive aggregates. The given algorithms correspond essentially to the ones used in the ASP grounder gringo.
Generating Music and Generative Art from Brain activity
Authors: Ricardo Andres Diaz Rincon
Date: 2021-08-09 00:00:00
Tasks: Brain Computer Interface

Nowadays, technological advances have influenced all human activities, creating new dynamics and ways of communication. In this context, some artists have incorporated these advances in their creative process, giving rise to unique aesthetic expressions referred to in the literature as Generative Art, which is characterized by assigning part of the creative process to a system that acts with certain autonomy (Galanter, 2003). This research work introduces a computational system for creating generative art using a Brain-Computer Interface (BCI) which portrays the user's brain activity in a digital artwork. In this way, the user takes an active role in the creative process. In aims of showing that the proposed system materializes in an artistic piece the user's mental states by means of a visual and sound representation, several tests are carried out to ensure the reliability of the BCI device sent data. The generated artwork uses brain signals and concepts of geometry, color and spatial location to give complexity to the autonomous construction. As an added value, the visual and auditory production is accompanied by an olfactory and kinesthetic component which complements the art pieces providing a multimodal communication character.
Sample Observed Effects: Enumeration, Randomization and Generalization
Authors: Andre F. Ribeiro
Date: 2021-08-09 00:00:00
Tasks: counterfactual

The widely used 'Counterfactual' definition of Causal Effects was derived for unbiasedness and accuracy - and not generalizability. We propose a Combinatorial definition for the External Validity (EV) of intervention effects. We first define the concept of an effect observation 'background'. We then formulate conditions for effect generalization based on samples' sets of (observed and unobserved) backgrounds. This reveals two limits for effect generalization: (1) when effects of a variable are observed under all their enumerable backgrounds, or, (2) when backgrounds have become sufficiently randomized. We use the resulting combinatorial framework to re-examine several issues in the original counterfactual formulation: out-of-sample validity, concurrent estimation of multiple effects, bias-variance tradeoffs, statistical power, and connections to current predictive and explaining techniques. Methodologically, the definitions also allow us to replace the parametric estimation problems that followed the counterfactual definition by combinatorial enumeration and randomization problems in non-experimental samples. We use the resulting non-parametric framework to demonstrate (External Validity, Unconfoundness and Precision) tradeoffs in the performance of popular supervised, explaining, and causal-effect estimators. We also illustrate how the approach allows for the use of supervised and explaining methods in non-i.i.d. samples. The COVID19 pandemic highlighted the need for learning solutions to provide predictions in severally incomplete samples. We demonstrate applications in this pressing problem.