pwc-archive/papers-with-abstracts huggingface dataset:
Comprehensive Movie Recommendation System
Authors: Hrisav Bhowmick, Ananda Chatterjee, Jaydip Sen
Date: 2021-12-23 00:00:00
Tasks: Collaborative Filtering, Movie Recommendation, Recommendation Systems, TAG

A recommender system, also known as a recommendation system, is a type of information filtering system that attempts to forecast a user's rating or preference for an item. This article designs and implements a complete movie recommendation system prototype based on the Genre, Pearson Correlation Coefficient, Cosine Similarity, KNN-Based, Content-Based Filtering using TFIDF and SVD, Collaborative Filtering using TFIDF and SVD, Surprise Library based recommendation system technology. Apart from that in this paper, we present a novel idea that applies machine learning techniques to construct a cluster for the movie based on genres and then observes the inertia value number of clusters were defined. The constraints of the approaches discussed in this work have been described, as well as how one strategy overcomes the disadvantages of another. The whole work has been done on the dataset Movie Lens present at the group lens website which contains 100836 ratings and 3683 tag applications across 9742 movies. These data were created by 610 users between March 29, 1996, and September 24, 2018.
Embedding Graph Convolutional Networks in Recurrent Neural Networks for Predictive Monitoring
Authors: Efrén Rama-Maneiro, Juan C. Vidal, Manuel Lama
Date: 2021-12-17 00:00:00

Predictive monitoring of business processes is a subfield of process mining that aims to predict, among other things, the characteristics of the next event or the sequence of next events. Although multiple approaches based on deep learning have been proposed, mainly recurrent neural networks and convolutional neural networks, none of them really exploit the structural information available in process models. This paper proposes an approach based on graph convolutional networks and recurrent neural networks that uses information directly from the process model. An experimental evaluation on real-life event logs shows that our approach is more consistent and outperforms the current state-of-the-art approaches.
Using Sequential Statistical Tests for Efficient Hyperparameter Tuning
Authors: Philip Buczak, Andreas Groll, Markus Pauly, Jakob Rehof, Daniel Horn
Date: 2021-12-23 00:00:00

Hyperparameter tuning is one of the the most time-consuming parts in machine learning. Despite the existence of modern optimization algorithms that minimize the number of evaluations needed, evaluations of a single setting may still be expensive. Usually a resampling technique is used, where the machine learning method has to be fitted a fixed number of k times on different training datasets. The respective mean performance of the k fits is then used as performance estimator. Many hyperparameter settings could be discarded after less than k resampling iterations if they are clearly inferior to high-performing settings. However, resampling is often performed until the very end, wasting a lot of computational effort. To this end, we propose the Sequential Random Search (SQRS) which extends the regular random search algorithm by a sequential testing procedure aimed at detecting and eliminating inferior parameter configurations early. We compared our SQRS with regular random search using multiple publicly available regression and classification datasets. Our simulation study showed that the SQRS is able to find similarly well-performing parameter settings while requiring noticeably fewer evaluations. Our results underscore the potential for integrating sequential tests into hyperparameter tuning.
Towards Learning Terminological Concept Systems from Multilingual Natural Language Text
Authors: Lennart Wachowiak, Christian Lang, Barbara Heinisch, Dagmar Gromann
Date: 2021-08-30 00:00:00
Proceeding: 3rd Conference on Language, Data and Knowledge 2021 8
Tasks: Relation Extraction, Term Extraction, Translation

Terminological Concept Systems (TCS) provide a means of organizing, structuring and representing domain-specific multilingual information and are important to ensure terminological consistency in many tasks, such as translation and cross-border communication. While several approaches to (semi-) automatic term extraction exist, learning their interrelations is vastly underexplored. We propose an automated method to extract terms and relations across natural languages and specialized domains. To this end, we adapt pretrained multilingual neural language models, which we evaluate on term extraction standard datasets with best performing results and a combination of relation extraction standard datasets with competitive results. Code and dataset are publicly available.
AST Indexing: A Near-Constant Time Solution to the Get-Descendants-by-Type Problem
Authors: Sam Kelly
Date: 2014-05-18 00:00:00
Proceeding: Dickinson College Honors Theses 2014 5

In this paper we present two novel abstract syntax tree (AST) indexing algorithms that solve the get-descendants-by-type problem in near constant time. This work has been implemented in the U.S. Department of Energy's ROSE compiler framework with plans to be officially integrated as an optimization library called "NodeFinder". ROSE is an open source software analysis platform and source-to-source compiler suited for large-scale C/C++, UPC, Java, Python, Fortran, OpenCL, CUDA, and OpenMP applications that has been actively developed at Lawrence Livermore National Laboratory for the last sixteen years. The get-descendants-by-type problem is the problem of efficiently answering queries of the form “given an arbitrary AST node 𝐴 and an arbitrary node type 𝑇, return all descendants of 𝐴 that are of type 𝑇". Our algorithms are generic in that they can also be applied to any tree that has a meaningful notion of node "type", so we also explore some potential applications in the fields of file systems and databases.
Advancing Spiking Neural Networks towards Deep Residual Learning
Authors: Yifan Hu, Lei Deng, Yujie Wu, Man Yao, Guoqi Li
Date: 2021-12-15 00:00:00

Despite the rapid progress of neuromorphic computing, inadequate capacity and insufficient representation power of spiking neural networks (SNNs) severely restrict their application scope in practice. Residual learning and shortcuts have been evidenced as an important approach for training deep neural networks, but rarely did previous work assess their applicability to the characteristics of spike-based communication and spatiotemporal dynamics. In this paper, we first identify that this negligence leads to impeded information flow and the accompanying degradation problem in previous residual SNNs. To address this issue, we propose a novel SNN-oriented residual architecture termed MS-ResNet, which establishes membrane-based shortcut pathways, and further prove that the gradient norm equality can be achieved in MS-ResNet by introducing block dynamical isometry theory, which ensures the network can be well-behaved in a depth-insensitive way. Thus we are able to significantly extend the depth of directly trained SNNs, e.g., up to 482 layers on CIFAR-10 and 104 layers on ImageNet, without observing any slight degradation problem. To validate the effectiveness of MS-ResNet, experiments on both frame-based and neuromorphic datasets are conducted. MS-ResNet104 achieves a superior result of 76.02% accuracy on ImageNet, which is the highest to our best knowledge in the domain of directly trained SNNs. Great energy efficiency is also observed, with an average of only one spike per neuron needed to classify an input sample. We believe our powerful and scalable models will provide a strong support for further exploration of SNNs.
Causal Explanation of Convolutional Neural Networks
Authors: Hichem Debbi
Date: 2021-09-13 00:00:00
Proceeding: ECML 2021 9
Tasks: counterfactual, Counterfactual Explanation, Counterfactual Reasoning, Model Compression, Network Pruning, Weakly-Supervised Object Localization

In this paper we introduce an explanation technique for Convolutional Neural Networks (CNNs) based on the theory of causality by Halpern and Pearl [12]. The causal explanation technique (CexCNN) is based on measuring the filter importance to a CNN decision, which is measured through counterfactual reasoning. In addition, we employ extended definitions of causality, which are responsibility and blame to weight the importance of such filters and project their contribution on input images. Since CNNs form a hierarchical structure, and since causal models can be hierarchically abstracted, we employ this similarity to perform the most important contribution of this paper, which is localizing the important features in the input image that contributed the most to a CNN’s decision. In addition to its ability in localization, we will show that CexCNN can be useful as well for model compression through pruning the less important filters. We tested CexCNN on several CNNs architectures and datasets. (The code is available on https://github.com/HichemDebbi/CexCNN)
Decentralized Multi-Task Stochastic Optimization With Compressed Communications
Authors: Navjot Singh, Xuanyu Cao, Suhas Diggavi, Tamer Basar
Date: 2021-12-23 00:00:00
Tasks: Stochastic Optimization

We consider a multi-agent network where each node has a stochastic (local) cost function that depends on the decision variable of that node and a random variable, and further the decision variables of neighboring nodes are pairwise constrained. There is an aggregate objective function for the network, composed additively of the expected values of the local cost functions at the nodes, and the overall goal of the network is to obtain the minimizing solution to this aggregate objective function subject to all the pairwise constraints. This is to be achieved at the node level using decentralized information and local computation, with exchanges of only compressed information allowed by neighboring nodes. The paper develops algorithms and obtains performance bounds for two different models of local information availability at the nodes: (i) sample feedback, where each node has direct access to samples of the local random variable to evaluate its local cost, and (ii) bandit feedback, where samples of the random variables are not available, but only the values of the local cost functions at two random points close to the decision are available to each node. For both models, with compressed communication between neighbors, we have developed decentralized saddle-point algorithms that deliver performances no different (in order sense) from those without communication compression; specifically, we show that deviation from the global minimum value and violations of the constraints are upper-bounded by $\mathcal{O}(T^{-\frac{1}{2}})$ and $\mathcal{O}(T^{-\frac{1}{4}})$, respectively, where $T$ is the number of iterations. Numerical examples provided in the paper corroborate these bounds and demonstrate the communication efficiency of the proposed method.
Learning with distributional inverters
Authors: Eric Binnendyk, Marco Carmosino, Antonina Kolokolova, Ramyaa Ramyaa, Manuel Sabin
Date: 2021-12-23 00:00:00

We generalize the "indirect learning" technique of Furst et. al., 1991 to reduce from learning a concept class over a samplable distribution $\mu$ to learning the same concept class over the uniform distribution. The reduction succeeds when the sampler for $\mu$ is both contained in the target concept class and efficiently invertible in the sense of Impagliazzo & Luby, 1989. We give two applications. - We show that AC0[q] is learnable over any succinctly-described product distribution. AC0[q] is the class of constant-depth Boolean circuits of polynomial size with AND, OR, NOT, and counting modulo $q$ gates of unbounded fanins. Our algorithm runs in randomized quasi-polynomial time and uses membership queries. - If there is a strongly useful natural property in the sense of Razborov & Rudich 1997 -- an efficient algorithm that can distinguish between random strings and strings of non-trivial circuit complexity -- then general polynomial-sized Boolean circuits are learnable over any efficiently samplable distribution in randomized polynomial time, given membership queries to the target function
Physics Constrained Flow Neural Network for Short-Timescale Predictions in Data Communications Networks
Authors: Xiangle Cheng, James He, Shihan Xiao, Yingxue Zhang, Zhitang Chen, Pascal Poupart, FengLin Li
Date: 2021-12-23 00:00:00
Tasks: Self-Supervised Learning

Machine learning is gaining growing momentum in various recent models for the dynamic analysis of information flows in data communications networks. These preliminary models often rely on off-the-shelf learning models to predict from historical statistics while disregarding the physics governing the generating behaviors of these flows. This paper instead introduces Flow Neural Network (FlowNN) to improve the feature representation with learned physical bias. This is implemented by an induction layer, working upon the embedding layer, to impose the physics connected data correlations, and a self-supervised learning strategy with stop-gradient to make the learned physics universal. For the short-timescale network prediction tasks, FlowNN achieves 17% - 71% of loss decrease than the state-of-the-art baselines on both synthetic and real-world networking datasets, which shows the strength of this new approach.
Model Selection in Batch Policy Optimization
Authors: Jonathan N. Lee, George Tucker, Ofir Nachum, Bo Dai
Date: 2021-12-23 00:00:00
Tasks: model, Model Selection

We study the problem of model selection in batch policy optimization: given a fixed, partial-feedback dataset and $M$ model classes, learn a policy with performance that is competitive with the policy derived from the best model class. We formalize the problem in the contextual bandit setting with linear model classes by identifying three sources of error that any model selection algorithm should optimally trade-off in order to be competitive: (1) approximation error, (2) statistical complexity, and (3) coverage. The first two sources are common in model selection for supervised learning, where optimally trading-off these properties is well-studied. In contrast, the third source is unique to batch policy optimization and is due to dataset shift inherent to the setting. We first show that no batch policy optimization algorithm can achieve a guarantee addressing all three simultaneously, revealing a stark contrast between difficulties in batch policy optimization and the positive results available in supervised learning. Despite this negative result, we show that relaxing any one of the three error sources enables the design of algorithms achieving near-oracle inequalities for the remaining two. We conclude with experiments demonstrating the efficacy of these algorithms.
GNN-Geo: A Graph Neural Network-based Fine-grained IP geolocation Framework
Authors: Shichang Ding, Xiangyang Luo, Jinwei Wang, XiaoMing Fu
Date: 2021-12-18 00:00:00
Tasks: Decoder, Graph Neural Network, Node Regression

Rule-based fine-grained IP geolocation methods are hard to generalize in computer networks which do not follow hypothetical rules. Recently, deep learning methods, like multi-layer perceptron (MLP), are tried to increase generalization capabilities. However, MLP is not so suitable for graph-structured data like networks. MLP treats IP addresses as isolated instances and ignores the connection information, which limits geolocation accuracy. In this work, we research how to increase the generalization capability with an emerging graph deep learning method -- Graph Neural Network (GNN). First, IP geolocation is re-formulated as an attributed graph node regression problem. Then, we propose a GNN-based IP geolocation framework named GNN-Geo. GNN-Geo consists of a preprocessor, an encoder, messaging passing (MP) layers and a decoder. The preprocessor and encoder transform measurement data into the initial node embeddings. MP layers refine the initial node embeddings by modeling the connection information. The decoder maps the refined embeddings to nodes' locations and relieves the convergence problem by considering prior knowledge. The experiments in 8 real-world IPv4/IPv6 networks in North America, Europe and Asia show the proposed GNN-Geo clearly outperforms the state-of-art rule-based and learning-based baselines. This work verifies the great potential of GNN for fine-grained IP geolocation.
Face alignment by explicit shape regression
Authors: Xudong Cao, Yichen Wei, Fang Wen, Jian Sun
Date: 2013-12-13 00:00:00
Proceeding: Int J Comput Vis 2013 12
Tasks: Face Alignment, feature selection, regression

We present a very efficient, highly accurate, “Explicit Shape Regression” approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape-indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 minutes for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.
Selective Multiple Power Iteration: from Tensor PCA to gradient-based exploration of landscapes
Authors: Mohamed Ouerfelli, Mohamed Tamaazousti, Vincent Rivasseau
Date: 2021-12-23 00:00:00
Tasks: Tensor Decomposition

We propose Selective Multiple Power Iterations (SMPI), a new algorithm to address the important Tensor PCA problem that consists in recovering a spike $\bf{v_0}^{\otimes k}$ corrupted by a Gaussian noise tensor $\bf{Z} \in (\mathbb{R}^n)^{\otimes k}$ such that $\bf{T}=\sqrt{n} \beta \bf{v_0}^{\otimes k} + \bf{Z}$ where $\beta$ is the signal-to-noise ratio (SNR). SMPI consists in generating a polynomial number of random initializations, performing a polynomial number of symmetrized tensor power iterations on each initialization, then selecting the one that maximizes $\langle \bf{T}, \bf{v}^{\otimes k} \rangle$. Various numerical simulations for $k=3$ in the conventionally considered range $n \leq 1000$ show that the experimental performances of SMPI improve drastically upon existent algorithms and becomes comparable to the theoretical optimal recovery. We show that these unexpected performances are due to a powerful mechanism in which the noise plays a key role for the signal recovery and that takes place at low $\beta$. Furthermore, this mechanism results from five essential features of SMPI that distinguish it from previous algorithms based on power iteration. These remarkable results may have strong impact on both practical and theoretical applications of Tensor PCA. (i) We provide a variant of this algorithm to tackle low-rank CP tensor decomposition. These proposed algorithms also outperforms existent methods even on real data which shows a huge potential impact for practical applications. (ii) We present new theoretical insights on the behavior of SMPI and gradient descent methods for the optimization in high-dimensional non-convex landscapes that are present in various machine learning problems. (iii) We expect that these results may help the discussion concerning the existence of the conjectured statistical-algorithmic gap.
Learning with Proper Partial Labels
Authors: Zhenguo Wu, Jiaqi Lv, Masashi Sugiyama
Date: 2021-12-23 00:00:00
Tasks: Partial Label Learning, Weakly-supervised Learning

Partial-label learning is a kind of weakly-supervised learning with inexact labels, where for each training example, we are given a set of candidate labels instead of only one true label. Recently, various approaches on partial-label learning have been proposed under different generation models of candidate label sets. However, these methods require relatively strong distributional assumptions on the generation models. When the assumptions do not hold, the performance of the methods is not guaranteed theoretically. In this paper, we propose the notion of properness on partial labels. We show that this proper partial-label learning framework requires a weaker distributional assumption and includes many previous partial-label learning settings as special cases. We then derive a unified unbiased estimator of the classification risk. We prove that our estimator is risk-consistent, and we also establish an estimation error bound. Finally, we validate the effectiveness of our algorithm through experiments.
A Robust Initialization of Residual Blocks for Effective ResNet Training without Batch Normalization
Authors: Enrico Civitelli, Alessio Sortino, Matteo Lapucci, Francesco Bagattini, Giulio Galvan
Date: 2021-12-23 00:00:00

Batch Normalization is an essential component of all state-of-the-art neural networks architectures. However, since it introduces many practical issues, much recent research has been devoted to designing normalization-free architectures. In this paper, we show that weights initialization is key to train ResNet-like normalization-free networks. In particular, we propose a slight modification to the summation operation of a block output to the skip-connection branch, so that the whole network is correctly initialized. We show that this modified architecture achieves competitive results on CIFAR-10, CIFAR-100 and ImageNet without further regularization nor algorithmic modifications.
Analysis of ECG data to detect Atrial Fibrillation
Authors: Arjun Sridharkumar, Sai Bhargav, Rahul Guntha
Date: 2021-12-23 00:00:00
Tasks: Rhythm

Atrial fibrillation(termed as AF/Afib henceforth) is a discrete and often rapid heart rhythm that can lead to clots near the heart. We can detect Afib by ECG signal by the absence of p and inconsistent intervals between R waves as shown in fig(1). Existing methods revolve around CNN that are used to detect afib but most of them work with 12 point lead ECG data where in our case the health gauge watch deals with single-point ECG data. Twelve-point lead ECG data is more accurate than a single point. Furthermore, the health gauge watch data is much noisier. Implementing a model to detect Afib for the watch is a test of how the CNN is changed/modified to work with real life data