pwc-archive/papers-with-abstracts huggingface dataset:
Image Classification using CNN for Traffic Signs in Pakistan
Authors: Abdul Azeem Sikander, Hamza Ali
Date: 2021-02-19 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

The autonomous automotive industry is one of the largest and most conventional projects worldwide, with many technology companies effectively designing and orienting their products towards automobile safety and accuracy. These products are performing very well over the roads in developed countries. But can fail in the first minute in an underdeveloped country because there is much difference between a developed country environment and an underdeveloped country environment. The following study proposed to train these Artificial intelligence models in environment space in an underdeveloped country like Pakistan. The proposed approach on image classification uses convolutional neural networks for image classification for the model. For model pre-training German traffic signs data set was selected then fine-tuned on Pakistan's dataset. The experimental setup showed the best results and accuracy from the previously conducted experiments. In this work to increase the accuracy, more dataset was collected to increase the size of images in every class in the data set. In the future, a low number of classes are required to be further increased where more images for traffic signs are required to be collected to get more accuracy on the training of the model over traffic signs of Pakistan's most used and popular roads motorway and national highway, whose traffic signs color, size, and shapes are different from common traffic signs.
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
Authors: Ning Liu, Geng Yuan, Zhengping Che, Xuan Shen, Xiaolong Ma, Qing Jin, Jian Ren, Jian Tang, Sijia Liu, Yanzhi Wang
Date: 2021-02-19 00:00:00
Tasks: Model Compression

In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., a properly pruned sub-network together with original weight initialization) that can achieve competitive performance than the original dense network. However, it is not easy to observe such winning property in many scenarios, where for example, a relatively large learning rate is used even if it benefits training the original dense model. In this work, we investigate the underlying condition and rationale behind the winning property, and find that the underlying reason is largely attributed to the correlation between initialized weights and final-trained weights when the learning rate is not sufficiently large. Thus, the existence of winning property is correlated with an insufficient DNN pretraining, and is unlikely to occur for a well-trained DNN. To overcome this limitation, we propose the "pruning & fine-tuning" method that consistently outperforms lottery ticket sparse training under the same pruning algorithm and the same total training epochs. Extensive experiments over multiple deep models (VGG, ResNet, MobileNet-v2) on different datasets have been conducted to justify our proposals.
A PAC-Bayes Analysis of Adversarial Robustness
Authors: Paul Viallard, Guillaume Vidot, Amaury Habrard, Emilie Morvant
Date: 2021-02-19 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Adversarial Robustness, Generalization Bounds, valid

We propose the first general PAC-Bayesian generalization bounds for adversarial robustness, that estimate, at test time, how much a model will be invariant to imperceptible perturbations in the input. Instead of deriving a worst-case analysis of the risk of a hypothesis over all the possible perturbations, we leverage the PAC-Bayesian framework to bound the averaged risk on the perturbations for majority votes (over the whole class of hypotheses). Our theoretically founded analysis has the advantage to provide general bounds (i) that are valid for any kind of attacks (i.e., the adversarial attacks), (ii) that are tight thanks to the PAC-Bayesian framework, (iii) that can be directly minimized during the learning phase to obtain a robust model on different attacks at test time.
Universal Approximation Theorem for Neural Networks
Authors: Takato Nishijima
Date: 2021-02-19 00:00:00

Is there any theoretical guarantee for the approximation ability of neural networks? The answer to this question is the "Universal Approximation Theorem for Neural Networks". This theorem states that a neural network is dense in a certain function space under an appropriate setting. This paper is a comprehensive explanation of the universal approximation theorem for feedforward neural networks, its approximation rate problem (the relation between the number of intermediate units and the approximation error), and Barron space in Japanese.
High-order Differentiable Autoencoder for Nonlinear Model Reduction
Authors: Siyuan Shen, Yang Yin, Tianjia Shao, He Wang, Chenfanfu Jiang, Lei Lan, Kun Zhou
Date: 2021-02-19 00:00:00
Tasks: GPU, model, Vocal Bursts Intensity Prediction

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton's method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., $10+$ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable \emph{weight-varying} Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.
Online Learning via Offline Greedy Algorithms: Applications in Market Design and Optimization
Authors: Rad Niazadeh, Negin Golrezaei, Joshua Wang, Fransisca Susan, Ashwinkumar Badanidiyuru
Date: 2021-02-18 00:00:00
Tasks: Decision Making, Management

Motivated by online decision-making in time-varying combinatorial environments, we study the problem of transforming offline algorithms to their online counterparts. We focus on offline combinatorial problems that are amenable to a constant factor approximation using a greedy algorithm that is robust to local errors. For such problems, we provide a general framework that efficiently transforms offline robust greedy algorithms to online ones using Blackwell approachability. We show that the resulting online algorithms have $O(\sqrt{T})$ (approximate) regret under the full information setting. We further introduce a bandit extension of Blackwell approachability that we call Bandit Blackwell approachability. We leverage this notion to transform greedy robust offline algorithms into a $O(T^{2/3})$ (approximate) regret in the bandit setting. Demonstrating the flexibility of our framework, we apply our offline-to-online transformation to several problems at the intersection of revenue management, market design, and online optimization, including product ranking optimization in online platforms, reserve price optimization in auctions, and submodular maximization. We also extend our reduction to greedy-like first order methods used in continuous optimization, such as those used for maximizing continuous strong DR monotone submodular functions subject to convex constraints. We show that our transformation, when applied to these applications, leads to new regret bounds or improves the current known bounds. We complement our theoretical studies by conducting numerical simulations for two of our applications, in both of which we observe that the numerical performance of our transformations outperforms the theoretical guarantees in practical instances.
JST-RR Model: Joint Modeling of Ratings and Reviews in Sentiment-Topic Prediction
Authors: Qiao Liang, Shyam Ranganathan, Kaibo Wang, Xinwei Deng
Date: 2021-02-18 00:00:00
Tasks: Prediction

Analysis of online reviews has attracted great attention with broad applications. Often times, the textual reviews are coupled with the numerical ratings in the data. In this work, we propose a probabilistic model to accommodate both textual reviews and overall ratings with consideration of their intrinsic connection for a joint sentiment-topic prediction. The key of the proposed method is to develop a unified generative model where the topic modeling is constructed based on review texts and the sentiment prediction is obtained by combining review texts and overall ratings. The inference of model parameters are obtained by an efficient Gibbs sampling procedure. The proposed method can enhance the prediction accuracy of review data and achieve an effective detection of interpretable topics and sentiments. The merits of the proposed method are elaborated by the case study from Amazon datasets and simulation studies.
Semantic Parsing to Manipulate Relational Database For a Management System
Authors: Muhammad Hamzah Mushtaq
Date: 2021-02-18 00:00:00
Tasks: Management, NER, Semantic Parsing

Chatbots and AI assistants have claimed their importance in today life. The main reason behind adopting this technology is to connect with the user, understand their requirements, and fulfill them. This has been achieved but at the cost of heavy training data and complex learning models. This work is carried out proposes a simple algorithm, a model which can be implemented in different fields each with its own work scope. The proposed model converts human language text to computer-understandable SQL queries. The model requires data only related to the specific field, saving data space. This model performs linear computation hence solving the computational complexity. This work also defines the stages where a new methodology is implemented and what previous method was adopted to fulfill the requirement at that stage. Two datasets available online will be used in this work, the ATIS dataset, and WikiSQL. This work compares the computation time among the 2 datasets and also compares the accuracy of both. This paper works over basic Natural language processing tasks like semantic parsing, NER, parts of speech and tends to achieve results through these simple methods.
Efficient Multi-stream Temporal Learning and Post-fusion Strategy for 3D Skeleton-based Hand Activity Recognition
Authors: Yasser Boutaleb, Catherine Soladie, Nam-Duong Duong, Amine Kacete, Jérôme Royan, Renaud Seguier
Date: 2021-02-10 00:00:00
Tasks: Activity Recognition, Skeleton Based Action Recognition

Recognizing first-person hand activity is a challenging task, especially when not enough data are available. In this paper, we tackle this challenge by proposing a new hybrid learning pipeline for skeleton-based hand activity recognition, which is composed of three blocks. First, for a given sequence of hand’s joint positions, the spatial features are extracted using a dedicated combination of local and global spatial hand-crafted features. Then, the temporal dependencies are learned using a multi-stream learning strategy. Finally, a hand activity sequence classifier is learned, via our Post-fusion strategy, applied to the previously learned temporal dependencies. The experiments, evaluated on two real-world data sets, show that our approach performs better than the state-of-the-art approaches. For more ablation studies, we compared our Post-fusion strategy with three traditional fusion baselines and showed an improvement above 2.4% of accuracy.
Tetrameric UvrD helicase is located at the E. coli replisome due to frequent replication blocks
Authors: Adam J. M Wollman, Aisha H. Syeda, Andrew Leech, Colin Guy, Peter McGlynn, Michelle Hawkins, Mark C. Leake
Date: 2021-02-22 00:00:00

DNA replication in all organisms must overcome nucleoprotein blocks to complete genome duplication. Accessory replicative helicases in Escherichia coli, Rep and UvrD, help replication machinery overcome blocks by removing incoming nucleoprotein complexes or aiding the re-initiation of replication. Mechanistic details of Rep function have emerged from recent live cell studies, however, the activities of UvrD in vivo remain unclear. Here, by integrating biochemical analysis and super-resolved single-molecule fluorescence microscopy, we discovered that UvrD self-associates into a tetramer and, unlike Rep, is not recruited to a specific replisome protein despite being found at approximately 80% of replication forks. By deleting rep and DNA repair factors mutS and uvrA, perturbing transcription by mutating RNA polymerase, and antibiotic inhibition; we show that the presence of UvrD at the fork is dependent on its activity. This is likely mediated by the very high frequency of replication blocks due to DNA bound proteins, including RNA polymerase, and DNA damage. UvrD is recruited to sites of nucleoprotein blocks via distinctly different mechanisms to Rep and therefore plays a more important and complementary role than previously realised in ensuring successful DNA replication.
On Stability and Convergence of Distributed Filters
Authors: Sayed Pouria Talebi, Stefan Werner, Vijay Gupta, Yih-Fang Huang
Date: 2021-02-22 00:00:00

Recent years have bore witness to the proliferation of distributed filtering techniques, where a collection of agents communicating over an ad-hoc network aim to collaboratively estimate and track the state of a system. These techniques form the enabling technology of modern multi-agent systems and have gained great importance in the engineering community. Although most distributed filtering techniques come with a set of stability and convergence criteria, the conditions imposed are found to be unnecessarily restrictive. The paradigm of stability and convergence in distributed filtering is revised in this manuscript. Accordingly, a general distributed filter is constructed and its estimation error dynamics is formulated. The conducted analysis demonstrates that conditions for achieving stable filtering operations are the same as those required in the centralized filtering setting. Finally, the concepts are demonstrated in a Kalman filtering framework and validated using simulation examples.
On Classifying the Effects of Policy Announcements on Volatility
Authors: Giampiero M. Gallo, Demetrio Lacava, Edoardo Otranto
Date: 2020-11-28 00:00:00
Tasks: Classification, Clustering, General Classification

The financial turmoil surrounding the Great Recession called for unprecedented intervention by Central Banks: unconventional policies affected various areas in the economy, including stock market volatility. In order to evaluate such effects, by including Markov Switching dynamics within a recent Multiplicative Error Model, we propose a model--based classification of the dates of a Central Bank's announcements to distinguish the cases where the announcement implies an increase or a decrease in volatility, or no effect. In detail, we propose two smoothed probability--based classification methods, obtained as a by--product of the model estimation, which provide very similar results to those coming from a classical k--means clustering procedure. The application on four Eurozone market volatility series shows a successful classification of 144 European Central Bank announcements.
Community Structure Recovery and Interaction Probability Estimation for Gossip Opinion Dynamics
Authors: Yu Xing, Xingkang He, Haitao Fang, Karl H. Johansson
Date: 2021-02-19 00:00:00
Tasks: Community Detection

We study how to jointly recover the community structure and estimate the interaction probabilities of gossip opinion dynamics. In this process, agents randomly interact pairwise, and there are stubborn agents never changing their states. Such a model illustrates how disagreement and opinion fluctuation arise in a social network. It is assumed that each agent is assigned with one of two community labels, and the agents interact with probabilities depending on their labels. The considered problem is to jointly recover the community labels of the agents and estimate interaction probabilities between the agents, based on a single trajectory of the model. We first study stability and limit theorems of the model, and then propose a joint recovery and estimation algorithm based on a trajectory. It is verified that the community recovery can be achieved in finite time, and the interaction estimator converges almost surely. We derive a sample-complexity result for the recovery, and analyze the estimator's convergence rate. Simulations are presented for illustration of the performance of the proposed algorithm.
Mixed-Precision Quantization and Parallel Implementation of Multispectral Riemannian Classification for Brain--Machine Interfaces
Authors: Xiaying Wang, Tibor Schneider, Michael Hersche, Lukas Cavigelli, Luca Benini
Date: 2021-02-22 00:00:00
Tasks: General Classification, Motor Imagery, Quantization

With Motor-Imagery (MI) Brain--Machine Interfaces (BMIs) we may control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost tradeoff for embedded BMI solutions. Our proposed Multispectral Riemannian Classifier reaches 75.1% accuracy on 4-class MI task. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1%, which is still 3.2% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU with parallel processing units taking only 33.39ms and consuming 1.304mJ per classification.
Coherence of Working Memory Study Between Deep Neural Network and Neurophysiology
Authors: Yurui Ming
Date: 2021-02-06 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

The auto feature extraction capability of deep neural networks (DNN) endows them the potentiality for analysing complicated electroencephalogram (EEG) data captured from brain functionality research. This work investigates the potential coherent correspondence between the region-of-interest (ROI) for DNN to explore, and ROI for conventional neurophysiological oriented methods to work with, exemplified in the case of working memory study. The attention mechanism induced by global average pooling (GAP) is applied to a public EEG dataset of working memory, to unveil these coherent ROIs via a classification problem. The result shows the alignment of ROIs from different research disciplines. This work asserts the confidence and promise of utilizing DNN for EEG data analysis, albeit in lack of the interpretation to network operations.
Towards Personalised and Document-level Machine Translation of Dialogue
Authors: Sebastian T. Vincent
Date: 2021-02-11 00:00:00
Tasks: Document Level Machine Translation, Machine Translation, NMT, Sentence, Translation

State-of-the-art (SOTA) neural machine translation (NMT) systems translate texts at sentence level, ignoring context: intra-textual information, like the previous sentence, and extra-textual information, like the gender of the speaker. Because of that, some sentences are translated incorrectly. Personalised NMT (PersNMT) and document-level NMT (DocNMT) incorporate this information into the translation process. Both fields are relatively new and previous work within them is limited. Moreover, there are no readily available robust evaluation metrics for them, which makes it difficult to develop better systems, as well as track global progress and compare different methods. This thesis proposal focuses on PersNMT and DocNMT for the domain of dialogue extracted from TV subtitles in five languages: English, Brazilian Portuguese, German, French and Polish. Three main challenges are addressed: (1) incorporating extra-textual information directly into NMT systems; (2) improving the machine translation of cohesion devices; (3) reliable evaluation for PersNMT and DocNMT.
Word frequency-rank relationship in tagged texts
Authors: A. Chacoma, D. H. Zanette
Date: 2021-02-07 00:00:00

We analyze the frequency-rank relationship in sub-vocabularies corresponding to three different grammatical classes (nouns, verbs, and others) in a collection of literary works in English, whose words have been automatically tagged according to their grammatical role. Comparing with a null hypothesis which assumes that words belonging to each class are uniformly distributed across the frequency-ranked vocabulary of the whole work, we disclose statistically significant differences between the three classes. This results point to the fact that frequency-rank relationships may reflect linguistic features associated with grammatical function.
An open access NLP dataset for Arabic dialects : Data collection, labeling, and model construction
Authors: ElMehdi Boujou, Hamza Chataoui, Abdellah El Mekki, Saad Benjelloun, Ikram Chairi, Ismail Berrada
Date: 2021-02-07 00:00:00
Tasks: Sentiment Analysis

Natural Language Processing (NLP) is today a very active field of research and innovation. Many applications need however big sets of data for supervised learning, suitably labelled for the training purpose. This includes applications for the Arabic language and its national dialects. However, such open access labeled data sets in Arabic and its dialects are lacking in the Data Science ecosystem and this lack can be a burden to innovation and research in this field. In this work, we present an open data set of social data content in several Arabic dialects. This data was collected from the Twitter social network and consists on +50K twits in five (5) national dialects. Furthermore, this data was labeled for several applications, namely dialect detection, topic detection and sentiment analysis. We publish this data as an open access data to encourage innovation and encourage other works in the field of NLP for Arabic dialects and social media. A selection of models were built using this data set and are presented in this paper along with their performances.