pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive Neuro Particle Swarm Optimization applied for diagnosing disorders
Authors: Majid Masoumi, Mina Rajabi
Date: 2019-10-14 00:00:00

A new Adaptive Neuro Particle Swarm Optimization (ANPSO) combined with a fuzzy inference system for diagnosing disorders is presented in this paper. The main contributions of the novel proposed method can be a global search across the whole search space with faster convergence rate. Moreover, it shows a better exploration and exploitation by applying the adaptive control parameters, automatic control of inertia weight and coefficient of personal and social behaviours. Utilizing such attributes lead to a fast and smart diagnosis mechanism which is able to diagnosis the diseases by the high accuracy. The ANPSO is associated with tuning the characteristics of the inference system to achieve the minimum diagnosis error as far as the optimized model is obtained. As a case study, we use liver disorders dataset called Bupa. According to the preliminary ramifications, the suggested adaptive PSO performance can overcome the traditional inference system and combined with other optimization methods substantially.
Scalable High Performance SDN Switch Architecture on FPGA for Core Networks
Authors: Sasindu Wijeratne, Ashen Ekanayake, Sandaruwan Jayaweera, Danuka Ravishan, Ajith Pasqual
Date: 2019-10-30 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Due to the increasing heterogeneity in network user requirements, dynamically varying day to day network traffic patterns and delay in-network service deployment, there is a huge demand for scalability and flexibility in modern networking infrastructure, which in return has paved way for the introduction of Software Defined Networking (SDN) in core networks. In this paper, we present an FPGA-based switch that is fully compliant with OpenFlow; the pioneering protocol for southbound interface of SDN. The switch architecture is completely implemented on hardware. The design consists of an OpenFlow Southbound agent which can process OpenFlow packets at a rate of 10Gbps. The proposed architecture speed scales up to 400Gbps while it consumes only 60% of resources on a Xilinx Virtex-7 featuring XC7VX485T FPGA.
Uplink-Downlink Tradeoff in Secure Distributed Matrix Multiplication
Authors: Jaber Kakar, Anton Khristoforov, Seyedhamed Ebadifar, Aydin Sezgin
Date: 2019-10-30 00:00:00

In secure distributed matrix multiplication (SDMM) the multiplication $\mathbf{A}\mathbf{B}$ from two private matrices $\mathbf{A}$ and $\mathbf{B}$ is outsourced by a user to $N$ distributed servers. In $\ell$-SDMM, the goal is to a design a joint communication-computation procedure that optimally balances conflicting communication and computation metrics without leaking any information on both $\mathbf{A}$ and $\mathbf{B}$ to any set of $\ell\leq N$ servers. To this end, the user applies coding with $\tilde{\mathbf{A}}_i$ and $\tilde{\mathbf{B}}_i$ representing encoded versions of $\mathbf{A}$ and $\mathbf{B}$ destined to the $i$-th server. Now, SDMM involves multiple tradeoffs. One such tradeoff is the tradeoff between uplink (UL) and downlink (DL) costs. To find a good balance between these two metrics, we propose two schemes which we term USCSA and GSCSA that are based on secure cross subspace alignment (SCSA). We show that there are various scenarios where they outperform existing SDMM schemes from the literature with respect to the UL-DL efficiency. Next, we implement schemes from the literature, including USCSA and GSCSA, and test their performance on Amazon EC2. Our numerical results show that USCSA and GSCSA establish a good balance between the time spend on the communication and computation in SDMMs. This is because they combine advantages of polynomial codes, namely low time for the upload of $\left(\tilde{\mathbf{A}}_i,\tilde{\mathbf{B}}_i\right)_{i=1}^{N}$ and the computation of $\mathbf{O}_i=\tilde{\mathbf{A}}_i\tilde{\mathbf{B}}_i$, with those of SCSA, being a low timing overhead for the download of $\left(\mathbf{O}_i\right)_{i=1}^{N}$ and the decoding of $\mathbf{A}\mathbf{B}$.
Review highlights: opinion mining on reviews: a hybrid model for rule selection in aspect extraction
Authors: Amit Kushwaha, Shubham Chaudhary
Date: 2017-10-18 00:00:00
Proceeding: IML ’17, October 17–18, 2017, Liverpool, United Kingdom 2017 10
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Extraction, BIG-bench Machine Learning, Chunking, Extract Aspect, Extract aspect-polarity tuple, Opinion Mining, Sentiment Analysis

This paper proposes a methodology to extract key insights from user generated reviews. This work is based on Aspect Based Sentiment Analysis (ABSA) which predicts the sentiment of aspects mentioned in the text documents. The extracted aspects are fine-grained for the presentation form known as Review Highlights.

The syntactic approach for extraction process suffers from the overlapping chunking rules which result in noise extraction. We introduce a hybrid technique which combines machine learning and rule based model. A multi-label classifier identifies the effective rules which efficiently parse aspects and opinions from texts. This selection of rules reduce the amount of noise in extraction tasks.

This is a novel attempt to learn syntactic rule fitness from a corpus using machine learning for accurate aspect extraction. As the model learns the syntactic rule prediction from the corpus, it makes the extraction method domain independent. It also allows studying the quality of syntactic rules in a different corpus.
Co-Generation with GANs using AIS based HMC
Authors: Tiantian Fang, Alexander G. Schwing
Date: 2019-10-31 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Structured Prediction

Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones - which we refer to as co-generation - is an important challenge that is computationally demanding for all but the simplest settings. This task has received a considerable amount of attention, particularly for classical ways of modeling distributions like structured prediction. In contrast, almost nothing is known about this task when considering recently proposed techniques for modeling high-dimensional distributions, particularly generative adversarial nets (GANs). Therefore, in this paper, we study the occurring challenges for co-generation with GANs. To address those challenges we develop an annealed importance sampling based Hamiltonian Monte Carlo co-generation algorithm. The presented approach significantly outperforms classical gradient based methods on a synthetic and on the CelebA and LSUN datasets.
Graph Structured Prediction Energy Networks
Authors: Colin Graber, Alexander Schwing
Date: 2019-10-31 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Prediction, Structured Prediction

For joint inference over multiple variables, a variety of structured prediction techniques have been developed to model correlations among variables and thereby improve predictions. However, many classical approaches suffer from one of two primary drawbacks: they either lack the ability to model high-order correlations among variables while maintaining computationally tractable inference, or they do not allow to explicitly model known correlations. To address this shortcoming, we introduce `Graph Structured Prediction Energy Networks,' for which we develop inference techniques that allow to both model explicit local and implicit higher-order correlations while maintaining tractability of inference. We apply the proposed method to tasks from the natural language processing and computer vision domain and demonstrate its general utility.
TAB-VCR: Tags and Attributes based Visual Commonsense Reasoning Baselines
Authors: Jingxiang Lin, Unnat Jain, Alexander G. Schwing
Date: 2019-10-31 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Attribute, Question Answering, Visual Commonsense Reasoning, Visual Dialog, Visual Question Answering, Visual Question Answering (VQA)

Reasoning is an important ability that we learn from a very early age. Yet, reasoning is extremely hard for algorithms. Despite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets. To develop models with better reasoning abilities, recently, the new visual commonsense reasoning (VCR) task has been introduced. Not only do models have to answer questions, but also do they have to provide a reason for the given answer. The proposed baseline achieved compelling results, leveraging a meticulously designed model composed of LSTM modules and attention nets. Here we show that a much simpler model obtained by ablating and pruning the existing intricate baseline can perform better with half the number of trainable parameters. By associating visual features with attribute information and better text to image grounding, we obtain further improvements for our simpler & effective baseline, TAB-VCR. We show that this approach results in a 5.3%, 4.4% and 6.5% absolute improvement over the previous state-of-the-art on question answering, answer justification and holistic VCR.
Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors
Authors: Zuxuan Wu, Ser-Nam Lim, Larry Davis, Tom Goldstein
Date: 2019-10-31 00:00:00
Proceeding: ECCV 2020 8
Tasks: Object, object-detection, Object Detection

We present a systematic study of adversarial attacks on state-of-the-art object detection frameworks. Using standard detection datasets, we train patterns that suppress the objectness scores produced by a range of commonly used detectors, and ensembles of detectors. Through extensive experiments, we benchmark the effectiveness of adversarially trained patches under both white-box and black-box settings, and quantify transferability of attacks between datasets, object classes, and detector models. Finally, we present a detailed study of physical world attacks using printed posters and wearable clothes, and rigorously quantify the performance of such attacks with different metrics.
Masked Language Model Scoring
Authors: Julian Salazar, Davis Liang, Toan Q. Nguyen, Katrin Kirchhoff
Date: 2019-10-31 00:00:00
Proceeding: ACL 2020 6
Tasks: Attribute, Domain Adaptation, Language Modeling, Language Modelling, Linguistic Acceptability, model, NMT, Translation

Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are computed by masking tokens one by one. We show that PLLs outperform scores from autoregressive language models like GPT-2 in a variety of tasks. By rescoring ASR and NMT hypotheses, RoBERTa reduces an end-to-end LibriSpeech model's WER by 30% relative and adds up to +1.7 BLEU on state-of-the-art baselines for low-resource translation pairs, with further gains from domain adaptation. We attribute this success to PLL's unsupervised expression of linguistic acceptability without a left-to-right bias, greatly improving on scores from GPT-2 (+10 points on island effects, NPI licensing in BLiMP). One can finetune MLMs to give scores without masking, enabling computation in a single inference pass. In all, PLLs and their associated pseudo-perplexities (PPPLs) enable plug-and-play use of the growing number of pretrained MLMs; e.g., we use a single cross-lingual model to rescore translations in multiple languages. We release our library for language model scoring at https://github.com/awslabs/mlm-scoring.
Enhancing Certifiable Robustness via a Deep Model Ensemble
Authors: Huan Zhang, Minhao Cheng, Cho-Jui Hsieh
Date: 2019-10-31 00:00:00
Tasks: Model Selection

We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model. Unlike previous works on using ensembles to empirically improve robustness, our algorithm is based on optimizing a guaranteed robustness certificate of neural networks. Our proposed ensemble framework with certified robustness, RobBoost, formulates the optimal model selection and weighting task as an optimization problem on a lower bound of classification margin, which can be efficiently solved using coordinate descent. Experiments show that our algorithm can form a more robust ensemble than naively averaging all available models using robustly trained MNIST or CIFAR base models. Additionally, our ensemble typically has better accuracy on clean (unperturbed) data. RobBoost allows us to further improve certified robustness and clean accuracy by creating an ensemble of already certified models.
Which Factors Impact Engagement on News Articles on Facebook?
Authors: Marc Faddoul
Date: 2019-10-31 00:00:00
Tasks: Articles

Social media is increasingly being used as a news-platform. To reach their intended audience, newspapers need for their articles to be well ranked by Facebook's news-feed algorithm. The number of likes, shares and other reactions determine the lead scoring criteria. This paper will try to assess how the reaction volume is impacted by the following criteria: (1) Delay between event and post release; (2) Time of the day the post is published; and (3) Post format: video, photo or text. To isolate the effect of the publication time and post format on a post, we need to control for the news-event and the publishing newspaper. For that end, a news-aggregator is designed and implemented, to group together posts that relate to the same news-event. This tool gave some spin-off results, allowing the ability to map newspapers by similarity and to detect some topic omissions.
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators
Authors: Reinhard Heckel, Mahdi Soltanolkotabi
Date: 2019-10-31 00:00:00
Proceeding: ICLR 2020 1
Tasks: Attribute, Denoising, Image Generation

Convolutional Neural Networks (CNNs) have emerged as highly successful tools for image generation, recovery, and restoration. A major contributing factor to this success is that convolutional networks impose strong prior assumptions about natural images. A surprising experiment that highlights this architectural bias towards natural images is that one can remove noise and corruptions from a natural image without using any training data, by simply fitting (via gradient descent) a randomly initialized, over-parameterized convolutional generator to the corrupted image. While this over-parameterized network can fit the corrupted image perfectly, surprisingly after a few iterations of gradient descent it generates an almost uncorrupted image. This intriguing phenomenon enables state-of-the-art CNN-based denoising and regularization of other inverse problems. In this paper, we attribute this effect to a particular architectural choice of convolutional networks, namely convolutions with fixed interpolating filters. We then formally characterize the dynamics of fitting a two-layer convolutional generator to a noisy signal and prove that early-stopped gradient descent denoises/regularizes. Our proof relies on showing that convolutional generators fit the structured part of an image significantly faster than the corrupted portion.
An Automatic Design Framework of Swarm Pattern Formation based on Multi-objective Genetic Programming
Authors: Zhun Fan, Zhaojun Wang, Xiaomin Zhu, Bingliang Hu, Anmin Zou, Dongwei Bao
Date: 2019-10-31 00:00:00

Most existing swarm pattern formation methods depend on a predefined gene regulatory network (GRN) structure that requires designers' priori knowledge, which is difficult to adapt to complex and changeable environments. To dynamically adapt to the complex and changeable environments, we propose an automatic design framework of swarm pattern formation based on multi-objective genetic programming. The proposed framework does not need to define the structure of the GRN-based model in advance, and it applies some basic network motifs to automatically structure the GRN-based model. In addition, a multi-objective genetic programming (MOGP) combines with NSGA-II, namely MOGP-NSGA-II, to balance the complexity and accuracy of the GRN-based model. In evolutionary process, an MOGP-NSGA-II and differential evolution (DE) are applied to optimize the structures and parameters of the GRN-based model in parallel. Simulation results demonstrate that the proposed framework can effectively evolve some novel GRN-based models, and these GRN-based models not only have a simpler structure and a better performance, but also are robust to the complex and changeable environments.
Mixing of Stochastic Accelerated Gradient Descent
Authors: Peiyuan Zhang, Hadi Daneshmand, Thomas Hofmann
Date: 2019-10-31 00:00:00
Tasks: Stochastic Optimization

We study the mixing properties for stochastic accelerated gradient descent (SAGD) on least-squares regression. First, we show that stochastic gradient descent (SGD) and SAGD are simulating the same invariant distribution. Motivated by this, we then establish mixing rate for SAGD-iterates and compare it with those of SGD-iterates. Theoretically, we prove that the chain of SAGD iterates is geometrically ergodic --using a proper choice of parameters and under regularity assumptions on the input distribution. More specifically, we derive an explicit mixing rate depending on the first 4 moments of the data distribution. By means of illustrative examples, we prove that SAGD-iterate chain mixes faster than the chain of iterates obtained by SGD. Furthermore, we highlight applications of the established mixing rate in the convergence analysis of SAGD on realizable objectives. The proposed analysis is based on a novel non-asymptotic analysis of products of random matrices. This theoretical result is substantiated and validated by experiments.
Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning
Authors: Arvind Neelakantan, Semih Yavuz, Sharan Narang, Vishaal Prasad, Ben Goodrich, Daniel Duckworth, Chinnadhurai Sankar, Xifeng Yan
Date: 2019-10-31 00:00:00
Tasks: Response Generation, Retrieval, Text Generation

Task-oriented dialog presents a difficult challenge encompassing multiple problems including multi-turn language understanding and generation, knowledge retrieval and reasoning, and action prediction. Modern dialog systems typically begin by converting conversation history to a symbolic object referred to as belief state by using supervised learning. The belief state is then used to reason on an external knowledge source whose result along with the conversation history is used in action prediction and response generation tasks independently. Such a pipeline of individually optimized components not only makes the development process cumbersome but also makes it non-trivial to leverage session-level user reinforcement signals. In this paper, we develop Neural Assistant: a single neural network model that takes conversation history and an external knowledge source as input and jointly produces both text response and action to be taken by the system as output. The model learns to reason on the provided knowledge source with weak supervision signal coming from the text generation and the action prediction tasks, hence removing the need for belief state annotations. In the MultiWOZ dataset, we study the effect of distant supervision, and the size of knowledge base on model performance. We find that the Neural Assistant without belief states is able to incorporate external knowledge information achieving higher factual accuracy scores compared to Transformer. In settings comparable to reported baseline systems, Neural Assistant when provided with oracle belief state significantly improves language generation performance.
Can adversarial training learn image captioning ?
Authors: Jean-Benoit Delbrouck, Bastien Vanderplaetse, Stéphane Dupont
Date: 2019-10-31 00:00:00
Tasks: Image Captioning, Text Generation

Recently, generative adversarial networks (GAN) have gathered a lot of interest. Their efficiency in generating unseen samples of high quality, especially images, has improved over the years. In the field of Natural Language Generation (NLG), the use of the adversarial setting to generate meaningful sentences has shown to be difficult for two reasons: the lack of existing architectures to produce realistic sentences and the lack of evaluation tools. In this paper, we propose an adversarial architecture related to the conditional GAN (cGAN) that generates sentences according to a given image (also called image captioning). This attempt is the first that uses no pre-training or reinforcement methods. We also explain why our experiment settings can be safely evaluated and interpreted for further works.
Adversarial NLI: A New Benchmark for Natural Language Understanding
Authors: Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, Douwe Kiela
Date: 2019-10-31 00:00:00
Proceeding: ACL 2020 6
Tasks: Natural Language Understanding

We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure. We show that training models on this new dataset leads to state-of-the-art performance on a variety of popular NLI benchmarks, while posing a more difficult challenge with its new test set. Our analysis sheds light on the shortcomings of current state-of-the-art models, and shows that non-expert annotators are successful at finding their weaknesses. The data collection method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate.
Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
Authors: Luca Della Libera, Vladimir Golkov, Yue Zhu, Arman Mielke, Daniel Cremers
Date: 2019-10-31 00:00:00
Tasks: Motion Estimation

Convolutional networks are successful due to their equivariance/invariance under translations. However, rotatable data such as images, volumes, shapes, or point clouds require processing with equivariance/invariance under rotations in cases where the rotational orientation of the coordinate system does not affect the meaning of the data (e.g. object classification). On the other hand, estimation/processing of rotations is necessary in cases where rotations are important (e.g. motion estimation). There has been recent progress in methods and theory in all these regards. Here we provide an overview of existing methods, both for 2D and 3D rotations (and translations), and identify commonalities and links between them.
Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness
Authors: Alexandre Bérard, Ioan Calapodescu, Marc Dymetman, Claude Roux, Jean-Luc Meunier, Vassilina Nikoulina
Date: 2019-10-31 00:00:00
Proceeding: WS 2019 11
Tasks: Domain Adaptation, Machine Translation, Sentiment Analysis, Translation

We share a French-English parallel corpus of Foursquare restaurant reviews (https://europe.naverlabs.com/research/natural-language-processing/machine-translation-of-restaurant-reviews), and define a new task to encourage research on Neural Machine Translation robustness and domain adaptation, in a real-world scenario where better-quality MT would be greatly beneficial. We discuss the challenges of such user-generated content, and train good baseline models that build upon the latest techniques for MT robustness. We also perform an extensive evaluation (automatic and human) that shows significant improvements over existing online systems. Finally, we propose task-specific metrics based on sentiment analysis or translation accuracy of domain-specific polysemous words.