pwc-archive/papers-with-abstracts huggingface dataset:
Comparative Analysis of Machine Learning and Deep Learning Algorithms for Detection of Online Hate Speech
Authors: Tashvik Dhamija, Anjum, Rahul Katarya
Date: 2021-04-23 00:00:00
Tasks: Feature Engineering, Sentence, Sentence Embeddings

In the day and age of social media, users have become prone to online hate speech. Several attempts have been made to classify hate speech using machine learning but the state-of-the-art models are not robust enough for practical applications. This is attributed to the use of primitive NLP feature engineering techniques. In this paper, we explored various feature engineering techniques ranging from different embeddings to conventional NLP algorithms. We also experimented with combinations of different features. From our experimentation, we realized that roBERTa (robustly optimized BERT approach) based sentence embeddings classified using decision trees gives the best results of 0.9998 F1 score. In our paper, we concluded that BERT based embeddings give the most useful features for this problem and have the capacity to be made into a practical robust model.
Subgraph-aware Few-Shot Inductive Link Prediction via Meta-Learning
Authors: Shuangjia Zheng, Sijie Mai, Ya Sun, Haifeng Hu, Yuedong Yang
Date: 2021-07-26 00:00:00
Tasks: Inductive Link Prediction, Knowledge Graphs, Link Prediction, Meta-Learning, Relation, Triplet

Link prediction for knowledge graphs aims to predict missing connections between entities. Prevailing methods are limited to a transductive setting and hard to process unseen entities. The recent proposed subgraph-based models provided alternatives to predict links from the subgraph structure surrounding a candidate triplet. However, these methods require abundant known facts of training triplets and perform poorly on relationships that only have a few triplets. In this paper, we propose Meta-iKG, a novel subgraph-based meta-learner for few-shot inductive relation reasoning. Meta-iKG utilizes local subgraphs to transfer subgraph-specific information and learn transferable patterns faster via meta gradients. In this way, we find the model can quickly adapt to few-shot relationships using only a handful of known facts with inductive settings. Moreover, we introduce a large-shot relation update procedure to traditional meta-learning to ensure that our model can generalize well both on few-shot and large-shot relations. We evaluate Meta-iKG on inductive benchmarks sampled from NELL and Freebase, and the results show that Meta-iKG outperforms the current state-of-the-art methods both in few-shot scenarios and standard inductive settings.
Analysis of Online Toxicity Detection Using Machine Learning Approaches
Authors: Anjum, Rahul Katarya
Date: 2021-04-23 00:00:00
Tasks: BIG-bench Machine Learning

Social media and the internet have become an integral part of how people spread and consume information. Over a period of time, social media evolved dramatically, and almost half of the population is using social media to express their views and opinions. Online hate speech is one of the drawbacks of social media nowadays, which needs to be controlled. In this paper, we will understand how hate speech originated and what are the consequences of it; Trends of machine-learning algorithms to solve an online hate speech problem. This study contributes by providing a systematic approach to help researchers to identify a new research direction and elucidating the shortcomings of the studies and model, as well as providing future directions to advance the field.
Automated News Summarization Using Transformers
Authors: Anushka Gupta, Diksha Chugh, Anjum, Rahul Katarya
Date: 2021-04-23 00:00:00
Tasks: Abstractive Text Summarization, Articles, BIG-bench Machine Learning, Extractive Summarization, News Summarization, text-classification, Text Classification, Text Summarization

The amount of text data available online is increasing at a very fast pace hence text summarization has become essential. Most of the modern recommender and text classification systems require going through a huge amount of data. Manually generating precise and fluent summaries of lengthy articles is a very tiresome and time-consuming task. Hence generating automated summaries for the data and using it to train machine learning models will make these models space and time-efficient. Extractive summarization and abstractive summarization are two separate methods of generating summaries. The extractive technique identifies the relevant sentences from the original document and extracts only those from the text. Whereas in abstractive summarization techniques, the summary is generated after interpreting the original text, hence making it more complicated. In this paper, we will be presenting a comprehensive comparison of a few transformer architecture based pre-trained models for text summarization. For analysis and comparison, we have used the BBC news dataset that contains text data that can be used for summarization and human generated summaries for evaluating and comparing the summaries generated by machine learning models.
Linking Attention-Based Multiscale CNN With Dynamical GCN for Driving Fatigue Detection
Authors: Hongtao Wang, Linfeng Xu, Anastasios Bezerianos, Chuangquan Chen, Zhiguo Zhang
Date: 2020-12-26 00:00:00
Proceeding: IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021 2020 12
Tasks: EEG, Electroencephalogram (EEG)

Electroencephalography (EEG) signals have been proven to be one of the most predictive and reliable indicators for estimating driving fatigue state. However, how to make full use of EEG data for driving fatigue detection remains a challenge.Many existing methods include a time-consuming manual process or tedious parameter tunings for feature extraction, which is inconvenient to train and implement. On the other hand, most models ignore or manually determine EEG connectivity features between different channels, thus failing to thoroughly exploit the intrinsic interchannel relations for classification. In this article, we introduce a new attention-based multiscale convolutional neural network-dynamical graph convolutional network (AMCNN-DGCN) model, aiming to conquer these two issues in a unified end-to-end model. AMCNN-DGCN starts with attention-based multiscale temporal convolutions to automatically learn frequency filters to extract the salient pattern from raw EEG data. Subsequently, AMCNN-DGCN uses dynamical graph convolutional networks (DGCNs) to learn spatial filters, in which the adjacency matrix is adaptively determined in a data-driven way to exploit the intrinsic relationship between channels effectively. With the temporal–spatial structure, AMCNN-DGCN can capture highly discriminative features. To verify the effectiveness of AMCNN-DGCN, we conduct a simulated fatigue driving environment to collect EEG signals from 29 healthy subjects (male/female = 17/12 and age = 23.28±2.70 years) through a remote wireless cap with 24 channels. The results demonstrate that our proposed model outperforms six widely used competitive EEG models with high accuracy of 95.65%. Finally, the critical brain regions and connections for driving fatigue detection were
investigated through the dynamically learned adjacency matrix.Index Terms— Attention-based multiscale convolutional neural network (CNN), driving fatigue, dynamical graph convolution network (GCN), electroencephalography (EEG), spatiotemporalstructure.
A TSK-type Convolutional Recurrent Fuzzy Network for Predicting Driving Fatigue
Authors: Guanglong Du, Zhiyao Wang, Chunquan Li, Peter X. Liu, Fellow
Date: 2020-05-06 00:00:00
Proceeding: 2020 2020 5
Tasks: EEG, Electroencephalogram (EEG), Vocal Bursts Type Prediction

Driver fatigue monitoring is very important for  driving safety, and many intricate factors in driving make fatigue 
monitoring harder. To effectively predict driving fatigue, this  paper proposes a new deep learning framework called TSK-type  Convolution Recurrent Fuzzy Network (TCRFN) based on the  spatial and temporal characteristics of EEG signals. In TCRFN, the convolution block is first introduced to extract spatial dependencies from EEG signals. Furthermore, since EEG noise has a strong spatial dependence, this Convolutional Neural Networks (CNN) is used to reduce the impact of noise. Additionally, a new local feedback method in Fuzzy Neural Network (FNN) is proposed to process the EEG signals, which can better capture the temporal dependencies from EEG signals. Finally, a logarithmic spatial firing layer function is used in the proposed TCRFN. The activation performance of this function is smoother, which allows more feature numbers and provides better prediction. The experimental results show that the proposed TCRFN model has better anti-noise performance and prediction accuracy compared with other widely-used and state-of-the-art models.
General Virtual Sketching Framework for Vector Line Art
Authors: Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, Ruomei Wang
Date: 2021-07-19 00:00:00
Proceeding: Transactions on Graphics (SIGGRAPH) 2021 7
Tasks: Diversity

Vector line art plays an important role in graphic design, however, it is tedious to manually create. We introduce a general framework to produce line drawings from a wide variety of images, by learning a mapping from raster image space to vector image space. Our approach is based on a recurrent neural network that draws the lines one by one. A differentiable rasterization module allows for training with only supervised raster data. We use a dynamic window around a virtual pen while drawing lines, implemented with a proposed aligned cropping and differentiable pasting modules. Furthermore, we develop a stroke regularization loss that encourages the model to use fewer and longer strokes to simplify the resulting vector image. Ablation studies and comparisons with existing methods corroborate the efficiency of our approach which is able to generate visually better results in less computation time, while generalizing better to a diversity of images and applications.
Inference via Sparse Coding in a Hierarchical Vision Model
Authors: Joshua Bowren, Luis Sanchez-Giraldo, Odelia Schwartz
Date: 2021-08-03 00:00:00
Tasks: image-classification, Image Classification, Sensitivity, Texture Classification

Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya and Hyv\"arinen, 2015), but replacing its independent component analysis (ICA) with an explicit sparse coding in which the degree of sparsity can be controlled. After training, the sparse coding basis functions with a higher degree of sparsity resembled qualitatively different structures, such as curves and corners. The contributions of the models were assessed with image classification tasks, specifically tasks associated with mid-level vision including figure-ground classification, texture classification, and angle prediction between two line stimuli. In addition, the models were assessed in comparison to a texture sensitivity measure that has been reported in V2 (Freeman et al., 2013), and a deleted-region inference task. The results from the experiments show that while sparse coding performed worse than ICA at classifying images, only sparse coding was able to better match the texture sensitivity level of V2 and infer deleted image regions, both by increasing the degree of sparsity in sparse coding. Higher degrees of sparsity allowed for inference over larger deleted image regions. The mechanism that allows for this inference capability in sparse coding is described here.
Using Query Expansion in Manifold Ranking for Query-Oriented Multi-Document Summarization
Authors: Quanye Jia, Rui Liu, Jianying Lin
Date: 2021-07-31 00:00:00
Proceeding: CCL 2021 8
Tasks: Document Summarization, Multi-Document Summarization

Manifold ranking has been successfully applied in query-oriented multi-document summarization. It not only makes use of the relationships among the sentences, but also the relationships between the given query and the sentences. However, the information of original query is often insufficient. So we present a query expansion method, which is combined in the manifold ranking to resolve this problem. Our method not only utilizes the information of the query term itself and the knowledge base WordNet to expand it by synonyms, but also uses the information of the document set itself to expand the query in various ways (mean expansion, variance expansion and TextRank expansion). Compared with the previous query expansion methods, our method combines multiple query expansion methods to better represent query information, and at the same time, it makes a useful attempt on manifold ranking. In addition, we use the degree of word overlap and the proximity between words to calculate the similarity between sentences. We performed experiments on the datasets of DUC 2006 and DUC2007, and the evaluation results show that the proposed query expansion method can significantly improve the system performance and make our system comparable to the state-of-the-art systems.
Uniform Sampling over Episode Difficulty
Authors: Sébastien M. R. Arnold, Guneet S. Dhillon, Avinash Ravichandran, Stefano Soatto
Date: 2021-08-03 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Few-Shot Learning

Episodic training is a core ingredient of few-shot learning to train models on tasks with limited labelled data. Despite its success, episodic training remains largely understudied, prompting us to ask the question: what is the best way to sample episodes? In this paper, we first propose a method to approximate episode sampling distributions based on their difficulty. Building on this method, we perform an extensive analysis and find that sampling uniformly over episode difficulty outperforms other sampling schemes, including curriculum and easy-/hard-mining. As the proposed sampling method is algorithm agnostic, we can leverage these insights to improve few-shot learning accuracies across many episodic training algorithms. We demonstrate the efficacy of our method across popular few-shot learning datasets, algorithms, network architectures, and protocols.
Graph Neural Networks With Lifting-based Adaptive Graph Wavelets
Authors: Mingxing Xu, Wenrui Dai, Chenglin Li, Junni Zou, Hongkai Xiong, Pascal Frossard
Date: 2021-08-03 00:00:00
Tasks: Graph Representation Learning, Representation Learning

Spectral-based graph neural networks (SGNNs) have been attracting increasing attention in graph representation learning. However, existing SGNNs are limited in implementing graph filters with rigid transforms (e.g., graph Fourier or predefined graph wavelet transforms) and cannot adapt to signals residing on graphs and tasks at hand. In this paper, we propose a novel class of graph neural networks that realizes graph filters with adaptive graph wavelets. Specifically, the adaptive graph wavelets are learned with neural network-parameterized lifting structures, where structure-aware attention-based lifting operations (i.e., prediction and update operations) are developed to jointly consider graph structures and node features. We propose to lift based on diffusion wavelets to alleviate the structural information loss induced by partitioning non-bipartite graphs. By design, the locality and sparsity of the resulting wavelet transform as well as the scalability of the lifting structure are guaranteed. We further derive a soft-thresholding filtering operation by learning sparse graph representations in terms of the learned wavelets, yielding a localized, efficient, and scalable wavelet-based graph filters. To ensure that the learned graph representations are invariant to node permutations, a layer is employed at the input of the networks to reorder the nodes according to their local topology information. We evaluate the proposed networks in both node-level and graph-level representation learning tasks on benchmark citation and bioinformatics graph datasets. Extensive experiments demonstrate the superiority of the proposed networks over existing SGNNs in terms of accuracy, efficiency, and scalability.
The Devil is in the GAN: Backdoor Attacks and Defenses in Deep Generative Models
Authors: Ambrish Rawat, Killian Levacher, Mathieu Sinn
Date: 2021-08-03 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation, Fairness

Deep Generative Models (DGMs) are a popular class of deep learning models which find widespread use because of their ability to synthesize data from complex, high-dimensional manifolds. However, even with their increasing industrial adoption, they haven't been subject to rigorous security and privacy analysis. In this work we examine one such aspect, namely backdoor attacks on DGMs which can significantly limit the applicability of pre-trained models within a model supply chain and at the very least cause massive reputation damage for companies outsourcing DGMs form third parties. While similar attacks scenarios have been studied in the context of classical prediction models, their manifestation in DGMs hasn't received the same attention. To this end we propose novel training-time attacks which result in corrupted DGMs that synthesize regular data under normal operations and designated target outputs for inputs sampled from a trigger distribution. These attacks are based on an adversarial loss function that combines the dual objectives of attack stealth and fidelity. We systematically analyze these attacks, and show their effectiveness for a variety of approaches like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as well as different data domains including images and audio. Our experiments show that - even for large-scale industry-grade DGMs (like StyleGAN) - our attacks can be mounted with only modest computational effort. We also motivate suitable defenses based on static/dynamic model and output inspections, demonstrate their usefulness, and prescribe a practical and comprehensive defense strategy that paves the way for safe usage of DGMs.
Scheduling Aerial Vehicles in an Urban Air Mobility Scheme
Authors: Emmanouil S. Rigas, Panayiotis Kolios, Georgios Ellinas
Date: 2021-08-03 00:00:00
Tasks: Scheduling

Highly populated cities face several challenges, one of them being the intense traffic congestion. In recent years, the concept of Urban Air Mobility has been put forward by large companies and organizations as a way to address this problem, and this approach has been rapidly gaining ground. This disruptive technology involves aerial vehicles (AVs) for hire than can be utilized by customers to travel between locations within large cities. This concept has the potential to drastically decrease traffic congestion and reduce air pollution, since these vehicles typically use electric motors powered by batteries. This work studies the problem of scheduling the assignment of AVs to customers, having as a goal to maximize the serviced customers and minimize the energy consumption of the AVs by forcing them to fly at the lowest possible altitude. Initially, an Integer Linear Program (ILP) formulation is presented, that is solved offline and optimally, followed by a near-optimal algorithm, that solves the problem incrementally, one AV at a time, to address scalability issues, allowing scheduling in problems involving large numbers of locations, AVs, and customer requests.
The application of artificial intelligence in software engineering: a review challenging conventional wisdom
Authors: Feras A. Batarseh, Rasika Mohod, Abhinav Kumar, Justin Bui
Date: 2021-08-03 00:00:00

The field of artificial intelligence (AI) is witnessing a recent upsurge in research, tools development, and deployment of applications. Multiple software companies are shifting their focus to developing intelligent systems; and many others are deploying AI paradigms to their existing processes. In parallel, the academic research community is injecting AI paradigms to provide solutions to traditional engineering problems. Similarly, AI has evidently been proved useful to software engineering (SE). When one observes the SE phases (requirements, design, development, testing, release, and maintenance), it becomes clear that multiple AI paradigms (such as neural networks, machine learning, knowledge-based systems, natural language processing) could be applied to improve the process and eliminate many of the major challenges that the SE field has been facing. This survey chapter is a review of the most commonplace methods of AI applied to SE. The review covers methods between years 1975-2017, for the requirements phase, 46 major AI-driven methods are found, 19 for design, 15 for development, 68 for testing, and 15 for release and maintenance. Furthermore, the purpose of this chapter is threefold; firstly, to answer the following questions: is there sufficient intelligence in the SE lifecycle? What does applying AI to SE entail? Secondly, to measure, formulize, and evaluate the overlap of SE phases and AI disciplines. Lastly, this chapter aims to provide serious questions to challenging the current conventional wisdom (i.e., status quo) of the state-of-the-art, craft a call for action, and to redefine the path forward.
Classification of Discrete Dynamical Systems Based on Transients
Authors: Barbora Hudcová, Tomáš Mikolov
Date: 2021-08-03 00:00:00
Tasks: Classification

In order to develop systems capable of artificial evolution, we need to identify which systems can produce complex behavior. We present a novel classification method applicable to any class of deterministic discrete space and time dynamical systems. The method is based on classifying the asymptotic behavior of the average computation time in a given system before entering a loop. We were able to identify a critical region of behavior that corresponds to a phase transition from ordered behavior to chaos across various classes of dynamical systems. To show that our approach can be applied to many different computational systems, we demonstrate the results of classifying cellular automata, Turing machines, and random Boolean networks. Further, we use this method to classify 2D cellular automata to automatically find those with interesting, complex dynamics. We believe that our work can be used to design systems in which complex structures emerge. Also, it can be used to compare various versions of existing attempts to model open-ended evolution (Ray (1991), Ofria et al. (2004), Channon (2006)).
EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
Authors: Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, Jie Tang
Date: 2021-08-03 00:00:00

Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones. In this paper, we propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. This dataset contains 1.4B context-response pairs and is used as the pre-training corpus of EVA. Extensive experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models especially in the multi-turn interaction of human-bot conversations.