pwc-archive/papers-with-abstracts huggingface dataset:
NCS4CVR: Neuron-Connection Sharing for Multi-Task Learning in Video Conversion Rate Prediction
Authors: Xuanji Xiao, Hua-Bin Chen, Yuzhen Liu, Xing Yao, Pei Liu, Chaosheng Fan, Nian Ji, Xirong Jiang
Date: 2020-08-22 00:00:00
Tasks: Click-Through Rate Prediction, Multi-Task Learning, Recommendation Systems

Click-through rate (CTR) and post-click conversion rate (CVR) predictions are two fundamental modules in industrial ranking systems such as recommender systems, advertising, and search engines. Since CVR involves much fewer samples than CTR (known as the CVR data sparsity problem), most of the existing works try to leverage CTR&CVR multi-task learning to improve CVR performance. However, typical coarse-grained sub-network/layer sharing methods may introduce conflicts and lead to performance degradation, since not every neuron or neuron connection in one layer should be shared between CVR and CTR tasks. This is because users may have different fine-grained content feature preferences between deep consumption and click behavior, represented by CVR and CTR, respectively. To address this sharing&conflict problem, we propose a novel multi-task CVR modeling scheme with neuron-connection level sharing named NCS4CVR, which can automatically and flexibly learn which neuron weights are shared or not shared without artificial experience. Compared with previous layer-level sharing methods, this is the first time that a fine-grained CTR&CVR sharing method at the neuron connection level is proposed, which is a research paradigm shift in the sharing level. Both offline and online experiments demonstrate that our method outperforms both the single-task model and the layer-level sharing model. Our proposed method has now been successfully deployed in an industry video recommender system serving major traffic.
UTMN at SemEval-2020 Task 11: A Kitchen Solution to Automatic Propaganda Detection
Authors: Elena Mikhalkova, Nadezhda Ganzherli, Anna Glazkova, Yuliya Bidulya
Date: 2020-08-22 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Propaganda detection

The article describes a fast solution to propaganda detection at SemEval-2020 Task 11, based onfeature adjustment. We use per-token vectorization of features and a simple Logistic Regressionclassifier to quickly test different hypotheses about our data. We come up with what seems to usthe best solution, however, we are unable to align it with the result of the metric suggested by theorganizers of the task. We test how our system handles class and feature imbalance by varying thenumber of samples of two classes (Propaganda and None) in the training set, the size of a contextwindow in which a token is vectorized and combination of vectorization means. The result of oursystem at SemEval2020 Task 11 is F-score=0.37.
Symbolic Semantic Segmentation and Interpretation of COVID-19 Lung Infections in Chest CT volumes based on Emergent Languages
Authors: Aritra Chowdhury, Alberto Santamaria-Pang, James R. Kubricht, Jianwei Qiu, Peter Tu
Date: 2020-08-22 00:00:00
Tasks: Segmentation, Semantic Segmentation, Sentence

The coronavirus disease (COVID-19) has resulted in a pandemic crippling the a breadth of services critical to daily life. Segmentation of lung infections in computerized tomography (CT) slices could be be used to improve diagnosis and understanding of COVID-19 in patients. Deep learning systems lack interpretability because of their black box nature. Inspired by human communication of complex ideas through language, we propose a symbolic framework based on emergent languages for the segmentation of COVID-19 infections in CT scans of lungs. We model the cooperation between two artificial agents - a Sender and a Receiver. These agents synergistically cooperate using emergent symbolic language to solve the task of semantic segmentation. Our game theoretic approach is to model the cooperation between agents unlike Generative Adversarial Networks (GANs). The Sender retrieves information from one of the higher layers of the deep network and generates a symbolic sentence sampled from a categorical distribution of vocabularies. The Receiver ingests the stream of symbols and cogenerates the segmentation mask. A private emergent language is developed that forms the communication channel used to describe the task of segmentation of COVID infections. We augment existing state of the art semantic segmentation architectures with our symbolic generator to form symbolic segmentation models. Our symbolic segmentation framework achieves state of the art performance for segmentation of lung infections caused by COVID-19. Our results show direct interpretation of symbolic sentences to discriminate between normal and infected regions, infection morphology and image characteristics. We show state of the art results for segmentation of COVID-19 lung infections in CT.
Tackling Over-Smoothing for General Graph Convolutional Networks
Authors: Wenbing Huang, Yu Rong, Tingyang Xu, Fuchun Sun, Junzhou Huang
Date: 2020-08-22 00:00:00
Tasks: Node Classification

Increasing the depth of GCN, which is expected to permit more expressivity, is shown to incur performance detriment especially on node classification. The main cause of this lies in over-smoothing. The over-smoothing issue drives the output of GCN towards a space that contains limited distinguished information among nodes, leading to poor expressivity. Several works on refining the architecture of deep GCN have been proposed, but it is still unknown in theory whether or not these refinements are able to relieve over-smoothing. In this paper, we first theoretically analyze how general GCNs act with the increase in depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We find that all these models are characterized by a universal process: all nodes converging to a cuboid. Upon this theorem, we propose DropEdge to alleviate over-smoothing by randomly removing a certain number of edges at each training epoch. Theoretically, DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by dimension collapse. Experimental evaluations on simulated dataset have visualized the difference in over-smoothing between different GCNs. Moreover, extensive experiments on several real benchmarks support that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs.
Emergent symbolic language based deep medical image classification
Authors: Aritra Chowdhury, Alberto Santamaria-Pang, James R. Kubricht, Peter Tu
Date: 2020-08-22 00:00:00
Tasks: Classification, Decision Making, General Classification, image-classification, Image Classification, Medical Image Classification, X-ray Classification

Modern deep learning systems for medical image classification have demonstrated exceptional capabilities for distinguishing between image based medical categories. However, they are severely hindered by their ina-bility to explain the reasoning behind their decision making. This is partly due to the uninterpretable continuous latent representations of neural net-works. Emergent languages (EL) have recently been shown to enhance the capabilities of neural networks by equipping them with symbolic represen-tations in the framework of referential games. Symbolic representations are one of the cornerstones of highly explainable good old fashioned AI (GOFAI) systems. In this work, we demonstrate for the first time, the emer-gence of deep symbolic representations of emergent language in the frame-work of image classification. We show that EL based classification models can perform as well as, if not better than state of the art deep learning mod-els. In addition, they provide a symbolic representation that opens up an entire field of possibilities of interpretable GOFAI methods involving symbol manipulation. We demonstrate the EL classification framework on immune cell marker based cell classification and chest X-ray classification using the CheXpert dataset. Code is available online at https://github.com/AriChow/EL.
CyberWallE at SemEval-2020 Task 11: An Analysis of Feature Engineering for Ensemble Models for Propaganda Detection
Authors: Verena Blaschke, Maxim Korniyenko, Sam Tureski
Date: 2020-08-22 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Propaganda detection, Propaganda span identification, Propaganda technique identification

This paper describes our participation in the SemEval-2020 task Detection of Propaganda Techniques in News Articles. We participate in both subtasks: Span Identification (SI) and Technique Classification (TC). We use a bi-LSTM architecture in the SI subtask and train a complex ensemble model for the TC subtask. Our architectures are built using embeddings from BERT in combination with additional lexical features and extensive label post-processing. Our systems achieve a rank of 8 out of 35 teams in the SI subtask (F1-score: 43.86%) and 8 out of 31 teams in the TC subtask (F1-score: 57.37%).
Hi-CI: Deep Causal Inference in High Dimensions
Authors: Ankit Sharma, Garima Gupta, Ranjitha Prasad, Arnab Chatterjee, Lovekesh Vig, Gautam Shroff
Date: 2020-08-22 00:00:00
Tasks: Causal Inference, counterfactual, Vocal Bursts Intensity Prediction

We address the problem of counterfactual regression using causal inference (CI) in observational studies consisting of high dimensional covariates and high cardinality treatments. Confounding bias, which leads to inaccurate treatment effect estimation, is attributed to covariates that affect both treatments and outcome. The presence of high-dimensional co-variates exacerbates the impact of bias as it is harder to isolate and measure the impact of these confounders. In the presence of high-cardinality treatment variables, CI is rendered ill-posed due to the increase in the number of counterfactual outcomes to be predicted. We propose Hi-CI, a deep neural network (DNN) based framework for estimating causal effects in the presence of large number of covariates, and high-cardinal and continuous treatment variables. The proposed architecture comprises of a decorrelation network and an outcome prediction network. In the decorrelation network, we learn a data representation in lower dimensions as compared to the original covariates and addresses confounding bias alongside. Subsequently, in the outcome prediction network, we learn an embedding of high-cardinality and continuous treatments, jointly with the data representation. We demonstrate the efficacy of causal effect prediction of the proposed Hi-CI network using synthetic and real-world NEWS datasets.
Data augmentation techniques for the Video Question Answering task
Authors: Alex Falcon, Oswald Lanz, Giuseppe Serra
Date: 2020-08-22 00:00:00
Tasks: Data Augmentation, Question Answering, Video Question Answering

Video Question Answering (VideoQA) is a task that requires a model to analyze and understand both the visual content given by the input video and the textual part given by the question, and the interaction between them in order to produce a meaningful answer. In our work we focus on the Egocentric VideoQA task, which exploits first-person videos, because of the importance of such task which can have impact on many different fields, such as those pertaining the social assistance and the industrial training. Recently, an Egocentric VideoQA dataset, called EgoVQA, has been released. Given its small size, models tend to overfit quickly. To alleviate this problem, we propose several augmentation techniques which give us a +5.5% improvement on the final accuracy over the considered baseline.
Fast Approximate Multi-output Gaussian Processes
Authors: Vladimir Joukov, Dana Kulić
Date: 2020-08-22 00:00:00
Tasks: Gaussian Processes, Hyperparameter Optimization, regression

Gaussian processes regression models are an appealing machine learning method as they learn expressive non-linear models from exemplar data with minimal parameter tuning and estimate both the mean and covariance of unseen points. However, exponential computational complexity growth with the number of training samples has been a long standing challenge. During training, one has to compute and invert an $N \times N$ kernel matrix at every iteration. Regression requires computation of an $m \times N$ kernel where $N$ and $m$ are the number of training and test points respectively. In this work we show how approximating the covariance kernel using eigenvalues and functions leads to an approximate Gaussian process with significant reduction in training and regression complexity. Training with the proposed approach requires computing only a $N \times n$ eigenfunction matrix and a $n \times n$ inverse where $n$ is a selected number of eigenvalues. Furthermore, regression now only requires an $m \times n$ matrix. Finally, in a special case the hyperparameter optimization is completely independent form the number of training samples. The proposed method can regress over multiple outputs, estimate the derivative of the regressor of any order, and learn the correlations between them. The computational complexity reduction, regression capabilities, and multioutput correlation learning are demonstrated in simulation examples.
On the Intrinsic Differential Privacy of Bagging
Authors: Hongbin Liu, Jinyuan Jia, Neil Zhenqiang Gong
Date: 2020-08-22 00:00:00
Tasks: BIG-bench Machine Learning, Ensemble Learning

Differentially private machine learning trains models while protecting privacy of the sensitive training data. The key to obtain differentially private models is to introduce noise/randomness to the training process. In particular, existing differentially private machine learning methods add noise to the training data, the gradients, the loss function, and/or the model itself. Bagging, a popular ensemble learning framework, randomly creates some subsamples of the training data, trains a base model for each subsample using a base learner, and takes majority vote among the base models when making predictions. Bagging has intrinsic randomness in the training process as it randomly creates subsamples. Our major theoretical results show that such intrinsic randomness already makes Bagging differentially private without the needs of additional noise. In particular, we prove that, for any base learner, Bagging with and without replacement respectively achieves $\left(N\cdot k \cdot \ln{\frac{n+1}{n}},1- (\frac{n-1}{n})^{N\cdot k}\right)$-differential privacy and $\left(\ln{\frac{n+1}{n+1-N\cdot k}}, \frac{N\cdot k}{n} \right)$-differential privacy, where $n$ is the training data size, $k$ is the subsample size, and $N$ is the number of base models. Moreover, we prove that if no assumptions about the base learner are made, our derived privacy guarantees are tight. We empirically evaluate Bagging on MNIST and CIFAR10. Our experimental results demonstrate that Bagging achieves significantly higher accuracies than state-of-the-art differentially private machine learning methods with the same privacy budgets.
Forecasting of the Montreal Subway Smart Card Entry Logs with Event Data
Authors: Florian Toqué, Etienne Côme, Martin Trépanier, Latifa Oukhellou
Date: 2020-08-22 00:00:00
Tasks: Scheduling

One of the major goals of transport operators is to adapt the transport supply scheduling to the passenger demand for existing transport networks during each specific period. Another problem mentioned by operators is accurately estimating the demand for disposable ticket or pass to adapt ticket availability to passenger demand. In this context, we propose generic data shaping, allowing the use of well-known regression models (basic, statistical and machine learning models) for the long-term forecasting of passenger demand with fine-grained temporal resolution. Specifically, this paper investigates the forecasting until one year ahead of the number of passengers entering each station of a transport network with a quarter-hour aggregation by taking planned events into account (e.g., concerts, shows, and so forth). To compare the models and the quality of the prediction, we use a real smart card and event data set from the city of Montr\'eal, Canada, that span a three-year period with two years for training and one year for testing.
Revisiting Anchor Mechanisms for Temporal Action Localization
Authors: Le Yang, Houwen Peng, Dingwen Zhang, Jianlong Fu, Junwei Han
Date: 2020-08-22 00:00:00
Tasks: Action Localization, Temporal Action Localization

Most of the current action localization methods follow an anchor-based pipeline: depicting action instances by pre-defined anchors, learning to select the anchors closest to the ground truth, and predicting the confidence of anchors with refinements. Pre-defined anchors set prior about the location and duration for action instances, which facilitates the localization for common action instances but limits the flexibility for tackling action instances with drastic varieties, especially for extremely short or extremely long ones. To address this problem, this paper proposes a novel anchor-free action localization module that assists action localization by temporal points. Specifically, this module represents an action instance as a point with its distances to the starting boundary and ending boundary, alleviating the pre-defined anchor restrictions in terms of action localization and duration. The proposed anchor-free module is capable of predicting the action instances whose duration is either extremely short or extremely long. By combining the proposed anchor-free module with a conventional anchor-based module, we propose a novel action localization framework, called A2Net. The cooperation between anchor-free and anchor-based modules achieves superior performance to the state-of-the-art on THUMOS14 (45.5% vs. 42.8%). Furthermore, comprehensive experiments demonstrate the complementarity between the anchor-free and the anchor-based module, making A2Net simple but effective.
From noisy point clouds to complete ear shapes: unsupervised pipeline
Authors: Filipa Valdeira, Ricardo Ferreira, Alessandra Micheletti, Cláudia Soares
Date: 2020-08-22 00:00:00

Ears are a particularly difficult region of the human face to model, not only due to the non-rigid deformations existing between shapes but also to the challenges in processing the retrieved data. The first step towards obtaining a good model is to have complete scans in correspondence, but these usually present a higher amount of occlusions, noise and outliers when compared to most face regions, thus requiring a specific procedure. Therefore, we propose a complete pipeline taking as input unordered 3D point clouds with the aforementioned problems, and producing as output a dataset in correspondence, with completion of the missing data. We provide a comparison of several state-of-the-art registration methods and propose a new approach for one of the steps of the pipeline, with better performance for our data.
Self-Competitive Neural Networks
Authors: Iman Saberi, Fathiyeh Faghih
Date: 2020-08-22 00:00:00
Tasks: Data Augmentation

Deep Neural Networks (DNNs) have improved the accuracy of classification problems in lots of applications. One of the challenges in training a DNN is its need to be fed by an enriched dataset to increase its accuracy and avoid it suffering from overfitting. One way to improve the generalization of DNNs is to augment the training data with new synthesized adversarial samples. Recently, researchers have worked extensively to propose methods for data augmentation. In this paper, we generate adversarial samples to refine the Domains of Attraction (DoAs) of each class. In this approach, at each stage, we use the model learned by the primary and generated adversarial data (up to that stage) to manipulate the primary data in a way that look complicated to the DNN. The DNN is then retrained using the augmented data and then it again generates adversarial data that are hard to predict for itself. As the DNN tries to improve its accuracy by competing with itself (generating hard samples and then learning them), the technique is called Self-Competitive Neural Network (SCNN). To generate such samples, we pose the problem as an optimization task, where the network weights are fixed and use a gradient descent based method to synthesize adversarial samples that are on the boundary of their true labels and the nearest wrong labels. Our experimental results show that data augmentation using SCNNs can significantly increase the accuracy of the original network. As an example, we can mention improving the accuracy of a CNN trained with 1000 limited training data of MNIST dataset from 94.26% to 98.25%.
HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection
Authors: Meghana Bhange, Nirant Kasliwal
Date: 2020-08-22 00:00:00
Tasks: BIG-bench Machine Learning, Sentiment Analysis

Sentiment analysis for code-mixed social media text continues to be an under-explored area. This work adds two common approaches: fine-tuning large transformer models and sample efficient methods like ULMFiT. Prior work demonstrates the efficacy of classical ML methods for polarity detection. Fine-tuned general-purpose language representation models, such as those of the BERT family are benchmarked along with classical machine learning and ensemble methods. We show that NB-SVM beats RoBERTa by 6.2% (relative) F1. The best performing model is a majority-vote ensemble which achieves an F1 of 0.707. The leaderboard submission was made under the codalab username nirantk, with F1 of 0.689.
Memory-based Jitter: Improving Visual Recognition on Long-tailed Data with Diversity In Memory
Authors: Jialun Liu, Jingwei Zhang, Yi Yang, Wenhui Li, Chi Zhang, Yifan Sun
Date: 2020-08-22 00:00:00
Tasks: Data Augmentation, Diversity, General Classification, image-classification, Image Classification, Long-tail Learning, Metric Learning, Person Re-Identification

This paper considers deep visual recognition on long-tailed data. To be general, we consider two applied scenarios, \ie, deep classification and deep metric learning. Under the long-tailed data distribution, the majority classes (\ie, tail classes) only occupy relatively few samples and are prone to lack of within-class diversity. A radical solution is to augment the tail classes with higher diversity. To this end, we introduce a simple and reliable method named Memory-based Jitter (MBJ). We observe that during training, the deep model constantly changes its parameters after every iteration, yielding the phenomenon of \emph{weight jitters}. Consequentially, given a same image as the input, two historical editions of the model generate two different features in the deeply-embedded space, resulting in \emph{feature jitters}. Using a memory bank, we collect these (model or feature) jitters across multiple training iterations and get the so-called Memory-based Jitter. The accumulated jitters enhance the within-class diversity for the tail classes and consequentially improves long-tailed visual recognition. With slight modifications, MBJ is applicable for two fundamental visual recognition tasks, \emph{i.e.}, deep image classification and deep metric learning (on long-tailed data). Extensive experiments on five long-tailed classification benchmarks and two deep metric learning benchmarks demonstrate significant improvement. Moreover, the achieved performance are on par with the state of the art on both tasks.