pwc-archive/papers-with-abstracts huggingface dataset:
Modeling Diverse Relevance Patterns in Ad-hoc Retrieval
Authors: Yixing Fan, Jiafeng Guo, Yanyan Lan, Jun Xu, ChengXiang Zhai, Xue-Qi Cheng
Date: 2018-05-15 00:00:00
Proceeding: SIGIR '18 2018 5
Tasks: Retrieval

Assessing relevance between a query and a document is challenging in ad-hoc retrieval due to its diverse patterns, i.e., a document could be relevant to a query as a whole or partially as long as it provides sufficient information for users' need. Such diverse relevance patterns require an ideal retrieval model to be able to assess relevance in the right granularity adaptively. Unfortunately, most existing retrieval models compute relevance at a single granularity, either document-wide or passage-level, or use fixed combination strategy, restricting their ability in capturing diverse relevance patterns. In this work, we propose a data-driven method to allow relevance signals at different granularities to compete with each other for final relevance assessment. Specifically, we propose a HIerarchical Neural maTching model (HiNT) which consists of two stacked components, namely local matching layer and global decision layer. The local matching layer focuses on producing a set of local relevance signals by modeling the semantic matching between a query and each passage of a document. The global decision layer accumulates local signals into different granularities and allows them to compete with each other to decide the final relevance score. Experimental results demonstrate that our HiNT model outperforms existing state-of-the-art retrieval models significantly on benchmark ad-hoc retrieval datasets.
A deep-learning-based approach for fast and robust steel surface defects classification
Authors: Guizhong Fu, Peize Sun a, Wenbin Zhu, Jiangxin Yang, Yanlong Cao, Michael Ying Yang, Yanpeng Cao
Date: 2019-05-15 00:00:00
Proceeding: elsevier journal 2019 5
Tasks: Diversity, General Classification

Automatic visual recognition of steel surface defects provides critical functionality to facilitate quality control of steel strip production. In this paper, we present a compact yet effective convolutional neural network (CNN) model, which emphasizes the training of low-level features and incorporates multiple receptive fields, to achieve fast and accurate steel surface defect classification. Our proposed method adopts the pre-trained SqueezeNet as the backbone architecture. It only requires a small amount of defect-specific training samples to achieve high- accuracy recognition on a diversity-enhanced testing dataset of steel surface defects which contains severe non- uniform illumination, camera noise, and motion blur. Moreover, our proposed light-weight CNN model can meet the requirement of real-time online inspection, running over 100 fps on a computer equipped with a single NVIDIA TITAN X Graphics Processing Unit (12G memory). Codes and a diversity-enhanced testing dataset will be made publicly available.
Deep Learning Framework for Steel Surface Defects Classification
Authors: Karun Singla1, Gangesh Chawla2, Ranganath M. Singari3
Date: 2019-01-25 00:00:00
Proceeding: International Journal of Advanced Production and Industrial Engineering 2019 1
Tasks: Classification, Deep Learning, Defect Detection, General Classification, Management

Deep learning has offered new avenues in the field of industrial management. Traditional
methods of quality inspection such as Acceptance Sampling relies on a probabilistic measure derived from
inspecting a sample of finished products. Evaluating a fixed number of products to derive the quality level for
the complete batch is not a robust approach. Visual inspection solutions based on deep learning can be
employed in the large manufacturing units to improve the quality inspection units for steel surface defect
detection. This leads to optimization of the human capital due to reduction in manual intervention and
turnaround time in the overall supply chain of the industry. Consequently, the sample size in the Acceptance
sampling can be increased with minimal effort vis-à-vis an increase in the overall accuracy of the inspection.
The learning curve of this work is supported by Convolutional Neural Network which has been used to extract
feature representations from grayscale images to classify the inputs into six types of surface defects. The neural
network architecture is compiled in Keras framework using Tensor flow backend with state of the art Adam
RMS Prop with Nesterov Momentum (NADAM) optimizer. The proposed classification algorithm holds the
potential to identify the dominant flaws in the manufacturing system responsible for leaking costs.
Incorporating Clicks, Attention and Satisfaction into a Search Engine Result Page Evaluation Model
Authors: Aleksandr Chuklin, Maarten de Rijke
Date: 2016-09-02 00:00:00
Tasks: Navigate

Modern search engine result pages often provide immediate value to users and organize information in such a way that it is easy to navigate. The core ranking function contributes to this and so do result snippets, smart organization of result blocks and extensive use of one-box answers or side panels. While they are useful to the user and help search engines to stand out, such features present two big challenges for evaluation. First, the presence of such elements on a search engine result page (SERP) may lead to the absence of clicks, which is, however, not related to dissatisfaction, so-called "good abandonments." Second, the non-linear layout and visual difference of SERP items may lead to non-trivial patterns of user attention, which is not captured by existing evaluation metrics. In this paper we propose a model of user behavior on a SERP that jointly captures click behavior, user attention and satisfaction, the CAS model, and demonstrate that it gives more accurate predictions of user actions and self-reported satisfaction than existing models based on clicks alone. We use the CAS model to build a novel evaluation metric that can be applied to non-linear SERP layouts and that can account for the utility that users obtain directly on a SERP. We demonstrate that this metric shows better agreement with user-reported satisfaction than conventional evaluation metrics.
New Methods for Metadata Extraction from Scientific Literature
Authors: Dominika Tkaczyk
Date: 2017-10-27 00:00:00
Tasks: Articles

Within the past few decades we have witnessed digital revolution, which moved scholarly communication to electronic media and also resulted in a substantial increase in its volume. Nowadays keeping track with the latest scientific achievements poses a major challenge for the researchers. Scientific information overload is a severe problem that slows down scholarly communication and knowledge propagation across the academia. Modern research infrastructures facilitate studying scientific literature by providing intelligent search tools, proposing similar and related documents, visualizing citation and author networks, assessing the quality and impact of the articles, and so on. In order to provide such high quality services the system requires the access not only to the text content of stored documents, but also to their machine-readable metadata. Since in practice good quality metadata is not always available, there is a strong demand for a reliable automatic method of extracting machine-readable metadata directly from source documents. This research addresses these problems by proposing an automatic, accurate and flexible algorithm for extracting wide range of metadata directly from scientific articles in born-digital form. Extracted information includes basic document metadata, structured full text and bibliography section. Designed as a universal solution, proposed algorithm is able to handle a vast variety of publication layouts with high precision and thus is well-suited for analyzing heterogeneous document collections. This was achieved by employing supervised and unsupervised machine-learning algorithms trained on large, diverse datasets. The evaluation we conducted showed good performance of proposed metadata extraction algorithm. The comparison with other similar solutions also proved our algorithm performs better than competition for most metadata types.
SimVAE: Simulator-Assisted Training forInterpretable Generative Models
Authors: Akash Srivastava, Jessie Rosenberg, Dan Gutfreund, David D. Cox
Date: 2019-11-19 00:00:00
Tasks: Decoder

This paper presents a simulator-assisted training method (SimVAE) for variational autoencoders (VAE) that leads to a disentangled and interpretable latent space. Training SimVAE is a two-step process in which first a deep generator network(decoder) is trained to approximate the simulator. During this step, the simulator acts as the data source or as a teacher network. Then an inference network (encoder)is trained to invert the decoder. As such, upon complete training, the encoder represents an approximately inverted simulator. By decoupling the training of the encoder and decoder we bypass some of the difficulties that arise in training generative models such as VAEs and generative adversarial networks (GANs). We show applications of our approach in a variety of domains such as circuit design, graphics de-rendering and other natural science problems that involve inference via simulation.
Learning Weighted Submanifolds with Variational Autoencoders and Riemannian Variational Autoencoders
Authors: Nina Miolane, Susan Holmes
Date: 2019-11-19 00:00:00
Proceeding: CVPR 2020 6
Tasks: Riemannian optimization, Time Series Analysis

Manifold-valued data naturally arises in medical imaging. In cognitive neuroscience, for instance, brain connectomes base the analysis of coactivation patterns between different brain regions on the analysis of the correlations of their functional Magnetic Resonance Imaging (fMRI) time series - an object thus constrained by construction to belong to the manifold of symmetric positive definite matrices. One of the challenges that naturally arises consists of finding a lower-dimensional subspace for representing such manifold-valued data. Traditional techniques, like principal component analysis, are ill-adapted to tackle non-Euclidean spaces and may fail to achieve a lower-dimensional representation of the data - thus potentially pointing to the absence of lower-dimensional representation of the data. However, these techniques are restricted in that: (i) they do not leverage the assumption that the connectomes belong on a pre-specified manifold, therefore discarding information; (ii) they can only fit a linear subspace to the data. In this paper, we are interested in variants to learn potentially highly curved submanifolds of manifold-valued data. Motivated by the brain connectomes example, we investigate a latent variable generative model, which has the added benefit of providing us with uncertainty estimates - a crucial quantity in the medical applications we are considering. While latent variable models have been proposed to learn linear and nonlinear spaces for Euclidean data, or geodesic subspaces for manifold data, no intrinsic latent variable model exists to learn nongeodesic subspaces for manifold data. This paper fills this gap and formulates a Riemannian variational autoencoder with an intrinsic generative model of manifold-valued data. We evaluate its performances on synthetic and real datasets by introducing the formalism of weighted Riemannian submanifolds.
Deep Unsupervised Clustering with Clustered Generator Model
Authors: Dandan Zhu, Tian Han, Linqi Zhou, Xiaokang Yang, Ying Nian Wu
Date: 2019-11-19 00:00:00
Tasks: Clustering, model

This paper addresses the problem of unsupervised clustering which remains one of the most fundamental challenges in machine learning and artificial intelligence. We propose the clustered generator model for clustering which contains both continuous and discrete latent variables. Discrete latent variables model the cluster label while the continuous ones model variations within each cluster. The learning of the model proceeds in a unified probabilistic framework and incorporates the unsupervised clustering as an inner step without the need for an extra inference model as in existing variational-based models. The latent variables learned serve as both observed data embedding or latent representation for data distribution. Our experiments show that the proposed model can achieve competitive unsupervised clustering accuracy and can learn disentangled latent representations to generate realistic samples. In addition, the model can be naturally extended to per-pixel unsupervised clustering which remains largely unexplored.
Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks
Authors: Tharindu Fernando, Clinton Fookes, Simon Denman, Sridha Sridharan
Date: 2019-11-17 00:00:00
Tasks: Face Detection

Advances in computer vision have brought us to the point where we have the ability to synthesise realistic fake content. Such approaches are seen as a source of disinformation and mistrust, and pose serious concerns to governments around the world. Convolutional Neural Networks (CNNs) demonstrate encouraging results when detecting fake images that arise from the specific type of manipulation they are trained on. However, this success has not transitioned to unseen manipulation types, resulting in a significant gap in the line-of-defense. We propose a Hierarchical Memory Network (HMN) architecture, which is able to successfully detect faked faces by utilising knowledge stored in neural memories as well as visual cues to reason about the perceived face and anticipate its future semantic embeddings. This renders a generalisable face tampering detection framework. Experimental results demonstrate the proposed approach achieves superior performance for fake and fraudulent face detection compared to the state-of-the-art.
Neural Random Subspace
Authors: Yun-Hao Cao, Jianxin Wu, Hanchen Wang, Joan Lasenby
Date: 2019-11-18 00:00:00
Tasks: Deep Learning, Representation Learning

The random subspace method, known as the pillar of random forests, is good at making precise and robust predictions. However, there is not a straightforward way yet to combine it with deep learning. In this paper, we therefore propose Neural Random Subspace (NRS), a novel deep learning based random subspace method. In contrast to previous forest methods, NRS enjoys the benefits of end-to-end, data-driven representation learning, as well as pervasive support from deep learning software and hardware platforms, hence achieving faster inference speed and higher accuracy. Furthermore, as a non-linear component to be encoded into Convolutional Neural Networks (CNNs), NRS learns non-linear feature representations in CNNs more efficiently than previous higher-order pooling methods, producing good results with negligible increase in parameters, floating point operations (FLOPs) and real running time. Compared with random subspaces, random forests and gradient boosting decision trees (GBDTs), NRS achieves superior performance on 35 machine learning datasets. Moreover, on both 2D image and 3D point cloud recognition tasks, integration of NRS with CNN architectures achieves consistent improvements with minor extra cost. Code is available at https://github.com/CupidJay/NRS_pytorch.
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
Authors: David W. Romero, Mark Hoogendoorn
Date: 2019-11-18 00:00:00
Proceeding: ICLR 2020 1
Tasks: Object Recognition, Rotated MNIST

Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architectures consider the set of all possible transformations in a transformation group when learning feature representations. Contrarily, the human visual system is able to attend to the set of relevant transformations occurring in the environment and utilizes this information to assist and improve object recognition. Based on this observation, we modify conventional equivariant feature mappings such that they are able to attend to the set of co-occurring transformations in data and generalize this notion to act on groups consisting of multiple symmetries. We show that our proposed co-attentive equivariant neural networks consistently outperform conventional rotation equivariant and rotation & reflection equivariant neural networks on rotated MNIST and CIFAR-10.
RWNE: A Scalable Random-Walk-Based Network Embedding Framework with Personalized Higher-Order Proximity Preserved
Authors: JianXin Li, Cheng Ji, Hao Peng, Yu He, Yangqiu Song, Xinmiao Zhang, Fanzhang Peng
Date: 2019-11-18 00:00:00
Tasks: Network Embedding

Higher-order proximity preserved network embedding has attracted increasing attention. In particular, due to the superior scalability, random-walk-based network embedding has also been well developed, which could efficiently explore higher-order neighborhoods via multi-hop random walks. However, despite the success of current random-walk-based methods, most of them are usually not expressive enough to preserve the personalized higher-order proximity and lack a straightforward objective to theoretically articulate what and how network proximity is preserved. In this paper, to address the above issues, we present a general scalable random-walk-based network embedding framework, in which random walk is explicitly incorporated into a sound objective designed theoretically to preserve arbitrary higher-order proximity. Further, we introduce the random walk with restart process into the framework to naturally and effectively achieve personalized-weighted preservation of proximities of different orders. We conduct extensive experiments on several real-world networks and demonstrate that our proposed method consistently and substantially outperforms the state-of-the-art network embedding methods.
Attribute noise robust binary classification
Authors: Aditya Petety, Sandhya Tripathi, N. Hemachandra
Date: 2019-11-18 00:00:00
Tasks: Attribute, Binary Classification, Classification, General Classification

We consider the problem of learning linear classifiers when both features and labels are binary. In addition, the features are noisy, i.e., they could be flipped with an unknown probability. In Sy-De attribute noise model, where all features could be noisy together with same probability, we show that $0$-$1$ loss ($l_{0-1}$) need not be robust but a popular surrogate, squared loss ($l_{sq}$) is. In Asy-In attribute noise model, we prove that $l_{0-1}$ is robust for any distribution over 2 dimensional feature space. However, due to computational intractability of $l_{0-1}$, we resort to $l_{sq}$ and observe that it need not be Asy-In noise robust. Our empirical results support Sy-De robustness of squared loss for low to moderate noise rates.
Basic Principles of Clustering Methods
Authors: Alexander Jung, Ivan Baranov
Date: 2019-11-18 00:00:00
Tasks: Clustering

Clustering methods group a set of data points into a few coherent groups or clusters of similar data points. As an example, consider clustering pixels in an image (or video) if they belong to the same object. Different clustering methods are obtained by using different notions of similarity and different representations of data points.
Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition
Authors: Chengjin Xu, Mojtaba Nayyeri, Fouad Alkhoury, Hamed Shariat Yazdi, Jens Lehmann
Date: 2019-11-18 00:00:00
Tasks: Graph Embedding, Knowledge Graph Completion, Knowledge Graph Embedding, Link Prediction, Position, Relation, Time Series, Time Series Analysis

Knowledge Graph (KG) embedding has attracted more attention in recent years. Most KG embedding models learn from time-unaware triples. However, the inclusion of temporal information beside triples would further improve the performance of a KGE model. In this regard, we propose ATiSE, a temporal KG embedding model which incorporates time information into entity/relation representations by using Additive Time Series decomposition. Moreover, considering the temporal uncertainty during the evolution of entity/relation representations over time, we map the representations of temporal KGs into the space of multi-dimensional Gaussian distributions. The mean of each entity/relation embedding at a time step shows the current expected position, whereas its covariance (which is temporally stationary) represents its temporal uncertainty. Experimental results show that ATiSE chieves the state-of-the-art on link prediction over four temporal KGs.
Comments on the Du-Kakade-Wang-Yang Lower Bounds
Authors: Benjamin Van Roy, Shi Dong
Date: 2019-11-18 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Du, Kakade, Wang, and Yang recently established intriguing lower bounds on sample complexity, which suggest that reinforcement learning with a misspecified representation is intractable. Another line of work, which centers around a statistic called the eluder dimension, establishes tractability of problems similar to those considered in the Du-Kakade-Wang-Yang paper. We compare these results and reconcile interpretations.
Face shape classification using Inception v3
Authors: Adonis Emmanuel Tio
Date: 2019-11-15 00:00:00
Tasks: Classification, General Classification

In this paper, we present experimental results obtained from retraining the last layer of the Inception v3 model in classifying images of human faces into one of five basic face shapes. The accuracy of the retrained Inception v3 model was compared with that of the following classification methods that uses facial landmark distance ratios and angles as features: linear discriminant analysis (LDA), support vector machines with linear kernel (SVM-LIN), support vector machines with radial basis function kernel (SVM-RBF), artificial neural networks or multilayer perceptron (MLP), and k-nearest neighbors (KNN). All classifiers were trained and tested using a total of 500 images of female celebrities with known face shapes collected from the Internet. Results show that training accuracy and overall accuracy ranges from 98.0% to 100% and from 84.4% to 84.8% for Inception v3 and from 50.6% to 73.0% and from 36.4% to 64.6% for the other classifiers depending on the training set size used. This result shows that the retrained Inception v3 model was able to fit the training data well and outperform the other classifiers without the need to handpick specific features to include in model training. Future work should consider expanding the labeled dataset, preferably one that can also be freely distributed to the research community, so that proper model cross-validation can be performed. As far as we know, this is the first in the literature to use convolutional neural networks in face-shape classification. The scripts are available at https://github.com/adonistio/inception-face-shape-classifier.
Privacy Leakage Avoidance with Switching Ensembles
Authors: Rauf Izmailov, Peter Lin, Chris Mesterharm, Samyadeep Basu
Date: 2019-11-18 00:00:00
Tasks: BIG-bench Machine Learning

We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. Several approaches have been developed to mitigate this privacy leakage but the tradeoff performance implications of these defensive mechanisms (i.e., accuracy and utility of the defended machine learning model) are not well studied yet. We propose a novel approach of privacy leakage avoidance with switching ensembles (PASE), which both protects against current membership inference attacks and does that with very small accuracy penalty, while requiring acceptable increase in training and inference time. We test our PASE method, along with the the current state-of-the-art PATE approach, on three calibration image datasets and analyze their tradeoffs.