pwc-archive/papers-with-abstracts huggingface dataset:
Exploring Properties of Icosoku by Constraint Satisfaction Approach
Authors: Ke Liu, Sven Löffler, Petra Hofstedt
Date: 2019-08-16 00:00:00

Icosoku is a challenging and interesting puzzle that exhibits highly symmetrical and combinatorial nature. In this paper, we pose the questions derived from the puzzle, but with more difficulty and generality. In addition, we also present a constraint programming model for the proposed questions, which can provide the answers to our first two questions. The purpose of this paper is to share our preliminary result and problems to encourage researchers in both group theory and constraint communities to consider this topic further.
Regularizing CNN Transfer Learning with Randomised Regression
Authors: Yang Zhong, Atsuto Maki
Date: 2019-08-16 00:00:00
Proceeding: CVPR 2020 6
Tasks: regression, Transfer Learning

This paper is about regularizing deep convolutional networks (CNNs) based on an adaptive framework for transfer learning with limited training data in the target domain. Recent advances of CNN regularization in this context are commonly due to the use of additional regularization objectives. They guide the training away from the target task using some forms of concrete tasks. Unlike those related approaches, we suggest that an objective without a concrete goal can still serve well as a regularized. In particular, we demonstrate Pseudo-task Regularization (PtR) which dynamically regularizes a network by simply attempting to regress image representations to pseudo-regression targets during fine-tuning. That is, a CNN is efficiently regularized without additional resources of data or prior domain expertise. In sum, the proposed PtR provides: a) an alternative for network regularization without dependence on the design of concrete regularization objectives or extra annotations; b) a dynamically adjusted and maintained strength of regularization effect by balancing the gradient norms between objectives on-line. Through numerous experiments, surprisingly, the improvements on classification accuracy by PtR are shown greater or on a par to the recent state-of-the-art methods.
Iterative Neural Networks with Bounded Weights
Authors: Tomasz Piotrowski, Krzysztof Rykaczewski
Date: 2019-08-16 00:00:00

A recent analysis of a model of iterative neural network in Hilbert spaces established fundamental properties of such networks, such as existence of the fixed points sets, convergence analysis, and Lipschitz continuity. Building on these results, we show that under a single mild condition on the weights of the network, one is guaranteed to obtain a neural network converging to its unique fixed point. We provide a bound on the norm of this fixed point in terms of norms of weights and biases of the network. We also show why this model of a feed-forward neural network is not able to accomodate Hopfield networks under our assumption.
The Partial Response Network: a neural network nomogram
Authors: Paulo J. G. Lisboa, Sandra Ortega-Martorell, Sadie Cashman, Ivan Olier
Date: 2019-08-16 00:00:00
Tasks: Additive models, Benchmarking, Binary Classification, feature selection, Interpretable Machine Learning

Among interpretable machine learning methods, the class of Generalised Additive Neural Networks (GANNs) is referred to as Self-Explaining Neural Networks (SENN) because of the linear dependence on explicit functions of the inputs. In binary classification this shows the precise weight that each input contributes towards the logit. The nomogram is a graphical representation of these weights. We show that functions of individual and pairs of variables can be derived from a functional Analysis of Variance (ANOVA) representation, enabling an efficient feature selection to be carried by application of the logistic Lasso. This process infers the structure of GANNs which otherwise needs to be predefined. As this method is particularly suited for tabular data, it starts by fitting a generic flexible model, in this case a Multi-layer Perceptron (MLP) to which the ANOVA decomposition is applied. This has the further advantage that the resulting GANN can be replicated as a SENN, enabling further refinement of the univariate and bivariate component functions to take place. The component functions are partial responses hence the SENN is a partial response network. The Partial Response Network (PRN) is equally as transparent as a traditional logistic regression model, but capable of non-linear classification with comparable or superior performance to the original MLP. In other words, the PRN is a fully interpretable representation of the MLP, at the level of univariate and bivariate effects. The performance of the PRN is shown to be competitive for benchmark data, against state-of-the-art machine learning methods including GBM, SVM and Random Forests. It is also compared with spline-based Sparse Additive Models (SAM) showing that a semi-parametric representation of the GAM as a neural network can be as effective as the SAM though less constrained by the need to set spline nodes.
HOTVis: Higher-Order Time-Aware Visualisation of Dynamic Graphs
Authors: Vincenzo Perri, Ingo Scholtes
Date: 2019-08-16 00:00:00
Tasks: Time Series, Time Series Analysis

Network visualisation techniques are important tools for the exploratory analysis of complex systems. While these methods are regularly applied to visualise data on complex networks, we increasingly have access to time series data that can be modelled as temporal networks or dynamic graphs. In dynamic graphs, the temporal ordering of time-stamped edges determines the causal topology of a system, i.e., which nodes can, directly and indirectly, influence each other via a so-called causal path. This causal topology is crucial to understand dynamical processes, assess the role of nodes, or detect clusters. However, we lack graph drawing techniques that incorporate this information into static visualisations. Addressing this gap, we present a novel dynamic graph visualisation algorithm that utilises higher-order graphical models of causal paths in time series data to compute time-aware static graph visualisations. These visualisations combine the simplicity and interpretability of static graphs with a time-aware layout algorithm that highlights patterns in the causal topology that result from the temporal dynamics of edges.
AI-based Prediction of Independent Construction Safety Outcomes from Universal Attributes
Authors: Henrietta Baker, Matthew R. Hallowell, Antoine J. -P. Tixier
Date: 2019-08-16 00:00:00
Tasks: Attribute, BIG-bench Machine Learning, Injury Prediction

This paper significantly improves on, and finishes to validate, an approach proposed in previous research in which safety outcomes were predicted from attributes with machine learning. Like in the original study, we use Natural Language Processing (NLP) to extract fundamental attributes from raw incident reports and machine learning models are trained to predict safety outcomes. The outcomes predicted here are injury severity, injury type, body part impacted, and incident type. However, unlike in the original study, safety outcomes were not extracted via NLP but were provided by independent human annotations, eliminating any potential source of artificial correlation between predictors and predictands. Results show that attributes are still highly predictive, confirming the validity of the original approach. Other improvements brought by the current study include the use of (1) a much larger dataset featuring more than 90,000 reports, (2) two new models, XGBoost and linear SVM (Support Vector Machines), (3) model stacking, (4) a more straightforward experimental setup with more appropriate performance metrics, and (5) an analysis of per-category attribute importance scores. Finally, the injury severity outcome is well predicted, which was not the case in the original study. This is a significant advancement.
Simplify the Usage of Lexicon in Chinese NER
Authors: Ruotian Ma, Minlong Peng, Qi Zhang, Xuanjing Huang
Date: 2019-08-16 00:00:00
Proceeding: ACL 2020 6
Tasks: Chinese Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Recently, many works have tried to augment the performance of Chinese named entity recognition (NER) using word lexicons. As a representative, Lattice-LSTM (Zhang and Yang, 2018) has achieved new benchmark results on several public Chinese NER datasets. However, Lattice-LSTM has a complex model architecture. This limits its application in many industrial areas where real-time NER responses are needed. In this work, we propose a simple but effective method for incorporating the word lexicon into the character representations. This method avoids designing a complicated sequence modeling architecture, and for any neural NER model, it requires only subtle adjustment of the character representation layer to introduce the lexicon information. Experimental studies on four benchmark Chinese NER datasets show that our method achieves an inference speed up to 6.15 times faster than those of state-ofthe-art methods, along with a better performance. The experimental results also show that the proposed method can be easily incorporated with pre-trained models like BERT.
N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding
Authors: Ryan McConville, Raul Santos-Rodriguez, Robert J. Piechocki, Ian Craddock
Date: 2019-08-16 00:00:00
Tasks: Clustering, Deep Clustering, Image Clustering, Representation Learning, Time Series, Time Series Analysis, Time Series Clustering

Deep clustering has increasingly been demonstrating superiority over conventional shallow clustering algorithms. Deep clustering algorithms usually combine representation learning with deep neural networks to achieve this performance, typically optimizing a clustering and non-clustering loss. In such cases, an autoencoder is typically connected with a clustering network, and the final clustering is jointly learned by both the autoencoder and clustering network. Instead, we propose to learn an autoencoded embedding and then search this further for the underlying manifold. For simplicity, we then cluster this with a shallow clustering algorithm, rather than a deeper network. We study a number of local and global manifold learning methods on both the raw data and autoencoded embedding, concluding that UMAP in our framework is best able to find the most clusterable manifold in the embedding, suggesting local manifold learning on an autoencoded embedding is effective for discovering higher quality discovering clusters. We quantitatively show across a range of image and time-series datasets that our method has competitive performance against the latest deep clustering algorithms, including out-performing current state-of-the-art on several. We postulate that these results show a promising research direction for deep clustering. The code can be found at https://github.com/rymc/n2d
Multi-Domain Adaptation in Brain MRI through Paired Consistency and Adversarial Learning
Authors: Mauricio Orbes-Arteaga, Thomas Varsavsky, Carole H. Sudre, Zach Eaton-Rosen, Lewis J. Haddow, Lauge Sørensen, Mads Nielsen, Akshay Pai, Sébastien Ourselin, Marc Modat, Parashkev Nachev, M. Jorge Cardoso
Date: 2019-08-16 00:00:00
Tasks: Domain Adaptation

Supervised learning algorithms trained on medical images will often fail to generalize across changes in acquisition parameters. Recent work in domain adaptation addresses this challenge and successfully leverages labeled data in a source domain to perform well on an unlabeled target domain. Inspired by recent work in semi-supervised learning we introduce a novel method to adapt from one source domain to $n$ target domains (as long as there is paired data covering all domains). Our multi-domain adaptation method utilises a consistency loss combined with adversarial learning. We provide results on white matter lesion hyperintensity segmentation from brain MRIs using the MICCAI 2017 challenge data as the source domain and two target domains. The proposed method significantly outperforms other domain adaptation baselines.
Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning
Authors: Zhijiang Guo, Yan Zhang, Zhiyang Teng, Wei Lu
Date: 2019-08-16 00:00:00
Proceeding: TACL 2019 3
Tasks: Graph-to-Sequence, Machine Translation, Text Generation, Translation

We focus on graph-to-sequence learning, which can be framed as transducing graph structures to sequences for text generation. To capture structural information associated with graphs, we investigate the problem of encoding graphs using graph convolutional networks (GCNs). Unlike various existing approaches where shallow architectures were used for capturing local structural information only, we introduce a dense connection strategy, proposing a novel Densely Connected Graph Convolutional Networks (DCGCNs). Such a deep architecture is able to integrate both local and non-local features to learn a better structural representation of a graph. Our model outperforms the state-of-the-art neural models significantly on AMRto-text generation and syntax-based neural machine translation.
How Sequence-to-Sequence Models Perceive Language Styles?
Authors: Ruozi Huang, Mi Zhang, Xudong Pan, Beina Sheng
Date: 2019-08-16 00:00:00
Tasks: Informativeness, Style Transfer, Text Style Transfer

Style is ubiquitous in our daily language uses, while what is language style to learning machines? In this paper, by exploiting the second-order statistics of semantic vectors of different corpora, we present a novel perspective on this question via style matrix, i.e. the covariance matrix of semantic vectors, and explain for the first time how Sequence-to-Sequence models encode style information innately in its semantic vectors. As an application, we devise a learning-free text style transfer algorithm, which explicitly constructs a pair of transfer operators from the style matrices for style transfer. Moreover, our algorithm is also observed to be flexible enough to transfer out-of-domain sentences. Extensive experimental evidence justifies the informativeness of style matrix and the competitive performance of our proposed style transfer algorithm with the state-of-the-art methods.
FSGAN: Subject Agnostic Face Swapping and Reenactment
Authors: Yuval Nirkin, Yosi Keller, Tal Hassner
Date: 2019-08-16 00:00:00
Proceeding: ICCV 2019 10
Tasks: Face Reenactment, Face Swapping, Facial Inpainting, Semantic Segmentation

We present Face Swapping GAN (FSGAN) for face swapping and reenactment. Unlike previous work, FSGAN is subject agnostic and can be applied to pairs of faces without requiring training on those faces. To this end, we describe a number of technical contributions. We derive a novel recurrent neural network (RNN)-based approach for face reenactment which adjusts for both pose and expression variations and can be applied to a single image or a video sequence. For video sequences, we introduce continuous interpolation of the face views based on reenactment, Delaunay Triangulation, and barycentric coordinates. Occluded face regions are handled by a face completion network. Finally, we use a face blending network for seamless blending of the two faces while preserving target skin color and lighting conditions. This network uses a novel Poisson blending loss which combines Poisson optimization with perceptual loss. We compare our approach to existing state-of-the-art systems and show our results to be both qualitatively and quantitatively superior.
Recommendation with Attribute-aware Product Networks: A Representation Learning Model
Authors: Guannan Liu, Liang Zhang, Junjie Wu, Xiao Fang
Date: 2019-08-16 00:00:00
Tasks: Attribute, Recommendation Systems, Representation Learning

With the prosperity of business intelligence, recommender systems have evolved into a new stage that we not only care about what to recommend, but why it is recommended. Explainability of recommendations thus emerges as a focal point of research and becomes extremely desired in e-commerce. Existent studies along this line often exploit item attributes and correlations from different perspectives, but they yet lack an effective way to combine both types of information for deep learning of personalized interests. In light of this, we propose a novel graph structure, \emph{attribute network}, based on both items' co-purchase network and important attributes. A novel neural model called \emph{eRAN} is then proposed to generate recommendations from attribute networks with explainability and cold-start capability. Specifically, eRAN first maps items connected in attribute networks to low-dimensional embedding vectors through a deep autoencoder, and then an attention mechanism is applied to model the attractions of attributes to users, from which personalized item representation can be derived. Moreover, a pairwise ranking loss is constructed into eRAN to improve recommendations, with the assumption that item pairs co-purchased by a user should be more similar than those non-paired with negative sampling in personalized view. Experiments on real-world datasets demonstrate the effectiveness of our method compared with some state-of-the-art competitors. In particular, eRAN shows its unique abilities in recommending cold-start items with higher accuracy, as well as in understanding user preferences underlying complicated co-purchasing behaviors.
Empirical Bayesian Mixture Models for Medical Image Translation
Authors: Mikael Brudfors, John Ashburner, Parashkev Nachev, Yael Balbastre
Date: 2019-08-16 00:00:00
Tasks: Translation

Automatically generating one medical imaging modality from another is known as medical image translation, and has numerous interesting applications. This paper presents an interpretable generative modelling approach to medical image translation. By allowing a common model for group-wise normalisation and segmentation of brain scans to handle missing data, the model allows for predicting entirely missing modalities from one, or a few, MR contrasts. Furthermore, the model can be trained on a fairly small number of subjects. The proposed model is validated on three clinically relevant scenarios. Results appear promising and show that a principled, probabilistic model of the relationship between multi-channel signal intensities can be used to infer missing modalities -- both MR contrasts and CT images.
Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
Authors: Zihan Liu, Yan Xu, Genta Indra Winata, Pascale Fung
Date: 2019-08-16 00:00:00
Proceeding: WS 2019 8
Tasks: Language Modeling, Language Modelling, Machine Translation, NMT, Translation, Unsupervised Machine Translation

This paper describes CAiRE's submission to the unsupervised machine translation track of the WMT'19 news shared task from German to Czech. We leverage a phrase-based statistical machine translation (PBSMT) model and a pre-trained language model to combine word-level neural machine translation (NMT) and subword-level NMT models without using any parallel data. We propose to solve the morphological richness problem of languages by training byte-pair encoding (BPE) embeddings for German and Czech separately, and they are aligned using MUSE (Conneau et al., 2018). To ensure the fluency and consistency of translations, a rescoring mechanism is proposed that reuses the pre-trained language model to select the translation candidates generated through beam search. Moreover, a series of pre-processing and post-processing approaches are applied to improve the quality of final translations.
Attending to Future Tokens For Bidirectional Sequence Generation
Authors: Carolin Lawrence, Bhushan Kotnis, Mathias Niepert
Date: 2019-08-16 00:00:00
Proceeding: IJCNLP 2019 11

Neural sequence generation is typically performed token-by-token and left-to-right. Whenever a token is generated only previously produced tokens are taken into consideration. In contrast, for problems such as sequence classification, bidirectional attention, which takes both past and future tokens into consideration, has been shown to perform much better. We propose to make the sequence generation process bidirectional by employing special placeholder tokens. Treated as a node in a fully connected graph, a placeholder token can take past and future tokens into consideration when generating the actual output token. We verify the effectiveness of our approach experimentally on two conversational tasks where the proposed bidirectional model outperforms competitive baselines by a large margin.
Context-Aware Emotion Recognition Networks
Authors: Jiyoung Lee, Seungryong Kim, Sunok Kim, Jungin Park, Kwanghoon Sohn
Date: 2019-08-16 00:00:00
Proceeding: ICCV 2019 10
Tasks: Emotion Classification, Emotion Recognition, Emotion Recognition in Context

Traditional techniques for emotion recognition have focused on the facial expression analysis only, thus providing limited ability to encode context that comprehensively represents the emotional responses. We present deep networks for context-aware emotion recognition, called CAER-Net, that exploit not only human facial expression but also context information in a joint and boosting manner. The key idea is to hide human faces in a visual scene and seek other contexts based on an attention mechanism. Our networks consist of two sub-networks, including two-stream encoding networks to seperately extract the features of face and context regions, and adaptive fusion networks to fuse such features in an adaptive fashion. We also introduce a novel benchmark for context-aware emotion recognition, called CAER, that is more appropriate than existing benchmarks both qualitatively and quantitatively. On several benchmarks, CAER-Net proves the effect of context for emotion recognition. Our dataset is available at http://caer-dataset.github.io.
BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction
Authors: Weipeng Huang, Xingyi Cheng, Taifeng Wang, Wei Chu
Date: 2019-08-16 00:00:00
Tasks: Relation, Relation Extraction

In this paper, we report our method for the Information Extraction task in 2019 Language and Intelligence Challenge. We incorporate BERT into the multi-head selection framework for joint entity-relation extraction. This model extends existing approaches from three perspectives. First, BERT is adopted as a feature extraction layer at the bottom of the multi-head selection framework. We further optimize BERT by introducing a semantic-enhanced task during BERT pre-training. Second, we introduce a large-scale Baidu Baike corpus for entity recognition pre-training, which is of weekly supervised learning since there is no actual named entity label. Third, soft label embedding is proposed to effectively transmit information between entity recognition and relation extraction. Combining these three contributions, we enhance the information extracting ability of the multi-head selection model and achieve F1-score 0.876 on testset-1 with a single model. By ensembling four variants of our model, we finally achieve F1 score 0.892 (1st place) on testset-1 and F1 score 0.8924 (2nd place) on testset-2.
The Regularization of Small Sub-Constraint Satisfaction Problems
Authors: Sven Löffler, Ke Liu, Petra Hofstedt
Date: 2019-08-16 00:00:00

This paper describes a new approach on optimization of constraint satisfaction problems (CSPs) by means of substituting sub-CSPs with locally consistent regular membership constraints. The purpose of this approach is to reduce the number of fails in the resolution process, to improve the inferences made during search by the constraint solver by strengthening constraint propagation, and to maintain the level of propagation while reducing the cost of propagating the constraints. Our experimental results show improvements in terms of the resolution speed compared to the original CSPs and a competitiveness to the recent tabulation approach. Besides, our approach can be realized in a preprocessing step, and therefore wouldn't collide with redundancy constraints or parallel computing if implemented.