bluuebunny/crossref_metadata_2025 huggingface dataset:
HST-LSTM: A Hierarchical Spatial-Temporal Long-Short Term Memory Network for Location Prediction
Authors: Kong Dejiang , Wu Fei
Published: July 2018

The widely use of positioning technology has made mining the movements of people feasible and plenty of trajectory data have been accumulated. How to efficiently leverage these data for location prediction has become an increasingly popular research topic as it is fundamental to location-based services (LBS). The existing methods often focus either on long time (days or months) visit prediction (i.e., the recommendation of point of interest) or on real time location prediction (i.e., trajectory prediction). In this paper, we are interested in the location prediction problem in a weak real time condition and aim to predict users' movement in next minutes or hours. We propose a Spatial-Temporal Long-Short Term Memory (ST-LSTM) model which naturally combines spatial-temporal influence into LSTM to mitigate the problem of data sparsity. Further, we employ a hierarchical extension of the proposed ST-LSTM (HST-LSTM) in an encoder-decoder manner which models the contextual historic visit information in order to boost the prediction performance. The proposed HST-LSTM is evaluated on a real world trajectory data set and the experimental results demonstrate the effectiveness of the proposed model.
Variance Reduction in Black-box Variational Inference by Adaptive Importance Sampling
Authors: Li Ximing , Li Changchun , Chi Jinjin , Ouyang Jihong
Published: July 2018

Overdispersed black-box variational inference employs importance sampling to reduce the variance of the Monte Carlo gradient in black-box variational inference. A simple overdispersed proposal distribution is used. This paper aims to investigate how to adaptively obtain better proposal distribution for lower variance. To this end, we directly approximate the optimal proposal in theory using a Monte Carlo moment matching step at each variational iteration. We call this adaptive proposal moment matching proposal (MMP). Experimental results on two Bayesian models show that the MMP can effectively reduce variance in black-box learning, and perform better than baseline inference algorithms.
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning
Authors: Lin Shaohui , Ji Rongrong , Li Yuchao , Wu Yongjian , Huang Feiyue , Zhang Baochang
Published: July 2018

Accelerating convolutional neural networks has recently received ever-increasing research focus. Among various approaches proposed in the literature, filter pruning has been regarded as a promising solution, which is due to its advantage in significant speedup and memory reduction of both network model and intermediate feature maps. To this end, most approaches tend to prune filters in a layer-wise fixed manner, which is incapable to dynamically recover the previously removed filter, as well as jointly optimize the pruned network across layers. In this paper, we propose a novel global & dynamic pruning (GDP) scheme to prune redundant filters for CNN acceleration. In particular, GDP first globally prunes the unsalient filters across all layers by proposing a global discriminative function based on prior knowledge of filters. Second, it dynamically updates the filter saliency all over the pruned sparse network, and then recover the mistakenly pruned filter, followed by a retraining phase to improve the model accuracy. Specially, we effectively solve the corresponding non-convex optimization problem of the proposed GDP via stochastic gradient descent with greedy alternative updating. Extensive experiments show that, comparing to the state-of-the-art filter pruning methods, the proposed approach achieves superior performance to accelerate several cutting-edge CNNs on the ILSVRC 2012 benchmark.
Multinomial Logit Bandit with Linear Utility Functions
Authors: Ou Mingdong , Li Nan , Zhu Shenghuo , Jin Rong
Published: July 2018

Multinomial logit bandit is a sequential subset selection problem which arises in many applications. In each round, the player selects a K-cardinality subset from N candidate items, and receives a reward which is governed by a multinomial logit (MNL) choice model considering both item utility and substitution property among items. The player's objective is to dynamically learn the parameters of MNL model and maximize cumulative reward over a finite horizon T. This problem faces the exploration-exploitation dilemma, and the involved combinatorial nature makes it non-trivial. In recent years, there have developed some algorithms by exploiting specific characteristics of the MNL model, but all of them estimate the parameters of MNL model separately and incur a regret bound which is not preferred for large candidate set size N. In this paper, we consider the linear utility MNL choice model whose item utilities are represented as linear functions of d-dimension item features, and propose an algorithm, titled LUMB, to exploit the underlying structure. It is proven that the proposed algorithm achieves regret which is free of candidate set size. Experiments show the superiority of the proposed algorithm.
Reachability Analysis of Deep Neural Networks with Provable Guarantees
Authors: Ruan Wenjie , Huang Xiaowei , Kwiatkowska Marta
Published: July 2018

Verifying correctness for deep neural networks (DNNs) is challenging. We study a generic reachability problem for feed-forward DNNs which, for a given set of inputs to the network and a Lipschitz-continuous function over its outputs computes the lower and upper bound on the function values. Because the network and the function are Lipschitz continuous, all values in the interval between the lower and upper bound are reachable. We show how to obtain the safety verification problem, the output range analysis problem and a robustness measure by instantiating the reachability problem. We present a novel algorithm based on adaptive nested optimisation to solve the reachability problem. The technique has been implemented and evaluated on a range of DNNs, demonstrating its efficiency, scalability and ability to handle a broader class of networks than state-of-the-art verification approaches.
Student-t Variational Autoencoder for Robust Density Estimation
Authors: Takahashi Hiroshi , Iwata Tomoharu , Yamanaka Yuki , Yamada Masanori , Yagi Satoshi
Published: July 2018

We propose a robust multivariate density estimator based on the variational autoencoder (VAE). The VAE is a powerful deep generative model, and used for multivariate density estimation. With the original VAE, the distribution of observed continuous variables is assumed to be a Gaussian, where its mean and variance are modeled by deep neural networks taking latent variables as their inputs. This distribution is called the decoder. However, the training of VAE often becomes unstable. One reason is that the decoder of VAE is sensitive to the error between the data point and its estimated mean when its estimated variance is almost zero. We solve this instability problem by making the decoder robust to the error using a Bayesian approach to the variance estimation: we set a prior for the variance of the Gaussian decoder, and marginalize it out analytically, which leads to proposing the Student-t VAE. Numerical experiments with various datasets show that training of the Student-t VAE is robust, and the Student-t VAE achieves high density estimation performance.
Exploration by Distributional Reinforcement Learning
Authors: Tang Yunhao , Agrawal Shipra
Published: July 2018

We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning. We show that our proposed framework conceptually unifies multiple previous methods in exploration. We also derive a practical algorithm that achieves efficient exploration on challenging control tasks.
Algorithms or Actions? A Study in Large-Scale Reinforcement Learning
Authors: Tavares Anderson Rocha , Anbalagan Sivasubramanian , Soriano Marcolino Leandro , Chaimowicz Luiz
Published: July 2018

Large state and action spaces are very challenging to reinforcement learning. However, in many domains there is a set of algorithms available, which estimate the best action given a state. Hence, agents can either directly learn a performance-maximizing mapping from states to actions, or from states to algorithms. We investigate several aspects of this dilemma, showing sufficient conditions for learning over algorithms to outperform over actions for a finite number of training iterations. We present synthetic experiments to further study such systems. Finally, we propose a function approximation approach, demonstrating the effectiveness of learning over algorithms in real-time strategy games.
Deep into Hypersphere: Robust and Unsupervised Anomaly Discovery in Dynamic Networks
Authors: Teng Xian , Yan Muheng , Ertugrul Ali Mert , Lin Yu-Ru
Published: July 2018

The increasing and flexible use of autonomous systems in many domains -- from intelligent transportation systems, information systems, to business transaction management -- has led to challenges in understanding the "normal" and "abnormal" behaviors of those systems. As the systems may be composed of internal states and relationships among sub-systems, it requires not only warning users to anomalous situations but also provides "transparency" about how the anomalies deviate from normalcy for more appropriate intervention. We propose a unified anomaly discovery framework "DeepSphere" that simultaneously meet the above two requirements -- identifying the anomalous cases and further exploring the cases' anomalous structure localized in spatial and temporal context. DeepSphere leverages deep autoencoders and hypersphere learning methods, having the capability of isolating anomaly pollution and reconstructing normal behaviors. DeepSphere does not rely on human annotated samples and can generalize to unseen data. Extensive experiments on both synthetic and real datasets demonstrate the consistent and robust performance of the proposed method.
Differentiable Submodular Maximization
Authors: Tschiatschek Sebastian , Sahin Aytunc , Krause Andreas
Published: July 2018

We consider learning of submodular functions from data. These functions are important in machine learning and have a wide range of applications, e.g. data summarization, feature selection and active learning. Despite their combinatorial nature, submodular functions can be maximized approximately with strong theoretical guarantees in polynomial time. Typically, learning the submodular function and optimization of that function are treated separately, i.e. the function is first learned using a proxy objective and subsequently maximized. In contrast, we show how to perform learning and optimization jointly. By interpreting the output of greedy maximization algorithms as distributions over sequences of items and smoothening these distributions, we obtain a differentiable objective. In this way, we can differentiate through the maximization algorithms and optimize the model to work well with the optimization algorithm. We theoretically characterize the error made by our approach, yielding insights into the tradeoff of smoothness and accuracy. We demonstrate the effectiveness of our approach for jointly learning and optimizing on synthetic maxcut data, and on real world applications such as product recommendation and image collection summarization.
Ranking Preserving Nonnegative Matrix Factorization
Authors: Wang Jing , Tian Feng , Liu Weiwei , Wang Xiao , Zhang Wenjie , Yamanishi Kenji
Published: July 2018

Nonnegative matrix factorization (NMF),  a well-known technique  to find  parts-based representations of nonnegative data, has been widely studied. In reality,  ordinal relations often exist among data,  such as data i is more related to j than to q.  Such relative order is naturally available, and more importantly, it truly reflects the latent data structure.  Preserving the ordinal relations enables us to find structured representations of data that are faithful to the relative order, so that the learned representations become  more discriminative. However, current NMFs pay no attention to this. In this paper, we make the first attempt towards incorporating the ordinal relations and  propose a novel ranking preserving nonnegative matrix factorization (RPNMF) approach, which enforces the learned representations to be ranked according to the relations. We derive  iterative updating rules to solve RPNMF's objective function with  convergence guaranteed.  Experimental results with several datasets for clustering and classification have demonstrated that RPNMF achieves greater performance against the state-of-the-arts,  not only  in terms of  accuracy, but also interpretation of orderly data structure.
Convolutional Memory Blocks for Depth Data Representation Learning
Authors: Wang Keze , Lin Liang , Ren Chuangjie , Zhang Wei , Sun Wenxiu
Published: July 2018

Compared to natural RGB images, data captured by 3D / depth sensors (e.g., Microsoft Kinect) have different properties, e.g., less discriminable in appearance due to lacking color / texture information. Applying convolutional neural networks (CNNs) on these depth data would lead to unsatisfying learning efficiency, i.e., requiring large amounts of annotated training data for convergence. To address this issue, this paper proposes a novel memory network module, called Convolutional Memory Block (CMB), which empowers CNNs with the memory mechanism on handling depth data. Different from the existing memory networks that store long / short term dependency from sequential data, our proposed CMB focuses on modeling the representative dependency (correlation) among non-sequential samples. Specifically, our CMB consists of one internal memory (i.e., a set of feature maps) and three specific controllers, which enable a powerful yet efficient memory manipulation mechanism. In this way, the internal memory, being implicitly aggregated from all previous inputted samples, can learn to store and utilize representative features among the samples. Furthermore, we employ our CMB to develop a concise framework for predicting articulated pose from still depth images. Comprehensive evaluations on three public benchmarks demonstrate significant superiority (about 6%) of our framework over all the compared methods. More importantly, thanks to the enhanced learning efficiency, our framework can still achieve satisfying results using 50% less training data.
Multi-Level Metric Learning via Smoothed Wasserstein Distance
Authors: Xu Jie , Luo Lei , Deng Cheng , Huang Heng
Published: July 2018

Traditional metric learning methods aim to learn a single Mahalanobis distance metric M, which, however, is not discriminative enough to characterize the complex and heterogeneous data. Besides, if the descriptors of the data are not strictly aligned, Mahalanobis distance would fail to exploit the relations among them. To tackle these problems, in this paper, we propose a multi-level metric learning method using a smoothed Wasserstein distance to characterize the errors between any two samples, where the ground distance is considered as a Mahalanobis distance. Since smoothed Wasserstein distance provides not only a distance value but also a flow-network indicating how the probability mass is optimally transported between the bins, it is very effective in comparing two samples whether they are aligned or not. In addition, to make full use of the global and local structures that exist in data features, we further model the commonalities between various classification through a shared distance matrix and the classification-specific idiosyncrasies with additional auxiliary distance matrices. An efficient algorithm is developed to solve the proposed new model. Experimental evaluations on four standard databases show that our method obviously outperforms other state-of-the-art methods.
Convergence Analysis of Gradient Descent for Eigenvector Computation
Authors: Xu Zhiqiang , Cao Xin , Gao Xin
Published: July 2018

We present a novel, simple and systematic convergence analysis of gradient descent for eigenvector computation. As a popular, practical, and provable approach to numerous machine learning problems, gradient descent has found successful applications to eigenvector computation as well. However, surprisingly, it lacks a thorough theoretical analysis for the underlying geodesically non-convex problem. In this work, the convergence of the gradient descent solver for the leading eigenvector computation is shown to be at a global rate O(min{ (lambda_1/Delta_p)^2 log(1/epsilon), 1/epsilon }), where Delta_p=lambda_p-lambda_p+1>0 represents the generalized positive eigengap and always exists without loss of generality with lambda_i being the i-th largest eigenvalue of the given real symmetric matrix and p being the multiplicity of lambda_1. The rate is linear at (lambda_1/Delta_p)^2 log(1/epsilon) if (lambda_1/Delta_p)^2=O(1), otherwise sub-linear at O(1/epsilon). We also show that the convergence only logarithmically instead of quadratically depends on the initial iterate. Particularly, this is the first time the linear convergence for the case that the conventionally considered eigengap Delta_1= lambda_1 - lambda_2=0 but the generalized eigengap Delta_p satisfies (lambda_1/Delta_p)^2=O(1), as well as the logarithmic dependence on the initial iterate are established for the gradient descent solver. We are also the first to leverage for analysis the log principal angle between the iterate and the space of globally optimal solutions. Theoretical properties are verified in experiments.
A Generic Approach for Accelerating Stochastic Zeroth-Order Convex Optimization
Authors: Yu Xiaotian , King Irwin , Lyu Michael R. , Yang Tianbao
Published: July 2018

In this paper, we propose a generic approach for accelerating the convergence of existing algorithms to solve the problem of stochastic zeroth-order convex optimization (SZCO). Standard techniques for accelerating the convergence of stochastic zeroth-order algorithms are by exploring multiple functional evaluations (e.g., two-point evaluations), or by exploiting global conditions of the problem (e.g., smoothness and strong convexity). Nevertheless, these classic acceleration techniques are necessarily restricting the applicability of newly developed algorithms. The key of our proposed generic approach is to explore a local growth condition  (or called local error bound condition) of the objective function in SZCO. The benefits of the proposed acceleration technique are: (i) it is applicable to both settings with one-point evaluation and two-point evaluations; (ii) it does not necessarily require strong convexity or smoothness condition of the objective function; (iii) it yields an improvement on convergence for a broad family of problems. Empirical studies in various settings demonstrate the effectiveness of the proposed acceleration approach.
Label-Sensitive Task Grouping by Bayesian Nonparametric Approach for Multi-Task Multi-Label Learning
Authors: Zhang Xiao , Li Wenzhong , Nguyen Vu , Zhuang Fuzhen , Xiong Hui , Lu Sanglu
Published: July 2018

Multi-label learning is widely applied in many real-world applications, such as image and gene annotation. While most of the existing multi-label learning models focus on the single-task learning problem, there are always some tasks that share some commonalities, which can help each other to improve the learning performances if the knowledge in the similar tasks can be smartly shared. In this paper, we propose a LABel-sensitive TAsk Grouping framework, named LABTAG, based on Bayesian nonparametric approach for multi-task multi-label classification. The proposed framework explores the label correlations to capture feature-label patterns, and clusters similar tasks into groups with shared knowledge, which are learned jointly to produce a strengthened multi-task multi-label model. We evaluate the model performance on three public multi-task multi-label data sets, and the results show that LABTAG outperforms the compared baselines with a significant margin.
ANRL: Attributed Network Representation Learning via Deep Neural Networks
Authors: Zhang Zhen , Yang Hongxia , Bu Jiajun , Zhou Sheng , Yu Pinggang , Zhang Jianwei , Ester Martin , Wang Can
Published: July 2018

Network representation learning (RL) aims

to transform the nodes in a network into low-dimensional vector spaces while

preserving the inherent properties of the network. Though network RL has been intensively

studied, most existing works focus on either network structure or node

attribute information. In this paper, we propose a novel framework, named ANRL,

to incorporate both the network structure and node attribute information in a principled

way. Specifically, we propose a neighbor enhancement autoencoder to model the

node attribute information, which reconstructs its target neighbors instead of

itself. To capture the network structure, attribute-aware skip-gram model is designed

based on the attribute encoder to formulate the correlations between each node and its

direct or indirect neighbors. We conduct extensive experiments on six

real-world networks, including two social networks, two citation networks and

two user behavior networks. The results empirically show that ANRL can achieve

relatively significant gains in node classification and link prediction tasks.
Symbolic Synthesis of Fault-Tolerance Ratios in Parameterised Multi-Agent Systems
Authors: Kouvaros Panagiotis , Lomuscio Alessio , Pirovano Edoardo
Published: July 2018

We study the problem of determining the robustness of a multi-agent system of unbounded size against specifications expressed in a temporal-epistemic logic. We introduce a procedure to synthesise automatically the maximal ratio of faulty agents that may be present at runtime for a specification to be satisfied in a multi-agent system. We show the procedure to be sound and amenable to symbolic implementation. We present an implementation and report the experimental results obtained by running this on a number of protocols from swarm robotics.
Fast Model Identification via Physics Engines for Data-Efficient Policy Search
Authors: Zhu Shaojun , Kimmel Andrew , Bekris Kostas E. , Boularias Abdeslam
Published: July 2018

This paper presents a method for identifying mechanical parameters of robots or objects, such as their mass and friction coefficients. Key features are the use of off-the-shelf physics engines and the adaptation of a Bayesian optimization technique towards minimizing the number of real-world experiments needed for model-based reinforcement learning. The proposed framework reproduces in a physics engine experiments performed on a real robot and optimizes the model's mechanical parameters so as to match real-world trajectories. The optimized model is then used for learning a policy in simulation, before real-world deployment. It is well understood, however, that it is hard to exactly reproduce real trajectories in simulation. Moreover, a near-optimal policy can be frequently found with an imperfect model. Therefore, this work proposes a strategy for identifying a model that is just good enough to approximate the value of a locally optimal policy with a certain confidence, instead of wasting effort on identifying the most accurate model. Evaluations, performed both in simulation and on a real robotic manipulation task, indicate that the proposed strategy results in an overall time-efficient, integrated model identification and learning solution, which significantly improves the data-efficiency of existing policy search algorithms.
Improving Deep Neural Network Sparsity through Decorrelation Regularization
Authors: Zhu Xiaotian , Zhou Wengang , Li Houqiang
Published: July 2018

Modern deep learning models usually suffer high complexity in model size and computation when transplanted to resource constrained platforms. To this end, many works are dedicated to compressing deep neural networks. Adding group LASSO regularization is one of the most effective model compression methods since it generates structured sparse networks. We investigate the deep neural networks trained by group LASSO constraint and observe that even with strong sparsity regularization imposed, there still exists substantial filter correlation among the convolution filters, which is undesired for a compact neural network. We propose to suppress such correlation with a new kind of constraint called decorrelation regularization, which explicitly forces the network to learn a set of less correlated filters. The experiments on CIFAR10/100 and ILSVRC2012 datasets show that when combined our decorrelation regularization with group LASSO, the correlation between filters could be effectively weakened, which increases the sparsity of the resulting model and leads to better compressing performance.
Explaining Multi-Criteria Decision Aiding Models with an Extended Shapley Value
Authors: Labreuche Christophe , Fossier Simon
Published: July 2018

The capability to explain the result of aggregation models to decision makers is key to reinforcing user trust. In practice, Multi-Criteria Decision Aiding models are often organized in a hierarchical way, based on a tree of criteria. We present an explanation approach usable with any hierarchical multi-criteria model, based on an influence index of each attribute on the decision. A set of desirable axioms are defined. We show that there is a unique index fulfilling these axioms. This new index is an extension of the Shapley value on trees. An efficient rewriting of this index, drastically reducing the computation time, is obtained. Finally, the use of the new index is illustrated on an example.
Aspect-Level Deep Collaborative Filtering via Heterogeneous Information Networks
Authors: Han Xiaotian , Shi Chuan , Wang Senzhang , Yu Philip S. , Song Li
Published: July 2018

Latent factor models have been widely used for recommendation. Most existing latent factor models mainly utilize the rating information between users and items, although some recently extended models add some auxiliary information to learn a unified latent factor between users and items.  The unified latent factor only represents the latent features of users and items from the aspect of purchase history. However, the latent features of users and items may stem from different aspects, e.g., the brand-aspect and category-aspect of items. In this paper, we propose a Neural network based Aspect-level Collaborative Filtering model (NeuACF) to exploit different aspect latent factors. Through modelling rich objects and relations in recommender system as a heterogeneous information network, NeuACF first extracts different aspect-level similarity matrices of users and items through different meta-paths and then feeds an elaborately designed deep neural network with these matrices to learn aspect-level latent factors. Finally, the aspect-level latent factors are effectively fused with an attention mechanism for the top-N recommendation. Extensive experiments on three real datasets show that NeuACF significantly outperforms both existing latent factor models and recent neural network models.
Interpretable Recommendation via Attraction Modeling: Learning Multilevel Attractiveness over Multimodal Movie Contents
Authors: Hu Liang , Jian Songlei , Cao Longbing , Chen Qingkui
Published: July 2018

New contents like blogs and online videos are produced in every second in the new media age. We argue that attraction is one of the decisive factors for user selection of new contents. However, collaborative filtering cannot work without user feedback; and the existing content-based recommender systems are ineligible to capture and interpret the attractive points on new contents. Accordingly, we propose attraction modeling to learn and interpret user attractiveness. Specially, we build a multilevel attraction model (MLAM) over the content features -- the story (textual data) and cast members (categorical data) of movies. In particular, we design multilevel personal filters to calculate users' attractiveness on words, sentences and cast members at different levels. The experimental results show the superiority of MLAM over the state-of-the-art methods. In addition, a case study is provided to demonstrate the interpretability of MLAM by visualizing user attractiveness on a movie.
Hashtag2Vec: Learning Hashtag Representation with Relational Hierarchical Embedding Model
Authors: Liu Jie , He Zhicheng , Huang Yalou
Published: July 2018

Hashtags have always been important elements in many social network platforms and micro-blog services. Semantic understanding of hashtags is a critical and fundamental task for many applications on social networks, such as event analysis, theme discovery, information retrieval, etc. However, this task is challenging due to the sparsity, polysemy, and synonymy of hashtags. In this paper, we investigate the problem of hashtag embedding by combining the short text content with the various heterogeneous relations in social networks. Specifically, we first establish a network with hashtags as its nodes. Hierarchically, each of the hashtag nodes is associated with a set of tweets and each tweet contains a set of words. Then we devise an embedding model, called Hashtag2Vec, which exploits multiple relations of hashtag-hashtag, hashtag-tweet, tweet-word, and word-word relations based on the hierarchical heterogeneous network. In addition to embedding the hashtags, our proposed framework is capable of embedding the short social texts as well. Extensive experiments are conducted on two real-world datasets, and the results demonstrate the effectiveness of the proposed method.
Your Tweets Reveal What You Like: Introducing Cross-media Content Information into Multi-domain Recommendation
Authors: Ma Weizhi , Zhang Min , Wang Chenyang , Luo Cheng , Liu Yiqun , Ma Shaoping
Published: July 2018

Cold start is a challenging problem in recommender systems. Many previous studies attempt to utilize extra information from other platforms to alleviate the problem. Most of the leveraged information is on-topic, directly related to users' preferences in the target domain. Thought to be unrelated, users' off-topic content information (such as user tweets) is usually omitted. However, the off-topic content information also helps to indicate the similarity of users on their tastes, interests, and opinions, which matches the underlying assumption of Collaborative Filtering (CF) algorithms. In this paper, we propose a framework to capture the features from user's off-topic content information in social media and introduce them into Matrix Factorization (MF) based algorithms. The framework is easy to understand and flexible in different embedding approaches and MF based algorithms. To the best of our knowledge, there is no previous study in which user's off-topic content in other platforms is taken into consideration. By capturing the cross-platform content including both on-topic and off-topic information, multiple algorithms with several embedding learning approaches have achieved significant improvements in rating prediction on three datasets. Especially in cold start scenarios, we observe greater enhancement. The results confirm our suggestion that off-topic cross-media information also contributes to the recommendation.
Power-law Distribution Aware Trust Prediction
Authors: Wang Xiao , Zhang Ziwei , Wang Jing , Cui Peng , Yang Shiqiang
Published: July 2018

Trust prediction, aiming to predict the trust relations between users in a social network, is a key to helping users discover the reliable information. Many trust prediction methods are proposed based on the low-rank assumption of a trust network. However, one typical property of the trust network is that the trust relations follow the power-law distribution, i.e., few users are trusted by many other users, while most tail users have few trustors. Due to these tail users, the fundamental low-rank assumption made by existing methods is seriously violated and becomes unrealistic. In this paper, we propose a simple yet effective method to address the problem of the violated low-rank assumption. Instead of discovering the low-rank component of the trust network alone, we learn a sparse component of the trust network to describe the tail users simultaneously. With both of the learned low-rank and sparse components, the trust relations in the whole network can be better captured. Moreover, the transitive closure structure of the trust relations is also integrated into our model. We then derive an effective iterative algorithm to infer the parameters of our model, along with the proof of correctness. Extensive experimental results on real-world trust networks demonstrate the superior performance of our proposed method over the state-of-the-arts.
DeepTravel: a Neural Network Based Travel Time Estimation Model with Auxiliary Supervision
Authors: Zhang Hanyuan , Wu Hao , Sun Weiwei , Zheng Baihua
Published: July 2018

Estimating the travel time of a path is of great importance to smart urban mobility. Existing approaches are either based on estimating the time cost of each road segment which are not able to capture many cross-segment complex factors, or designed heuristically in a non-learning-based way which fail to leverage the natural abundant temporal labels of the data, i.e., the time stamp of each trajectory point. In this paper, we leverage on new development of deep neural networks and propose a novel auxiliary supervision model, namely DeepTravel, that can automatically and effectively extract different features, as well as make full use of the temporal labels of the trajectory data. We have conducted comprehensive experiments on real datasets to demonstrate the out-performance of DeepTravel over existing approaches.
JUMP: a Jointly Predictor for User Click and Dwell Time
Authors: Zhou Tengfei , Qian Hui , Shen Zebang , Zhang Chao , Wang Chengwei , Liu Shichen , Ou Wenwu
Published: July 2018

With the recent proliferation of recommendation system, there have been a lot of interests in session-based prediction methods, particularly those based on Recurrent Neural Network (RNN) and their variants. However, existing methods either ignore the dwell time prediction that plays an important role in measuring user's engagement on the content, or fail to process very short or noisy sessions. In this paper, we propose a joint predictor, JUMP, for both user click and dwell time in session-based settings. To map its input into a feature vector, JUMP adopts a novel three-layered RNN structure which includes a fast-slow layer for very short sessions and an attention layer for noisy sessions. Experiments demonstrate that JUMP outperforms state-of-the-art methods in both user click and dwell time prediction.
Curriculum Adversarial Training
Authors: Cai Qi-Zhi , Liu Chang , Song Dawn
Published: July 2018

Recently, deep learning has been applied to many security-sensitive applications, such as facial authentication. The existence of adversarial examples hinders such applications. The state-of-the-art result on defense shows that adversarial training can be applied to train a robust model on MNIST against adversarial examples; but it fails to achieve a high empirical worst-case accuracy on a more complex task, such as CIFAR-10 and SVHN.

In our work, we propose curriculum adversarial training (CAT) to resolve this issue. The basic idea is to develop a curriculum of adversarial examples generated by attacks with a wide range of strengths. With two techniques to mitigate the catastrophic forgetting and the generalization issues, we demonstrate that CAT can improve the prior art's empirical worst-case accuracy by a large margin of 25% on CIFAR-10 and 35% on SVHN. At the same, the model's performance on non-adversarial inputs is comparable to the state-of-the-art models.
Three-Head Neural Network Architecture for Monte Carlo Tree Search
Authors: Gao Chao , Müller Martin , Hayward Ryan
Published: July 2018

AlphaGo Zero pioneered the concept of two-head neural networks in Monte Carlo Tree Search (MCTS), where the policy output is used for prior action

probability and the state-value estimate is used for leaf node evaluation.

We propose a three-head neural net architecture with policy, state- and action-value outputs, which could lead to more efficient MCTS since

neural leaf estimate can still be back-propagated in tree with delayed node expansion and evaluation. To effectively train

the newly introduced action-value head on the same game dataset as for two-head nets, we exploit the optimal relations between parent

and children nodes for data augmentation and regularization. In our experiments for the game of Hex, the action-value head learning

achieves similar error as the state-value prediction of a two-head architecture. The resulting neural net models are then combined with

the same Policy Value MCTS (PV-MCTS) implementation. We show that, due to more efficient use of neural net evaluations, PV-MCTS with

three-head neural nets consistently performs better than the two-head ones, significantly outplaying the state-of-the-art player MoHex-CNN.
Towards Better Representation Learning for Personalized News Recommendation: a Multi-Channel Deep Fusion Approach
Authors: Lian Jianxun , Zhang Fuzheng , Xie Xing , Sun Guangzhong
Published: July 2018

Millions of news articles emerge every day. How to provide personalized news recommendations has become a critical task for service providers. In the past few decades, latent factor models has been widely used for building recommender systems (RSs). With the remarkable success of deep learning techniques especially in visual computing and natural language understanding, more and more researchers have been trying to leverage deep neural networks to learn latent representations for advanced RSs. Following mainstream deep learning-based RSs, we propose a novel deep fusion model (DFM), which aims to improve the representation learning abilities in deep RSs and can be used for both candidate retrieval and item re-ranking. There are two key components in our DFM approach, namely an inception module and an attention mechanism. The inception module improves the plain multi-layer network via leveraging of various levels of interaction simultaneously, while the attention mechanism merges latent representations learnt from different channels in a customized fashion. We conduct extensive experiments on a commercial news reading dataset, and the results demonstrate that the proposed DFM is superior to several state-of-the-art models.
Integrating Demand Response and Renewable Energy In Wholesale Market
Authors: Li Chaojie , Liu Chen , Yu Xinghuo , Deng Ke , Huang Tingwen , Liu Liangchen
Published: July 2018

Demand response (DR) can provide a cost-effect approach for reducing peak loads while renewable energy sources (RES) can result in an environmental-friendly solution for solving the problem of power shortage. The increasingly integration of DR and renewable energy bring challenging issues for energy policy makers, and electricity market regulators in the main power grid. In this paper, a new two-stage stochastic game model is introduced to operate the electricity market, where Stochastic Stackelberg-Cournot-Nash (SSCN) equilibrium  is applied to characterize the optimal energy bidding strategy of the forward market and the optimal energy trading strategy of the spot market. To obtain a SSCN equilibrium, sampling average approximation (SAA) technique is harnessed to address the stochastic game model in a distributed way. By this game model, the participation ratio of demand response can be significantly increased while the unreliability of power system caused by renewable energy resources can be considerably reduced. The effectiveness of proposed model is illustrated by extensive simulations.
A Brand-level Ranking System with the Customized Attention-GRU Model
Authors: Zhu Yu , Zhu Junxiong , Hou Jie , Li Yongliang , Wang Beidou , Guan Ziyu , Cai Deng
Published: July 2018

In e-commerce websites like Taobao, brand is playing a more important role in influencing users' decision of click/purchase, partly because users are now attaching more importance to the quality of products and brand is an indicator of quality. However, existing ranking systems are not specifically designed to satisfy this kind of demand. Some design tricks may partially alleviate this problem, but still cannot provide satisfactory results or may create additional interaction cost. In this paper, we design the first brand-level ranking system to address this problem. The key challenge of this system is how to sufficiently exploit users' rich behavior in e-commerce websites to rank the brands. In our solution, we firstly conduct the feature engineering specifically tailored for the personalized brand ranking problem and then rank the brands by an adapted Attention-GRU model containing three important modifications. Note that our proposed modifications can also apply to many other machine learning models on various tasks. We conduct a series of experiments to evaluate the effectiveness of our proposed ranking model and test the response to the brand-level ranking system from real users on a large-scale e-commerce platform, i.e. Taobao.
Translations as Additional Contexts for Sentence Classification
Authors: Amplayo Reinald Kim , Lee Kyungjae , Yeo Jinyoung , Hwang Seung-won
Published: July 2018

In sentence classification tasks, additional contexts, such as the neighboring sentences, may improve the accuracy of the classifier. However, such contexts are domain-dependent and thus cannot be used for another classification task with an inappropriate domain. In contrast, we propose the use of translated sentences as domain-free context that is always available regardless of the domain. We find that naive feature expansion of translations gains only marginal improvements and may decrease the performance of the classifier, due to possible inaccurate translations thus producing noisy sentence vectors. To this end, we present multiple context fixing attachment (MCFA), a series of modules attached to multiple sentence vectors to fix the noise in the vectors using the other sentence vectors as context. We show that our method performs competitively compared to previous models, achieving best classification performance on multiple data sets. We are the first to use translations as domain-free contexts for sentence classification.
Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words
Authors: Bollegala Danushka , Hayashi Kohei , Kawarabayashi Ken-ichi
Published: July 2018

Distributed word embeddings have shown superior performances in numerous Natural Language Processing (NLP) tasks.

However, their performances vary significantly across different tasks,

implying that the word embeddings learnt by those methods capture complementary aspects of lexical semantics.

Therefore, we believe that it is important to combine the existing word embeddings to produce more accurate and complete

meta-embeddings of words.

For this purpose, we propose an unsupervised locally linear meta-embedding learning method that takes

pre-trained word embeddings as the input, and produces more accurate meta embeddings.

Unlike previously proposed meta-embedding learning methods that learn a global projection over all words in a vocabulary,

our proposed method is sensitive to the differences in local neighbourhoods of the individual source word embeddings.

Moreover, we show that vector concatenation, a previously proposed highly competitive baseline

approach for integrating word embeddings, can be derived as a special case of the proposed method.

Experimental results on semantic similarity, word analogy, relation classification, and short-text classification tasks show that our

meta-embeddings to significantly outperform prior methods in several benchmark datasets,

establishing a new state of the art for meta-embeddings.
Adversarial Active Learning for Sequences Labeling and Generation
Authors: Deng Yue , Chen KaWai , Shen Yilin , Jin Hongxia
Published: July 2018

We introduce an active learning framework for general sequence learning tasks including sequence labeling and generation. Most existing active learning algorithms mainly rely on an uncertainty measure derived from the probabilistic classifier for query sample selection. However, such approaches suffer from two shortcomings in the context of sequence learning including 1) cold start problem and 2) label sampling dilemma. To overcome these shortcomings, we propose a deep-learning-based active learning framework to directly identify query samples from the perspective of adversarial learning.  Our approach intends to offer labeling  priorities for sequences whose information content are least covered by existing labeled data. We verify our sequence-based active learning approach  on two tasks including sequence labeling and sequence generation.
A Deep Modular RNN Approach for Ethos Mining
Authors: Duthie Rory , Budzynska Katarzyna
Published: July 2018

Automatically recognising and extracting the reasoning expressed in natural language text is extremely

demanding and only very recently has there been significant headway. While such argument mining focuses on logos (the content of what is said) evidence has demonstrated that using ethos (the character of the speaker) can sometimes be an even more powerful tool of influence. We study the UK parliamentary debates which furnish a rich source of ethos with linguistic material signalling the ethotic relationships between politicians. We then develop a novel deep modular recurrent neural network, DMRNN, approach and employ proven methods from argument mining and sentiment analysis to create an ethos mining pipeline. Annotation of ethotic statements is reliable and its extraction is robust (macro-F1 = 0.83), while annotation of polarity is perfect and its extraction is solid (macro-F1 = 0.84). By exploring correspondences between ethos in political discourse and major events in the political landscape through ethos analytics, we uncover tantalising evidence

that identifying expressions of positive and negative ethotic sentiment is a powerful instrument for understanding the dynamics of governments.
Efficient Pruning of Large Knowledge Graphs
Authors: Faralli Stefano , Finocchi Irene , Ponzetto Simone Paolo , Velardi Paola
Published: July 2018

In this paper we present an efficient and highly accurate algorithm to prune noisy or over-ambiguous knowledge graphs given as input an extensional definition of a domain of interest, namely as a set of instances or concepts. Our method climbs the graph in a bottom-up fashion, iteratively layering the graph and pruning nodes and edges in each layer while not compromising the connectivity of the set of input nodes. Iterative layering and protection of pre-defined nodes allow to extract semantically coherent DAG structures from noisy or over-ambiguous cyclic graphs, without loss of information and without incurring in computational bottlenecks, which are the main problem of state-of-the-art methods for cleaning large, i.e., Web-scale, knowledge graphs. We apply our algorithm to the tasks of pruning automatically acquired taxonomies using benchmarking data from a SemEval evaluation exercise, as well as the extraction of a domain-adapted taxonomy from the Wikipedia category hierarchy. The results show the superiority of our approach over state-of-art algorithms in terms of both output quality and computational efficiency.