pwc-archive/papers-with-abstracts huggingface dataset:
SDM: Sequential Deep Matching Model for Online Large-scale Recommender System
Authors: Fuyu Lv, Taiwei Jin, Changlong Yu, Fei Sun, Quan Lin, Keping Yang, Wilfred Ng
Date: 2019-09-01 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Capturing users' precise preferences is a fundamental problem in large-scale recommender system. Currently, item-based Collaborative Filtering (CF) methods are common matching approaches in industry. However, they are not effective to model dynamic and evolving preferences of users. In this paper, we propose a new sequential deep matching (SDM) model to capture users' dynamic preferences by combining short-term sessions and long-term behaviors. Compared with existing sequence-aware recommendation methods, we tackle the following two inherent problems in real-world applications: (1) there could exist multiple interest tendencies in one session. (2) long-term preferences may not be effectively fused with current session interests. Long-term behaviors are various and complex, hence those highly related to the short-term session should be kept for fusion. We propose to encode behavior sequences with two corresponding components: multi-head self-attention module to capture multiple types of interests and long-short term gated fusion module to incorporate long-term preferences. Successive items are recommended after matching between sequential user behavior vector and item embedding vectors. Offline experiments on real-world datasets show the superior performance of the proposed SDM. Moreover, SDM has been successfully deployed on online large-scale recommender system at Taobao and achieves improvements in terms of a range of commercial metrics.
All You Need is Ratings: A Clustering Approach to Synthetic Rating Datasets Generation
Authors: Diego Monti, Giuseppe Rizzo, Maurizio Morisio
Date: 2019-09-02 00:00:00
Tasks: All, Clustering, Recommendation Systems

The public availability of collections containing user preferences is of vital importance for performing offline evaluations in the field of recommender systems. However, the number of rating datasets is limited because of the costs required for their creation and the fear of violating the privacy of the users by sharing them. For this reason, numerous research attempts investigated the creation of synthetic collections of ratings using generative approaches. Nevertheless, these datasets are usually not reliable enough for conducting an evaluation campaign. In this paper, we propose a method for creating synthetic datasets with a configurable number of users that mimic the characteristics of already existing ones. We empirically validated the proposed approach by exploiting the synthetic datasets for evaluating different recommenders and by comparing the results with the ones obtained using real datasets.
Know2Look: Commonsense Knowledge for Visual Search
Authors: Sreyasi Nag Chowdhury, Niket Tandon, Gerhard Weikum
Date: 2019-09-02 00:00:00
Proceeding: WS 2016 6
Tasks: Retrieval

With the rise in popularity of social media, images accompanied by contextual text form a huge section of the web. However, search and retrieval of documents are still largely dependent on solely textual cues. Although visual cues have started to gain focus, the imperfection in object/scene detection do not lead to significantly improved results. We hypothesize that the use of background commonsense knowledge on query terms can significantly aid in retrieval of documents with associated images. To this end we deploy three different modalities - text, visual cues, and commonsense knowledge pertaining to the query - as a recipe for efficient search and retrieval.
Algorithms and System Architecture for Immediate Personalized News Recommendations
Authors: Takeshi Yoneda, Shunsuke Kozawa, Keisuke Osone, Yukinori Koide, Yosuke Abe, Yoshifumi Seki
Date: 2019-09-03 00:00:00
Tasks: Articles, Collaborative Filtering

Personalization plays an important role in many services, just as news does. Many studies have examined news personalization algorithms, but few have considered practical environments. This paper provides algorithms and system architecture for generating immediate personalized news in a practical environment. Immediacy means changes in news trends and user interests are reflected in recommended news lists quickly. Since news trends and user interests rapidly change, immediacy is critical in news personalization applications. We develop algorithms and system architecture to realize immediacy. Our algorithms are based on collaborative filtering of user clusters and evaluate news articles using click-through rate and decay scores based on the time elapsed since the user's last access. Existing studies have not fully discussed system architecture, so a major contribution of this paper is that we demonstrate a system architecture and realize our algorithms and a configuration example implemented on top of Amazon Web Services. We evaluate the proposed method both offline and online. The offline experiments are conducted through a real-world dataset from a commercial news delivery service, and online experiments are conducted via A/B testing on production environments. We confirm the effectiveness of our proposed method and also that our system architecture can operate in large-scale production environments.
Finding Salient Context based on Semantic Matching for Relevance Ranking
Authors: Yuanyuan Qi, Jiayue Zhang, Weiran Xu, Jun Guo
Date: 2019-09-03 00:00:00
Tasks: Information Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity

In this paper, we propose a salient-context based semantic matching method to improve relevance ranking in information retrieval. We first propose a new notion of salient context and then define how to measure it. Then we show how the most salient context can be located with a sliding window technique. Finally, we use the semantic similarity between a query term and the most salient context terms in a corpus of documents to rank those documents. Experiments on various collections from TREC show the effectiveness of our model compared to the state-of-the-art methods.
READ: Recursive Autoencoders for Document Layout Generation
Authors: Akshay Gadi Patil, Omri Ben-Eliezer, Or Perel, Hadar Averbuch-Elor
Date: 2019-09-01 00:00:00
Tasks: Layout Generation

Layout is a fundamental component of any graphic design. Creating large varieties of plausible document layouts can be a tedious task, requiring numerous constraints to be satisfied, including local ones relating different semantic elements and global constraints on the general appearance and spacing. In this paper, we present a novel framework, coined READ, for REcursive Autoencoders for Document layout generation, to generate plausible 2D layouts of documents in large quantities and varieties. First, we devise an exploratory recursive method to extract a structural decomposition of a single document. Leveraging a dataset of documents annotated with labeled bounding boxes, our recursive neural network learns to map the structural representation, given in the form of a simple hierarchy, to a compact code, the space of which is approximated by a Gaussian distribution. Novel hierarchies can be sampled from this space, obtaining new document layouts. Moreover, we introduce a combinatorial metric to measure structural similarity among document layouts. We deploy it to show that our method is able to generate highly variable and realistic layouts. We further demonstrate the utility of our generated layouts in the context of standard detection tasks on documents, showing that detection performance improves when the training data is augmented with generated documents whose layouts are produced by READ.
VISIR: Visual and Semantic Image Label Refinement
Authors: Sreyasi Nag Chowdhury, Niket Tandon, Hakan Ferhatosmanoglu, Gerhard Weikum
Date: 2019-09-02 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, object-detection, Object Detection, Retrieval, TAG

The social media explosion has populated the Internet with a wealth of images. There are two existing paradigms for image retrieval: 1) content-based image retrieval (CBIR), which has traditionally used visual features for similarity search (e.g., SIFT features), and 2) tag-based image retrieval (TBIR), which has relied on user tagging (e.g., Flickr tags). CBIR now gains semantic expressiveness by advances in deep-learning-based detection of visual labels. TBIR benefits from query-and-click logs to automatically infer more informative labels. However, learning-based tagging still yields noisy labels and is restricted to concrete objects, missing out on generalizations and abstractions. Click-based tagging is limited to terms that appear in the textual context of an image or in queries that lead to a click. This paper addresses the above limitations by semantically refining and expanding the labels suggested by learning-based object detection. We consider the semantic coherence between the labels for different objects, leverage lexical and commonsense knowledge, and cast the label assignment into a constrained optimization problem solved by an integer linear program. Experiments show that our method, called VISIR, improves the quality of the state-of-the-art visual labeling tools like LSDA and YOLO.
Metric Learning for Adversarial Robustness
Authors: Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, Baishakhi Ray
Date: 2019-09-03 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Adversarial Robustness, Metric Learning

Deep networks are well-known to be fragile to adversarial attacks. We conduct an empirical analysis of deep representations under the state-of-the-art attack method called PGD, and find that the attack causes the internal representation to shift closer to the "false" class. Motivated by this observation, we propose to regularize the representation space under attack with metric learning to produce more robust classifiers. By carefully sampling examples for metric learning, our learned representation not only increases robustness, but also detects previously unseen adversarial samples. Quantitative experiments show improvement of robustness accuracy by up to 4% and detection efficiency by up to 6% according to Area Under Curve score over prior work. The code of our work is available at https://github.com/columbia/Metric_Learning_Adversarial_Robustness.
Social Influence-based Attentive Mavens Mining and Aggregative Representation Learning for Group Recommendation
Authors: Peipei Wang, Lin Li, Yi Yu, Guandong Xu
Date: 2019-08-10 00:00:00
Tasks: Collaborative Filtering, Decision Making, Recommendation Systems, Representation Learning

Frequent group activities of human beings have become an indispensable part in their daily life. Group recommendation can recommend satisfactory activities to group members in the recommender systems, and the key issue is how to aggregate preferences in different group members. Most existing group recommendation employed the predefined static aggregation strategies to aggregate the preferences of different group members, but these static strategies cannot simulate the dynamic group decision-making. Meanwhile, most of these methods depend on intuitions or assumptions to analyze the influence of group members and lack of convincing theoretical support. We argue that the influence of group members plays a particularly important role in group decision-making and it can better assist group profile modeling and perform more accurate group recommendation. To tackle the issue of preference aggregation for group recommendation, we propose a novel attentive aggregation representation learning method based on sociological theory for group recommendation, namely SIAGR (short for "Social Influence-based Attentive Group Recommendation"), which takes attention mechanisms and the popular method (BERT) as the aggregation representation for group profile modeling. Specifically, we analyze the influence of group members based on social identity theory and two-step flow theory and exploit an attentive mavens mining method. In addition, we develop a BERT-based representation method to learn the interaction of group members. Lastly, we complete the group recommendation under the neural collaborative filtering framework and verify the effectiveness of the proposed method by experimenting.
Triclustering of Gene Expression Microarray Data Using Coarse-Grained Parallel Genetic Algorithm
Authors: Shubhankar Mohapatra, Moumita Sarkar, Anjali Mohapatra, Bhawani Sankar Biswal
Date: 2019-08-31 00:00:00
Tasks: Clustering, Drug Discovery

Microarray data analysis is one of the major area of research in the field computational biology. Numerous techniques like clustering, biclustering are often applied to microarray data to extract meaningful outcomes which play key roles in practical healthcare affairs like disease identification, drug discovery etc. But these techniques become obsolete when time as an another factor is considered for evaluation in such data. This problem motivates to use triclustering method on gene expression 3D microarray data. In this article, a new methodology based on coarse-grained parallel genetic approach is proposed to locate meaningful triclusters in gene expression data. The outcomes are quite impressive as they are more effective as compared to traditional state of the art genetic approaches previously applied for triclustering of 3D GCT microarray data.
Hierarchical Control for Bipedal Locomotion using Central Pattern Generators and Neural Networks
Authors: Sayantan Auddy, Sven Magg, Stefan Wermter
Date: 2019-09-02 00:00:00
Tasks: Motion Generation

The complexity of bipedal locomotion may be attributed to the difficulty in synchronizing joint movements while at the same time achieving high-level objectives such as walking in a particular direction. Artificial central pattern generators (CPGs) can produce synchronized joint movements and have been used in the past for bipedal locomotion. However, most existing CPG-based approaches do not address the problem of high-level control explicitly. We propose a novel hierarchical control mechanism for bipedal locomotion where an optimized CPG network is used for joint control and a neural network acts as a high-level controller for modulating the CPG network. By separating motion generation from motion modulation, the high-level controller does not need to control individual joints directly but instead can develop to achieve a higher goal using a low-dimensional control signal. The feasibility of the hierarchical controller is demonstrated through simulation experiments using the Neuro-Inspired Companion (NICO) robot. Experimental results demonstrate the controller's ability to function even without the availability of an exact robot model.
Error Analysis of Elitist Randomized Search Heuristics
Authors: Cong Wang, Yu Chen, Jun He, Chengwang Xie
Date: 2019-09-03 00:00:00
Tasks: Evolutionary Algorithms

When globally optimal solutions of complicated optimization problems cannot be located by evolutionary algorithms (EAs) in polynomial expected running time, the hitting time/running time analysis is not flexible enough to accommodate the requirement of theoretical study, because sometimes we have no idea on what approximation ratio is available in polynomial expected running time. Thus, it is necessary to propose an alternative routine for the theoretical analysis of EAs. To bridge the gap between theoretical analysis and algorithm implementation, in this paper we perform an error analysis where expected approximation error is estimated to evaluate performances of randomized search heuristics (RSHs). Based on the Markov chain model of RSHs, the multi-step transition matrix can be computed by diagonalizing the one-step transition matrix, and a general framework for estimation of expected approximation errors is proposed. Case studies indicate that the error analysis works well for both uni- and multi-modal benchmark problems. It leads to precise estimations of approximation error instead of asymptotic results on fitness values, which demonstrates its competitiveness to fixed budget analysis.
Meta-Learning with Warped Gradient Descent
Authors: Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, Raia Hadsell
Date: 2019-08-30 00:00:00
Proceeding: ICLR 2020 1
Tasks: Few-Shot Learning, Inductive Bias, Meta-Learning, Reinforcement Learning

Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works have approached this issue either by attempting to train a neural network that directly produces updates or by attempting to learn better initialisations or scaling factors for a gradient-based update rule. Both of these approaches pose challenges. On one hand, directly producing an update forgoes a useful inductive bias and can easily lead to non-converging behaviour. On the other hand, approaches that try to control a gradient-based update rule typically resort to computing gradients through the learning process to obtain their meta-gradients, leading to methods that can not scale beyond few-shot task adaptation. In this work, we propose Warped Gradient Descent (WarpGrad), a method that intersects these approaches to mitigate their limitations. WarpGrad meta-learns an efficiently parameterised preconditioning matrix that facilitates gradient descent across the task distribution. Preconditioning arises by interleaving non-linear layers, referred to as warp-layers, between the layers of a task-learner. Warp-layers are meta-learned without backpropagating through the task training process in a manner similar to methods that learn to directly produce updates. WarpGrad is computationally efficient, easy to implement, and can scale to arbitrarily large meta-learning problems. We provide a geometrical interpretation of the approach and evaluate its effectiveness in a variety of settings, including few-shot, standard supervised, continual and reinforcement learning.
Learning Digital Circuits: A Journey Through Weight Invariant Self-Pruning Neural Networks
Authors: Amey Agrawal, Rohit Karlupia
Date: 2019-08-30 00:00:00

Recently, in the paper "Weight Agnostic Neural Networks" Gaier & Ha utilized architecture search to find networks where the topology completely encodes the knowledge. However, architecture search in topology space is expensive. We use the existing framework of binarized networks to find performant topologies by constraining the weights to be either, zero or one. We show that such topologies achieve performance similar to standard networks while pruning more than 99% weights. We further demonstrate that these topologies can perform tasks using constant weights without any explicit tuning. Finally, we discover that in our setup each neuron acts like a NOR gate, virtually learning a digital circuit. We demonstrate the efficacy of our approach on computer vision datasets.
Ultra-Low Energy and High Speed LIF Neuron using Silicon Bipolar Impact Ionization MOSFET for Spiking Neural Networks
Authors: Alok Kumar Kamal, Jawar Singh
Date: 2019-09-02 00:00:00

Silicon bipolar impact ionization MOSFET offers the potential for realization of leaky integrated fire (LIF) neuron due to the presence of parasitic BJT in the floating body. In this work, we have proposed an L shaped gate bipolar impact ionization MOS (L-BIMOS), with reduced breakdown voltage ($V_{B}$ = 1.68 V) and demonstrated the functioning of LIF neuron based on positive feedback mechanism of parasitic BJT. Using 2-D TCAD simulations, we manifest that the proposed L-BIMOS exhibits a low threshold voltage (0.2 V) for firing a spike, and the minimum energy required to fire a single spike for L-BIMOS is calculated to be 0.18 pJ, which makes proposed device $194\times$ more energy efficient than PD-SOI MOSFET silicon neuron (MOSFET silicon neuron) and $5\times10^{3}$ times more energy efficient than analog/digital circuit based conventional neuron. Furthermore, the proposed L-BIMOS silicon neuron exhibits spiking frequency in the GHz range, when the drain is biased at $V_{DG}$ = 2.0 V.
Evolutionary reinforcement learning of dynamical large deviations
Authors: Stephen Whitelam, Daniel Jacobson, Isaac Tamblyn
Date: 2019-09-02 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We show how to calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, eventually allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces the evolutionary process acts directly on rates, and for models with large state spaces the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.
An efficient and perceptually motivated auditory neural encoding and decoding algorithm for spiking neural networks
Authors: Zihan Pan, Yansong Chua, Jibin Wu, Malu Zhang, Haizhou Li, Eliathamby Ambikairajah
Date: 2019-09-03 00:00:00
Tasks: Benchmarking, speech-recognition, Speech Recognition

Auditory front-end is an integral part of a spiking neural network (SNN) when performing auditory cognitive tasks. It encodes the temporal dynamic stimulus, such as speech and audio, into an efficient, effective and reconstructable spike pattern to facilitate the subsequent processing. However, most of the auditory front-ends in current studies have not made use of recent findings in psychoacoustics and physiology concerning human listening. In this paper, we propose a neural encoding and decoding scheme that is optimized for speech processing. The neural encoding scheme, that we call Biologically plausible Auditory Encoding (BAE), emulates the functions of the perceptual components of the human auditory system, that include the cochlear filter bank, the inner hair cells, auditory masking effects from psychoacoustic models, and the spike neural encoding by the auditory nerve. We evaluate the perceptual quality of the BAE scheme using PESQ; the performance of the BAE based on speech recognition experiments. Finally, we also built and published two spike-version of speech datasets: the Spike-TIDIGITS and the Spike-TIMIT, for researchers to use and benchmarking of future SNN research.
Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks
Authors: Charlotte Frenkel, Martin Lefebvre, David Bol
Date: 2019-09-03 00:00:00
Tasks: Edge-computing

While the backpropagation of error algorithm enables deep neural network training, it implies (i) bidirectional synaptic weight transport and (ii) update locking until the forward and backward passes are completed. Not only do these constraints preclude biological plausibility, but they also hinder the development of low-cost adaptive smart sensors at the edge, as they severely constrain memory accesses and entail buffering overhead. In this work, we show that the one-hot-encoded labels provided in supervised classification problems, denoted as targets, can be viewed as a proxy for the error sign. Therefore, their fixed random projections enable a layerwise feedforward training of the hidden layers, thus solving the weight transport and update locking problems while relaxing the computational and memory requirements. Based on these observations, we propose the direct random target projection (DRTP) algorithm and demonstrate that it provides a tradeoff between accuracy and computational cost that is suitable for adaptive edge computing devices.