OpenRaiser/Intern-Atlas huggingface dataset:
Neural Collaborative Filtering
Venue: arXiv cs.IR 2017
Year: 2017

In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation -- collaborative filtering -- on the basis of implicit feedback. Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in collaborative filtering -- the interaction between user and item features, they still resorted to matrix factorization and applied an inner product on the latent features of users and items. By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network-based Collaborative Filtering. NCF is generic and can express and generalize matrix factorization under its framework. To supercharge NCF modelling with non-linearities, we propose to leverage a multi-layer perceptron to learn the user-item interaction function. Extensive experiments on two real-world datasets show significant improvements of our proposed NCF framework over the state-of-the-art methods. Empirical evidence shows that using deeper layers of neural networks offers better recommendation performance.
Analysis of QoS of VoIP Traffic through WiFi-UMTS Networks
Venue: arXiv cs.NI 2017
Year: 2017

Simulation of VoIP (Voice over Internet Protocol) traffic through UMTS (Universal Mobile Telecommunication System) and WiFi (IEEE 802.11x) alone and together are analysed for Quality of Service (QoS) performance. The average jitter of VoIP transiting the WiFi-UMTS network has been found to be lower than that of either solely through the WiFi and the UMTS networks. It is normally expected to be higher than traversing through the WiFi network only. Both the MOS (Mean Opinion Score) and the packet end-to-end delay were also found to be much lower than expected through the heterogeneous WiFi-UMTS network.
SOCRATES: A System For Scalable Graph Analytics
Venue: arXiv cs.DC 2017
Year: 2017

A distributed semantic graph processing system that provides locality control, indexing, graph query, and parallel processing capabilities is presented.
Deep & Cross Network for Ad Click Predictions
Venue: arXiv cs.LG 2017
Year: 2017

Feature engineering has been the key to the success of many prediction models. However, the process is non-trivial and often requires manual feature engineering or exhaustive searching. DNNs are able to automatically learn feature interactions; however, they generate all the interactions implicitly, and are not necessarily efficient in learning all types of cross features. In this paper, we propose the Deep & Cross Network (DCN) which keeps the benefits of a DNN model, and beyond that, it introduces a novel cross network that is more efficient in learning certain bounded-degree feature interactions. In particular, DCN explicitly applies feature crossing at each layer, requires no manual feature engineering, and adds negligible extra complexity to the DNN model. Our experimental results have demonstrated its superiority over the state-of-art algorithms on the CTR prediction dataset and dense classification dataset, in terms of both model accuracy and memory usage.
Triangle-Free Penny Graphs: Degeneracy, Choosability, and Edge Count
Venue: arXiv cs.DM 2017
Year: 2017

We show that triangle-free penny graphs have degeneracy at most two, list coloring number (choosability) at most three, diameter $D=Ω(\sqrt n)$, and at most $\min\bigl(2n-Ω(\sqrt n),2n-D-2\bigr)$ edges.
High Efficient Reconstruction of Single-shot T2 Mapping from OverLapping-Echo Detachment Planar Imaging Based on Deep Residual Network
Venue: arXiv cs.CV 2017
Year: 2017

Purpose: An end-to-end deep convolutional neural network (CNN) based on deep residual network (ResNet) was proposed to efficiently reconstruct reliable T2 mapping from single-shot OverLapping-Echo Detachment (OLED) planar imaging. Methods: The training dataset was obtained from simulations carried out on SPROM software developed by our group. The relationship between the original OLED image containing two echo signals and the corresponded T2 mapping was learned by ResNet training. After the ResNet was trained, it was applied to reconstruct the T2 mapping from simulation and in vivo human brain data. Results: Though the ResNet was trained entirely on simulated data, the trained network was generalized well to real human brain data. The results from simulation and in vivo human brain experiments show that the proposed method significantly outperformed the echo-detachment-based method. Reliable T2 mapping was achieved within tens of milliseconds after the network had been trained while the echo-detachment-based OLED reconstruction method took minutes. Conclusion: The proposed method will greatly facilitate real-time dynamic and quantitative MR imaging via OLED sequence, and ResNet has the potential to reconstruct images from complex MRI sequence efficiently.
Open storm: a complete framework for sensing and control of urban watersheds
Venue: arXiv cs.SY 2017
Year: 2017

Leveraging recent advances in technologies surrounding the Internet of Things, "smart" water systems are poised to transform water resources management by enabling ubiquitous real-time sensing and control. Recent applications have demonstrated the potential to improve flood forecasting, enhance rainwater harvesting, and prevent combined sewer overflows. However, adoption of smart water systems has been hindered by a limited number of proven case studies, along with a lack of guidance on how smart water systems should be built. To this end, we review existing solutions, and introduce open storm---an open-source, end-to-end platform for real-time monitoring and control of watersheds. Open storm includes (i) a robust hardware stack for distributed sensing and control in harsh environments (ii) a cloud services platform that enables system-level supervision and coordination of water assets, and (iii) a comprehensive, web-based "how-to" guide, available on open-storm.org, that empowers newcomers to develop and deploy their own smart water networks. We illustrate the capabilities of the open storm platform through two ongoing deployments: (i) a high-resolution flash-flood monitoring network that detects and communicates flood hazards at the level of individual roadways and (ii) a real-time stormwater control network that actively modulates discharges from stormwater facilities to improve water quality and reduce stream erosion. Through these case studies, we demonstrate the real-world potential for smart water systems to enable sustainable management of water resources.
LSCHC: Layered Static Context Header Compression for LPWANs
Venue: arXiv cs.NI 2017
Year: 2017

Supporting IPv6/UDP/CoAP protocols over Low Power Wide Area Networks (LPWANs) can bring open networking, interconnection, and cooperation to this new type of Internet of Things networks. However, accommodating these protocols over these very low bandwidth networks requires efficient header compression schemes to meet the limited frame size of these networks, where only one or two octets are available to transmit all headers. Recently, the Internet Engineering Task Force (IETF) LPWAN working group drafted the Static Context Header Compression (SCHC), a new header compression scheme for LPWANs, which can provide a good compression factor without complex synchronization. In this paper, we present an implementation and evaluation of SCHC. We compare SCHC with IPHC, which also targets constrained networks. Additionally, we propose an enhancement of SCHC, Layered SCHC (LSCHC). LSCHC is a layered context that reduces memory consumption and processing complexity, and adds flexibility when compressing packets. Finally, we perform calculations to show the impact of SCHC/LSCHC on an example LPWAN technology, e.g. LoRaWAN, from the point of view of transmission time and reliability.
Towards Syntactic Iberian Polarity Classification
Venue: arXiv cs.CL 2017
Year: 2017

Lexicon-based methods using syntactic rules for polarity classification rely on parsers that are dependent on the language and on treebank guidelines. Thus, rules are also dependent and require adaptation, especially in multilingual scenarios. We tackle this challenge in the context of the Iberian Peninsula, releasing the first symbolic syntax-based Iberian system with rules shared across five official languages: Basque, Catalan, Galician, Portuguese and Spanish. The model is made available.
Learning Musical Relations using Gated Autoencoders
Venue: arXiv cs.SD 2017
Year: 2017

Music is usually highly structured and it is still an open question how to design models which can successfully learn to recognize and represent musical structure. A fundamental problem is that structurally related patterns can have very distinct appearances, because the structural relationships are often based on transformations of musical material, like chromatic or diatonic transposition, inversion, retrograde, or rhythm change. In this preliminary work, we study the potential of two unsupervised learning techniques - Restricted Boltzmann Machines (RBMs) and Gated Autoencoders (GAEs) - to capture pre-defined transformations from constructed data pairs. We evaluate the models by using the learned representations as inputs in a discriminative task where for a given type of transformation (e.g. diatonic transposition), the specific relation between two musical patterns must be recognized (e.g. an upward transposition of diatonic steps). Furthermore, we measure the reconstruction error of models when reconstructing musical transformed patterns. Lastly, we test the models in an analogy-making task. We find that it is difficult to learn musical transformations with the RBM and that the GAE is much more adequate for this task, since it is able to learn representations of specific transformations that are largely content-invariant. We believe these results show that models such as GAEs may provide the basis for more encompassing music analysis systems, by endowing them with a better understanding of the structures underlying music.
Isometries and Binary Images of Linear Block Codes over $Z_4+uZ_4$ and $Z_8+uZ_8$
Venue: arXiv cs.IT 2017
Year: 2017

Let $F_2$ be the binary field and $Z_{2^r}$ the residue class ring of integers modulo $2^r$, where $r$ is a positive integer. For the finite $16$-element commutative local Frobenius non-chain ring $Z_4+uZ_4$, where $u$ is nilpotent of index $2$, two weight functions are considered, namely the Lee weight and the homogeneous weight. With the appropriate application of these weights, isometric maps from $Z_4+uZ_4$ to the binary spaces $F_2^4$ and $F_2^8$, respectively, are established via the composition of other weight-based isometries. The classical Hamming weight is used on the binary space. The resulting isometries are then applied to linear block codes over $Z_4+uZ_4$ whose images are binary codes of predicted length, which may or may not be linear. Certain lower and upper bounds on the minimum distances of the binary images are also derived in terms of the parameters of the $Z_4+uZ_4$ codes. Several new codes and their images are constructed as illustrative examples. An analogous procedure is performed successfully on the ring $Z_8+uZ_8$, where $u^2=0$, which is a commutative local Frobenius non-chain ring of order $64$. It turns out that the method is possible in general for the class of rings $Z_{2^r}+uZ_{2^r}$, where $u^2=0$, for any positive integer $r$, using the generalized Gray map from $Z_{2^r}$ to $F_2^{2^{r-1}}$.
SMASH: One-Shot Model Architecture Search through HyperNetworks
Venue: arXiv cs.LG 2017
Year: 2017

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMASH
Phase Noise Compensation for OFDM Systems
Venue: arXiv cs.IT 2017
Year: 2017

We describe a low complexity method for time domain compensation of phase noise in OFDM systems. We extend existing methods in several respects. First we suggest using the Karhunen-Loéve representation of the phase noise process to estimate the phase noise. We then derive an improved datadirected choice of basis elements for LS phase noise estimation and present its total least square counterpart problem. The proposed method helps overcome one of the major weaknesses of OFDM systems. We also generalize the time domain phase noise compensation to the multiuser MIMO context. Finally we present simulation results using both simulated and measured phased noise. We quantify the tracking performance in the presence of residual carrier offset.
Eigen Evolution Pooling for Human Action Recognition
Venue: arXiv cs.CV 2017
Year: 2017

We introduce Eigen Evolution Pooling, an efficient method to aggregate a sequence of feature vectors. Eigen evolution pooling is designed to produce compact feature representations for a sequence of feature vectors, while maximally preserving as much information about the sequence as possible, especially the temporal evolution of the features over time. Eigen evolution pooling is a general pooling method that can be applied to any sequence of feature vectors, from low-level RGB values to high-level Convolutional Neural Network (CNN) feature vectors. We show that eigen evolution pooling is more effective than average, max, and rank pooling for encoding the dynamics of human actions in video. We demonstrate the power of eigen evolution pooling on UCF101 and Hollywood2 datasets, two human action recognition benchmarks, and achieve state-of-the-art performance.
A Question Answering Approach to Emotion Cause Extraction
Venue: arXiv cs.CL 2017
Year: 2017

Emotion cause extraction aims to identify the reasons behind a certain emotion expressed in text. It is a much more difficult task compared to emotion classification. Inspired by recent advances in using deep memory networks for question answering (QA), we propose a new approach which considers emotion cause identification as a reading comprehension task in QA. Inspired by convolutional neural networks, we propose a new mechanism to store relevant context in different memory slots to model context information. Our proposed approach can extract both word level sequence features and lexical features. Performance evaluation shows that our method achieves the state-of-the-art performance on a recently released emotion cause dataset, outperforming a number of competitive baselines by at least 3.01% in F-measure.
Syllable-level Neural Language Model for Agglutinative Language
Venue: arXiv cs.CL 2017
Year: 2017

Language models for agglutinative languages have always been hindered in past due to myriad of agglutinations possible to any given word through various affixes. We propose a method to diminish the problem of out-of-vocabulary words by introducing an embedding derived from syllables and morphemes which leverages the agglutinative property. Our model outperforms character-level embedding in perplexity by 16.87 with 9.50M parameters. Proposed method achieves state of the art performance over existing input prediction methods in terms of Key Stroke Saving and has been commercialized.
On Input Design for Regularized LTI System Identification: Power-constrained Input
Venue: arXiv cs.SY 2017
Year: 2017

Input design is an important issue for classical system identification methods but has not been investigated for the kernel-based regularization method (KRM) until very recently. In this paper, we consider in the time domain the input design problem of KRMs for LTI system identification. Different from the recent result, we adopt a Bayesian perspective and in particular make use of scalar measures (e.g., the $A$-optimality, $D$-optimality, and $E$-optimality) of the Bayesian mean square error matrix as the design criteria subject to power-constraint on the input. Instead to solve the optimization problem directly, we propose a two-step procedure. In the first step, by making suitable assumptions on the unknown input, we construct a quadratic map (transformation) of the input such that the transformed input design problems are convex, the number of optimization variables is independent of the number of input data, and their global minima can be found efficiently by applying well-developed convex optimization software packages. In the second step, we derive the expression of the optimal input based on the global minima found in the first step by solving the inverse image of the quadratic map. In addition, we derive analytic results for some special types of fixed kernels, which provide insights on the input design and also its dependence on the kernel structure.
Moving object tracking employing rigid body motion on matrix Lie groups
Venue: arXiv cs.RO 2017
Year: 2017

In this paper we propose a novel method for estimating rigid body motion by modeling the object state directly in the space of the rigid body motion group SE(2). It has been recently observed that a noisy manoeuvring object in SE(2) exhibits banana-shaped probability density contours in its pose. For this reason, we propose and investigate two state space models for moving object tracking: (i) a direct product SE(2)xR3 and (ii) a direct product of the two rigid body motion groups SE(2)xSE(2). The first term within these two state space constructions describes the current pose of the rigid body, while the second one employs its second order dynamics, i.e., the velocities. By this, we gain the flexibility of tracking omnidirectional motion in the vein of a constant velocity model, but also accounting for the dynamics in the rotation component. Since the SE(2) group is a matrix Lie group, we solve this problem by using the extended Kalman filter on matrix Lie groups and provide a detailed derivation of the proposed filters. We analyze the performance of the filters on a large number of synthetic trajectories and compare them with (i) the extended Kalman filter based constant velocity and turn rate model and (ii) the linear Kalman filter based constant velocity model. The results show that the proposed filters outperform the other two filters on a wide spectrum of types of motion.
Induction of Decision Trees based on Generalized Graph Queries
Venue: arXiv cs.LG 2017
Year: 2017

Usually, decision tree induction algorithms are limited to work with non relational data. Given a record, they do not take into account other objects attributes even though they can provide valuable information for the learning task. In this paper we present GGQ-ID3, a multi-relational decision tree learning algorithm that uses Generalized Graph Queries (GGQ) as predicates in the decision nodes. GGQs allow to express complex patterns (including cycles) and they can be refined step-by-step. Also, they can evaluate structures (not only single records) and perform Regular Pattern Matching. GGQ are built dynamically (pattern mining) during the GGQ-ID3 tree construction process. We will show how to use GGQ-ID3 to perform multi-relational machine learning keeping complexity under control. Finally, some real examples of automatically obtained classification trees and semantic patterns are shown. ----- Normalmente, los algoritmos de inducción de árboles de decisión trabajan con datos no relacionales. Dado un registro, no tienen en cuenta los atributos de otros objetos a pesar de que éstos pueden proporcionar información útil para la tarea de aprendizaje. En este artículo presentamos GGQ-ID3, un algoritmo de aprendizaje de árboles de decisiones multi-relacional que utiliza Generalized Graph Queries (GGQ) como predicados en los nodos de decisión. Los GGQs permiten expresar patrones complejos (incluyendo ciclos) y pueden ser refinados paso a paso. Además, pueden evaluar estructuras (no solo registros) y llevar a cabo Regular Pattern Matching. En GGQ-ID3, los GGQ son construidos dinámicamente (pattern mining) durante el proceso de construcción del árbol. Además, se muestran algunos ejemplos reales de árboles de clasificación multi-relacionales y patrones semánticos obtenidos automáticamente.
LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions
Venue: arXiv cs.LG 2017
Year: 2017

We present LADDER, the first deep reinforcement learning agent that can successfully learn control policies for large-scale real-world problems directly from raw inputs composed of high-level semantic information. The agent is based on an asynchronous stochastic variant of DQN (Deep Q Network) named DASQN. The inputs of the agent are plain-text descriptions of states of a game of incomplete information, i.e. real-time large scale online auctions, and the rewards are auction profits of very large scale. We apply the agent to an essential portion of JD's online RTB (real-time bidding) advertising business and find that it easily beats the former state-of-the-art bidding policy that had been carefully engineered and calibrated by human experts: during JD.com's June 18th anniversary sale, the agent increased the company's ads revenue from the portion by more than 50%, while the advertisers' ROI (return on investment) also improved significantly.
Future Word Contexts in Neural Network Language Models
Venue: arXiv cs.CL 2017
Year: 2017

Recently, bidirectional recurrent network language models (bi-RNNLMs) have been shown to outperform standard, unidirectional, recurrent neural network language models (uni-RNNLMs) on a range of speech recognition tasks. This indicates that future word context information beyond the word history can be useful. However, bi-RNNLMs pose a number of challenges as they make use of the complete previous and future word context information. This impacts both training efficiency and their use within a lattice rescoring framework. In this paper these issues are addressed by proposing a novel neural network structure, succeeding word RNNLMs (su-RNNLMs). Instead of using a recurrent unit to capture the complete future word contexts, a feedforward unit is used to model a finite number of succeeding, future, words. This model can be trained much more efficiently than bi-RNNLMs and can also be used for lattice rescoring. Experimental results on a meeting transcription task (AMI) show the proposed model consistently outperformed uni-RNNLMs and yield only a slight degradation compared to bi-RNNLMs in N-best rescoring. Additionally, performance improvements can be obtained using lattice rescoring and subsequent confusion network decoding.