OpenRaiser/Intern-Atlas huggingface dataset:
Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems
Venue: arXiv cs.IR 2018
Year: 2018

We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models for reliably recommending the N top-ranked items. The proposed models optimize a variant of the widely used discounted cumulative gain (DCG) objective function which differs from DCG in two important aspects: (i) It limits the evaluation of DCG only on the top N items in the ranked lists, thereby eliminating the impact of low-ranked items on the learned ranking function; and (ii) it incorporates weights that allow the model to leverage multiple types of implicit feedback with differing levels of reliability or trustworthiness. Because the resulting objective function is non-smooth and hence challenging to optimize, we consider two smooth approximations of the objective function, using the traditional sigmoid function and the rectified linear unit (ReLU). We propose a family of learning-to-rank algorithms (Top-N-Rank) that work with any smooth objective function. Then, a more efficient variant, Top-N-Rank.ReLU, is introduced, which effectively exploits the properties of ReLU function to reduce the computational complexity of Top-N-Rank from quadratic to linear in the average number of items rated by users. The results of our experiments using two widely used benchmarks, namely, the MovieLens data set and the Amazon Video Games data set demonstrate that: (i) The `top-N truncation' of the objective function substantially improves the ranking quality of the top N recommendations; (ii) using the ReLU for smoothing the objective function yields significant improvement in both ranking quality as well as runtime as compared to using the sigmoid; and (iii) Top-N-Rank.ReLU substantially outperforms the well-performing list-wise ranking methods in terms of ranking quality.
Smart Stamp Duty
Venue: arXiv cs.CY 2018
Year: 2018

Blockchain technology has enjoyed a massive adoption in cryptocurrencies such as Bitcoin. Following the success, many people have started to explore the possibility of implementing blockchain technology in different fields. We propose smart stamp duty, a system which can revolutionize the way stamp duty is managed and paid. The smart stamp duty offers significant improvements on the convenience when paying stamp duty. At the same time, the blockchain technology also provides the auditability of the transaction data. Smart stamp duty enables the expansion of the existing electronic stamp duty application to retail level as well as allows the taxpayers to pay the stamp duty of their electronic documents. Our proposed system also enables the taxpayers to print their electronic documents without losing the paid electronic-based stamps.
An Efficient Hybrid I/O Caching Architecture Using Heterogeneous SSDs
Venue: arXiv cs.PF 2018
Year: 2018

SSDs are emerging storage devices which unlike HDDs, do not have mechanical parts and therefore, have superior performance compared to HDDs. Due to the high cost of SSDs, entirely replacing HDDs with SSDs is not economically justified. Additionally, SSDs can endure a limited number of writes before failing. To mitigate the shortcomings of SSDs while taking advantage of their high performance, SSD caching is practiced in both academia and industry. Previously proposed caching architectures have only focused on either performance or endurance and neglected to address both parameters in suggested architectures. Moreover, the cost, reliability, and power consumption of such architectures is not evaluated. This paper proposes a hybrid I/O caching architecture that while offers higher performance than previous studies, it also improves power consumption with a similar budget. The proposed architecture uses DRAM, Read-Optimized SSD, and Write-Optimized SSD in a three-level cache hierarchy and tries to efficiently redirect read requests to either DRAM or RO-SSD while sending writes to WO-SSD. To provide high reliability, dirty pages are written to at least two devices which removes any single point of failure. The power consumption is also managed by reducing the number of accesses issued to SSDs. The proposed architecture reconfigures itself between performance- and endurance-optimized policies based on the workload characteristics to maintain an effective tradeoff between performance and endurance. We have implemented the proposed architecture on a server equipped with industrial SSDs and HDDs. The experimental results show that as compared to state-of-the-art studies, the proposed architecture improves performance and power consumption by an average of 8% and 28%, respectively, and reduces the cost by 5% while increasing the endurance cost by 4.7% and negligible reliability penalty.
Learning Discriminative Motion Features Through Detection
Venue: arXiv cs.CV 2018
Year: 2018

Despite huge success in the image domain, modern detection models such as Faster R-CNN have not been used nearly as much for video analysis. This is arguably due to the fact that detection models are designed to operate on single frames and as a result do not have a mechanism for learning motion representations directly from video. We propose a learning procedure that allows detection models such as Faster R-CNN to learn motion features directly from the RGB video data while being optimized with respect to a pose estimation task. Given a pair of video frames---Frame A and Frame B---we force our model to predict human pose in Frame A using the features from Frame B. We do so by leveraging deformable convolutions across space and time. Our network learns to spatially sample features from Frame B in order to maximize pose detection accuracy in Frame A. This naturally encourages our network to learn motion offsets encoding the spatial correspondences between the two frames. We refer to these motion offsets as DiMoFs (Discriminative Motion Features). In our experiments we show that our training scheme helps learn effective motion cues, which can be used to estimate and localize salient human motion. Furthermore, we demonstrate that as a byproduct, our model also learns features that lead to improved pose detection in still-images, and better keypoint tracking. Finally, we show how to leverage our learned model for the tasks of spatiotemporal action localization and fine-grained action recognition.
KF-LAX: Kronecker-factored curvature estimation for control variate optimization in reinforcement learning
Venue: arXiv cs.LG 2018
Year: 2018

A key challenge for gradient based optimization methods in model-free reinforcement learning is to develop an approach that is sample efficient and has low variance. In this work, we apply Kronecker-factored curvature estimation technique (KFAC) to a recently proposed gradient estimator for control variate optimization, RELAX, to increase the sample efficiency of using this gradient estimation method in reinforcement learning. The performance of the proposed method is demonstrated on a synthetic problem and a set of three discrete control task Atari games.
A Main/Subsidiary Network Framework for Simplifying Binary Neural Network
Venue: arXiv cs.CV 2018
Year: 2018

To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately. However, it is unclear how to combine the best of the two worlds to get extremely small and efficient models. In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models. A novel learning-based approach is proposed to prune filters in our main/subsidiary network framework, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme. We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods. Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models, including binarized NIN, VGG-11, and ResNet-18, on various image classification datasets.
Efficient Model-Free Reinforcement Learning Using Gaussian Process
Venue: arXiv cs.LG 2018
Year: 2018

Efficient Reinforcement Learning usually takes advantage of demonstration or good exploration strategy. By applying posterior sampling in model-free RL under the hypothesis of GP, we propose Gaussian Process Posterior Sampling Reinforcement Learning(GPPSTD) algorithm in continuous state space, giving theoretical justifications and empirical results. We also provide theoretical and empirical results that various demonstration could lower expected uncertainty and benefit posterior sampling exploration. In this way, we combined the demonstration and exploration process together to achieve a more efficient reinforcement learning.
Conditional Variational Autoencoder for Neural Machine Translation
Venue: arXiv cs.CL 2018
Year: 2018

We explore the performance of latent variable models for conditional text generation in the context of neural machine translation (NMT). Similar to Zhang et al., we augment the encoder-decoder NMT paradigm by introducing a continuous latent variable to model features of the translation process. We extend this model with a co-attention mechanism motivated by Parikh et al. in the inference network. Compared to the vision domain, latent variable models for text face additional challenges due to the discrete nature of language, namely posterior collapse. We experiment with different approaches to mitigate this issue. We show that our conditional variational model improves upon both discriminative attention-based translation and the variational baseline presented in Zhang et al. Finally, we present some exploration of the learned latent space to illustrate what the latent variable is capable of capturing. This is the first reported conditional variational model for text that meaningfully utilizes the latent variable without weakening the translation model.
Learning Item-Interaction Embeddings for User Recommendations
Venue: arXiv cs.IR 2018
Year: 2018

Industry-scale recommendation systems have become a cornerstone of the e-commerce shopping experience. For Etsy, an online marketplace with over 50 million handmade and vintage items, users come to rely on personalized recommendations to surface relevant items from its massive inventory. One hallmark of Etsy's shopping experience is the multitude of ways in which a user can interact with an item they are interested in: they can view it, favorite it, add it to a collection, add it to cart, purchase it, etc. We hypothesize that the different ways in which a user interacts with an item indicates different kinds of intent. Consequently, a user's recommendations should be based not only on the item from their past activity, but also the way in which they interacted with that item. In this paper, we propose a novel method for learning interaction-based item embeddings that encode the co-occurrence patterns of not only the item itself, but also the interaction type. The learned embeddings give us a convenient way of approximating the likelihood that one item-interaction pair would co-occur with another by way of a simple inner product. Because of its computational efficiency, our model lends itself naturally as a candidate set selection method, and we evaluate it as such in an industry-scale recommendation system that serves live traffic on Etsy.com. Our experiments reveal that taking interaction type into account shows promising results in improving the accuracy of modeling user shopping behavior.
Towards Automatic Identification of Elephants in the Wild
Venue: arXiv cs.CV 2018
Year: 2018

Identifying animals from a large group of possible individuals is very important for biodiversity monitoring and especially for collecting data on a small number of particularly interesting individuals, as these have to be identified first before this can be done. Identifying them can be a very time-consuming task. This is especially true, if the animals look very similar and have only a small number of distinctive features, like elephants do. In most cases the animals stay at one place only for a short period of time during which the animal needs to be identified for knowing whether it is important to collect new data on it. For this reason, a system supporting the researchers in identifying elephants to speed up this process would be of great benefit. In this paper, we present such a system for identifying elephants in the face of a large number of individuals with only few training images per individual. For that purpose, we combine object part localization, off-the-shelf CNN features, and support vector machine classification to provide field researches with proposals of possible individuals given new images of an elephant. The performance of our system is demonstrated on a dataset comprising a total of 2078 images of 276 individual elephants, where we achieve 56% top-1 test accuracy and 80% top-10 accuracy. To deal with occlusion, varying viewpoints, and different poses present in the dataset, we furthermore enable the analysts to provide the system with multiple images of the same elephant to be identified and aggregate confidence values generated by the classifier. With that, our system achieves a top-1 accuracy of 74% and a top-10 accuracy of 88% on the held-out test dataset.
Time-Bandwidth Product Perspective for Multi-Soliton Phase Modulation
Venue: arXiv cs.IT 2018
Year: 2018

Multi-soliton pulses are potential candidates for fiber optical transmission where the information is modulated and recovered in the so-called nonlinear Fourier domain. While this is an elegant technique to account for the channel nonlinearity, the obtained spectral efficiency, so far, is not competitive with classic Nyquist-based schemes. This is especially due to the observation that soliton pulses generally exhibit a large time-bandwidth product. We consider the phase modulation of spectral amplitudes of higher order solitons, taking into account their varying spectral and temporal behavior when propagating along the fiber. For second and third order solitons, we numerically optimize the pulse shapes to minimize the time-bandwidth product. We study the behavior of multi-soliton pulse duration and bandwidth and generally observe two corner cases where we approximate them analytically. We use these results to give an estimate on the minimal achievable time-bandwidth product per eigenvalue.
Towards the average-case analysis of substitution resolution in $λ$-calculus
Venue: arXiv cs.LO 2018
Year: 2018

Substitution resolution supports the computational character of $β$-reduction, complementing its execution with a capture-avoiding exchange of terms for bound variables. Alas, the meta-level definition of substitution, masking a non-trivial computation, turns $β$-reduction into an atomic rewriting rule, despite its varying operational complexity. In the current paper we propose a somewhat indirect average-case analysis of substitution resolution in the classic $λ$-calculus, based on the quantitative analysis of substitution in $λ\upsilon$, an extension of $λ$-calculus internalising the $\upsilon$-calculus of explicit substitutions. Within this framework, we show that for any fixed $n \geq 0$, the probability that a uniformly random, conditioned on size, $λ\upsilon$-term $\upsilon$-normalises in $n$ normal-order (i.e. leftmost-outermost) reduction steps tends to a computable limit as the term size tends to infinity. For that purpose, we establish an effective hierarchy $\left(\mathscr{G}_n\right)_n$ of regular tree grammars partitioning $\upsilon$-normalisable terms into classes of terms normalising in $n$ normal-order rewriting steps. The main technical ingredient in our construction is an inductive approach to the construction of $\mathscr{G}_{n+1}$ out of $\mathscr{G}_n$ based, in turn, on the algorithmic construction of finite intersection partitions, inspired by Robinson's unification algorithm. Finally, we briefly discuss applications of our approach to other term rewriting systems, focusing on two closely related formalisms, i.e. the full $λ\upsilon$-calculus and combinatory logic.
Trade Selection with Supervised Learning and OCA
Venue: arXiv cs.LG 2018
Year: 2018

In recent years, state-of-the-art methods for supervised learning have exploited increasingly gradient boosting techniques, with mainstream efficient implementations such as xgboost or lightgbm. One of the key points in generating proficient methods is Feature Selection (FS). It consists in selecting the right valuable effective features. When facing hundreds of these features, it becomes critical to select best features. While filter and wrappers methods have come to some maturity, embedded methods are truly necessary to find the best features set as they are hybrid methods combining features filtering and wrapping. In this work, we tackle the problem of finding through machine learning best a priori trades from an algorithmic strategy. We derive this new method using coordinate ascent optimization and using block variables. We compare our method to Recursive Feature Elimination (RFE) and Binary Coordinate Ascent (BCA). We show on a real life example the capacity of this method to select good trades a priori. Not only this method outperforms the initial trading strategy as it avoids taking loosing trades, it also surpasses other method, having the smallest feature set and the highest score at the same time. The interest of this method goes beyond this simple trade classification problem as it is a very general method to determine the optimal feature set using some information about features relationship as well as using coordinate ascent optimization.
BOSPHORUS: Bridging ANF and CNF Solvers
Venue: arXiv cs.LO 2018
Year: 2018

Algebraic Normal Form (ANF) and Conjunctive Normal Form (CNF) are commonly used to encode problems in Boolean algebra. ANFs are typically solved via Gr"obner basis algorithms, often using more memory than is feasible; while CNFs are solved using SAT solvers, which cannot exploit the algebra of polynomials naturally. We propose a paradigm that bridges between ANF and CNF solving techniques: the techniques are applied in an iterative manner to emph{learn facts} to augment the original problems. Experiments on over 1,100 benchmarks arising from four different applications domains demonstrate that learnt facts can significantly improve runtime and enable more benchmarks to be solved.
Scalable language model adaptation for spoken dialogue systems
Venue: arXiv cs.CL 2018
Year: 2018

Language models (LM) for interactive speech recognition systems are trained on large amounts of data and the model parameters are optimized on past user data. New application intents and interaction types are released for these systems over time, imposing challenges to adapt the LMs since the existing training data is no longer sufficient to model the future user interactions. It is unclear how to adapt LMs to new application intents without degrading the performance on existing applications. In this paper, we propose a solution to (a) estimate n-gram counts directly from the hand-written grammar for training LMs and (b) use constrained optimization to optimize the system parameters for future use cases, while not degrading the performance on past usage. We evaluated our approach on new applications intents for a personal assistant system and find that the adaptation improves the word error rate by up to 15% on new applications even when there is no adaptation data available for an application.
Deep Neural Networks Meet CSI-Based Authentication
Venue: arXiv cs.NI 2018
Year: 2018

The first step of a secure communication is authenticating legible users and detecting the malicious ones. In the last recent years, some promising schemes proposed using wireless medium network's features, in particular, channel state information (CSI) as a means for authentication. These schemes mainly compare user's previous CSI with the new received CSI to determine if the user is in fact what it is claiming to be. Despite high accuracy, these approaches lack the stability in authentication when the users rotate in their positions. This is due to a significant change in CSI when a user rotates which mislead the authenticator when it compares the new CSI with the previous ones. Our approach presents a way of extracting features from raw CSI measurements which are stable towards rotation. We extract these features by the means of a deep neural network. We also present a scenario in which users can be {efficiently} authenticated while they are at certain locations in an environment (even if they rotate); and, they will be rejected if they change their location. Also, experimental results are presented to show the performance of the proposed scheme.
Context is Key: New Approaches to Neural Coherence Modeling
Venue: arXiv cs.CL 2018
Year: 2018

We formulate coherence modeling as a regression task and propose two novel methods to combine techniques from our setup with pairwise approaches. The first of our methods is a model that we call "first-next," which operates similarly to selection sorting but conditions decision-making on information about already-sorted sentences. The second consists of a technique for adding context to regression-based models by concatenating sentence-level representations with an encoding of its corresponding out-of-order paragraph. This latter model achieves Kendall-tau distance and positional accuracy scores that match or exceed the current state-of-the-art on these metrics. Our results suggest that many of the gains that come from more complex, machine-translation inspired approaches can be achieved with simpler, more efficient models.
The 2017 ISO New England System Operational Analysis and Renewable Energy Integration Study (SOARES)
Venue: arXiv cs.SY 2018
Year: 2018

The bulk electric power system in New England is fundamentally changing. The representation of nuclear, coal and oil generation facilities is set to dramatically fall, and natural gas, wind and solar facilities will come to fill their place. The introduction of variable energy resources (VERs) like solar and wind, however, necessitates fundamental changes in the power grid's dynamic operation. VER forecasts are uncertain and their profiles are intermittent thus requiring greater quantities of operating reserves. This paper describes the methodology and the key findings of the 2017 ISO New England System Operational Analysis and Renewable Energy Integration Study (SOARES). This study was commissioned by the ISO New England stakeholders to investigate the effect of several scenarios of varying generation mix on normal operating reserves. The project was conducted using the holistic assessment approach called the Electric Power Enterprise Control System (EPECS) simulator.The EPECS characterizes a power system in terms of the physical power grid and its multiple layers of control including commitment decisions, economic dispatch, and regulation services. This paper provides precise definitions of how variable energy resources and operating reserves are modeled. It also includes detailed models of the day-ahead resource scheduling, the same-day resource scheduling, the real-time balancing operations and the regulation service. Among the key findings, the reports shows that scenarios with high penetrations of VERs exhaust their operating reserves for part of the year and all scenarios curtail semi-dispatchable resources both to complement operating reserves and to mitigate some of the topological limitations of the system. Overall, curtailment emerges as a key balancing performance control lever and there is a clear need for higher amounts of operating reserves.