OpenRaiser/Intern-Atlas huggingface dataset:
Retraining-Based Iterative Weight Quantization for Deep Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage requirement but also for fast inference operations. Quantization is known to be an effective model compression method and researchers are interested in minimizing the number of bits to represent parameters. In this work, we introduce an iterative technique to apply quantization, presenting high compression ratio without any modifications to the training algorithm. In the proposed technique, weight quantization is followed by retraining the model with full precision weights. We show that iterative retraining generates new sets of weights which can be quantized with decreasing quantization loss at each iteration. We also show that quantization is efficiently able to leverage pruning, another effective model compression method. Implementation issues on combining the two methods are also addressed. Our experimental results demonstrate that an LSTM model using 1-bit quantized weights is sufficient for PTB dataset without any accuracy degradation while previous methods demand at least 2-4 bits for quantized weights.
Multi-hop Inference for Sentence-level TextGraphs: How Challenging is Meaningfully Combining Information for Science Question Answering?
Venue: arXiv cs.CL 2018
Year: 2018

Question Answering for complex questions is often modeled as a graph construction or traversal task, where a solver must build or traverse a graph of facts that answer and explain a given question. This "multi-hop" inference has been shown to be extremely challenging, with few models able to aggregate more than two facts before being overwhelmed by "semantic drift", or the tendency for long chains of facts to quickly drift off topic. This is a major barrier to current inference models, as even elementary science questions require an average of 4 to 6 facts to answer and explain. In this work we empirically characterize the difficulty of building or traversing a graph of sentences connected by lexical overlap, by evaluating chance sentence aggregation quality through 9,784 manually-annotated judgments across knowledge graphs built from three free-text corpora (including study guides and Simple Wikipedia). We demonstrate semantic drift tends to be high and aggregation quality low, at between 0.04% and 3%, and highlight scenarios that maximize the likelihood of meaningfully combining information.
Scheduling under dynamic speed-scaling for minimizing weighted completion time and energy consumption
Venue: arXiv cs.DS 2018
Year: 2018

Since a few years there is an increasing interest in minimizing the energy consumption of computing systems. However in a shared computing system, users want to optimize their experienced quality of service, at the price of a high energy consumption. In this work, we address the problem of optimizing and designing mechanisms for a linear combination of weighted completion time and energy consumption on a single machine with dynamic speed-scaling. We show that minimizing linear combination reduces to a unit speed scheduling problem under a polynomial penalty function. In the mechanism design setting, we define a cost share mechanism and studied its properties, showing that it is truthful and the overcharging of total cost share is bounded by a constant.
Trust-based dynamic linear threshold models for non-competitive and competitive influence propagation
Venue: arXiv cs.SI 2018
Year: 2018

What are the key-features that enable an information diffusion model to explain the inherent dynamic, and often competitive, nature of real-world propagation phenomena? In this paper we aim to answer this question by proposing a novel class of diffusion models, inspired by the classic Linear Threshold model, and built around the following aspects: trust/distrust in the user relationships, which is leveraged to model different effects of social influence on the decisions taken by an individual; changes in adopting one or alternative information items; hesitation towards adopting an information item over time; latency in the propagation; time horizon for the unfolding of the diffusion process; and multiple cascades of information that might occur competitively. To the best of our knowledge, the above aspects have never been unified into the same LT-based diffusion model. We also define different strategies for the selection of the initial influencers to simulate non-competitive and competitive diffusion scenarios, particularly related to the problem of limitation of misinformation spread. Results on publicly available networks have shown the meaningfulness and uniqueness of our models.
CNN-Based Detection of Generic Constrast Adjustment with JPEG Post-processing
Venue: arXiv cs.CR 2018
Year: 2018

Detection of contrast adjustments in the presence of JPEG postprocessing is known to be a challenging task. JPEG post processing is often applied innocently, as JPEG is the most common image format, or it may correspond to a laundering attack, when it is purposely applied to erase the traces of manipulation. In this paper, we propose a CNN-based detector for generic contrast adjustment, which is robust to JPEG compression. The proposed system relies on a patch-based Convolutional Neural Network (CNN), trained to distinguish pristine images from contrast adjusted images, for some selected adjustment operators of different nature. Robustness to JPEG compression is achieved by training the CNN with JPEG examples, compressed over a range of Quality Factors (QFs). Experimental results show that the detector works very well and scales well with respect to the adjustment type, yielding very good performance under a large variety of unseen tonal adjustments.
LaKe: An Energy Efficient, Low Latency, Accelerated Key-Value Store
Venue: arXiv cs.AR 2018
Year: 2018

Key-value store is a popular type of cloud computing applications. The performance of key-value store applications have been shown to be very sensitive to load within the data center, and in particular to latency. As load within data center increases, it is becoming hard to maintain key-value store applications' performance, without exceeding both the processing capacity of hosts and the power budgets of racks. In this paper, we present LaKe: a low latency, power efficient key-value store design for cloud applications. LaKe is a modular design, combining multiple cores and cache layering, both in hardware and software. LaKe achieves full line rate throughput, while maintaining a latency of 1.1us and better power efficiency than existing hardware based memcached designs. Using the modularity of our design, we study trade-offs in the use of on-chip memory, SRAM and DRAM in accelerated designs and provide insights for future architectures.
Quantum-inspired Complex Word Embedding
Venue: arXiv cs.CL 2018
Year: 2018

A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words. For example, existing approaches in word embeddings will assign high probabilities to the words "Penguin" and "Fly" if they frequently co-occur, but it fails to capture the fact that they occur in an opposite sense - Penguins do not fly. We hypothesize that humans do not associate a single polarity or sentiment to each word. The word contributes to the overall polarity of a combination of words depending upon which other words it is combined with. This is analogous to the behavior of microscopic particles which exist in all possible states at the same time and interfere with each other to give rise to new states depending upon their relative phases. We make use of the Hilbert Space representation of such particles in Quantum Mechanics where we subscribe a relative phase to each word, which is a complex number, and investigate two such quantum inspired models to derive the meaning of a combination of words. The proposed models achieve better performances than state-of-the-art non-quantum models on the binary sentence classification task.
Webpage Saliency Prediction with Two-stage Generative Adversarial Networks
Venue: arXiv cs.CV 2018
Year: 2018

Web page saliency prediction is a challenge problem in image transformation and computer vision. In this paper, we propose a new model combined with web page outline information to prediction people's interest region in web page. For each web page image, our model can generate the saliency map which indicates the region of interest for people. A two-stage generative adversarial networks are proposed and image outline information is introduced for better transferring. Experiment results on FIWI dataset show that our model have better performance in terms of saliency prediction.
Representational Power of ReLU Networks and Polynomial Kernels: Beyond Worst-Case Analysis
Venue: arXiv cs.LG 2018
Year: 2018

There has been a large amount of interest, both in the past and particularly recently, into the power of different families of universal approximators, e.g. ReLU networks, polynomials, rational functions. However, current research has focused almost exclusively on understanding this problem in a worst-case setting, e.g. bounding the error of the best infinity-norm approximation in a box. In this setting a high-degree polynomial is required to even approximate a single ReLU. However, in real applications with high dimensional data we expect it is only important to approximate the desired function well on certain relevant parts of its domain. With this motivation, we analyze the ability of neural networks and polynomial kernels of bounded degree to achieve good statistical performance on a simple, natural inference problem with sparse latent structure. We give almost-tight bounds on the performance of both neural networks and low degree polynomials for this problem. Our bounds for polynomials involve new techniques which may be of independent interest and show major qualitative differences with what is known in the worst-case setting.
Performance Evaluation in High-Speed Networks by the Example of Intrusion Detection
Venue: arXiv cs.CR 2018
Year: 2018

Purchase decisions for devices in high-throughput networks as well as scientific evaluations of algorithms and technologies need to be based in measurements and clear procedures. Therefore, evaluation of network devices and their performance in high-throughput networks is an important part of research. In this paper, we document our approach and show its applicability for our purpose in an evaluation of two of the most well-known and common open source intrusion detection systems, Snort and Suricata. We used a hardware network testing setup to ensure a realistic environment and documented our testing approach. In our work, we focus on accuracy of the detection especially dependent on bandwidth. We would like to pass on our experiences and considerations.
CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation
Venue: arXiv cs.CL 2018
Year: 2018

Dating and romantic relationships not only play a huge role in our personal lives but also collectively influence and shape society. Today, many romantic partnerships originate from the Internet, signifying the importance of technology and the web in modern dating. In this paper, we present a text-based computational approach for estimating the relationship compatibility of two users on social media. Unlike many previous works that propose reciprocal recommender systems for online dating websites, we devise a distant supervision heuristic to obtain real world couples from social platforms such as Twitter. Our approach, the CoupleNet is an end-to-end deep learning based estimator that analyzes the social profiles of two users and subsequently performs a similarity match between the users. Intuitively, our approach performs both user profiling and match-making within a unified end-to-end framework. CoupleNet utilizes hierarchical recurrent neural models for learning representations of user profiles and subsequently coupled attention mechanisms to fuse information aggregated from two users. To the best of our knowledge, our approach is the first data-driven deep learning approach for our novel relationship recommendation problem. We benchmark our CoupleNet against several machine learning and deep learning baselines. Experimental results show that our approach outperforms all approaches significantly in terms of precision. Qualitative analysis shows that our model is capable of also producing explainable results to users.
Decision Making of Maximizers and Satisficers Based on Collaborative Explanations
Venue: arXiv cs.IR 2018
Year: 2018

Rating-based summary statistics are ubiquitous in e-commerce, and often are crucial components in personalized recommendation mechanisms. Largely left unexplored, however, is the issue to what extent the descriptives of rating distributions influence the decision making of online consumers. We conducted a conjoint experiment to explore how different summarizations of rating distributions (i.e., in the form of the number of ratings, mean, variance, skewness or the origin of the ratings) impact users' decision making. Results from over 200 participants indicate that users are primarily guided by the mean and the number of ratings and to a lesser degree by the variance, and the origin of a rating. We also looked into the maximizing behavioral tendencies of our participants, and found that in particular participants scoring high on the Decision Difficulty subscale displayed other sensitivities regarding the way in which rating distributions were summarized than others.
Observe and Look Further: Achieving Consistent Performance on Atari
Venue: arXiv cs.LG 2018
Year: 2018

Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games. We identify three key challenges that any algorithm needs to master in order to perform well on all games: processing diverse reward distributions, reasoning over long time horizons, and exploring efficiently. In this paper, we propose an algorithm that addresses each of these challenges and is able to learn human-level policies on nearly all Atari games. A new transformed Bellman operator allows our algorithm to process rewards of varying densities and scales; an auxiliary temporal consistency loss allows us to train stably using a discount factor of $γ= 0.999$ (instead of $γ= 0.99$) extending the effective planning horizon by an order of magnitude; and we ease the exploration problem by using human demonstrations that guide the agent towards rewarding states. When tested on a set of 42 Atari games, our algorithm exceeds the performance of an average human on 40 games using a common set of hyper parameters. Furthermore, it is the first deep RL algorithm to solve the first level of Montezuma's Revenge.
Deep Neural Networks for Swept Volume Prediction Between Configurations
Venue: arXiv cs.RO 2018
Year: 2018

Swept Volume (SV), the volume displaced by an object when it is moving along a trajectory, is considered a useful metric for motion planning. First, SV has been used to identify collisions along a trajectory, because it directly measures the amount of space required for an object to move. Second, in sampling-based motion planning, SV is an ideal distance metric, because it correlates to the likelihood of success of the expensive local planning step between two sampled configurations. However, in both of these applications, traditional SV algorithms are too computationally expensive for efficient motion planning. In this work, we train Deep Neural Networks (DNNs) to learn the size of SV for specific robot geometries. Results for two robots, a 6 degree of freedom (DOF) rigid body and a 7 DOF fixed-based manipulator, indicate that the network estimations are very close to the true size of SV and is more than 1500 times faster than a state of the art SV estimation algorithm.
Splitting source code identifiers using Bidirectional LSTM Recurrent Neural Network
Venue: arXiv cs.CL 2018
Year: 2018

Programmers make rich use of natural language in the source code they write through identifiers and comments. Source code identifiers are selected from a pool of tokens which are strongly related to the meaning, naming conventions, and context. These tokens are often combined to produce more precise and obvious designations. Such multi-part identifiers count for 97% of all naming tokens in the Public Git Archive - the largest dataset of Git repositories to date. We introduce a bidirectional LSTM recurrent neural network to detect subtokens in source code identifiers. We trained that network on 41.7 million distinct splittable identifiers collected from 182,014 open source projects in Public Git Archive, and show that it outperforms several other machine learning models. The proposed network can be used to improve the upstream models which are based on source code identifiers, as well as improving developer experience allowing writing code without switching the keyboard case.
Process Algebraic Architectural Description Languages: Generalizing Component-Oriented Mismatch Detection in the Presence of Nonsynchronous Communications
Venue: arXiv cs.SE 2018
Year: 2018

In the original paper, we showed how to enhance the expressiveness of a typical process algebraic architectural description language by including the capability of representing nonsynchronous communications. In particular, we extended the language by means of additional qualifiers enabling the designer to distinguish among synchronous, semi-synchronous, and asynchronous ports. Moreover, we showed how to modify techniques for detecting coordination mismatches such as the compatibility check for star topologies and the interoperability check for cycle topologies, in such a way that those two checks are applicable also in the presence of nonsynchronous communications. In this addendum, we generalize those results by showing that it is possible to verify in a component-oriented way an arbitrary property of a certain class (not only deadlock) over an entire architectural type having an arbitrary topology (not only stars and cycles) by considering also behavioral variations, exogenous variations, endogenous variations, and multiplicity variations, so to deal with the possible presence of nonsynchronous communications. The proofs are at the basis of some results mentioned in the book "A Process Algebraic Approach to Software Architecture Design" by Alessandro Aldini, Marco Bernardo, and Flavio Corradini, published by Springer in 2010.
Planning, Inference and Pragmatics in Sequential Language Games
Venue: arXiv cs.CL 2018
Year: 2018

We study sequential language games in which two players, each with private information, communicate to achieve a common goal. In such games, a successful player must (i) infer the partner's private information from the partner's messages, (ii) generate messages that are most likely to help with the goal, and (iii) reason pragmatically about the partner's strategy. We propose a model that captures all three characteristics and demonstrate their importance in capturing human behavior on a new goal-oriented dataset we collected using crowdsourcing.
Object Detection using Domain Randomization and Generative Adversarial Refinement of Synthetic Images
Venue: arXiv cs.CV 2018
Year: 2018

In this work, we present an application of domain randomization and generative adversarial networks (GAN) to train a near real-time object detector for industrial electric parts, entirely in a simulated environment. Large scale availability of labelled real world data is typically rare and difficult to obtain in many industrial settings. As such here, only a few hundred of unlabelled real images are used to train a Cyclic-GAN network, in combination with various degree of domain randomization procedures. We demonstrate that this enables robust translation of synthetic images to the real world domain. We show that a combination of the original synthetic (simulation) and GAN translated images, when used for training a Mask-RCNN object detection network achieves greater than 0.95 mean average precision in detecting and classifying a collection of industrial electric parts. We evaluate the performance across different combinations of training data.
Hyperspectral Imaging Technology and Transfer Learning Utilized in Identification Haploid Maize Seeds
Venue: arXiv cs.CV 2018
Year: 2018

It is extremely important to correctly identify the cultivars of maize seeds in the breeding process of maize. In this paper, the transfer learning as a method of deep learning is adopted to establish a model by combining with the hyperspectral imaging technology. The haploid seeds can be recognized from large amount of diploid maize ones with great accuracy through the model. First, the information of maize seeds on each wave band is collected using the hyperspectral imaging technology, and then the recognition model is built on VGG-19 network, which is pre-trained by large-scale computer vision database (Image-Net). The correct identification rate of model utilizing seed spectral images containing 256 wave bands (862.5-1704.2nm) reaches 96.32%, and the correct identification rate of the model utilizing the seed spectral images containing single-band reaches 95.75%. The experimental results show that, CNN model which is pre-trained by visible light image database can be applied to the near-infrared hyperspectral imaging-based identification of maize seeds, and high accurate identification rate can be achieved. Meanwhile, when there is small amount of data samples, it can still realize high recognition by using transfer learning. The model not only meets the requirements of breeding recognition, but also greatly reduce the cost occurred in sample collection.
Anaphora and Coreference Resolution: A Review
Venue: arXiv cs.CL 2018
Year: 2018

Entity resolution aims at resolving repeated references to an entity in a document and forms a core component of natural language processing (NLP) research. This field possesses immense potential to improve the performance of other NLP fields like machine translation, sentiment analysis, paraphrase detection, summarization, etc. The area of entity resolution in NLP has seen proliferation of research in two separate sub-areas namely: anaphora resolution and coreference resolution. Through this review article, we aim at clarifying the scope of these two tasks in entity resolution. We also carry out a detailed analysis of the datasets, evaluation metrics and research methods that have been adopted to tackle this NLP problem. This survey is motivated with the aim of providing the reader with a clear understanding of what constitutes this NLP problem and the issues that require attention.
Learning multiple non-mutually-exclusive tasks for improved classification of inherently ordered labels
Venue: arXiv cs.CV 2018
Year: 2018

Medical image classification involves thresholding of labels that represent malignancy risk levels. Usually, a task defines a single threshold, and when developing computer-aided diagnosis tools, a single network is trained per such threshold, e.g. as screening out healthy (very low risk) patients to leave possibly sick ones for further analysis (low threshold), or trying to find malignant cases among those marked as non-risk by the radiologist ("second reading", high threshold). We propose a way to rephrase the classification problem in a manner that yields several problems (corresponding to different thresholds) to be solved simultaneously. This allows the use of Multiple Task Learning (MTL) methods, significantly improving the performance of the original classifier, by facilitating effective extraction of information from existing data.