OpenRaiser/Intern-Atlas huggingface dataset:
Sentiment Classification using Images and Label Embeddings
Venue: arXiv cs.CL 2017
Year: 2017

In this project we analysed how much semantic information images carry, and how much value image data can add to sentiment analysis of the text associated with the images. To better understand the contribution from images, we compared models which only made use of image data, models which only made use of text data, and models which combined both data types. We also analysed if this approach could help sentiment classifiers generalize to unknown sentiments.
Cascade R-CNN: Delving into High Quality Object Detection
Venue: arXiv cs.CV 2017
Year: 2017

In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference-time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R-CNN is shown to surpass all single-model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R-CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code will be made available at https://github.com/zhaoweicai/cascade-rcnn.
Data Dropout in Arbitrary Basis for Deep Network Regularization
Venue: arXiv cs.CV 2017
Year: 2017

An important problem in training deep networks with high capacity is to ensure that the trained network works well when presented with new inputs outside the training dataset. Dropout is an effective regularization technique to boost the network generalization in which a random subset of the elements of the given data and the extracted features are set to zero during the training process. In this paper, a new randomized regularization technique in which we withhold a random part of the data without necessarily turning off the neurons/data-elements is proposed. In the proposed method, of which the conventional dropout is shown to be a special case, random data dropout is performed in an arbitrary basis, hence the designation Generalized Dropout. We also present a framework whereby the proposed technique can be applied efficiently to convolutional neural networks. The presented numerical experiments demonstrate that the proposed technique yields notable performance gain. Generalized Dropout provides new insight into the idea of dropout, shows that we can achieve different performance gains by using different bases matrices, and opens up a new research question as of how to choose optimal bases matrices that achieve maximal performance gain.
Cascading Behavior in Yelp Reviews
Venue: arXiv cs.SI 2017
Year: 2017

Social media has changed the landscape of marketing and consumer research as the adoption and promotion of businesses is becoming more and more dependent on how the customers are interacting and feeling about the business on platforms like Facebook, Twitter, Yelp etc. Social review websites like Yelp have become an important source of information about different businesses. Social influence on these online platforms can result in individuals adopting or promoting ideas and actions resulting in information cascades. Research on information cascades have been gaining popularity over the last few years but most of the research has been focused on platforms like Twitter and Facebook. Research on the adoption or promotion of product using cascades can help determine important latent patterns of social influence. In this work, we have analyzed the spread of information i.e. cascades in Yelp across different cities in Europe and North America. We have enumerated and analyzed different cascade topologies that occur in the Yelp social networks. Some of our significant findings include the presence of a significant number of cascades in Yelp reviews indicating the importance of social influence, heavy-tailed distribution of cascades and possibility to accurately predict the size of cascades on the basis of initial reviews. In addition, we have also found that the characteristics of the non-root nodes and the non-root reviews are much more important type of feature as compared to the properties of the root nodes and root reviews of the cascade. These findings can help social scientists to analyze customer behavior across different cities in a much more systematic way. Furthermore, it can also help the businesses in a city to figure out different consumer trends and hence improve their processes and offerings.
Data Cache Prefetching with Perceptron Learning
Venue: arXiv cs.AR 2017
Year: 2017

Cache prefetcher greatly eliminates compulsory cache misses, by fetching data from slower memory to faster cache before it is actually required by processors. Sophisticated prefetchers predict next use cache line by repeating program's historical spatial and temporal memory access pattern. However, they are error prone and the mis-predictions lead to cache pollution and exert extra pressure on memory subsystem. In this paper, a novel scheme of data cache prefetching with perceptron learning is proposed. The key idea is a two-level prefetching mechanism. A primary decision is made by utilizing previous table-based prefetching mechanism, e.g. stride prefetching or Markov prefetching, and then, a neural network, perceptron is taken to detect and trace program memory access patterns, to help reject those unnecessary prefetching decisions. The perceptron can learn from both local and global history in time and space, and can be easily implemented by hardware. This mechanism boost execution performance by ideally mitigating cache pollution and eliminating redundant memory request issued by prefetcher. Detailed evaluation and analysis were conducted based on SPEC CPU 2006 benchmarks. The simulation results show that generally the proposed scheme yields a geometric mean of 60.64%-83.84% decrease in prefetching memory requests without loss in instruction per cycle(IPC)(floating between -2.22% and 2.55%) and cache hit rate(floating between -1.67% and 2.46%). Though it is possible that perceptron may refuse useful blocks and thus cause minor raise in cache miss rate, lower memory request count can decrease average memory access latency, which compensate for the loss, and in the meantime, enhance overall performance in multi-programmed workloads.
Boolean function analysis meets stochastic optimization: An approximation scheme for stochastic knapsack
Venue: arXiv cs.DS 2017
Year: 2017

The stochastic knapsack problem is the stochastic variant of the classical knapsack problem in which the algorithm designer is given a a knapsack with a given capacity and a collection of items where each item is associated with a profit and a probability distribution on its size. The goal is to select a subset of items with maximum profit and violate the capacity constraint with probability at most $p$ (referred to as the overflow probability). While several approximation algorithms have been developed for this problem, most of these algorithms relax the capacity constraint of the knapsack. In this paper, we design efficient approximation schemes for this problem without relaxing the capacity constraint. (i) Our first result is in the case when item sizes are Bernoulli random variables. In this case, we design a (nearly) fully polynomial time approximation scheme (FPTAS) which only relaxes the overflow probability. (ii) Our second result generalizes the first result to the case when all the item sizes are supported on a (common) set of constant size. (iii) Our third result is in the case when item sizes are so-called "hypercontractive" random variables i.e., random variables whose second and fourth moments are within constant factors of each other. In other words, the kurtosis of the random variable is upper bounded by a constant. Crucially, all of our algorithms meet the capacity constraint exactly, a result which was previously known only when the item sizes were Poisson or Gaussian random variables. Our results rely on new connections between Boolean function analysis and stochastic optimization. We believe that these ideas and techniques may prove to be useful in other stochastic optimization problems as well.
End-to-End Relation Extraction using Markov Logic Networks
Venue: arXiv cs.AI 2017
Year: 2017

The task of end-to-end relation extraction consists of two sub-tasks: i) identifying entity mentions along with their types and ii) recognizing semantic relations among the entity mention pairs. %Identifying entity mentions along with their types and recognizing semantic relations among the entity mentions, are two very important problems in Information Extraction. It has been shown that for better performance, it is necessary to address these two sub-tasks jointly. We propose an approach for simultaneous extraction of entity mentions and relations in a sentence, by using inference in Markov Logic Networks (MLN). We learn three different classifiers : i) local entity classifier, ii) local relation classifier and iii) "pipeline" relation classifier which uses predictions of the local entity classifier. Predictions of these classifiers may be inconsistent with each other. We represent these predictions along with some domain knowledge using weighted first-order logic rules in an MLN and perform joint inference over the MLN to obtain a global output with minimum inconsistencies. Experiments on the ACE (Automatic Content Extraction) 2004 dataset demonstrate that our approach of joint extraction using MLNs outperforms the baselines of individual classifiers. Our end-to-end relation extraction performance is better than 2 out of 3 previous results reported on the ACE 2004 dataset.
Model Checking in multiplayer games development
Venue: arXiv cs.SE 2017
Year: 2017

Multiplayer computer games play a big role in the ever-growing entertainment industry. Being competitive in this industry means releasing the best possible software, and reliability is a key feature to win the market. Computer games are also actively used to simulate different robotic systems where reliability is even more important, and potentially critical. Traditional software testing approaches can check a subset of all the possible program executions, and they can never guarantee complete absence of errors in the source code. On the other hand, during more than twenty years, Model Checking has demonstrated to be a powerful instrument for formal verification of large hardware and software components. In this paper, we contribute with a novel approach to formally verify computer games. We propose a method of model construction that starts from a computer game description and utilizes Model Checking technique. We apply the method on a case study: the game Penguin Clash. Finally, an approach to game model reduction (and its implementation) is introduced in order to address the state explosion problem.
An Encoder-Decoder Model for ICD-10 Coding of Death Certificates
Venue: arXiv cs.CL 2017
Year: 2017

Information extraction from textual documents such as hospital records and healthrelated user discussions has become a topic of intense interest. The task of medical concept coding is to map a variable length text to medical concepts and corresponding classification codes in some external system or ontology. In this work, we utilize recurrent neural networks to automatically assign ICD-10 codes to fragments of death certificates written in English. We develop end-to-end neural architectures directly tailored to the task, including basic encoder-decoder architecture for statistical translation. In order to incorporate prior knowledge, we concatenate cosine similarities vector among the text and dictionary entry to the encoded state. Being applied to a standard benchmark from CLEF eHealth 2017 challenge, our model achieved F-measure of 85.01% on a full test set with significant improvement as compared to the average score of 62.2% for all official participants approaches.
On Out-of-Band Emissions of Quantized Precoding in Massive MU-MIMO-OFDM
Venue: arXiv cs.IT 2017
Year: 2017

We analyze out-of-band (OOB) emissions in the massive multi-user (MU) multiple-input multiple-output (MIMO) downlink. We focus on systems in which the base station (BS) is equipped with low-resolution digital-to-analog converters (DACs) and orthogonal frequency-division multiplexing (OFDM) is used to communicate to the user equipments (UEs) over frequency-selective channels. We demonstrate that analog filtering in combination with simple frequency-domain digital predistortion (DPD) at the BS enables a significant reduction of OOB emissions, but degrades the signal-to-interference-noise-and-distortion ratio (SINDR) at the UEs and increases the peak-to-average power ratio (PAR) at the BS. We use Bussgang's theorem to characterize the tradeoffs between OOB emissions, SINDR, and PAR, and to study the impact of analog filters and DPD on the error-rate performance of the massive MU-MIMO-OFDM downlink. Our results show that by carefully tuning the parameters of the analog filters, one can achieve a significant reduction in OOB emissions with only a moderate degradation of error-rate performance and PAR.
A Context-aware Recommender System for Hyperlocal News: A Conceptual Framework
Venue: arXiv cs.IR 2017
Year: 2017

Recommender systems (RSs) have been popular in variety of application domains due to the increased demand for filtering and sorting items and information. Today, there is a numerous approaches and algorithms of data filtering and recommendations. This works presents a conceptual framework for constructing a mobile RS in hyper-local news domain. The mobile RS is designed to deal with specific requirements of news readers, such as spatial- temporal relevance, recency, real-time update and validated news. The implementation of the RS in a distributed file system is also discussed.
Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars
Venue: arXiv cs.CV 2017
Year: 2017

As an initial assessment, over 480,000 labeled virtual images of normal highway driving were readily generated in Grand Theft Auto V's virtual environment. Using these images, a CNN was trained to detect following distance to cars/objects ahead, lane markings, and driving angle (angular heading relative to lane centerline): all variables necessary for basic autonomous driving. Encouraging results were obtained when tested on over 50,000 labeled virtual images from substantially different GTA-V driving environments. This initial assessment begins to define both the range and scope of the labeled images needed for training as well as the range and scope of labeled images needed for testing the definition of boundaries and limitations of trained networks. It is the efficacy and flexibility of a "GTA-V"-like virtual environment that is expected to provide an efficient well-defined foundation for the training and testing of Convolutional Neural Networks for safe driving. Additionally, described is the Princeton Virtual Environment (PVE) for the training, testing and enhancement of safe driving AI, which is being developed using the video-game engine Unity. PVE is being developed to recreate rare but critical corner cases that can be used in re-training and enhancing machine learning models and understanding the limitations of current self driving models. The Florida Tesla crash is being used as an initial reference.
AI Oriented Large-Scale Video Management for Smart City: Technologies, Standards and Beyond
Venue: arXiv cs.CV 2017
Year: 2017

Deep learning has achieved substantial success in a series of tasks in computer vision. Intelligent video analysis, which can be broadly applied to video surveillance in various smart city applications, can also be driven by such powerful deep learning engines. To practically facilitate deep neural network models in the large-scale video analysis, there are still unprecedented challenges for the large-scale video data management. Deep feature coding, instead of video coding, provides a practical solution for handling the large-scale video surveillance data. To enable interoperability in the context of deep feature coding, standardization is urgent and important. However, due to the explosion of deep learning algorithms and the particularity of feature coding, there are numerous remaining problems in the standardization process. This paper envisions the future deep feature coding standard for the AI oriented large-scale video management, and discusses existing techniques, standards and possible solutions for these open problems.
AWE-CM Vectors: Augmenting Word Embeddings with a Clinical Metathesaurus
Venue: arXiv cs.CL 2017
Year: 2017

In recent years, word embeddings have been surprisingly effective at capturing intuitive characteristics of the words they represent. These vectors achieve the best results when training corpora are extremely large, sometimes billions of words. Clinical natural language processing datasets, however, tend to be much smaller. Even the largest publicly-available dataset of medical notes is three orders of magnitude smaller than the dataset of the oft-used "Google News" word vectors. In order to make up for limited training data sizes, we encode expert domain knowledge into our embeddings. Building on a previous extension of word2vec, we show that generalizing the notion of a word's "context" to include arbitrary features creates an avenue for encoding domain knowledge into word embeddings. We show that the word vectors produced by this method outperform their text-only counterparts across the board in correlation with clinical experts.
Broadcast Caching Networks with Two Receivers and Multiple Correlated Sources
Venue: arXiv cs.IT 2017
Year: 2017

The correlation among the content distributed across a cache-aided broadcast network can be exploited to reduce the delivery load on the shared wireless link. This paper considers a two-user three-file network with correlated content, and studies its fundamental limits for the worst-case demand. A class of achievable schemes based on a two-step source coding approach is proposed. Library files are first compressed using Gray-Wyner source coding, and then cached and delivered using a combination of correlation-unaware cache-aided coded multicast schemes. The second step is interesting in its own right and considers a multiple-request caching problem, whose solution requires coding in the placement phase. A lower bound on the optimal peak rate-memory trade-off is derived, which is used to evaluate the performance of the proposed scheme. It is shown that for symmetric sources the two-step strategy achieves the lower bound for large cache capacities, and it is within half of the joint entropy of two of the sources conditioned on the third source for all other cache sizes.
SAFEBIKE: A Bike-sharing Route Recommender with Availability Prediction and Safe Routing
Venue: arXiv cs.CY 2017
Year: 2017

This paper presents SAFEBIKE, a novel route recommendation system for bike-sharing service that utilizes station information to infer the number of available bikes in dock and recommend bike routes according to multiple factors such as distance and safety level. The system consists of a station level availability predictor that predicts bikes and docks amount at each station, and an efficient route recommendation service that considers safety and bike/dock availability factors. It targets users who are concerned about route safeness and station availability. We demonstrate the system by utilizing Citi Bike station availability and New York City crime data of Manhattan to show the effectiveness of our approach. Integrated with real-time station availability and historical crime data resources, our proposed system can effectively recommend an optimal bike route and improve the travel experience of bike users.
Determinism in the Certification of UNSAT Proofs
Venue: arXiv cs.LO 2017
Year: 2017

The search for increased trustworthiness of SAT solvers is very active and uses various methods. Some of these methods obtain a proof from the provers then check it, normally by replicating the search based on the proof's information. Because the certification process involves another nontrivial proof search, the trust we can place in it is decreased. Some attempts to amend this use certifiers which have been verified by proofs assistants such as Isabelle/HOL and Coq. Our approach is different because it is based on an extremely simplified certifier. This certifier enjoys a very high level of trust but is very inefficient. In this paper, we experiment with this approach and conclude that by placing some restrictions on the formats, one can mostly eliminate the need for search and in principle, can certify proofs of arbitrary size.
EmTaggeR: A Word Embedding Based Novel Method for Hashtag Recommendation on Twitter
Venue: arXiv cs.CL 2017
Year: 2017

The hashtag recommendation problem addresses recommending (suggesting) one or more hashtags to explicitly tag a post made on a given social network platform, based upon the content and context of the post. In this work, we propose a novel methodology for hashtag recommendation for microblog posts, specifically Twitter. The methodology, EmTaggeR, is built upon a training-testing framework that builds on the top of the concept of word embedding. The training phase comprises of learning word vectors associated with each hashtag, and deriving a word embedding for each hashtag. We provide two training procedures, one in which each hashtag is trained with a separate word embedding model applicable in the context of that hashtag, and another in which each hashtag obtains its embedding from a global context. The testing phase constitutes computing the average word embedding of the test post, and finding the similarity of this embedding with the known embeddings of the hashtags. The tweets that contain the most-similar hashtag are extracted, and all the hashtags that appear in these tweets are ranked in terms of embedding similarity scores. The top-K hashtags that appear in this ranked list, are recommended for the given test post. Our system produces F1 score of 50.83%, improving over the LDA baseline by around 6.53 times, outperforming the best-performing system known in the literature that provides a lift of 6.42 times. EmTaggeR is a fast, scalable and lightweight system, which makes it practical to deploy in real-life applications.