OpenRaiser/Intern-Atlas huggingface dataset:
Fishing in the Stream: Similarity Search over Endless Data
Venue: arXiv cs.IR 2017
Year: 2017

Similarity search is the task of retrieving data items that are similar to a given query. In this paper, we introduce the time-sensitive notion of similarity search over endless data-streams (SSDS), which takes into account data quality and temporal characteristics in addition to similarity. SSDS is challenging as it needs to process unbounded data, while computation resources are bounded. We propose Stream-LSH, a randomized SSDS algorithm that bounds the index size by retaining items according to their freshness, quality, and dynamic popularity attributes. We analytically show that Stream-LSH increases the probability to find similar items compared to alternative approaches using the same space capacity. We further conduct an empirical study using real world stream datasets, which confirms our theoretical results.
Efficient Noisy Optimisation with the Sliding Window Compact Genetic Algorithm
Venue: arXiv cs.NE 2017
Year: 2017

The compact genetic algorithm is an Estimation of Distribution Algorithm for binary optimisation problems. Unlike the standard Genetic Algorithm, no cross-over or mutation is involved. Instead, the compact Genetic Algorithm uses a virtual population represented as a probability distribution over the set of binary strings. At each optimisation iteration, exactly two individuals are generated by sampling from the distribution, and compared exactly once to determine a winner and a loser. The probability distribution is then adjusted to increase the likelihood of generating individuals similar to the winner. This paper introduces two straightforward variations of the compact Genetic Algorithm, each of which lead to a significant improvement in performance. The main idea is to make better use of each fitness evaluation, by ensuring that each evaluated individual is used in multiple win/loss comparisons. The first variation is to sample $n>2$ individuals at each iteration to make $n(n-1)/2$ comparisons. The second variation only samples one individual at each iteration but keeps a sliding history window of previous individuals to compare with. We evaluate methods on two noisy test problems and show that in each case they significantly outperform the compact Genetic Algorithm, while maintaining the simplicity of the algorithm.
MoPS: A Modular Protection Scheme for Long-Term Storage
Venue: arXiv cs.CR 2017
Year: 2017

Current trends in technology, such as cloud computing, allow outsourcing the storage, backup, and archiving of data. This provides efficiency and flexibility, but also poses new risks for data security. It in particular became crucial to develop protection schemes that ensure security even in the long-term, i.e. beyond the lifetime of keys, certificates, and cryptographic primitives. However, all current solutions fail to provide optimal performance for different application scenarios. Thus, in this work, we present MoPS, a modular protection scheme to ensure authenticity and integrity for data stored over long periods of time. MoPS does not come with any requirements regarding the storage architecture and can therefore be used together with existing archiving or storage systems. It supports a set of techniques which can be plugged together, combined, and migrated in order to create customized solutions that fulfill the requirements of different application scenarios in the best possible way. As a proof of concept we implemented MoPS and provide performance measurements. Furthermore, our implementation provides additional features, such as guidance for non-expert users and export functionalities for external verifiers.
Multimodal Classification for Analysing Social Media
Venue: arXiv cs.CL 2017
Year: 2017

Classification of social media data is an important approach in understanding user behavior on the Web. Although information on social media can be of different modalities such as texts, images, audio or videos, traditional approaches in classification usually leverage only one prominent modality. Techniques that are able to leverage multiple modalities are often complex and susceptible to the absence of some modalities. In this paper, we present simple models that combine information from different modalities to classify social media content and are able to handle the above problems with existing techniques. Our models combine information from different modalities using a pooling layer and an auxiliary learning task is used to learn a common feature space. We demonstrate the performance of our models and their robustness to the missing of some modalities in the emotion classification domain. Our approaches, although being simple, can not only achieve significantly higher accuracies than traditional fusion approaches but also have comparable results when only one modality is available.
Lewis meets Brouwer: constructive strict implication
Venue: arXiv cs.LO 2017
Year: 2017

C. I. Lewis invented modern modal logic as a theory of "strict implication". Over the classical propositional calculus one can as well work with the unary box connective. Intuitionistically, however, the strict implication has greater expressive power than the box and allows to make distinctions invisible in the ordinary syntax. In particular, the logic determined by the most popular semantics of intuitionistic K becomes a proper extension of the minimal normal logic of the binary connective. Even an extension of this minimal logic with the "strength" axiom, classically near-trivial, preserves the distinction between the binary and the unary setting. In fact, this distinction and the strong constructive strict implication itself has been also discovered by the functional programming community in their study of "arrows" as contrasted with "idioms". Our particular focus is on arithmetical interpretations of the intuitionistic strict implication in terms of preservativity in extensions of Heyting's Arithmetic.
PowerAI DDL
Venue: arXiv cs.DC 2017
Year: 2017

As deep neural networks become more complex and input datasets grow larger, it can take days or even weeks to train a deep neural network to the desired accuracy. Therefore, distributed Deep Learning at a massive scale is a critical capability, since it offers the potential to reduce the training time from weeks to hours. In this paper, we present a software-hardware co-optimized distributed Deep Learning system that can achieve near-linear scaling up to hundreds of GPUs. The core algorithm is a multi-ring communication pattern that provides a good tradeoff between latency and bandwidth and adapts to a variety of system configurations. The communication algorithm is implemented as a library for easy use. This library has been integrated into Tensorflow, Caffe, and Torch. We train Resnet-101 on Imagenet 22K with 64 IBM Power8 S822LC servers (256 GPUs) in about 7 hours to an accuracy of 33.8 % validation accuracy. Microsoft's ADAM and Google's DistBelief results did not reach 30 % validation accuracy for Imagenet 22K. Compared to Facebook AI Research's recent paper on 256 GPU training, we use a different communication algorithm, and our combined software and hardware system offers better communication overhead for Resnet-50. A PowerAI DDL enabled version of Torch completed 90 epochs of training on Resnet 50 for 1K classes in 50 minutes using 64 IBM Power8 S822LC servers (256 GPUs).
Learning a CNN-based End-to-End Controller for a Formula SAE Racecar
Venue: arXiv cs.CV 2017
Year: 2017

We present a set of CNN-based end-to-end models for controls of a Formula SAE racecar, along with various benchmarking and visualization tools to understand model performance. We tackled three main problems in the context of cone-delineated racetrack driving: (1) discretized steering, which translates a first-person frame along to the track to a predicted steering direction. (2) real-value steering, which translates a frame view to a real-value steering angle, and (3) a network design for predicting brake and throttle. We demonstrate high accuracy on our discretization task, low theoretical testing errors with our model for real-value steering, and a starting point for future work regarding a controller for our vehicle's brake and throttle. Timing benchmarks suggests that the networks we propose have the latency and throughput required for real-time controllers, when run on GPU-enabled hardware.
Real-Time Visual Localisation in a Tagged Environment
Venue: arXiv cs.CV 2017
Year: 2017

In a robotised warehouse a major issue is the safety of human operators in case of intervention in the work area of the robots. The current solution is to shut down every robot but it causes a loss of productivity, especially for large robotised warehouses. In order to avoid this loss we need to ensure the operator's security during his/her intervention in the warehouse without powering off the robots. The human operator needs to be localised in the warehouse and the trajectories of the robots have to be modified so that they do not interfere with the human. The purpose of this paper is to demonstrate a visual localisation method with visual elements that are already available in the current warehouse setup.
An Approximate ML Detector for MIMO Channels Corrupted by Phase Noise
Venue: arXiv cs.IT 2017
Year: 2017

We consider the multiple-input multiple-output (MIMO) communication channel impaired by phase noises at both the transmitter and receiver. We focus on the maximum likelihood (ML) detection problem for uncoded single-carrier transmission. We derive an approximation of the likelihood function, based on which we propose an efficient detection algorithm. The proposed algorithm, named self-interference whitening (SIW), consists in 1) estimating the self-interference caused by the phase noise perturbation, then 2) whitening the said interference, and finally 3) detecting the transmitted vector. While the exact ML solution is computationally intractable, we construct a simulation-based lower bound on the error probability of ML detection. Leveraging this lower bound, we perform extensive numerical experiments demonstrating that SIW is, in most cases of interest, very close to optimal with moderate phase noise. More importantly and perhaps surprisingly, such near-ML performance can be achieved by applying only twice the nearest neighbor detection algorithm. In this sense, our results reveal a striking fact: near-ML detection of phase noise corrupted MIMO channels can be done as efficiently as for conventional MIMO channels without phase noise.
Beyond the technical challenges for deploying Machine Learning solutions in a software company
Venue: arXiv cs.HC 2017
Year: 2017

Recently software development companies started to embrace Machine Learning (ML) techniques for introducing a series of advanced functionality in their products such as personalisation of the user experience, improved search, content recommendation and automation. The technical challenges for tackling these problems are heavily researched in literature. A less studied area is a pragmatic approach to the role of humans in a complex modern industrial environment where ML based systems are developed. Key stakeholders affect the system from inception and up to operation and maintenance. Product managers want to embed "smart" experiences for their users and drive the decisions on what should be built next; software engineers are challenged to build or utilise ML software tools that require skills that are well outside of their comfort zone; legal and risk departments may influence design choices and data access; operations teams are requested to maintain ML systems which are non-stationary in their nature and change behaviour over time; and finally ML practitioners should communicate with all these stakeholders to successfully build a reliable system. This paper discusses some of the challenges we faced in Atlassian as we started investing more in the ML space.
Mining fine-grained opinions on closed captions of YouTube videos with an attention-RNN
Venue: arXiv cs.CL 2017
Year: 2017

Video reviews are the natural evolution of written product reviews. In this paper we target this phenomenon and introduce the first dataset created from closed captions of YouTube product review videos as well as a new attention-RNN model for aspect extraction and joint aspect extraction and sentiment classification. Our model provides state-of-the-art performance on aspect extraction without requiring the usage of hand-crafted features on the SemEval ABSA corpus, while it outperforms the baseline on the joint task. In our dataset, the attention-RNN model outperforms the baseline for both tasks, but we observe important performance drops for all models in comparison to SemEval. These results, as well as further experiments on domain adaptation for aspect extraction, suggest that differences between speech and written text, which have been discussed extensively in the literature, also extend to the domain of product reviews, where they are relevant for fine-grained opinion mining.
An Effective Feature Selection Method Based on Pair-Wise Feature Proximity for High Dimensional Low Sample Size Data
Venue: arXiv cs.CV 2017
Year: 2017

Feature selection has been studied widely in the literature. However, the efficacy of the selection criteria for low sample size applications is neglected in most cases. Most of the existing feature selection criteria are based on the sample similarity. However, the distance measures become insignificant for high dimensional low sample size (HDLSS) data. Moreover, the variance of a feature with a few samples is pointless unless it represents the data distribution efficiently. Instead of looking at the samples in groups, we evaluate their efficiency based on pairwise fashion. In our investigation, we noticed that considering a pair of samples at a time and selecting the features that bring them closer or put them far away is a better choice for feature selection. Experimental results on benchmark data sets demonstrate the effectiveness of the proposed method with low sample size, which outperforms many other state-of-the-art feature selection methods.
Impossibility of $n-1$-strong-equllibrium for Distributed Consensus with Rational Agents
Venue: arXiv cs.DC 2017
Year: 2017

An algorithm for $n-1$-strong-equillibrium for distributed consensus in a ring with rational agents was proposed by Afek et al. (2014). A proof of impossibility of $n-1$-strong-equillibrium for distributed consensus in every topology with rational agents, when $n$ is even, is presented. Furthermore, we show that the algorithm proposed by Afek et al. is the only algorithm which can solve the problem when $n$ is odd. Finally, we prove that the proposed algorithm provides a $n-2$-strong-equillibrium in a synchronous ring when $n$ is even.
Multi-Generator Generative Adversarial Nets
Venue: arXiv cs.LG 2017
Year: 2017

We propose a new approach to train the Generative Adversarial Nets (GANs) with a mixture of generators to overcome the mode collapsing problem. The main intuition is to employ multiple generators, instead of using a single one as in the original GAN. The idea is simple, yet proven to be extremely effective at covering diverse data modes, easily overcoming the mode collapse and delivering state-of-the-art results. A minimax formulation is able to establish among a classifier, a discriminator, and a set of generators in a similar spirit with GAN. Generators create samples that are intended to come from the same distribution as the training data, whilst the discriminator determines whether samples are true data or generated by generators, and the classifier specifies which generator a sample comes from. The distinguishing feature is that internal samples are created from multiple generators, and then one of them will be randomly selected as final output similar to the mechanism of a probabilistic mixture model. We term our method Mixture GAN (MGAN). We develop theoretical analysis to prove that, at the equilibrium, the Jensen-Shannon divergence (JSD) between the mixture of generators' distributions and the empirical data distribution is minimal, whilst the JSD among generators' distributions is maximal, hence effectively avoiding the mode collapse. By utilizing parameter sharing, our proposed model adds minimal computational cost to the standard GAN, and thus can also efficiently scale to large-scale datasets. We conduct extensive experiments on synthetic 2D data and natural image databases (CIFAR-10, STL-10 and ImageNet) to demonstrate the superior performance of our MGAN in achieving state-of-the-art Inception scores over latest baselines, generating diverse and appealing recognizable objects at different resolutions, and specializing in capturing different types of objects by generators.
Which Encoding is the Best for Text Classification in Chinese, English, Japanese and Korean?
Venue: arXiv cs.CL 2017
Year: 2017

This article offers an empirical study on the different ways of encoding Chinese, Japanese, Korean (CJK) and English languages for text classification. Different encoding levels are studied, including UTF-8 bytes, characters, words, romanized characters and romanized words. For all encoding levels, whenever applicable, we provide comparisons with linear models, fastText and convolutional networks. For convolutional networks, we compare between encoding mechanisms using character glyph images, one-hot (or one-of-n) encoding, and embedding. In total there are 473 models, using 14 large-scale text classification datasets in 4 languages including Chinese, English, Japanese and Korean. Some conclusions from these results include that byte-level one-hot encoding based on UTF-8 consistently produces competitive results for convolutional networks, that word-level n-grams linear models are competitive even without perfect word segmentation, and that fastText provides the best result using character-level n-gram encoding but can overfit when the features are overly rich.
Randomly coloring graphs of bounded treewidth
Venue: arXiv cs.DS 2017
Year: 2017

We consider the problem of sampling a proper $k$-coloring of a graph of maximal degree $Δ$ uniformly at random. We describe a new Markov chain for sampling colorings, and show that it mixes rapidly on graphs of bounded treewidth if $k\geq(1+ε)Δ$, for any $ε>0$.
A Data Prism: Semi-Verified Learning in the Small-Alpha Regime
Venue: arXiv cs.LG 2017
Year: 2017

We consider a model of unreliable or crowdsourced data where there is an underlying set of $n$ binary variables, each evaluator contributes a (possibly unreliable or adversarial) estimate of the values of some subset of $r$ of the variables, and the learner is given the true value of a constant number of variables. We show that, provided an $α$-fraction of the evaluators are "good" (either correct, or with independent noise rate $p < 1/2$), then the true values of a $(1-ε)$ fraction of the $n$ underlying variables can be deduced as long as $α> 1/(2-2p)^r$. This setting can be viewed as an instance of the semi-verified learning model introduced in [CSV17], which explores the tradeoff between the number of items evaluated by each worker and the fraction of good evaluators. Our results require the number of evaluators to be extremely large, $>n^r$, although our algorithm runs in linear time, $O_{r,ε}(n)$, given query access to the large dataset of evaluations. This setting and results can also be viewed as examining a general class of semi-adversarial CSPs with a planted assignment. This parameter regime where the fraction of reliable data is small, is relevant to a number of practical settings. For example, settings where one has a large dataset of customer preferences, with each customer specifying preferences for a small (constant) number of items, and the goal is to ascertain the preferences of a specific demographic of interest. Our results show that this large dataset (which lacks demographic information) can be leveraged together with the preferences of the demographic of interest for a constant number of randomly selected items, to recover an accurate estimate of the entire set of preferences. In this sense, our results can be viewed as a "data prism" allowing one to extract the behavior of specific cohorts from a large, mixed, dataset.
Analysis of Analog Network Coding noise in Multiuser Cooperative Relaying for Spatially Correlated Environment
Venue: arXiv cs.NI 2017
Year: 2017

Analog Network Coding (ANC) is proposed in literature to improve the network throughput by exploiting channel diversity. In practical scenarios, due to the difference in channel characteristics, an extra residual component, termed as ANC noise, appears during the processing of the received signal. This ANC noise component may suppress the ANC gain. None of the existing literature to our knowledge considers the effect of spatial correlation among channels on ANC noise. This paper develops a generic framework to investigate the effect of channel characteristics on ANC noise. We have modelled the channels as spatially correlated to take the (dis)similarity among them into account. Per node power constraint is also taken into consideration. In this work, we have characterized the behaviour of ANC noise and presented the results to analyze the network performance in terms of outage probability. Outcomes of our investigation show that spatial correlation among channels significantly affects the variance of ANC noise as well as the outage performance of the system. The proposed framework can provide better insights while selecting the system parameters in a correlated environment.
Impacts of Culture and Socio-Economic Circumstances on Users' Behavior and Mobile Broadband Technology Diffusion Trends
Venue: arXiv cs.CY 2017
Year: 2017

The use of Internet and Internet-based services on PCs, Laptops, Net Pads, Mobile Phones, PDAs etc have not only changed the global economy but also the way people communicate and their life styles. It also has evolved people from different origins, cultures, beliefs across the national boundaries. As a result it has become an absolute necessity to address the cross-cultural issues of information systems (IS) reflecting the user behaviours and influencing the way the mobile broadband technology is being accepted as well as the way it is changing the life styles of different groups of people. This paper reports on an on-going research effort which studies the impacts of culture and socio-economic circumstances on users' behavior and mobile broadband technology diffusion trends.
Enhancing Cellular M2M Random Access with Binary Countdown Contention Resolution
Venue: arXiv cs.NI 2017
Year: 2017

Accommodating Machine-to-Machine applications and their requirements is one of the challenges on the way from LTE towards 5G networks. The envisioned high density of devices, alongside with their sporadic and synchronized transmission patterns, might create signaling storms and overload in the current LTE network. Here, we address the notorious random access (RA) challenge, namely, scalability of the radio link connection establishment protocol in LTE networks. We revisit the binary countdown technique for contention resolution (BCCR), and apply it to the LTE RA procedure. We analytically investigate the performance gains and trade-offs of applying BCCR in LTE. We further simulatively compare BCCR RA with the state-of-the-art RA techniques, and demonstrate its advantages in terms of delay and throughput.