OpenRaiser/Intern-Atlas huggingface dataset:
Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization
Venue: arXiv cs.LG 2017
Year: 2017

An efficient algorithm for recurrent neural network training is presented. The approach increases the training speed for tasks where a length of the input sequence may vary significantly. The proposed approach is based on the optimal batch bucketing by input sequence length and data parallelization on multiple graphical processing units. The baseline training performance without sequence bucketing is compared with the proposed solution for a different number of buckets. An example is given for the online handwriting recognition task using an LSTM recurrent neural network. The evaluation is performed in terms of the wall clock time, number of epochs, and validation loss value.
Learning to Transfer
Venue: arXiv cs.AI 2017
Year: 2017

Transfer learning borrows knowledge from a source domain to facilitate learning in a target domain. Two primary issues to be addressed in transfer learning are what and how to transfer. For a pair of domains, adopting different transfer learning algorithms results in different knowledge transferred between them. To discover the optimal transfer learning algorithm that maximally improves the learning performance in the target domain, researchers have to exhaustively explore all existing transfer learning algorithms, which is computationally intractable. As a trade-off, a sub-optimal algorithm is selected, which requires considerable expertise in an ad-hoc way. Meanwhile, it is widely accepted in educational psychology that human beings improve transfer learning skills of deciding what to transfer through meta-cognitive reflection on inductive transfer learning practices. Motivated by this, we propose a novel transfer learning framework known as Learning to Transfer (L2T) to automatically determine what and how to transfer are the best by leveraging previous transfer learning experiences. We establish the L2T framework in two stages: 1) we first learn a reflection function encrypting transfer learning skills from experiences; and 2) we infer what and how to transfer for a newly arrived pair of domains by optimizing the reflection function. Extensive experiments demonstrate the L2T's superiority over several state-of-the-art transfer learning algorithms and its effectiveness on discovering more transferable knowledge.
What does a convolutional neural network recognize in the moon?
Venue: arXiv cs.CV 2017
Year: 2017

Many people see a human face or animals in the pattern of the maria on the moon. Although the pattern corresponds to the actual variation in composition of the lunar surface, the culture and environment of each society influence the recognition of these objects (i.e., symbols) as specific entities. In contrast, a convolutional neural network (CNN) recognizes objects from characteristic shapes in a training data set. Using CNN, this study evaluates the probabilities of the pattern of lunar maria categorized into the shape of a crab, a lion and a hare. If Mare Frigoris (a dark band on the moon) is included in the lunar image, the lion is recognized. However, in an image without Mare Frigoris, the hare has the highest probability of recognition. Thus, the recognition of objects similar to the lunar pattern depends on which part of the lunar maria is taken into account. In human recognition, before we find similarities between the lunar maria and objects such as animals, we may be persuaded in advance to see a particular image from our culture and environment and then adjust the lunar pattern to the shape of the imagined object.
Modeling Spread of Preferences in Social Networks for Sampling-based Preference Aggregation
Venue: arXiv cs.SI 2017
Year: 2017

Given a large population, it is an intensive task to gather individual preferences over a set of alternatives and arrive at an aggregate or collective preference of the population. We show that social network underlying the population can be harnessed to accomplish this task effectively, by sampling preferences of a small subset of representative nodes. We first develop a Facebook app to create a dataset consisting of preferences of nodes and the underlying social network, using which, we develop models that capture how preferences are distributed among nodes in a typical social network. We hence propose an appropriate objective function for the problem of selecting best representative nodes. We devise two algorithms, namely, Greedy-min which provides a performance guarantee for a wide class of popular voting rules, and Greedy-sum which exhibits excellent performance in practice. We compare the performance of these proposed algorithms against random-polling and popular centrality measures, and provide a detailed analysis of the obtained results. Our analysis suggests that selecting representatives using social network information is advantageous for aggregating preferences related to personal topics (e.g., lifestyle), while random polling with a reasonable sample size is good enough for aggregating preferences related to social topics (e.g., government policies).
Cross-Lingual Dependency Parsing for Closely Related Languages - Helsinki's Submission to VarDial 2017
Venue: arXiv cs.CL 2017
Year: 2017

This paper describes the submission from the University of Helsinki to the shared task on cross-lingual dependency parsing at VarDial 2017. We present work on annotation projection and treebank translation that gave good results for all three target languages in the test set. In particular, Slovak seems to work well with information coming from the Czech treebank, which is in line with related work. The attachment scores for cross-lingual models even surpass the fully supervised models trained on the target language treebank. Croatian is the most difficult language in the test set and the improvements over the baseline are rather modest. Norwegian works best with information coming from Swedish whereas Danish contributes surprisingly little.
Dynamic Connectivity Game for Adversarial Internet of Battlefield Things Systems
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, the problem of network connectivity is studied for an adversarial Internet of Battlefield Things (IoBT) system in which an attacker aims at disrupting the connectivity of the network by choosing to compromise one of the IoBT nodes at each time epoch. To counter such attacks, an IoBT defender attempts to reestablish the IoBT connectivity by either deploying new IoBT nodes or by changing the roles of existing nodes. This problem is formulated as a dynamic multistage Stackelberg connectivity game that extends classical connectivity games and that explicitly takes into account the characteristics and requirements of the IoBT network. In particular, the defender's payoff captures the IoBT latency as well as the sum of weights of disconnected nodes at each stage of the game. Due to the dependence of the attacker's and defender's actions at each stage of the game on the network state, the feedback Stackelberg solution (FSE) is used to solve the IoBT connectivity game. Then, sufficient conditions under which the IoBT system will remain connected, when the FSE solution is used, are determined analytically. Numerical results show that the expected number of disconnected sensors, when the FSE solution is used, decreases up to 62% compared to a baseline scenario in which a Stackelberg game with no feedback is used, and up to 57% compared to a baseline equal probability policy.
University Twitter Engagement: Using Twitter Followers to Rank Universities
Venue: arXiv cs.DL 2017
Year: 2017

We examine and rank a set of 264 U.S. universities extracted from the National Collegiate Athletic Association (NCAA) Division I membership and global lists published in U.S. News, Times Higher Education, Academic Ranking of World Universities, and Money Magazine. Our University Twitter Engagement (UTE) rank is based on the friend and extended follower network of primary and affiliated secondary Twitter accounts referenced on a university's home page. In rank-to-rank comparisons we observed a significant, positive rank correlation (τ=0.6018) between UTE and an aggregate reputation ranking which indicates that UTE could be a viable proxy for ranking atypical institutions normally excluded from traditional lists. In addition, we significantly reduce the cost of data collection needed to rank each institution by using only web-based artifacts and a publicly accessible Twitter application programming interface (API).
CLaC @ QATS: Quality Assessment for Text Simplification
Venue: arXiv cs.CL 2017
Year: 2017

This paper describes our approach to the 2016 QATS quality assessment shared task. We trained three independent Random Forest classifiers in order to assess the quality of the simplified texts in terms of grammaticality, meaning preservation and simplicity. We used the language model of Google-Ngram as feature to predict the grammaticality. Meaning preservation is predicted using two complementary approaches based on word embedding and WordNet synonyms. A wider range of features including TF-IDF, sentence length and frequency of cue phrases are used to evaluate the simplicity aspect. Overall, the accuracy of the system ranges from 33.33% for the overall aspect to 58.73% for grammaticality.
Design Space of Programming Tools on Mobile Touchscreen Devices
Venue: arXiv cs.HC 2017
Year: 2017

While mobile touchscreen devices are ubiquitous and present opportunities for novel applications, they have seen little adoption as tools for computer programming. In this literature survey, we bring together the diverse research work on programming-related tasks supported by mobile touchscreen devices to explore the design space for applying them to programming situations. We used the Grounded theory approach to identify themes and classify previous work. We present these themes and how each paper contributes to the theme, and we outline the remaining challenges in and opportunities for using mobile touchscreen devices in programming applications.
Secure Search on the Cloud via Coresets and Sketches
Venue: arXiv cs.CR 2017
Year: 2017

\emph{Secure Search} is the problem of retrieving from a database table (or any unsorted array) the records matching specified attributes, as in SQL SELECT queries, but where the database and the query are encrypted. Secure search has been the leading example for practical applications of Fully Homomorphic Encryption (FHE) starting in Gentry's seminal work; however, to the best of our knowledge all state-of-the-art secure search algorithms to date are realized by a polynomial of degree $Ω(m)$ for $m$ the number of records, which is typically too slow in practice even for moderate size $m$. In this work we present the first algorithm for secure search that is realized by a polynomial of degree polynomial in $\log m$. We implemented our algorithm in an open source library based on HELib implementation for the Brakerski-Gentry-Vaikuntanthan's FHE scheme, and ran experiments on Amazon's EC2 cloud. Our experiments show that we can retrieve the first match in a database of millions of entries in less than an hour using a single machine; the time reduced almost linearly with the number of machines. Our result utilizes a new paradigm of employing coresets and sketches, which are modern data summarization techniques common in computational geometry and machine learning, for efficiency enhancement for homomorphic encryption. As a central tool we design a novel sketch that returns the first positive entry in a (not necessarily sparse) array; this sketch may be of independent interest.
Real Time Prediction of Drive by Download Attacks on Twitter
Venue: arXiv cs.SI 2017
Year: 2017

The popularity of Twitter for information discovery, coupled with the automatic shortening of URLs to save space, given the 140 character limit, provides cyber criminals with an opportunity to obfuscate the URL of a malicious Web page within a tweet. Once the URL is obfuscated the cyber criminal can lure a user to click on it with enticing text and images before carrying out a cyber attack using a malicious Web server. This is known as a drive-by- download. In a drive-by-download a user's computer system is infected while interacting with the malicious endpoint, often without them being made aware, the attack has taken place. An attacker can gain control of the system by exploiting unpatched system vulnerabilities and this form of attack currently represents one of the most common methods employed. In this paper, we build a machine learning model using machine activity data and tweet meta data to move beyond post-execution classification of such URLs as malicious, to predict a URL will be malicious with 99.2% F-measure (using 10-fold cross validation) and 83.98% (using an unseen test set) at 1 second into the interaction with the URL. Thus providing a basis from which to kill the connection to the server before an attack has completed and proactively blocking and preventing an attack, rather than reacting and repairing at a later date.
A Brief Survey of Deep Reinforcement Learning
Venue: arXiv cs.LG 2017
Year: 2017

Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels. Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep reinforcement learning, including the deep $Q$-network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning. To conclude, we describe several current areas of research within the field.
What Drives the International Development Agenda? An NLP Analysis of the United Nations General Debate 1970-2016
Venue: arXiv cs.CL 2017
Year: 2017

There is surprisingly little known about agenda setting for international development in the United Nations (UN) despite it having a significant influence on the process and outcomes of development efforts. This paper addresses this shortcoming using a novel approach that applies natural language processing techniques to countries' annual statements in the UN General Debate. Every year UN member states deliver statements during the General Debate on their governments' perspective on major issues in world politics. These speeches provide invaluable information on state preferences on a wide range of issues, including international development, but have largely been overlooked in the study of global politics. This paper identifies the main international development topics that states raise in these speeches between 1970 and 2016, and examine the country-specific drivers of international development rhetoric.
Electricity Theft Detection using Machine Learning
Venue: arXiv cs.CR 2017
Year: 2017

Non-technical losses (NTL) in electric power grids arise through electricity theft, broken electric meters or billing errors. They can harm the power supplier as well as the whole economy of a country through losses of up to 40% of the total power distribution. For NTL detection, researchers use artificial intelligence to analyse data. This work is about improving the extraction of more meaningful features from a data set. With these features, the prediction quality will increase.
Designing virus-resistant, high-performance networks: a game-formation approach
Venue: arXiv cs.GT 2017
Year: 2017

Designing an optimal network topology while balancing multiple, possibly conflicting objectives like cost, performance, and resiliency to viruses is a challenging endeavor, let alone in the case of decentralized network formation. We therefore propose a game-formation technique where each player aims to minimize its cost in installing links, the probability of being infected by a virus and the sum of hopcounts on its shortest paths to all other nodes. In this article, we (1) determine the Nash Equilibria and the Price of Anarchy for our novel network formation game, (2) demonstrate that the Price of Anarchy (PoA) is usually low, which suggests that (near-)optimal topologies can be formed in a decentralized way, and (3) give suggestions for practitioners for those cases where the PoA is high and some centralized control/incentives are advisable.
360-degree Video Stitching for Dual-fisheye Lens Cameras Based On Rigid Moving Least Squares
Venue: arXiv cs.MM 2017
Year: 2017

Dual-fisheye lens cameras are becoming popular for 360-degree video capture, especially for User-generated content (UGC), since they are affordable and portable. Images generated by the dual-fisheye cameras have limited overlap and hence require non-conventional stitching techniques to produce high-quality 360x180-degree panoramas. This paper introduces a novel method to align these images using interpolation grids based on rigid moving least squares. Furthermore, jitter is the critical issue arising when one applies the image-based stitching algorithms to video. It stems from the unconstrained movement of stitching boundary from one frame to another. Therefore, we also propose a new algorithm to maintain the temporal coherence of stitching boundary to provide jitter-free 360-degree videos. Results show that the method proposed in this paper can produce higher quality stitched images and videos than prior work.
Software-Defined Robotics -- Idea & Approach
Venue: arXiv cs.RO 2017
Year: 2017

The methodology of Software-Defined Robotics hierarchical-based and stand-alone framework can be designed and implemented to program and control different sets of robots, regardless of their manufacturers' parameters and specifications, with unified commands and communications. This framework approach will increase the capability of (re)programming a specific group of robots during the runtime without affecting the others as desired in the critical missions and industrial operations, expand the shared bandwidth, enhance the reusability of code, leverage the computational processing power, decrease the unnecessary analyses of vast supplemental electrical components for each robot, as well as get advantages of the most state-of-the-art industrial trends in the cloud-based computing, Virtual Machines (VM), and Robot-as-a-Service (RaaS) technologies.
Message Passing in C-RAN: Joint User Activity and Signal Detection
Venue: arXiv cs.IT 2017
Year: 2017

In cloud radio access network (C-RAN), remote radio heads (RRHs) and users are uniformly distributed in a large area such that the channel matrix can be considered as sparse. Based on this phenomenon, RRHs only need to detect the relatively strong signals from nearby users and ignore the weak signals from far users, which is helpful to develop low-complexity detection algorithms without causing much performance loss. However, before detection, RRHs require to obtain the realtime user activity information by the dynamic grant procedure, which causes the enormous latency. To address this issue, in this paper, we consider a grant-free C-RAN system and propose a low-complexity Bernoulli-Gaussian message passing (BGMP) algorithm based on the sparsified channel, which jointly detects the user activity and signal. Since active users are assumed to transmit Gaussian signals at any time, the user activity can be regarded as a Bernoulli variable and the signals from all users obey a Bernoulli-Gaussian distribution. In the BGMP, the detection functions for signals are designed with respect to the Bernoulli-Gaussian variable. Numerical results demonstrate the robustness and effectivity of the BGMP. That is, for different sparsified channels, the BGMP can approach the mean-square error (MSE) of the genie-aided sparse minimum mean-square error (GA-SMMSE) which exactly knows the user activity information. Meanwhile, the fast convergence and strong recovery capability for user activity of the BGMP are also verified.
An improved watermarking scheme for Internet applications
Venue: arXiv cs.MM 2017
Year: 2017

In this paper, a data hiding scheme ready for Internet applications is proposed. An existing scheme based on chaotic iterations is improved, to respond to some major Internet security concerns, such as digital rights management, communication over hidden channels, and social search engines. By using Reed Solomon error correcting codes and wavelets domain, we show that this data hiding scheme can be improved to solve issues and requirements raised by these Internet fields.
Joint Multi-view Face Alignment in the Wild
Venue: arXiv cs.CV 2017
Year: 2017

The de facto algorithm for facial landmark estimation involves running a face detector with a subsequent deformable model fitting on the bounding box. This encompasses two basic problems: i) the detection and deformable fitting steps are performed independently, while the detector might not provide best-suited initialisation for the fitting step, ii) the face appearance varies hugely across different poses, which makes the deformable face fitting very challenging and thus distinct models have to be used (\eg, one for profile and one for frontal faces). In this work, we propose the first, to the best of our knowledge, joint multi-view convolutional network to handle large pose variations across faces in-the-wild, and elegantly bridge face detection and facial landmark localisation tasks. Existing joint face detection and landmark localisation methods focus only on a very small set of landmarks. By contrast, our method can detect and align a large number of landmarks for semi-frontal (68 landmarks) and profile (39 landmarks) faces. We evaluate our model on a plethora of datasets including standard static image datasets such as IBUG, 300W, COFW, and the latest Menpo Benchmark for both semi-frontal and profile faces. Significant improvement over state-of-the-art methods on deformable face tracking is witnessed on 300VW benchmark. We also demonstrate state-of-the-art results for face detection on FDDB and MALF datasets.
Geodesic Order Types
Venue: arXiv cs.CG 2017
Year: 2017

The geodesic between two points $a$ and $b$ in the interior of a simple polygon~$P$ is the shortest polygonal path inside $P$ that connects $a$ to $b$. It is thus the natural generalization of straight line segments on unconstrained point sets to polygonal environments. In this paper we use this extension to generalize the concept of the order type of a set of points in the Euclidean plane to geodesic order types. In particular, we show that, for any set $S$ of points and an ordered subset $\mathcal{B} \subseteq S$ of at least four points, one can always construct a polygon $P$ such that the points of $\mathcal{B}$ define the geodesic hull of~$S$ w.r.t.~$P$, in the specified order. Moreover, we show that an abstract order type derived from the dual of the Pappus arrangement can be realized as a geodesic order type.
Finding shorter paths for robot arms using their redundancy
Venue: arXiv cs.RO 2017
Year: 2017

Many robot arms can accomplish one task using many different joint configurations. Often only one of these configurations is used as a goal by the path planner. Ideally the robot's path planner would be able to use the extra configurations to find higher quality paths. In this paper we use the extra goal configurations to find significantly shorter paths that are faster to execute compared to a planner that chooses one goal configuration arbitrarily. In a grape vine pruning robot arm experiment our proposed approach reduced execution times by 58%.