OpenRaiser/Intern-Atlas huggingface dataset:
A New Data Representation Based on Training Data Characteristics to Extract Drug Named-Entity in Medical Text
Venue: arXiv cs.CL 2016
Year: 2016

One essential task in information extraction from the medical corpus is drug name recognition. Compared with text sources come from other domains, the medical text is special and has unique characteristics. In addition, the medical text mining poses more challenges, e.g., more unstructured text, the fast growing of new terms addition, a wide range of name variation for the same drug. The mining is even more challenging due to the lack of labeled dataset sources and external knowledge, as well as multiple token representations for a single drug name that is more common in the real application setting. Although many approaches have been proposed to overwhelm the task, some problems remained with poor F-score performance (less than 0.75). This paper presents a new treatment in data representation techniques to overcome some of those challenges. We propose three data representation techniques based on the characteristics of word distribution and word similarities as a result of word embedding training. The first technique is evaluated with the standard NN model, i.e., MLP (Multi-Layer Perceptrons). The second technique involves two deep network classifiers, i.e., DBN (Deep Belief Networks), and SAE (Stacked Denoising Encoders). The third technique represents the sentence as a sequence that is evaluated with a recurrent NN model, i.e., LSTM (Long Short Term Memory). In extracting the drug name entities, the third technique gives the best F-score performance compared to the state of the art, with its average F-score being 0.8645.
Trial-Offer Markets with Continuation
Venue: arXiv cs.GT 2016
Year: 2016

Trial-offer markets, where customers can sample a product before deciding whether to buy it, are ubiquitous in the online experience. Their static and dynamic properties are often studied by assuming that consumers follow a multinomial logit model and try exactly one product. In this paper, we study how to generalize existing results to a more realistic setting where consumers can try multiple products. We show that a multinomial logit model with continuation can be reduced to a standard multinomial logit model with different appeal and product qualities. We examine the consequences of this reduction on the performance and predictability of the market, the role of social influence, and the ranking policies.
MoveSteg: A Method of Network Steganography Detection
Venue: arXiv cs.MM 2016
Year: 2016

This article presents a new method for detecting a source point of time based network steganography - MoveSteg. A steganography carrier could be an example of multimedia stream made with packets. These packets are then delayed intentionally to send hidden information using time based steganography methods. The presented analysis describes a method that allows finding the source of steganography stream in network that is under our management.
DeepDGA: Adversarially-Tuned Domain Generation and Detection
Venue: arXiv cs.CR 2016
Year: 2016

Many malware families utilize domain generation algorithms (DGAs) to establish command and control (C&C) connections. While there are many methods to pseudorandomly generate domains, we focus in this paper on detecting (and generating) domains on a per-domain basis which provides a simple and flexible means to detect known DGA families. Recent machine learning approaches to DGA detection have been successful on fairly simplistic DGAs, many of which produce names of fixed length. However, models trained on limited datasets are somewhat blind to new DGA variants. In this paper, we leverage the concept of generative adversarial networks to construct a deep learning based DGA that is designed to intentionally bypass a deep learning based detector. In a series of adversarial rounds, the generator learns to generate domain names that are increasingly more difficult to detect. In turn, a detector model updates its parameters to compensate for the adversarially generated domains. We test the hypothesis of whether adversarially generated domains may be used to augment training sets in order to harden other machine learning models against yet-to-be-observed DGAs. We detail solutions to several challenges in training this character-based generative adversarial network (GAN). In particular, our deep learning architecture begins as a domain name auto-encoder (encoder + decoder) trained on domains in the Alexa one million. Then the encoder and decoder are reassembled competitively in a generative adversarial network (detector + generator), with novel neural architectures and training strategies to improve convergence.
Safe Human-Inspired Mesoscopic Hybrid Automaton for Autonomous Vehicles
Venue: arXiv cs.SY 2016
Year: 2016

In this paper a mesoscopic hybrid model, i.e. a microscopic hybrid model that takes into account macroscopic parameters, is introduced for designing a human-inspired Adaptive Cruise Control. A control law is proposed with the design goal of replacing and imitating the behaviour of a human driver in a car-following situation where lane changes are possible. First, a microscopic hybrid automaton model is presented, based on human psycho-physical behavior, for both longitudinal and lateral vehicle control. Then a rule for changing time headway on the basis of macroscopic quantities is used to describe the interaction among next vehicles and their impact on driver performance. Simulation results show the advantages of the mesoscopic model. A feasibility analysis of the needed communication network is also presented.
Regularized Dynamic Boltzmann Machine with Delay Pruning for Unsupervised Learning of Temporal Sequences
Venue: arXiv cs.LG 2016
Year: 2016

We introduce Delay Pruning, a simple yet powerful technique to regularize dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a particularly structured Boltzmann machine, as a generative model of a multi-dimensional time-series. This Boltzmann machine can have infinitely many layers of units but allows exact inference and learning based on its biologically motivated structure. DyBM uses the idea of conduction delays in the form of fixed length first-in first-out (FIFO) queues, with a neuron connected to another via this FIFO queue, and spikes from a pre-synaptic neuron travel along the queue to the post-synaptic neuron with a constant period of delay. Here, we present Delay Pruning as a mechanism to prune the lengths of the FIFO queues (making them zero) by setting some delay lengths to one with a fixed probability, and finally selecting the best performing model with fixed delays. The uniqueness of structure and a non-sampling based learning rule in DyBM, make the application of previously proposed regularization techniques like Dropout or DropConnect difficult, leading to poor generalization. First, we evaluate the performance of Delay Pruning to let DyBM learn a multidimensional temporal sequence generated by a Markov chain. Finally, we show the effectiveness of delay pruning in learning high dimensional sequences using the moving MNIST dataset, and compare it with Dropout and DropConnect methods.
Scalable Machine Translation in Memory Constrained Environments
Venue: arXiv cs.CL 2016
Year: 2016

Machine translation is the discipline concerned with developing automated tools for translating from one human language to another. Statistical machine translation (SMT) is the dominant paradigm in this field. In SMT, translations are generated by means of statistical models whose parameters are learned from bilingual data. Scalability is a key concern in SMT, as one would like to make use of as much data as possible to train better translation systems. In recent years, mobile devices with adequate computing power have become widely available. Despite being very successful, mobile applications relying on NLP systems continue to follow a client-server architecture, which is of limited use because access to internet is often limited and expensive. The goal of this dissertation is to show how to construct a scalable machine translation system that can operate with the limited resources available on a mobile device. The main challenge for porting translation systems on mobile devices is memory usage. The amount of memory available on a mobile device is far less than what is typically available on the server side of a client-server application. In this thesis, we investigate alternatives for the two components which prevent standard translation systems from working on mobile devices due to high memory usage. We show that once these standard components are replaced with our proposed alternatives, we obtain a scalable translation system that can work on a device with limited memory.
On the automated verification of web applications with embedded SQL
Venue: arXiv cs.LO 2016
Year: 2016

A large number of web applications is based on a relational database together with a program, typically a script, that enables the user to interact with the database through embedded SQL queries and commands. In this paper, we introduce a method for formal automated verification of such systems which connects database theory to mainstream program analysis. We identify a fragment of SQL which captures the behavior of the queries in our case studies, is algorithmically decidable, and facilitates the construction of weakest preconditions. Thus, we can integrate the analysis of SQL queries into a program analysis tool chain. To this end, we implement a new decision procedure for the SQL fragment that we introduce. We demonstrate practical applicability of our results with three case studies, a web administrator, a simple firewall, and a conference management system.
Logic as a distributive law
Venue: arXiv cs.LO 2016
Year: 2016

We present an algorithm for deriving a spatial-behavioral type system from a formal presentation of a computational calculus. Given a 2-monad Calc: Catv$\to$ Cat for the free calculus on a category of terms and rewrites and a 2-monad BoolAlg for the free Boolean algebra on a category, we get a 2-monad Form = BoolAlg + Calc for the free category of formulae and proofs. We also get the 2-monad BoolAlg $\circ$ Calc for subsets of terms. The interpretation of formulae is a natural transformation $\interp{-}$: Form $\Rightarrow$ BoolAlg $\circ$ Calc defined by the units and multiplications of the monads and a distributive law transformation $δ$: Calc $\circ$ BoolAlg $\Rightarrow$ BoolAlg $\circ$ Calc. This interpretation is consistent both with the Curry-Howard isomorphism and with realizability. We give an implementation of the "possibly" modal operator parametrized by a two-hole term context and show that, surprisingly, the arrow type constructor in the $λ$-calculus is a specific case. We also exhibit nontrivial formulae encoding confinement and liveness properties for a reflective higher-order variant of the $π$-calculus.
Backward-Shifted Coding (BSC) based on Scalable Video Coding for HAS
Venue: arXiv cs.MM 2016
Year: 2016

The main task of HTTP Adaptive Streaming is to adapt video quality dynamically under variable network conditions. This is a key feature for multimedia delivery especially when quality of service cannot be granted network-wide and, e.g., throughput may suffer short term fluctuations. Hence, robust bitrate adaptation schemes become crucial in order to improve video quality. The objective, in this context, is to control the filling level of the playback buffer and maximize the quality of the video, while avoiding unnecessary video quality variations. In this paper we study bitrate adaptation algorithms based on Backward-Shifted Coding (BSC), a scalable video coding scheme able to greatly improve video quality. We design bitrate adaptation algorithms that balance video rate smoothness and high network capacity utilization, leveraging both on throughput-based and buffer-based adaptation mechanisms. Extensive simulations using synthetic and real-world video traffic traces show that the proposed scheme performs remarkably well even under challenging network conditions.
Optimization of Convolutional Neural Network using Microcanonical Annealing Algorithm
Venue: arXiv cs.CV 2016
Year: 2016

Convolutional neural network (CNN) is one of the most prominent architectures and algorithm in Deep Learning. It shows a remarkable improvement in the recognition and classification of objects. This method has also been proven to be very effective in a variety of computer vision and machine learning problems. As in other deep learning, however, training the CNN is interesting yet challenging. Recently, some metaheuristic algorithms have been used to optimize CNN using Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing and Harmony Search. In this paper, another type of metaheuristic algorithms with different strategy has been proposed, i.e. Microcanonical Annealing to optimize Convolutional Neural Network. The performance of the proposed method is tested using the MNIST and CIFAR-10 datasets. Although experiment results of MNIST dataset indicate the increase in computation time (1.02x - 1.38x), nevertheless this proposed method can considerably enhance the performance of the original CNN (up to 4.60\%). On the CIFAR10 dataset, currently, state of the art is 96.53\% using fractional pooling, while this proposed method achieves 99.14\%.
Adaptive Convolutional ELM For Concept Drift Handling in Online Stream Data
Venue: arXiv cs.AI 2016
Year: 2016

In big data era, the data continuously generated and its distribution may keep changes overtime. These challenges in online stream of data are known as concept drift. In this paper, we proposed the Adaptive Convolutional ELM method (ACNNELM) as enhancement of Convolutional Neural Network (CNN) with a hybrid Extreme Learning Machine (ELM) model plus adaptive capability. This method is aimed for concept drift handling. We enhanced the CNN as convolutional hiererchical features representation learner combined with Elastic ELM (E$^2$LM) as a parallel supervised classifier. We propose an Adaptive OS-ELM (AOS-ELM) for concept drift adaptability in classifier level (named ACNNELM-1) and matrices concatenation ensembles for concept drift adaptability in ensemble level (named ACNNELM-2). Our proposed Adaptive CNNELM is flexible that works well in classifier level and ensemble level while most current methods only proposed to work on either one of the levels. We verified our method in extended MNIST data set and not MNIST data set. We set the experiment to simulate virtual drift, real drift, and hybrid drift event and we demonstrated how our CNNELM adaptability works. Our proposed method works well and gives better accuracy, computation scalability, and concept drifts adaptability compared to the regular ELM and CNN. Further researches are still required to study the optimum parameters and to use more varied image data set.
Instagram Post Data Analysis
Venue: arXiv cs.HC 2016
Year: 2016

Because of the spread of the Internet, social platforms become big data pools. From there we can learn about the trends, culture and hot topics. This project focuses on analyzing the data from Instagram. It shows the relationship of Instagram filter data with location and number of likes to give users filter suggestion on achieving more likes based on their location. It also analyzes the popular hashtags in different locations to show visual culture differences between different cities.
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Venue: arXiv cs.LG 2016
Year: 2016

We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are unevenly distributed over an extremely large number of nodes. The goal is to train a high-quality centralized model. We refer to this setting as Federated Optimization. In this setting, communication efficiency is of the utmost importance and minimizing the number of rounds of communication is the principal goal. A motivating example arises when we keep the training data locally on users' mobile devices instead of logging it to a data center for training. In federated optimziation, the devices are used as compute nodes performing computation on their local data in order to update a global model. We suppose that we have extremely large number of devices in the network --- as many as the number of users of a given service, each of which has only a tiny fraction of the total data available. In particular, we expect the number of data points available locally to be much smaller than the number of devices. Additionally, since different users generate data with different patterns, it is reasonable to assume that no device has a representative sample of the overall distribution. We show that existing algorithms are not suitable for this setting, and propose a new algorithm which shows encouraging experimental results for sparse convex problems. This work also sets a path for future research needed in the context of \federated optimization.
Correction to the article "Dynamic power management in energy-aware computer networks and data intensive computing systems" published in "Future Generation Computer Systems" journal
Venue: arXiv cs.DC 2016
Year: 2016

This paper indicates two errors in the formulation of the main optimization model in the article "Dynamic power management in energy-aware computer networks and data intensive computing systems" by Niewiadomska-Szynkiewicz et al. [FGCS, vol.37 (2014), pp.284-296] and shows how to fix them.
Reconstruction of signals from their autocorrelation and cross-correlation vectors, with applications to phase retrieval and blind channel estimation
Venue: arXiv cs.IT 2016
Year: 2016

We consider the problem of reconstructing two signals from the autocorrelation and cross-correlation measurements. This inverse problem is a fundamental one in signal processing, and arises in many applications, including phase retrieval and blind channel estimation. In a typical phase retrieval setup, only the autocorrelation measurements are obtainable. We show that, when the measurements are obtained using three simple "masks", phase retrieval reduces to the aforementioned reconstruction problem. The classic solution to this problem is based on finding common factors between the $z$-transforms of the autocorrelation and cross-correlation vectors. This solution has enjoyed limited practical success, mainly due to the fact that it is not sufficiently stable in the noisy setting. In this work, inspired by the success of convex programming in provably and stably solving various quadratic constrained problems, we develop a semidefinite programming-based algorithm and provide theoretical guarantees. In particular, we show that almost all signals can be uniquely recovered by this algorithm (up to a global phase). Comparative numerical studies demonstrate that the proposed method significantly outperforms the classic method in the noisy setting.
Multi-Objective Deep Reinforcement Learning
Venue: arXiv cs.AI 2016
Year: 2016

We propose Deep Optimistic Linear Support Learning (DOL) to solve high-dimensional multi-objective decision problems where the relative importances of the objectives are not known a priori. Using features from the high-dimensional inputs, DOL computes the convex coverage set containing all potential optimal solutions of the convex combinations of the objectives. To our knowledge, this is the first time that deep reinforcement learning has succeeded in learning multi-objective policies. In addition, we provide a testbed with two experiments to be used as a benchmark for deep multi-objective reinforcement learning.
Electoral Competition under Best-Worst Voting Rules
Venue: arXiv cs.GT 2016
Year: 2016

We characterise multi-candidate pure-strategy equilibria in the Hotelling-Downs spatial election model for the class of best-worst voting rules, in which each voter is endowed with both a positive and a negative vote, i.e., each voter can vote in favour of one candidate and against another one. The weights attached to positive and negative votes in calculating a candidate's net score may be different, so that a negative vote and a positive vote need not cancel out exactly. These rules combine the first-place seeking incentives of plurality with the incentives to avoid being ranked last of anti-plurality. We show that these rules generally admit equilibria, which are nonconvergent if and only if the importance of a positive vote exceeds that of a negative vote. The set of equilibria in the latter case is very similar to that of plurality, except that the platforms are less extreme due to the moderating effect of negative votes. Moreover, any degree of dispersion between plurality, at one extreme, and full convergence, at the other, can be attained for the correct choice of the weights.
Energy Efficient Design for Tactile Internet
Venue: arXiv cs.IT 2016
Year: 2016

Ensuring the ultra-low end-to-end latency and ultrahigh reliability required by tactile internet is challenging. This is especially true when the stringent Quality-of-Service (QoS) requirement is expected to be satisfied not at the cost of significantly reducing spectral efficiency and energy efficiency (EE). In this paper, we study how to maximize the EE for tactile internet under the stringent QoS constraint, where both queueing delay and transmission delay are taken into account. We first validate that the upper bound of queueing delay violation probability derived from the effective bandwidth can be used to characterize the queueing delay violation probability in the short delay regime for Poisson arrival process. However, the upper bound is not tight for short delay, which leads to conservative designs and hence leads to wasting energy. To avoid this, we optimize resource allocation that depends on the queue state information and channel state information. Analytical results show that with a large number of transmit antennas the EE achieved by the proposed policy approaches to the EE limit achieved for infinite delay bound, which implies that the policy does not lead to any EE loss. Simulation and numerical results show that even for not-so-large number of antennas, the EE achieved by the proposed policy is still close to the EE limit.
Redundancies in Linear Systems with two Variables per Inequality
Venue: arXiv cs.DS 2016
Year: 2016

The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs), given by $d$ variables with $n$ inequality constraints. A constraint is called \emph{redundant}, if after its removal, the LP still has the same feasible region. The currently fastest method to detect all redundancies is due to Clarkson: it solves $n$ linear programs, but each of them has at most $s$ constraints, where $s$ is the number of nonredundant constraints. In this paper, we study the special case where every constraint has at most two variables with nonzero coefficients. This family, denoted by $LI(2)$, has some nice properties. Namely, as shown by Aspvall and Shiloach, given a variable $x_i$ and a value $λ$, we can test in time $O(nd)$ whether there is a feasible solution with $x_i = λ$. Hochbaum and Naor present an $O(d^2 n \log n)$ algorithm for solving the feasibility problem in $LI(2)$. Their technique makes use of the Fourier-Motzkin elimination method and the earlier mentioned result by Aspvall and Shiloach. We present a strongly polynomial algorithm that solves redundancy detection in time $O(n d^2 s \log s)$. It uses a modification of Clarkson's algorithm, together with a revised version of Hochbaum and Naor's technique. Finally we show that dimensionality testing can be done with the same running time as solving feasibility.