OpenRaiser/Intern-Atlas huggingface dataset:
A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
Venue: arXiv cs.CV 2018
Year: 2018

We propose a new flexible deep convolutional neural network (convnet) to perform fast visual style transfer. In contrast to existing convnets that address the same task, our architecture derives directly from the structure of the gradient descent originally used to solve the style transfer problem [Gatys et al., 2016]. Like existing convnets, ours approximately solves the original problem much faster than the gradient descent. However, our network is uniquely flexible by design: it can be manipulated at runtime to enforce new constraints on the final solution. In particular, we show how to modify it to obtain a photorealistic result with no retraining. We study the modifications made by [Luan et al., 2017] to the original cost function of [Gatys et al., 2016] to achieve photorealistic style transfer. These modifications affect directly the gradient descent and can be reported on-the-fly in our network. These modifications are possible as the proposed architecture stems from unrolling the gradient descent.
Rate-Splitting Robustness in Multi-Pair Massive MIMO Relay Systems
Venue: arXiv cs.IT 2018
Year: 2018

Relay systems improve both coverage and system capacity. Towards this direction, full-duplex (FD) technology, being able to boost the spectral efficiency by transmitting and receiving simultaneously on the same frequency and time resources, is envisaged to play a key role in future networks. However, its benefits come at the expense of self-interference (SI) from their own transmit signal. At the same time, massive MIMO systems, bringing unconventionally many antennas, emerge as a promising technology with huge degrees-of-freedom (DoF). To this end, this paper considers a multi-pair decode-and-forward FD relay channel, where the relay station is deployed with a large number of antennas. Moreover, the rate-splitting (RS) transmission has recently been shown to provide significant performance benefits in various multi-user scenarios with imperfect channel state information at the transmitter (CSIT). Engaging the RS approach, we employ the deterministic equivalent (DE) analysis to derive the corresponding sum-rates in the presence of interferences. Initially, numerical results demonstrate the robustness of RS in half-duplex (HD) systems, since the achievable sum-rate increases without bound, i.e., it does not saturate at high signal-to-noise ratio (SNR). Next, we tackle the detrimental effect of SI in FD. In particular, and most importantly, not only FD outperforms HD, but also RS enables increasing the range of SI over which FD outperforms HD. Furthermore, increasing the number of relay station antennas, RS appears to be more efficacious due to imperfect CSIT, since SI decreases. Interestingly, increasing the number of users, the efficiency of RS worsens and its implementation becomes less favorable under these conditions. Finally, we verify that the proposed DEs, being accurate for a large number of relay station antennas, are tight approximations even for realistic system dimensions.
Towards Implementation of Robust and Low-Cost Security Primitives for Resource-Constrained IoT Devices
Venue: arXiv cs.CR 2018
Year: 2018

In recent years, due to the trend in globalization, system integrators have had to deal with integrated circuit (IC)/intellectual property (IP) counterfeiting more than ever. These counterfeit hardware issues counterfeit hardware that have driven the need for more secure chip authentication. High entropy random numbers from physical sources are a critical component in authentication and encryption processes within secure systems [6]. Secure encryption is dependent on sources of truly random numbers for generating keys, and there is a need for an on chip random number generator to achieve adequate security. Furthermore, the Internet of Things (IoT) adopts a large number of these hardware-based security and prevention solutions in order to securely exchange data in resource efficient manner. In this work, we have developed several methodologies of hardware-based random functions in order to address the issues and enhance the security and trust of ICs: a novel DRAM-based intrinsic Physical Unclonable Function (PUF) [13] for system level security and authentication along with analysis of the impact of various environmental conditions, particularly silicon aging; a DRAM remanence based True Random Number Generation (TRNG) to produce random sequences with a very low overhead; a DRAM TRNG model using its startup value behavior for creating random bit streams; an efficient power supply noise based TRNG model for generating an infinite number of random bits which has been evaluated as a cost effective technique; architectures and hardware security solutions for the Internet of Things (IoT) environment. Since IoT devices are heavily resource constrained, our proposed designs can alleviate the concerns of establishing trustworthy and security in an efficient and low-cost manner.
From Trailers to Storylines: An Efficient Way to Learn from Movies
Venue: arXiv cs.CV 2018
Year: 2018

The millions of movies produced in the human history are valuable resources for computer vision research. However, learning a vision model from movie data would meet with serious difficulties. A major obstacle is the computational cost -- the length of a movie is often over one hour, which is substantially longer than the short video clips that previous study mostly focuses on. In this paper, we explore an alternative approach to learning vision models from movies. Specifically, we consider a framework comprised of a visual module and a temporal analysis module. Unlike conventional learning methods, the proposed approach learns these modules from different sets of data -- the former from trailers while the latter from movies. This allows distinctive visual features to be learned within a reasonable budget while still preserving long-term temporal structures across an entire movie. We construct a large-scale dataset for this study and define a series of tasks on top. Experiments on this dataset showed that the proposed method can substantially reduce the training time while obtaining highly effective features and coherent temporal structures.
Fire SSD: Wide Fire Modules based Single Shot Detector on Edge Device
Venue: arXiv cs.CV 2018
Year: 2018

With the emergence of edge computing, there is an increasing need for running convolutional neural network based object detection on small form factor edge computing devices with limited compute and thermal budget for applications such as video surveillance. To address this problem, efficient object detection frameworks such as YOLO and SSD were proposed. However, SSD based object detection that uses VGG16 as backend network is insufficient to achieve real time speed on edge devices. To further improve the detection speed, the backend network is replaced by more efficient networks such as SqueezeNet and MobileNet. Although the speed is greatly improved, it comes with a price of lower accuracy. In this paper, we propose an efficient SSD named Fire SSD. Fire SSD achieves 70.7mAP on Pascal VOC 2007 test set. Fire SSD achieves the speed of 30.6FPS on low power mainstream CPU and is about 6 times faster than SSD300 and has about 4 times smaller model size. Fire SSD also achieves 22.2FPS on integrated GPU.
Theory of Estimation-of-Distribution Algorithms
Venue: arXiv cs.NE 2018
Year: 2018

Estimation-of-distribution algorithms (EDAs) are general metaheuristics used in optimization that represent a more recent alternative to classical approaches like evolutionary algorithms. In a nutshell, EDAs typically do not directly evolve populations of search points but build probabilistic models of promising solutions by repeatedly sampling and selecting points from the underlying search space. Recently, there has been made significant progress in the theoretical understanding of EDAs. This article provides an up-to-date overview of the most commonly analyzed EDAs and the most recent theoretical results in this area. In particular, emphasis is put on the runtime analysis of simple univariate EDAs, including a description of typical benchmark functions and tools for the analysis. Along the way, open problems and directions for future research are described.
Micro Congestion Control: Every Flow Deserves a Second Chance
Venue: arXiv cs.NI 2018
Year: 2018

Today, considerable Internet traffic is sent from the datacenter and heads for users. The characteristics of connections served by servers in datacenters are usually diverse and varied over time, with continuous upgrades in network infrastructure and user devices. As a result, a specific congestion control algorithm hardly accommodates the heterogeneity and performs well in various scenarios. In this work, we present Micro Congestion Control (MCC) --- a novel framework for Internet congestion control. With MCC, diverse algorithms can be assigned purposely to connections in one server to adapt to heterogeneity, and different algorithms can be chosen in each connection's life cycle to keep pace with the dynamic of network. We design and implement MCC in Linux, and the experiments validate that MCC is capable of smoothly switching among various candidate algorithms on the fly to achieve potential performance gain in the real world. Meanwhile, the overheads introduced by MCC are moderate and acceptable.
Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
Venue: arXiv cs.CV 2018
Year: 2018

Recent advances in deep learning led to novel generative modeling techniques that achieve unprecedented quality in generated samples and performance in learning complex distributions in imaging data. These new models in medical image computing have important applications that form clinically relevant and very challenging unsupervised learning problems. In this paper, we explore the feasibility of using state-of-the-art auto-encoder-based deep generative models, such as variational and adversarial auto-encoders, for one such task: abnormality detection in medical imaging. We utilize typical, publicly available datasets with brain scans from healthy subjects and patients with stroke lesions and brain tumors. We use the data from healthy subjects to train different auto-encoder based models to learn the distribution of healthy images and detect pathologies as outliers. Models that can better learn the data distribution should be able to detect outliers more accurately. We evaluate the detection performance of deep generative models and compare them with non-deep learning based approaches to provide a benchmark of the current state of research. We conclude that abnormality detection is a challenging task for deep generative models and large room exists for improvement. In order to facilitate further research, we aim to provide carefully pre-processed imaging data available to the research community.
Low-rank geometric mean metric learning
Venue: arXiv cs.LG 2018
Year: 2018

We propose a low-rank approach to learning a Mahalanobis metric from data. Inspired by the recent geometric mean metric learning (GMML) algorithm, we propose a low-rank variant of the algorithm. This allows to jointly learn a low-dimensional subspace where the data reside and the Mahalanobis metric that appropriately fits the data. Our results show that we compete effectively with GMML at lower ranks.
Distributed Hypothesis Testing based on Unequal-Error Protection Codes
Venue: arXiv cs.IT 2018
Year: 2018

Coding and testing schemes for binary hypothesis testing over noisy networks are proposed and their corresponding type-II error exponents are derived. When communication is over a discrete memoryless channel (DMC), our scheme combines Shimokawa-Han-Amari's hypothesis testing scheme with Borade's unequal error protection (UEP) for channel coding. A separate source channel coding architecture is employed. The resulting exponent is optimal for the newly introduced class of \emph{generalized testing against conditional independence}. When communication is over a MAC or a BC, our scheme combines hybrid coding with UEP. The resulting error exponent over the MAC is optimal in the case of generalized testing against conditional independence with independent observations at the two sensors, when the MAC decomposes into two individual DMCs. In this case, separate source-channel coding is sufficient; this same conclusion holds also under arbitrarily correlated sensor observations when testing is against independence. For the BC, the error exponents region of hybrid coding with UEP exhibits a tradeoff between the exponents attained at the two decision centers. When both receivers aim at maximizing the error exponents under different hypotheses and the marginal distributions of the sensors' observations are different under these hypotheses, then this tradeoff can be mitigated with the following strategy. The sensor makes a tentative guess on the hypothesis, submits this guess, and applies our coding and testing scheme for the DMC only for the decision center that is not interested in maximizing the exponent under the guessed hypothesis.
Tract orientation mapping for bundle-specific tractography
Venue: arXiv cs.CV 2018
Year: 2018

While the major white matter tracts are of great interest to numerous studies in neuroscience and medicine, their manual dissection in larger cohorts from diffusion MRI tractograms is time-consuming, requires expert knowledge and is hard to reproduce. Tract orientation mapping (TOM) is a novel concept that facilitates bundle-specific tractography based on a learned mapping from the original fiber orientation distribution function (fODF) peaks to a list of tract orientation maps (also abbr. TOM). Each TOM represents one of the known tracts with each voxel containing no more than one orientation vector. TOMs can act as a prior or even as direct input for tractography. We use an encoder-decoder fully-convolutional neural network architecture to learn the required mapping. In comparison to previous concepts for the reconstruction of specific bundles, the presented one avoids various cumbersome processing steps like whole brain tractography, atlas registration or clustering. We compare it to four state of the art bundle recognition methods on 20 different bundles in a total of 105 subjects from the Human Connectome Project. Results are anatomically convincing even for difficult tracts, while reaching low angular errors, unprecedented runtimes and top accuracy values (Dice). Our code and our data are openly available.
From Self-ception to Image Self-ception: A method to represent an image with its own approximations
Venue: arXiv cs.CV 2018
Year: 2018

A concept of defining images based on its own approximate ones is proposed here, which is called 'Self-ception'. In this regard, an algorithm is proposed to implement the self-ception for images, which we call it 'Image Self-ception' since we use it for images. We can control the accuracy of this self-ception representation by deciding how many segments or regions we want to use for the representation. Some self-ception images are included in the paper. The video versions of the proposed image self-ception algorithm in action are shown in a YouTube channel (find it by Googling image self-ception).
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
Venue: arXiv cs.LG 2018
Year: 2018

Modern deep transfer learning approaches have mainly focused on learning generic feature vectors from one task that are transferable to other tasks, such as word embeddings in language and pretrained convolutional features in vision. However, these approaches usually transfer unary features and largely ignore more structured graphical representations. This work explores the possibility of learning generic latent relational graphs that capture dependencies between pairs of data units (e.g., words or pixels) from large-scale unlabeled data and transferring the graphs to downstream tasks. Our proposed transfer learning framework improves performance on various tasks including question answering, natural language inference, sentiment analysis, and image classification. We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden unit), or embedding-free units such as image pixels.
An eco-friendly Ecash with recycled banknotes
Venue: arXiv cs.CR 2018
Year: 2018

By comparing cryptocurrencies with other existing payment methods, including banknotes and bank cards, it is clear that the use of Bitcoin and its competitors (Ethereum, \dots) is almost insignificant in world trade. We may also note that these cryptocurrencies have become tools of speculation, which is the antithesis of their primary purpose. Based essentially on the security of electronic signatures, the Ecash introduced here will put the users back to the center of the game and exclude miners and their enormous waste of power energy. Thus, the purpose of this paper is to show that even a piece of paper can be recycled into a secure Ecash, while remaining environmentally friendly. Hence, we create here a cryptocurrency that would use a slight modification of the current banknotes to set up anonymous electronic transactions. By trading with banknotes, we mechanically transfer ownership of the paper money from one owner to another, hence, in this scheme, we introduce the notion of ownership transfer. It implies that at each transaction the elements allowing to authenticate the Ecash does not change, while the ownership certificate will change since the Ecash is transfer towards someone else.
Generative Adversarial Networks and Perceptual Losses for Video Super-Resolution
Venue: arXiv cs.CV 2018
Year: 2018

Video super-resolution (VSR) has become one of the most critical problems in video processing. In the deep learning literature, recent works have shown the benefits of using adversarial-based and perceptual losses to improve the performance on various image restoration tasks; however, these have yet to be applied for video super-resolution. In this work, we propose a Generative Adversarial Network(GAN)-based formulation for VSR. We introduce a new generator network optimized for the VSR problem, named VSRResNet, along with a new discriminator architecture to properly guide VSRResNet during the GAN training. We further enhance our VSR GAN formulation with two regularizers, a distance loss in feature-space and pixel-space, to obtain our final VSRResFeatGAN model. We show that pre-training our generator with the Mean-Squared-Error loss only quantitatively surpasses the current state-of-the-art VSR models. Finally, we employ the PercepDist metric (Zhang et al., 2018) to compare state-of-the-art VSR models. We show that this metric more accurately evaluates the perceptual quality of SR solutions obtained from neural networks, compared with the commonly used PSNR/SSIM metrics. Finally, we show that our proposed model, the VSRResFeatGAN model, outperforms current state-of-the-art SR models, both quantitatively and qualitatively.
Safe Active Feature Selection for Sparse Learning
Venue: arXiv cs.LG 2018
Year: 2018

We present safe active incremental feature selection~(SAIF) to scale up the computation of LASSO solutions. SAIF does not require a solution from a heavier penalty parameter as in sequential screening or updating the full model for each iteration as in dynamic screening. Different from these existing screening methods, SAIF starts from a small number of features and incrementally recruits active features and updates the significantly reduced model. Hence, it is much more computationally efficient and scalable with the number of features. More critically, SAIF has the safe guarantee as it has the convergence guarantee to the optimal solution to the original full LASSO problem. Such an incremental procedure and theoretical convergence guarantee can be extended to fused LASSO problems. Compared with state-of-the-art screening methods as well as working set and homotopy methods, which may not always guarantee the optimal solution, SAIF can achieve superior or comparable efficiency and high scalability with the safe guarantee when facing extremely high dimensional data sets. Experiments with both synthetic and real-world data sets show that SAIF can be up to 50 times faster than dynamic screening, and hundreds of times faster than computing LASSO or fused LASSO solutions without screening.
Financial Risk and Returns Prediction with Modular Networked Learning
Venue: arXiv cs.LG 2018
Year: 2018

An artificial agent for financial risk and returns' prediction is built with a modular cognitive system comprised of interconnected recurrent neural networks, such that the agent learns to predict the financial returns, and learns to predict the squared deviation around these predicted returns. These two expectations are used to build a volatility-sensitive interval prediction for financial returns, which is evaluated on three major financial indices and shown to be able to predict financial returns with higher than 80% success rate in interval prediction in both training and testing, raising into question the Efficient Market Hypothesis. The agent is introduced as an example of a class of artificial intelligent systems that are equipped with a Modular Networked Learning cognitive system, defined as an integrated networked system of machine learning modules, where each module constitutes a functional unit that is trained for a given specific task that solves a subproblem of a complex main problem expressed as a network of linked subproblems. In the case of neural networks, these systems function as a form of an "artificial brain", where each module is like a specialized brain region comprised of a neural network with a specific architecture.
Efficient Handling of SPARQL OPTIONAL for OBDA (Extended Version)
Venue: arXiv cs.DB 2018
Year: 2018

OPTIONAL is a key feature in SPARQL for dealing with missing information. While this operator is used extensively, it is also known for its complexity, which can make efficient evaluation of queries with OPTIONAL challenging. We tackle this problem in the Ontology-Based Data Access (OBDA) setting, where the data is stored in a SQL relational database and exposed as a virtual RDF graph by means of an R2RML mapping. We start with a succinct translation of a SPARQL fragment into SQL. It fully respects bag semantics and three-valued logic and relies on the extensive use of the LEFT JOIN operator and COALESCE function. We then propose optimisation techniques for reducing the size and improving the structure of generated SQL queries. Our optimisations capture interactions between JOIN, LEFT JOIN, COALESCE and integrity constraints such as attribute nullability, uniqueness and foreign key constraints. Finally, we empirically verify effectiveness of our techniques on the BSBM OBDA benchmark.
Discovering User Groups for Natural Language Generation
Venue: arXiv cs.CL 2018
Year: 2018

We present a model which predicts how individual users of a dialog system understand and produce utterances based on user groups. In contrast to previous work, these user groups are not specified beforehand, but learned in training. We evaluate on two referring expression (RE) generation tasks; our experiments show that our model can identify user groups and learn how to most effectively talk to them, and can dynamically assign unseen users to the correct groups as they interact with the system.