OpenRaiser/Intern-Atlas huggingface dataset:
How to select the best set of ads: Can we do better than Greedy Algorithm?
Venue: arXiv cs.DS 2018
Year: 2018

Selecting the best set of ads is critical for advertisers for a given set of keywords, which involves the composition of ads from millions of candidates. While click through rates (CTRs) are important, there could be high correlation among different ads, therefore the set of ads with top CTRs does not necessarily maximize the number of clicks. Greedy algorithm has been a standard and straightforward way to find out a decent enough solution, however, it is not guaranteed to be the global optimum. In fact, it proves not to be the global optimum more than 70% of the time across all our simulations, implying that it's very likely to be trapped at a local optimum. In this paper, we propose a Greedy-Power Algorithm to find out the best set of creatives, that is starting with the solution from the conventional Greedy Algorithm, one can perform another Greedy Algorithm search on top of it, with the option of a few or even infinite rounds. The Greedy-Power algorithm is guaranteed to be not worse, as it only moves in the direction to increase the goal function. We show that Greedy-Power Algorithm's performance is consistently better, and reach the conclusion that it is able to perform better than the Greedy Algorithm systematically.
Mixed Link Networks
Venue: arXiv cs.LG 2018
Year: 2018

Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same "dense topology", yet they only differ in the form of connection -- addition (dubbed "inner link") vs. concatenation (dubbed "outer link"). However, both two forms of connections have the superiority and insufficiency. To combine their advantages and avoid certain limitations on representation learning, we present a highly efficient and modularized Mixed Link Network (MixNet) which is equipped with flexible inner link and outer link modules. Consequently, ResNet, DenseNet and Dual Path Network (DPN) can be regarded as a special case of MixNet, respectively. Furthermore, we demonstrate that MixNets can achieve superior efficiency in parameter over the state-of-the-art architectures on many competitive datasets like CIFAR-10/100, SVHN and ImageNet.
Rollable Latent Space for Azimuth Invariant SAR Target Recognition
Venue: arXiv cs.CV 2018
Year: 2018

This paper proposes rollable latent space (RLS) for an azimuth invariant synthetic aperture radar (SAR) target recognition. Scarce labeled data and limited viewing direction are critical issues in SAR target recognition.The RLS is a designed space in which rolling of latent features corresponds to 3D rotation of an object. Thus latent features of an arbitrary view can be inferred using those of different views. This characteristic further enables us to augment data from limited viewing in RLS. RLS-based classifiers with and without data augmentation and a conventional classifier trained with target front shots are evaluated over untrained target back shots. Results show that the RLS-based classifier with augmentation improves an accuracy by 30% compared to the conventional classifier.
Learning Image Representations by Completing Damaged Jigsaw Puzzles
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we explore methods of complicating self-supervised tasks for representation learning. That is, we do severe damage to data and encourage a network to recover them. First, we complicate each of three powerful self-supervised task candidates: jigsaw puzzle, inpainting, and colorization. In addition, we introduce a novel complicated self-supervised task called "Completing damaged jigsaw puzzles" which is puzzles with one piece missing and the other pieces without color. We train a convolutional neural network not only to solve the puzzles, but also generate the missing content and colorize the puzzles. The recovery of the aforementioned damage pushes the network to obtain robust and general-purpose representations. We demonstrate that complicating the self-supervised tasks improves their original versions and that our final task learns more robust and transferable representations compared to the previous methods, as well as the simple combination of our candidate tasks. Our approach achieves state-of-the-art performance in transfer learning on PASCAL classification and semantic segmentation.
The performances of R GPU implementations of the GMRES method
Venue: arXiv cs.DC 2018
Year: 2018

Although the performance of commodity computers has improved drastically with the introduction of multicore processors and GPU computing, the standard R distribution is still based on single-threaded model of computation, using only a small fraction of the computational power available now for most desktops and laptops. Modern statistical software packages rely on high performance implementations of the linear algebra routines there are at the core of several important leading edge statistical methods. In this paper we present a GPU implementation of the GMRES iterative method for solving linear systems. We compare the performance of this implementation with a pure single threaded version of the CPU. We also investigate the performance of our implementation using different GPU packages available now for R such as gmatrix, gputools or gpuR which are based on CUDA or OpenCL frameworks.
Uplink Channel Estimation and Data Transmission in Millimeter-Wave CRAN with Lens Antenna Arrays
Venue: arXiv cs.IT 2018
Year: 2018

Millimeter-wave (mmWave) communication and network densification hold great promise for achieving high-rate communication in next-generation wireless networks. Cloud radio access network (CRAN), in which low-complexity remote radio heads (RRHs) coordinated by a central unit (CU) are deployed to serve users in a distributed manner, is a cost-effective solution to achieve network densification. However, when operating over a large bandwidth in the mmWave frequencies, the digital fronthaul links in a CRAN would be easily saturated by the large amount of sampled and quantized signals to be transferred between RRHs and the CU. To tackle this challenge, we propose in this paper a new architecture for mmWave-based CRAN with advanced lens antenna arrays at the RRHs. Due to the energy focusing property, lens antenna arrays are effective in exploiting the angular sparsity of mmWave channels, and thus help in substantially reducing the fronthaul rate and simplifying the signal processing at the multi-antenna RRHs and the CU, even when the channels are frequency-selective. We consider the uplink transmission in a mmWave CRAN with lens antenna arrays and propose a low-complexity quantization bit allocation scheme for multiple antennas at each RRH to meet the given fronthaul rate constraint. Further, we propose a channel estimation technique that exploits the energy focusing property of the lens array and can be implemented at the CU with low complexity. Finally, we compare the proposed mmWave CRAN using lens antenna arrays with a conventional CRAN using uniform planar arrays at the RRHs, and show that the proposed design achieves significant throughput gains, yet with much lower complexity.
A Systematic Analysis for State-of-the-Art 3D Lung Nodule Proposals Generation
Venue: arXiv cs.CV 2018
Year: 2018

Lung nodule proposals generation is the primary step of lung nodule detection and has received much attention in recent years . In this paper, we first construct a model of 3-dimension Convolutional Neural Network (3D CNN) to generate lung nodule proposals, which can achieve the state-of-the-art performance. Then, we analyze a series of key problems concerning the training performance and efficiency. Firstly, we train the 3D CNN model with data in different resolutions and find out that models trained by high resolution input data achieve better lung nodule proposals generation performances especially for nodules in too small sizes, while consumes much more memory at the same time. Then, we analyze the memory consumptions on different platforms and the experimental results indicate that CPU architecture can provide us with larger memory and enables us to explore more possibilities of 3D applications. We implement the 3D CNN model on CPU platform and propose an Intel Extended-Caffe framework which supports many highly-efficient 3D computations, which is opened source at https://github.com/extendedcaffe/extended-caffe.
2D-Densely Connected Convolution Neural Networks for automatic Liver and Tumor Segmentation
Venue: arXiv cs.CV 2018
Year: 2018

In this paper we propose a fully automatic 2-stage cascaded approach for segmentation of liver and its tumors in CT (Computed Tomography) images using densely connected fully convolutional neural network (DenseNet). We independently train liver and tumor segmentation models and cascade them for a combined segmentation of the liver and its tumor. The first stage involves segmentation of liver and the second stage uses the first stage's segmentation results for localization of liver and henceforth tumor segmentations inside liver region. The liver model was trained on the down-sampled axial slices $(256 \times 256)$, whereas for the tumor model no down-sampling of slices was done, but instead it was trained on the CT axial slices windowed at three different Hounsfield (HU) levels. On the test set our model achieved a global dice score of 0.923 and 0.625 on liver and tumor respectively. The computed tumor burden had an rmse of 0.044.
Uptake and outcome of manuscripts in Nature journals by review model and author characteristics
Venue: arXiv cs.DL 2018
Year: 2018

Double-blind peer review has been proposed as a possible solution to avoid implicit referee bias in academic publishing. The aims of this study are to analyse the demographics of corresponding authors choosing double blind peer review, and to identify differences in the editorial outcome of manuscripts depending on their review model. Data includes 128,454 manuscripts received between March 2015 and February 2017 by 25 Nature-branded journals. Author uptake for double-blind was 12%. We found a small but significant association between journal tier and review type. We found no statistically significant difference in the distribution of peer review model between males and females. We found that corresponding authors from the less prestigious institutions are more likely to choose double-blind review. In the ten countries with the highest number of submissions, we found a small but significant association between country and review type. The outcome at both first decision and post review is significantly more negative (i.e. a higher likelihood for rejection) for double than single-blind papers. Authors choose double-blind review more frequently when they submit to more prestigious journals, they are affiliated with less prestigious institutions or they are from specific countries; the double-blind option is also linked to less successful editorial outcomes.
Automated dataset generation for image recognition using the example of taxonomy
Venue: arXiv cs.CV 2018
Year: 2018

This master thesis addresses the subject of automatically generating a dataset for image recognition, which takes a lot of time when being done manually. As the thesis was written with motivation from the context of the biodiversity workgroup at the City University of Applied Sciences Bremen, the classification of taxonomic entries was chosen as an exemplary use case. In order to automate the dataset creation, a prototype was conceptualized and implemented after working out knowledge basics and analyzing requirements for it. It makes use of an pre-trained abstract artificial intelligence which is able to sort out images that do not contain the desired content. Subsequent to the implementation and the automated dataset creation resulting from it, an evaluation was performed. Other, manually collected datasets were compared to the one the prototype produced in means of specifications and accuracy. The results were more than satisfactory and showed that automatically generating a dataset for image recognition is not only possible, but also might be a decent alternative to spending time and money in doing this task manually. At the very end of this work, an idea of how to use the principle of employing abstract artificial intelligences for step-by-step classification of deeper taxonomic layers in a productive system is presented and discussed.
Describing Semantic Representations of Brain Activity Evoked by Visual Stimuli
Venue: arXiv cs.CV 2018
Year: 2018

Quantitative modeling of human brain activity based on language representations has been actively studied in systems neuroscience. However, previous studies examined word-level representation, and little is known about whether we could recover structured sentences from brain activity. This study attempts to generate natural language descriptions of semantic contents from human brain activity evoked by visual stimuli. To effectively use a small amount of available brain activity data, our proposed method employs a pre-trained image-captioning network model using a deep learning framework. To apply brain activity to the image-captioning network, we train regression models that learn the relationship between brain activity and deep-layer image features. The results demonstrate that the proposed model can decode brain activity and generate descriptions using natural language sentences. We also conducted several experiments with data from different subsets of brain regions known to process visual stimuli. The results suggest that semantic information for sentence generations is widespread across the entire cortex.
From Game-theoretic Multi-agent Log Linear Learning to Reinforcement Learning
Venue: arXiv cs.LG 2018
Year: 2018

The main focus of this paper is on enhancement of two types of game-theoretic learning algorithms: log-linear learning and reinforcement learning. The standard analysis of log-linear learning needs a highly structured environment, i.e. strong assumptions about the game from an implementation perspective. In this paper, we introduce a variant of log-linear learning that provides asymptotic guarantees while relaxing the structural assumptions to include synchronous updates and limitations in information available to the players. On the other hand, model-free reinforcement learning is able to perform even under weaker assumptions on players' knowledge about the environment and other players' strategies. We propose a reinforcement algorithm that uses a double-aggregation scheme in order to deepen players' insight about the environment and constant learning step-size which achieves a higher convergence rate. Numerical experiments are conducted to verify each algorithm's robustness and performance.
Computer-Aided Annotation for Video Tampering Dataset of Forensic Research
Venue: arXiv cs.MM 2018
Year: 2018

The annotation of video tampering dataset is a boring task that takes a lot of manpower and financial resources. At present, there is no published literature which is capable to improve the annotation efficiency of forged videos. We presented a computer-aided annotation method for video tampering dataset in this paper. This annotation method can be utilized to label the frames of forged video sequences. By means of comparing the original video frames with the forged video frames, we can locate the position and the trajectory of the forged areas of the forged video frames. Then, we select several key points on the temporal domain according to the trajectory of the forged areas, and mark the forged area of the forged frames in the key point with a mouse. Finally, we use the linear prediction algorithm based on the coordinates of the key positions in the temporal domain to generate the annotation information of forged areas in other video frames which without manually labeled. If the bounding box generated by the computer-aided algorithm deviates from the actual location of the forged area, we can use the mouse to change the position of the bounding box during the preview period. This method combines the manual annotation with computer-aided annotation. It solves the problems of the inaccuracy of annotation by computer-aided as well as the low efficiency of annotation manually, and meet the needs of annotation for an enormous amount of forged videos in the research of video passive forensics.
MAGNETO: Covert Channel between Air-Gapped Systems and Nearby Smartphones via CPU-Generated Magnetic Fields
Venue: arXiv cs.CR 2018
Year: 2018

In this paper, we show that attackers can leak data from isolated, air-gapped computers to nearby smartphones via covert magnetic signals. The proposed covert channel works even if a smartphone is kept inside a Faraday shielding case, which aims to block any type of inbound and outbound wireless communication (Wi-Fi, cellular, Bluetooth, etc.). The channel also works if the smartphone is set in airplane mode in order to block any communication with the device. We implement a malware that controls the magnetic fields emanating from the computer by regulating workloads on the CPU cores. Sensitive data such as encryption keys, passwords, or keylogging data is encoded and transmitted over the magnetic signals. A smartphone located near the computer receives the covert signals with its magnetic sensor. We present technical background, and discuss signal generation, data encoding, and signal reception. We show that the proposed covert channel works from a user-level process, without requiring special privileges, and can successfully operate from within an isolated virtual machine (VM).
A Novel Co-design Peta-scale Heterogeneous Cluster for Deep Learning Training
Venue: arXiv cs.CV 2018
Year: 2018

Large scale deep Convolution Neural Networks (CNNs) increasingly demands the computing power. It is key for researchers to own a great powerful computing platform to leverage deep learning (DL) advancing.On the other hand, as the commonly-used accelerator, the commodity GPUs cards of new generations are more and more expensive. Consequently, it is of importance to design an affordable distributed heterogeneous system that provides powerful computational capacity and develop a well-suited software that efficiently utilizes its computational capacity. In this paper, we present our co-design distributed system including a peta-scale GPU cluster, called "Manoa". Based on properties and topology of Manoa, we first propose job server framework and implement it, named "MiMatrix". The central node of MiMatrix, referred to as the job server, undertakes all of controlling, scheduling and monitoring, and I/O tasks without weight data transfer for AllReduce processing in each iteration. Therefore, MiMatrix intrinsically solves the bandwidth bottleneck of central node in parameter server framework that is widely used in distributed DL tasks. Meanwhile, we also propose a new AllReduce algorithm, GPUDirect RDMA-Aware AllReduce~(GDRAA), in which both computation and handshake message are O(1) and the number of synchronization is two in each iteration that is a theoretical minimum number. Owe to the dedicated co-design distributed system, MiMatrix efficiently makes use of the Manoa's computational capacity and bandwidth. We benchmark Manoa Resnet50 and Resenet101 on Imagenet-1K dataset. Some of results have demonstrated state-of-the-art.
Road Network Fusion for Incremental Map Updates
Venue: arXiv cs.OH 2018
Year: 2018

In the recent years a number of novel, automatic map-inference techniques have been proposed, which derive road-network from a cohort of GPS traces collected by a fleet of vehicles. In spite of considerable attention, these maps are imperfect in many ways: they create an abundance of spurious connections, have poor coverage, and are visually confusing. Hence, commercial and crowd-sourced mapping services heavily use human annotation to minimize the mapping errors. Consequently, their response to changes in the road network is inevitably slow. In this paper we describe \mapfuse, a system which fuses a human-annotated map (e.g., OpenStreetMap) with any automatically inferred map, thus effectively enabling quick map updates. In addition to new road creation, we study in depth road closure, which have not been examined in the past. By leveraging solid, human-annotated maps with minor corrections, we derive maps which minimize the trajectory matching errors due to both road network change and imperfect map inference of fully-automatic approaches.
The $b$-branching problem in digraphs
Venue: arXiv cs.DM 2018
Year: 2018

In this paper, we introduce the concept of $b$-branchings in digraphs, which is a generalization of branchings serving as a counterpart of $b$-matchings. Here $b$ is a positive integer vector on the vertex set of a digraph, and a $b$-branching is defined as a common independent set of two matroids defined by $b$: an arc set is a $b$-branching if it has at most $b(v)$ arcs sharing the terminal vertex $v$, and it is an independent set of a certain sparsity matroid defined by $b$. We demonstrate that $b$-branchings yield an appropriate generalization of branchings by extending several classical results on branchings. We first present a multi-phase greedy algorithm for finding a maximum-weight $b$-branching. We then prove a packing theorem extending Edmonds' disjoint branchings theorem, and provide a strongly polynomial algorithm for finding optimal disjoint $b$-branchings. As a consequence of the packing theorem, we prove the integer decomposition property of the $b$-branching polytope. Finally, we deal with a further generalization in which a matroid constraint is imposed on the $b(v)$ arcs sharing the terminal vertex $v$.
Cramer-Rao Bound for Constrained Parameter Estimation Using Lehmann-Unbiasedness
Venue: arXiv cs.IT 2018
Year: 2018

The constrained Cramer-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the non-asymptotic region these conditions are usually too strict and the commonly-used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-unbiasedness, which is based on using Lehmann-unbiasedness with a weighted MSE (WMSE) risk and taking into account the parametric constraints. In addition to C-unbiasedness, a Cramer-Rao-type bound on the WMSE of C-unbiased estimators, denoted as Lehmann-unbiased CCRB (LU-CCRB), is derived. This bound is a scalar bound that depends on the chosen weighted combination of estimation errors. It is shown that C-unbiasedness is less restrictive than the CCRB unbiasedness conditions. Thus, the set of estimators that satisfy the CCRB unbiasedness conditions is a subset of the set of C-unbiased estimators and the proposed LU-CCRB may be an informative lower bound in cases where the corresponding CCRB is not. In the simulations, we examine linear and nonlinear estimation problems under nonlinear constraints in which the CML estimator is shown to be C-unbiased and the LU-CCRB is an informative lower bound on the WMSE, while the corresponding CCRB on the WMSE is not a lower bound and is not informative in the non-asymptotic region.