OpenRaiser/Intern-Atlas huggingface dataset:
Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections
Venue: arXiv cs.CV 2017
Year: 2017

Unsupervised pre-training was a critical technique for training deep neural networks years ago. With sufficient labeled data and modern training techniques, it is possible to train very deep neural networks from scratch in a purely supervised manner nowadays. However, unlabeled data is easier to obtain and usually of very large scale. How to make use of them better to help supervised learning is still a well-valued topic. In this paper, we investigate convolutional denoising auto-encoders to show that unsupervised pre-training can still improve the performance of high-level image related tasks such as image classification and semantic segmentation. The architecture we use is a convolutional auto-encoder network with symmetric shortcut connections. We empirically show that symmetric shortcut connections are very important for learning abstract representations via image reconstruction. When no extra unlabeled data are available, unsupervised pre-training with our network can regularize the supervised training and therefore lead to better generalization performance. With the help of unsupervised pre-training, our method achieves very competitive results in image classification using very simple all-convolution networks. When labeled data are limited but extra unlabeled data are available, our method achieves good results in several semi-supervised learning tasks.
Pilot Contamination is Not a Fundamental Asymptotic Limitation in Massive MIMO
Venue: arXiv cs.IT 2017
Year: 2017

Massive MIMO (multiple-input multiple-output) provides great improvements in spectral efficiency over legacy cellular networks, by coherent combining of the signals over a large antenna array and by spatial multiplexing of many users. Since its inception, the coherent interference caused by pilot contamination has been believed to be an impairment that does not vanish, even with an unlimited number of antennas. In this work, we show that this belief is incorrect and an artifact from using simplistic channel models and suboptimal signal processing schemes. We focus on the uplink and prove that with multicell MMSE combining, the spectral efficiency grows without bound as the number of antennas increases, even under pilot contamination, under a condition of linear independence between the channel covariance matrices. This condition is generally satisfied, except in special cases that are hardly found in practice.
Improving Variational Auto-Encoders using Householder Flow
Venue: arXiv cs.LG 2017
Year: 2017

Variational auto-encoders (VAE) are scalable and powerful generative models. However, the choice of the variational posterior determines tractability and flexibility of the VAE. Commonly, latent variables are modeled using the normal distribution with a diagonal covariance matrix. This results in computational efficiency but typically it is not flexible enough to match the true posterior distribution. One fashion of enriching the variational posterior distribution is application of normalizing flows, i.e., a series of invertible transformations to latent variables with a simple posterior. In this paper, we follow this line of thinking and propose a volume-preserving flow that uses a series of Householder transformations. We show empirically on MNIST dataset and histopathology data that the proposed flow allows to obtain more flexible variational posterior and competitive results comparing to other normalizing flows.
Perturbation-Based Regularization for Signal Estimation in Linear Discrete Ill-posed Problems
Venue: arXiv cs.IT 2017
Year: 2017

Estimating the values of unknown parameters from corrupted measured data faces a lot of challenges in ill-posed problems. In such problems, many fundamental estimation methods fail to provide a meaningful stabilized solution. In this work, we propose a new regularization approach and a new regularization parameter selection approach for linear least-squares discrete ill-posed problems. The proposed approach is based on enhancing the singular-value structure of the ill-posed model matrix to acquire a better solution. Unlike many other regularization algorithms that seek to minimize the estimated data error, the proposed approach is developed to minimize the mean-squared error of the estimator which is the objective in many typical estimation scenarios. The performance of the proposed approach is demonstrated by applying it to a large set of real-world discrete ill-posed problems. Simulation results demonstrate that the proposed approach outperforms a set of benchmark regularization methods in most cases. In addition, the approach also enjoys the lowest runtime and offers the highest level of robustness amongst all the tested benchmark regularization methods.
Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
Venue: arXiv cs.CV 2017
Year: 2017

We propose a CNN-based approach for 3D human body pose estimation from single RGB images that addresses the issue of limited generalizability of models trained solely on the starkly limited publicly available 3D pose data. Using only the existing 3D pose data and 2D pose data, we show state-of-the-art performance on established benchmarks through transfer of learned features, while also generalizing to in-the-wild scenes. We further introduce a new training set for human body pose estimation from monocular images of real humans that has the ground truth captured with a multi-camera marker-less motion capture system. It complements existing corpora with greater diversity in pose, human appearance, clothing, occlusion, and viewpoints, and enables an increased scope of augmentation. We also contribute a new benchmark that covers outdoor and indoor scenes, and demonstrate that our 3D pose dataset shows better in-the-wild performance than existing annotated data, which is further improved in conjunction with transfer learning from 2D pose data. All in all, we argue that the use of transfer learning of representations in tandem with algorithmic and data contributions is crucial for general 3D body pose estimation.
Android Inter-App Communication Threats and Detection Techniques
Venue: arXiv cs.CR 2017
Year: 2017

With the digital breakthrough, smart phones have become very essential component. Mobile devices are very attractive attack surface for cyber thieves as they hold personal details (accounts, locations, contacts, photos) and have potential capabilities for eavesdropping (with cameras/microphone, wireless connections). Android, being the most popular, is the target of malicious hackers who are trying to use Android app as a tool to break into and control device. Android malware authors use many anti-analysis techniques to hide from analysis tools. Academic researchers and commercial anti-malware companies are putting great effort to detect such malicious apps. They are making use of the combinations of static, dynamic and behavior based analysis techniques. Despite of all the security mechanisms provided by Android, apps can carry out malicious actions through collusion. In collusion malicious functionality is divided across multiple apps. Each participating app accomplish its part and communicate information to another app through Inter Component Communication (ICC). ICC do not require any special permissions. Also, there is no compulsion to inform user about the communication. Each participating app needs to request a minimal set of privileges, which may make it appear benign to current state-of-the-art techniques that analyze one app at a time. There are many surveys on app analysis techniques in Android; however they focus on single-app analysis. This survey augments this through focusing only on collusion among multiple-apps. In this paper, we present Android vulnerabilities that may be exploited for a possible collusion attack. We cover the existing threat analysis, scenarios, and a detailed comparison of tools for intra and inter-app analysis. To the best of our knowledge this is the first survey on app collusion and state-of-the-art detection tools in Android.
Predicting the Category and Attributes of Visual Search Targets Using Deep Gaze Pooling
Venue: arXiv cs.CV 2017
Year: 2017

Predicting the target of visual search from eye fixation (gaze) data is a challenging problem with many applications in human-computer interaction. In contrast to previous work that has focused on individual instances as a search target, we propose the first approach to predict categories and attributes of search targets based on gaze data. However, state of the art models for categorical recognition, in general, require large amounts of training data, which is prohibitive for gaze data. To address this challenge, we propose a novel Gaze Pooling Layer that integrates gaze information into CNN-based architectures as an attention mechanism - incorporating both spatial and temporal aspects of human gaze behavior. We show that our approach is effective even when the gaze pooling layer is added to an already trained CNN, thus eliminating the need for expensive joint data collection of visual and gaze data. We propose an experimental setup and data set and demonstrate the effectiveness of our method for search target prediction based on gaze behavior. We further study how to integrate temporal and spatial gaze information most effectively, and indicate directions for future research in the gaze-based prediction of mental states.
Capacity limit for faster-than-Nyquist non-orthogonal frequency-division multiplexing signaling
Venue: arXiv cs.IT 2017
Year: 2017

Faster-than-Nyquist (FTN) signal achieves higher spectral efficiency and capacity compared to Nyquist signal due to its smaller pulse interval or narrower subcarrier spacing. Shannon limit typically defines the upper-limit capacity of Nyquist signal. To the best of our knowledge, the mathematical expression for the capacity limit of FTN non-orthogonal frequency-division multiplexing (NOFDM) signal is first demonstrated in this paper. The mathematical expression shows that FTN NOFDM signal has the potential to achieve a higher capacity limit compared to Nyquist signal. In this paper, we demonstrate the principle of FTN NOFDM by taking fractional cosine transform-based NOFDM (FrCT-NOFDM) for instance. FrCT-NOFDM is first proposed and implemented by both simulation and experiment. When the bandwidth compression factor $α$ is set to $0.8$ in FrCT-NOFDM, the subcarrier spacing is equal to $40\%$ of the symbol rate per subcarrier, thus the transmission rate is about $25\%$ faster than Nyquist rate. FTN NOFDM with higher capacity would be promising in the future communication systems, especially in the bandwidth-limited applications.
Optimizing Governed Blockchains for Financial Process Authentications
Venue: arXiv cs.CR 2017
Year: 2017

We propose the formal study of governed blockchains that are owned and controlled by organizations and that neither create cryptocurrencies nor provide any incentives to solvers of cryptographic puzzles. We view such approaches as frameworks in which system parts, such as the cryptographic puzzle, may be instantiated with different technology. Owners of such a blockchain procure puzzle solvers as resources they control, and use a mathematical model to compute optimal parameters for the cryptographic puzzle mechanism or other parts of the blockchain. We illustrate this approach with a use case in which blockchains record hashes of financial process transactions to increase their trustworthiness and that of their audits. For Proof of Work as cryptographic puzzle, we develop a detailed mathematical model to derive MINLP optimization problems for computing optimal Proof of Work configuration parameters that trade off potentially conflicting aspects such as availability, resiliency, security, and cost in this governed setting. We demonstrate the utility of such a mining calculus by solving some instances of this problem. This experimental validation is strengthened by statistical experiments that confirm the validity of random variables used in formulating our mathematical model. We hope that our work may facilitate the creation of domain-specific blockchains for a wide range of applications such as trustworthy information in Internet of Things systems and bespoke improvements of legacy financial services.
Object Detection via Aspect Ratio and Context Aware Region-based Convolutional Networks
Venue: arXiv cs.CV 2017
Year: 2017

Jointly integrating aspect ratio and context has been extensively studied and shown performance improvement in traditional object detection systems such as the DPMs. It, however, has been largely ignored in deep neural network based detection systems. This paper presents a method of integrating a mixture of object models and region-based convolutional networks for accurate object detection. Each mixture component accounts for both object aspect ratio and multi-scale contextual information explicitly: (i) it exploits a mixture of tiling configurations in the RoI pooling to remedy the warping artifacts caused by a single type RoI pooling (e.g., with equally-sized 7 x 7 cells), and to respect the underlying object shapes more; (ii) it "looks from both the inside and the outside of a RoI" by incorporating contextual information at two scales: global context pooled from the whole image and local context pooled from the surrounding of a RoI. To facilitate accurate detection, this paper proposes a multi-stage detection scheme for integrating the mixture of object models, which utilizes the detection results of the model at the previous stage as the proposals for the current in both training and testing. The proposed method is called the aspect ratio and context aware region-based convolutional network (ARC-R-CNN). In experiments, ARC-R-CNN shows very competitive results with Faster R-CNN [41] and R-FCN [10] on two datasets: the PASCAL VOC and the Microsoft COCO. It obtains significantly better mAP performance using high IoU thresholds on both datasets.
Zero-Shot Learning posed as a Missing Data Problem
Venue: arXiv cs.CV 2017
Year: 2017

This paper presents a method of zero-shot learning (ZSL) which poses ZSL as the missing data problem, rather than the missing label problem. Specifically, most existing ZSL methods focus on learning mapping functions from the image feature space to the label embedding space. Whereas, the proposed method explores a simple yet effective transductive framework in the reverse way \--- our method estimates data distribution of unseen classes in the image feature space by transferring knowledge from the label embedding space. In experiments, our method outperforms the state-of-the-art on two popular datasets.
Action Recognition with Dynamic Image Networks
Venue: arXiv cs.CV 2017
Year: 2017

We introduce the concept of "dynamic image", a novel compact representation of videos useful for video analysis, particularly in combination with convolutional neural networks (CNNs). A dynamic image encodes temporal data such as RGB or optical flow videos by using the concept of `rank pooling'. The idea is to learn a ranking machine that captures the temporal evolution of the data and to use the parameters of the latter as a representation. When a linear ranking machine is used, the resulting representation is in the form of an image, which we call dynamic because it summarizes the video dynamics in addition of appearance. This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos. We also present an efficient and effective approximate rank pooling operator, accelerating standard rank pooling algorithms by orders of magnitude, and formulate that as a CNN layer. This new layer allows generalizing dynamic images to dynamic feature maps. We demonstrate the power of the new representations on standard benchmarks in action recognition achieving state-of-the-art performance.
Estimating latent feature-feature interactions in large feature-rich graphs
Venue: arXiv cs.SI 2017
Year: 2017

Real-world complex networks describe connections between objects; in reality, those objects are often endowed with some kind of features. How does the presence or absence of such features interplay with the network link structure? Although the situation here described is truly ubiquitous, there is a limited body of research dealing with large graphs of this kind. Many previous works considered homophily as the only possible transmission mechanism translating node features into links. Other authors, instead, developed more sophisticated models, that are able to handle complex feature interactions, but are unfit to scale to very large networks. We expand on the MGJ model, where interactions between pairs of features can foster or discourage link formation. In this work, we will investigate how to estimate the latent feature-feature interactions in this model. We shall propose two solutions: the first one assumes feature independence and it is essentially based on Naive Bayes; the second one, which relaxes the independence assumption assumption, is based on perceptrons. In fact, we show it is possible to cast the model equation in order to see it as the prediction rule of a perceptron. We analyze how classical results for the perceptrons can be interpreted in this context; then, we define a fast and simple perceptron-like algorithm for this task, which can process $10^8$ links in minutes. We then compare these two techniques, first with synthetic datasets that follows our model, gaining evidence that the Naive independence assumptions are detrimental in practice. Secondly, we consider a real, large-scale citation network where each node (i.e., paper) can be described by different types of characteristics; there, our algorithm can assess how well each set of features can explain the links, and thus finding meaningful latent feature-feature interactions.
The Complexity of Bayesian Networks Specified by Propositional and Relational Languages
Venue: arXiv cs.AI 2017
Year: 2017

We examine the complexity of inference in Bayesian networks specified by logical languages. We consider representations that range from fragments of propositional logic to function-free first-order logic with equality; in doing so we cover a variety of plate models and of probabilistic relational models. We study the complexity of inferences when network, query and domain are the input (the inferential and the combined complexity), when the network is fixed and query and domain are the input (the query/data complexity), and when the network and query are fixed and the domain is the input (the domain complexity). We draw connections with probabilistic databases and liftability results, and obtain complexity classes that range from polynomial to exponential levels.
In-network Compression for Multiterminal Cascade MIMO Systems
Venue: arXiv cs.IT 2017
Year: 2017

We study the problem of receive beamforming in uplink cascade multiple-input multiple-output (MIMO) systems as an instance of that of cascade multiterminal source coding for lossy function computation. Using this connection, we develop two coding schemes for the second and show that their application leads to beamforming schemes for the first. In the first coding scheme, each terminal in the cascade sends a description of the source that it observes; the decoder reconstructs all sources, lossily, and then computes an estimate of the desired function. This scheme improves upon standard routing in that every terminal only compresses the innovation of its source w.r.t. the descriptions that are sent by the previous terminals in the cascade. In the second scheme, the desired function is computed gradually in the cascade network, and each terminal sends a finer description of it. In the context of uplink cascade MIMO systems, the application of these two schemes leads to centralized receive-beamforming and distributed receive-beamforming, respectively. Numerical results illustrate the performance of the proposed methods and show that they outperform standard routing.
Rate-Compatible Punctured Polar Codes: Optimal Construction Based on Polar Spectra
Venue: arXiv cs.IT 2017
Year: 2017

Polar codes are the first class of constructive channel codes achieving the symmetric capacity of the binary-input discrete memoryless channels. But the corresponding code length is limited to the power of two. In this paper, we establish a systematic framework to design the rate-compatible punctured polar (RCPP) codes with arbitrary code length. A new theoretic tool, called polar spectra, is proposed to count the number of paths on the code tree with the same number of zeros or ones respectively. Furthermore, a spectrum distance SD0 (SD1) and a joint spectrum distance (JSD) are presented as performance criteria to optimize the puncturing tables. For the capacity-zero puncturing mode (punctured bits are unknown to the decoder), we propose a quasi-uniform puncturing algorithm, analyze the number of equivalent puncturings and prove that this scheme can maximize SD1 and JSD. Similarly, for the capacity-one mode (punctured bits are known to the decoder), we also devise a reversal quasi-uniform puncturing scheme and prove that it has the maximum SD0 and JSD. Both schemes have a universal puncturing table without any exhausted search. These optimal RCPP codes outperform the performance of turbo codes in LTE wireless communication systems.
An Asymptotically Optimal Contextual Bandit Algorithm Using Hierarchical Structures
Venue: arXiv cs.LG 2017
Year: 2017

We propose online algorithms for sequential learning in the contextual multi-armed bandit setting. Our approach is to partition the context space and then optimally combine all of the possible mappings between the partition regions and the set of bandit arms in a data driven manner. We show that in our approach, the best mapping is able to approximate the best arm selection policy to any desired degree under mild Lipschitz conditions. Therefore, we design our algorithms based on the optimal adaptive combination and asymptotically achieve the performance of the best mapping as well as the best arm selection policy. This optimality is also guaranteed to hold even in adversarial environments since we do not rely on any statistical assumptions regarding the contexts or the loss of the bandit arms. Moreover, we design efficient implementations for our algorithms in various hierarchical partitioning structures such as lexicographical or arbitrary position splitting and binary trees (and several other partitioning examples). For instance, in the case of binary tree partitioning, the computational complexity is only log-linear in the number of regions in the finest partition. In conclusion, we provide significant performance improvements by introducing upper bounds (w.r.t. the best arm selection policy) that are mathematically proven to vanish in the average loss per round sense at a faster rate compared to the state-of-the-art. Our experimental work extensively covers various scenarios ranging from bandit settings to multi-class classification with real and synthetic data. In these experiments, we show that our algorithms are highly superior over the state-of-the-art techniques while maintaining the introduced mathematical guarantees and a computationally decent scalability.