OpenRaiser/Intern-Atlas huggingface dataset:
Geometric Operator Convolutional Neural Network
Venue: arXiv cs.CV 2018
Year: 2018

The Convolutional Neural Network (CNN) has been successfully applied in many fields during recent decades; however it lacks the ability to utilize prior domain knowledge when dealing with many realistic problems. We present a framework called Geometric Operator Convolutional Neural Network (GO-CNN) that uses domain knowledge, wherein the kernel of the first convolutional layer is replaced with a kernel generated by a geometric operator function. This framework integrates many conventional geometric operators, which allows it to adapt to a diverse range of problems. Under certain conditions, we theoretically analyze the convergence and the bound of the generalization errors between GO-CNNs and common CNNs. Although the geometric operator convolution kernels have fewer trainable parameters than common convolution kernels, the experimental results indicate that GO-CNN performs more accurately than common CNN on CIFAR-10/100. Furthermore, GO-CNN reduces dependence on the amount of training examples and enhances adversarial stability. In the practical task of medically diagnosing bone fractures, GO-CNN obtains 3% improvement in terms of the recall.
Iris recognition in cases of eye pathology
Venue: arXiv cs.CV 2018
Year: 2018

This chapter provides insight on how iris recognition, one of the leading biometric identification technologies in the world, can be impacted by pathologies and illnesses present in the eye, what are the possible repercussions of this influence, and what are the possible means for taking such effects into account when matching iris samples. To make this study possible, a special database of iris images has been used, representing more than 20 different medical conditions of the ocular region (including cataract, glaucoma, rubeosis iridis, synechiae, iris defects, corneal pathologies and other) and containing almost 3000 samples collected from 230 distinct irises. Then, with the use of four different iris recognition methods, a series of experiments has been conducted, concluding in several important observations. One of the most popular ocular disorders worldwide - the cataract - is shown to worsen genuine comparison scores when results obtained from cataract-affected eyes are compared to those coming from healthy irises. An analysis devoted to different types of impact on eye structures caused by diseases is also carried out with significant results. The enrollment process is highly sensitive to those eye conditions that make the iris obstructed or introduce geometrical distortions. Disorders affecting iris geometry, or producing obstructions are exceptionally capable of degrading the genuine comparison scores, so that the performance of the entire biometric system can be influenced. Experiments also reveal that imperfect execution of the image segmentation stage is the most prominent contributor to recognition errors.
The Effect of Context on Metaphor Paraphrase Aptness Judgments
Venue: arXiv cs.CL 2018
Year: 2018

We conduct two experiments to study the effect of context on metaphor paraphrase aptness judgments. The first is an AMT crowd source task in which speakers rank metaphor paraphrase candidate sentence pairs in short document contexts for paraphrase aptness. In the second we train a composite DNN to predict these human judgments, first in binary classifier mode, and then as gradient ratings. We found that for both mean human judgments and our DNN's predictions, adding document context compresses the aptness scores towards the center of the scale, raising low out of context ratings and decreasing high out of context scores. We offer a provisional explanation for this compression effect.
Optimal Steerable mmWave Mesh Backhaul Reconfiguration
Venue: arXiv cs.NI 2018
Year: 2018

Future 5G mobile networks will require increased backhaul (BH) capacity to connect a massive amount of high capacity small cells (SCs) to the network. Because having an optical connection to each SC might be infeasible, mmWave-based (e.g. 60 GHz) BH links are an interesting alternative due to their large available bandwidth. To cope with the increased path loss, mmWave links require directional antennas that should be able to direct their beams to different neighbors, to dynamically change the BH topology, in case new nodes are powered on/off or the traffic demand has changed. Such BH adaptation needs to be orchestrated to minimize the impact on existing traffic. This paper develops a Software-defined networking-based framework that guides the optimal reconfiguration of mesh BH networks composed by mmWave links, where antennas need to be mechanically aligned. By modelling the problem as a Mixed Integer Linear Program (MILP), its solution returns the optimal ordering of events necessary to transition between two BH network configurations. The model creates backup paths whenever it is possible, while minimizing the packet loss of ongoing flows. A numerical evaluation with different topologies and traffic demands shows that increasing the number of BH interfaces per SC from 2 to 4 can decrease the total loss by more than 50%. Moreover, when increasing the total reconfiguration time, additional backup paths can be created, consequently reducing the reconfiguration impact on existing traffic.
Compositional Stochastic Average Gradient for Machine Learning and Related Applications
Venue: arXiv cs.LG 2018
Year: 2018

Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compositional optimization problems (FS-CEVF). Compositional stochastic variance reduced gradient (C-SVRG) methods that combine stochastic compositional gradient descent (SCGD) and stochastic variance reduced gradient descent (SVRG) methods are the state-of-the-art methods for FS-CEVF problems. We introduce compositional stochastic average gradient descent (C-SAG) a novel extension of the stochastic average gradient method (SAG) to minimize composition of finite-sum functions. C-SAG, like SAG, estimates gradient by incorporating memory of previous gradient information. We present theoretical analyses of C-SAG which show that C-SAG, like SAG, and C-SVRG, achieves a linear convergence rate when the objective function is strongly convex; However, C-CAG achieves lower oracle query complexity per iteration than C-SVRG. Finally, we present results of experiments showing that C-SAG converges substantially faster than full gradient (FG), as well as C-SVRG.
t-Exponential Memory Networks for Question-Answering Machines
Venue: arXiv cs.LG 2018
Year: 2018

Recent advances in deep learning have brought to the fore models that can make multiple computational steps in the service of completing a task; these are capable of describ- ing long-term dependencies in sequential data. Novel recurrent attention models over possibly large external memory modules constitute the core mechanisms that enable these capabilities. Our work addresses learning subtler and more complex underlying temporal dynamics in language modeling tasks that deal with sparse sequential data. To this end, we improve upon these recent advances, by adopting concepts from the field of Bayesian statistics, namely variational inference. Our proposed approach consists in treating the network parameters as latent variables with a prior distribution imposed over them. Our statistical assumptions go beyond the standard practice of postulating Gaussian priors. Indeed, to allow for handling outliers, which are prevalent in long observed sequences of multivariate data, multivariate t-exponential distributions are imposed. On this basis, we proceed to infer corresponding posteriors; these can be used for inference and prediction at test time, in a way that accounts for the uncertainty in the available sparse training data. Specifically, to allow for our approach to best exploit the merits of the t-exponential family, our method considers a new t-divergence measure, which generalizes the concept of the Kullback-Leibler divergence. We perform an extensive experimental evaluation of our approach, using challenging language modeling benchmarks, and illustrate its superiority over existing state-of-the-art techniques.
Deep Priority Hashing
Venue: arXiv cs.CV 2018
Year: 2018

Deep hashing enables image retrieval by end-to-end learning of deep representations and hash codes from training data with pairwise similarity information. Subject to the distribution skewness underlying the similarity information, most existing deep hashing methods may underperform for imbalanced data due to misspecified loss functions. This paper presents Deep Priority Hashing (DPH), an end-to-end architecture that generates compact and balanced hash codes in a Bayesian learning framework. The main idea is to reshape the standard cross-entropy loss for similarity-preserving learning such that it down-weighs the loss associated to highly-confident pairs. This idea leads to a novel priority cross-entropy loss, which prioritizes the training on uncertain pairs over confident pairs. Also, we propose another priority quantization loss, which prioritizes hard-to-quantize examples for generation of nearly lossless hash codes. Extensive experiments demonstrate that DPH can generate high-quality hash codes and yield state-of-the-art image retrieval results on three datasets, ImageNet, NUS-WIDE, and MS-COCO.
Collaborative Artificial Intelligence (AI) for User-Cell association in Ultra-Dense Cellular Systems
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the problem of cell association between small base stations (SBSs) and users in dense wireless networks is studied using artificial intelligence (AI) techniques. The problem is formulated as a mean-field game in which the users' goal is to maximize their data rate by exploiting local data and the data available at neighboring users via an imitation process. Such a collaborative learning process prevents the users from exchanging their data directly via the cellular network's limited backhaul links and, thus, allows them to improve their cell association policy collaboratively with minimum computing. To solve this problem, a neural Q-learning learning algorithm is proposed that enables the users to predict their reward function using a neural network whose input is the SBSs selected by neighboring users and the local data of the considered user. Simulation results show that the proposed imitation-based mechanism for cell association converges faster to the optimal solution, compared with conventional cell association mechanisms without imitation.
Joint Trajectory and Resource Allocation Design for UAV Communication Systems
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, we investigate resource allocation design for unmanned aerial vehicle (UAV)-enabled communication systems, where a UAV is dispatched to provide communications to multiple user nodes. Our objective is to maximize the communication system throughput by jointly optimizing the subcarrier allocation policy and the trajectory of the UAV, while taking into account the minimum required data rate for each user node, no-fly zones (NFZs), the maximum UAV cruising speed, and initial/final UAV locations. The design is formulated as a mixed integer non-convex optimization problem which is generally intractable. Subsequently, a computationally-efficient iterative algorithm is proposed to obtain a locally optimal solution. Simulation results illustrate that the performance of the proposed iterative algorithm approaches closely to that of the system without NFZ. In addition, the proposed algorithm can achieve a significant throughput gain compared to various benchmark schemes.
Retinal Vessel Segmentation under Extreme Low Annotation: A Generative Adversarial Network Approach
Venue: arXiv cs.CV 2018
Year: 2018

Contemporary deep learning based medical image segmentation algorithms require hours of annotation labor by domain experts. These data hungry deep models perform sub-optimally in the presence of limited amount of labeled data. In this paper, we present a data efficient learning framework using the recent concept of Generative Adversarial Networks; this allows a deep neural network to perform significantly better than its fully supervised counterpart in low annotation regime. The proposed method is an extension of our previous work with the addition of a new unsupervised adversarial loss and a structured prediction based architecture. To the best of our knowledge, this work is the first demonstration of an adversarial framework based structured prediction model for medical image segmentation. Though generic, we apply our method for segmentation of blood vessels in retinal fundus images. We experiment with extreme low annotation budget (0.8 - 1.6% of contemporary annotation size). On DRIVE and STARE datasets, the proposed method outperforms our previous method and other fully supervised benchmark models by significant margins especially with very low number of annotated examples. In addition, our systematic ablation studies suggest some key recipes for successfully training GAN based semi-supervised algorithms with an encoder-decoder style network architecture.
Towards quantitative methods to assess network generative models
Venue: arXiv cs.LG 2018
Year: 2018

Assessing generative models is not an easy task. Generative models should synthesize graphs which are not replicates of real networks but show topological features similar to real graphs. We introduce an approach for assessing graph generative models using graph classifiers. The inability of an established graph classifier for distinguishing real and synthesized graphs could be considered as a performance measurement for graph generators.
Modified Diversity of Class Probability Estimation Co-training for Hyperspectral Image Classification
Venue: arXiv cs.CV 2018
Year: 2018

Due to the limited amount and imbalanced classes of labeled training data, the conventional supervised learning can not ensure the discrimination of the learned feature for hyperspectral image (HSI) classification. In this paper, we propose a modified diversity of class probability estimation (MDCPE) with two deep neural networks to learn spectral-spatial feature for HSI classification. In co-training phase, recurrent neural network (RNN) and convolutional neural network (CNN) are utilized as two learners to extract features from labeled and unlabeled data. Based on the extracted features, MDCPE selects most credible samples to update initial labeled data by combining k-means clustering with the traditional diversity of class probability estimation (DCPE) co-training. In this way, MDCPE can keep new labeled data class-balanced and extract discriminative features for both the minority and majority classes. During testing process, classification results are acquired by co-decision of the two learners. Experimental results demonstrate that the proposed semi-supervised co-training method can make full use of unlabeled information to enhance generality of the learners and achieve favorable accuracies on all three widely used data sets: Salinas, Pavia University and Pavia Center.
How is Contrast Encoded in Deep Neural Networks?
Venue: arXiv cs.CV 2018
Year: 2018

Contrast is a crucial factor in visual information processing. It is desired for a visual system - irrespective of being biological or artificial - to "perceive" the world robustly under large potential changes in illumination. In this work, we studied the responses of deep neural networks (DNN) to identical images at different levels of contrast. We analysed the activation of kernels in the convolutional layers of eight prominent networks with distinct architectures (e.g. VGG and Inception). The results of our experiments indicate that those networks with a higher tolerance to alteration of contrast have more than one convolutional layer prior to the first max-pooling operator. It appears that the last convolutional layer before the first max-pooling acts as a mitigator of contrast variation in input images. In our investigation, interestingly, we observed many similarities between the mechanisms of these DNNs and biological visual systems. These comparisons allow us to understand more profoundly the underlying mechanisms of a visual system that is grounded on the basis of "data-analysis".
Chest X-ray Inpainting with Deep Generative Models
Venue: arXiv cs.GR 2018
Year: 2018

Generative adversarial networks have been successfully applied to inpainting in natural images. However, the current state-of-the-art models have not yet been widely adopted in the medical imaging domain. In this paper, we investigate the performance of three recently published deep learning based inpainting models: context encoders, semantic image inpainting, and the contextual attention model, applied to chest x-rays, as the chest exam is the most commonly performed radiological procedure. We train these generative models on 1.2M 128 $\times$ 128 patches from 60K healthy x-rays, and learn to predict the center 64 $\times$ 64 region in each patch. We test the models on both the healthy and abnormal radiographs. We evaluate the results by visual inspection and comparing the PSNR scores. The outputs of the models are in most cases highly realistic. We show that the methods have potential to enhance and detect abnormalities. In addition, we perform a 2AFC observer study and show that an experienced human observer performs poorly in detecting inpainted regions, particularly those generated by the contextual attention model.
Geodesic farthest-point Voronoi diagram in linear time
Venue: arXiv cs.CG 2018
Year: 2018

Let $P$ be a simple polygon with $n$ vertices. For any two points in $P$, the geodesic distance between them is the length of the shortest path that connects them among all paths contained in $P$. Given a set $S$ of $m$ sites being a subset of the vertices of $P$, we present a randomized algorithm to compute the geodesic farthest-point Voronoi diagram of $S$ in $P$ running in expected $O(n + m)$ time. That is, a partition of $P$ into cells, at most one cell per site, such that every point in a cell has the same farthest site with respect to the geodesic distance. In particular, this algorithm can be extended to run in expected $O(n + m\log m)$ time when $S$ is an arbitrary set of $m$ sites contained in $P$, thereby solving the open problem posed by Mitchell in Chapter 27 of the Handbook of Computational Geometry.
CNNs-based Acoustic Scene Classification using Multi-Spectrogram Fusion and Label Expansions
Venue: arXiv cs.CV 2018
Year: 2018

Spectrograms have been widely used in Convolutional Neural Networks based schemes for acoustic scene classification, such as the STFT spectrogram and the MFCC spectrogram, etc. They have different time-frequency characteristics, contributing to their own advantages and disadvantages in recognizing acoustic scenes. In this letter, a novel multi-spectrogram fusion framework is proposed, making the spectrograms complement each other. In the framework, a single CNN architecture is applied onto multiple spectrograms for feature extraction. The deep features extracted from multiple spectrograms are then fused to discriminate the acoustic scenes. Moreover, motivated by the inter-class similarities in acoustic scene datasets, a label expansion method is further proposed in which super-class labels are constructed upon the original classes. On the help of the expanded labels, the CNN models are transformed into the multitask learning form to improve the acoustic scene classification by appending the auxiliary task of super-class classification. To verify the effectiveness of the proposed methods, intensive experiments have been performed on the DCASE2017 and the LITIS Rouen datasets. Experimental results show that the proposed method can achieve promising accuracies on both datasets. Specifically, accuracies of 0.9744, 0.8865 and 0.7778 are obtained for the LITIS Rouen dataset, the DCASE Development set and Evaluation set respectively.
Deep Depth from Defocus: how can defocus blur improve 3D estimation using dense neural networks?
Venue: arXiv cs.CV 2018
Year: 2018

Depth estimation is of critical interest for scene understanding and accurate 3D reconstruction. Most recent approaches in depth estimation with deep learning exploit geometrical structures of standard sharp images to predict corresponding depth maps. However, cameras can also produce images with defocus blur depending on the depth of the objects and camera settings. Hence, these features may represent an important hint for learning to predict depth. In this paper, we propose a full system for single-image depth prediction in the wild using depth-from-defocus and neural networks. We carry out thorough experiments to test deep convolutional networks on real and simulated defocused images using a realistic model of blur variation with respect to depth. We also investigate the influence of blur on depth prediction observing model uncertainty with a Bayesian neural network approach. From these studies, we show that out-of-focus blur greatly improves the depth-prediction network performances. Furthermore, we transfer the ability learned on a synthetic, indoor dataset to real, indoor and outdoor images. For this purpose, we present a new dataset containing real all-focus and defocused images from a Digital Single-Lens Reflex (DSLR) camera, paired with ground truth depth maps obtained with an active 3D sensor for indoor scenes. The proposed approach is successfully validated on both this new dataset and standard ones as NYUv2 or Depth-in-the-Wild. Code and new datasets are available at https://github.com/marcelampc/d3net_depth_estimation
Blockmania: from Block DAGs to Consensus
Venue: arXiv cs.CR 2018
Year: 2018

Blockmania is a byzantine consensus protocol. Nodes emit blocks forming a directed acyclic graph (block DAG) that is subsequently interpreted by each node separately to ensure consensus with safety, liveness and finality. The resulting system has communication complexity $O(N^2)$ even in the worse case, and very low constant factors --- as compared to $O(N^4)$ for PBFT; it is leaderless; and network operations do not depend on the composition of the quorum or node stake. This makes Blockmania very efficient (leading to over 400K transactions per second on a wide area network), and ideal for dynamic membership and flexible and non-interrupted proof-of-stake protocols. A X-Blockmania variant, has $O(N)$ communication cost but also higher latency $O(\log N)$.
A Differentially Private Wilcoxon Signed-Rank Test
Venue: arXiv cs.CR 2018
Year: 2018

Hypothesis tests are a crucial statistical tool for data mining and are the workhorse of scientific research in many fields. Here we present a differentially private analogue of the classic Wilcoxon signed-rank hypothesis test, which is used when comparing sets of paired (e.g., before-and-after) data values. We present not only a private estimate of the test statistic, but a method to accurately compute a p-value and assess statistical significance. We evaluate our test on both simulated and real data. Compared to the only existing private test for this situation, that of Task and Clifton, we find that our test requires less than half as much data to achieve the same statistical power.
Web Based Information System for Heat Supply Monitoring
Venue: arXiv cs.OH 2018
Year: 2018

The paper presents web based information system for heat supply monitoring. The proposed model and information system for gathering data from heating station heat-flow meters and regulators is software realized. The novel system with proved functionality can be commercialized at the cost of minimal investments, finding wildly use on Bulgarian market as cheap and quality alternative of the western systems.