OpenRaiser/Intern-Atlas huggingface dataset:
Understanding Deep Learning Generalization by Maximum Entropy
Venue: arXiv cs.LG 2017
Year: 2017

Deep learning achieves remarkable generalization capability with overwhelming number of model parameters. Theoretical understanding of deep learning generalization receives recent attention yet remains not fully explored. This paper attempts to provide an alternative understanding from the perspective of maximum entropy. We first derive two feature conditions that softmax regression strictly apply maximum entropy principle. DNN is then regarded as approximating the feature conditions with multilayer feature learning, and proved to be a recursive solution towards maximum entropy principle. The connection between DNN and maximum entropy well explains why typical designs such as shortcut and regularization improves model generalization, and provides instructions for future model development.
Hidden Tree Markov Networks: Deep and Wide Learning for Structured Data
Venue: arXiv cs.LG 2017
Year: 2017

The paper introduces the Hidden Tree Markov Network (HTN), a neuro-probabilistic hybrid fusing the representation power of generative models for trees with the incremental and discriminative learning capabilities of neural networks. We put forward a modular architecture in which multiple generative models of limited complexity are trained to learn structural feature detectors whose outputs are then combined and integrated by neural layers at a later stage. In this respect, the model is both deep, thanks to the unfolding of the generative models on the input structures, as well as wide, given the potentially large number of generative modules that can be trained in parallel. Experimental results show that the proposed approach can outperform state-of-the-art syntactic kernels as well as generative kernels built on the same probabilistic model as the HTN.
UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss
Venue: arXiv cs.CV 2017
Year: 2017

In the era of end-to-end deep learning, many advances in computer vision are driven by large amounts of labeled data. In the optical flow setting, however, obtaining dense per-pixel ground truth for real scenes is difficult and thus such data is rare. Therefore, recent end-to-end convolutional networks for optical flow rely on synthetic datasets for supervision, but the domain mismatch between training and test scenarios continues to be a challenge. Inspired by classical energy-based optical flow methods, we design an unsupervised loss based on occlusion-aware bidirectional flow estimation and the robust census transform to circumvent the need for ground truth flow. On the KITTI benchmarks, our unsupervised approach outperforms previous unsupervised deep networks by a large margin, and is even more accurate than similar supervised methods trained on synthetic datasets alone. By optionally fine-tuning on the KITTI training data, our method achieves competitive optical flow accuracy on the KITTI 2012 and 2015 benchmarks, thus in addition enabling generic pre-training of supervised networks for datasets with limited amounts of ground truth.
Training large margin host-pathogen protein-protein interaction predictors
Venue: arXiv cs.LG 2017
Year: 2017

Detection of protein-protein interactions (PPIs) plays a vital role in molecular biology. Particularly, infections are caused by the interactions of host and pathogen proteins. It is important to identify host-pathogen interactions (HPIs) to discover new drugs to counter infectious diseases. Conventional wet lab PPI prediction techniques have limitations in terms of large scale application and budget. Hence, computational approaches are developed to predict PPIs. This study aims to develop large margin machine learning models to predict interspecies PPIs with a special interest in host-pathogen protein interactions (HPIs). Especially, we focus on seeking answers to three queries that arise while developing an HPI predictor. 1) How should we select negative samples? 2) What should be the size of negative samples as compared to the positive samples? 3) What type of margin violation penalty should be used to train the predictor? We compare two available methods for negative sampling. Moreover, we propose a new method of assigning weights to each training example in weighted SVM depending on the distance of the negative examples from the positive examples. We have also developed a web server for our HPI predictor called HoPItor (Host Pathogen Interaction predicTOR) that can predict interactions between human and viral proteins. This webserver can be accessed at the URL: http://faculty.pieas.edu.pk/fayyaz/software.html#HoPItor.
Optimal Sleeping Mechanism for Multiple Servers with MMPP-Based Bursty Traffic Arrival
Venue: arXiv cs.IT 2017
Year: 2017

An important fundamental problem in green communications and networking is the operation of servers (routers or base stations) with sleeping mechanism to optimize energy-delay tradeoffs. This problem is very challenging when considering realistic bursty, non-Poisson traffic. We prove for the first time the optimal structure of such a sleep mechanism for multiple servers when the arrival of jobs is modeled by a bursty Markov-modulated Poisson process (MMPP). It is shown that the optimal operation, which determines the number of active (or sleeping) servers dynamically, is hysteretic and monotone, and hence it is a queue-threshold-based policy. This work settles a conjecture in the literature that the optimal sleeping mechanism for a single server with interrupted Poisson arrival process, which can be treated as a special case of MMPP, is queue-threshold-based. The exact thresholds are given by numerically solving the Markov decision process.
A Compositional Treatment of Iterated Open Games
Venue: arXiv cs.GT 2017
Year: 2017

Compositional Game Theory is a new, recently introduced model of economic games based upon the computer science idea of compositionality. In it, complex and irregular games can be built up from smaller and simpler games, and the equilibria of these complex games can be defined recursively from the equilibria of their simpler subgames. This paper extends the model by providing a final coalgebra semantics for infinite games. In the course of this, we introduce a new operator on games to model the economic concept of subgame perfection.
Finding Algebraic Structure of Care in Time: A Deep Learning Approach
Venue: arXiv cs.LG 2017
Year: 2017

Understanding the latent processes from Electronic Medical Records could be a game changer in modern healthcare. However, the processes are complex due to the interaction between at least three dynamic components: the illness, the care and the recording practice. Existing methods are inadequate in capturing the dynamic structure of care. We propose an end-to-end model that reads medical record and predicts future risk. The model adopts the algebraic view in that discrete medical objects are embedded into continuous vectors lying in the same space. The bag of disease and comorbidities recorded at each hospital visit are modeled as function of sets. The same holds for the bag of treatments. The interaction between diseases and treatments at a visit is modeled as the residual of the diseases minus the treatments. Finally, the health trajectory, which is a sequence of visits, is modeled using a recurrent neural network. We report preliminary results on chronic diseases - diabetes and mental health - for predicting unplanned readmission.
Toward Foraging for Understanding of StarCraft Agents: An Empirical Study
Venue: arXiv cs.HC 2017
Year: 2017

Assessing and understanding intelligent agents is a difficult task for users that lack an AI background. A relatively new area, called "Explainable AI," is emerging to help address this problem, but little is known about how users would forage through information an explanation system might offer. To inform the development of Explainable AI systems, we conducted a formative study, using the lens of Information Foraging Theory, into how experienced users foraged in the domain of StarCraft to assess an agent. Our results showed that participants faced difficult foraging problems. These foraging problems caused participants to entirely miss events that were important to them, reluctantly choose to ignore actions they did not want to ignore, and bear high cognitive, navigation, and information costs to access the information they needed.
Standards for enabling heterogeneous IaaS cloud federations
Venue: arXiv cs.NI 2017
Year: 2017

Technology market is continuing a rapid growth phase where different resource providers and Cloud Management Frameworks are positioning to provide ad-hoc solutions -in terms of management interfaces, information discovery or billing- trying to differentiate from competitors but that as a result remain incompatible between them when addressing more complex scenarios like federated clouds. Grasping interoperability problems present in current infrastructures is then a must-do, tackled by studying how existing and emerging standards could enhance user experience in the cloud ecosystem. In this paper we will review the current open challenges in Infrastructure as a Service cloud interoperability and federation, as well as point to the potential standards that should alleviate these problems.
Investigating the Best Radio Propagation Model for 4G - WiMAX Networks Deployment in 2530MHz Band in Sub- Saharan Africa
Venue: arXiv cs.NI 2017
Year: 2017

One of the salient factors to look at during wireless network planning is developing a modified path loss prediction models to suit a new environment other than the one it was originally designed for. This helps to give accurate predictive outcomes. This paper seeks to demonstrate the effects of applying correction factors on radio propagation model used in planning for 4G-WiMAX network through a comparative analysis between estimated and field data collected on received power for a 4G-WiMAX site. Four existing models were considered for this research; COST 231 Hata, Extended COST 231 Hata, SUI (Stanford University Interim) and Ericsson models. In order to optimize and validate the effectiveness of the proposed models, the mean square error (MSE) and correlation co-efficient were calculated for each model between the predicted and the measured received power for the selected area before and after applying an appropriate correction factor. Based on this, the Extended COST-231 Hata prediction model proved to correlate well with the measured values since it showed least Mean Square Error (MSE) but with highest correlation co-efficient. Through comparative analysis of the corrected models, the Extended COST-231 Hata model could be applied for effective planning of the radio systems in Ghana and the subregion at large.
Link Selection in Hybrid RF/VLC Systems under Statistical Queueing Constraints
Venue: arXiv cs.IT 2017
Year: 2017

The co-deployment of radio frequency (RF) and visible light communications (VLC) technologies has been investigated in indoor environments to enhance network performances and to address specific quality-of-service (QoS) constraints. In this paper, we explore the benefits of employing both technologies when the QoS requirements are imposed as limits on the buffer overflow and delay violation probabilities, which are important metrics in designing low latency wireless networks. Particularly, we consider a multi-mechanism scenario that utilizes RF and VLC links for data transmission in an indoor environment, and then propose a link selection process through which the transmitter sends data over the link that sustains the desired QoS guarantees the most. Considering an ON-OFF data source, we employ the maximum average data arrival rate at the transmitter buffer and the non-asymptotic bounds on data buffering delay as the main performance measures. We formulate the performance measures under the assumption that both links are subject to average and peak power constraints. Furthermore, we investigate the performance levels when either one of the two links is used for data transmission, or when both are used simultaneously. Finally, we show the impacts of different physical layer parameters on the system performance through numerical analysis.
Improving local clustering based top-L link prediction methods via asymmetrical link clustering information
Venue: arXiv cs.SI 2017
Year: 2017

Networks can represent a wide range of complex systems, such as social, biological and technological systems. Link prediction is one of the most important problems in network analysis, and has attracted much research interest recently. Many link prediction methods have been proposed to solve this problem with various technics. We can note that clustering information plays an important role in solving the link prediction problem. In previous literatures, we find node clustering coefficient appears frequently in many link prediction methods. However, node clustering coefficient is limited to describe the role of a common-neighbor in different local networks, because it can not distinguish different clustering abilities of a node to different node pairs. In this paper, we shift our focus from nodes to links, and propose the concept of asymmetric link clustering (ALC) coefficient. Further, we improve three node clustering based link prediction methods via the concept of ALC. The experimental results demonstrate that ALC-based methods outperform node clustering based methods, especially achieving remarkable improvements on food web, hamster friendship and Internet networks. Besides, comparing with other methods, the performance of ALC-based methods are very stable in both globalized and personalized top-L link prediction tasks.
Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
Venue: arXiv cs.CV 2017
Year: 2017

Automatic diagnosing lung cancer from Computed Tomography (CT) scans involves two steps: detect all suspicious lesions (pulmonary nodules) and evaluate the whole-lung/pulmonary malignancy. Currently, there are many studies about the first step, but few about the second step. Since the existence of nodule does not definitely indicate cancer, and the morphology of nodule has a complicated relationship with cancer, the diagnosis of lung cancer demands careful investigations on every suspicious nodule and integration of information of all nodules. We propose a 3D deep neural network to solve this problem. The model consists of two modules. The first one is a 3D region proposal network for nodule detection, which outputs all suspicious nodules for a subject. The second one selects the top five nodules based on the detection confidence, evaluates their cancer probabilities and combines them with a leaky noisy-or gate to obtain the probability of lung cancer for the subject. The two modules share the same backbone network, a modified U-net. The over-fitting caused by the shortage of training data is alleviated by training the two modules alternately. The proposed model won the first place in the Data Science Bowl 2017 competition. The code has been made publicly available.
Adaptive Cardinality Estimation
Venue: arXiv cs.DB 2017
Year: 2017

In this paper we address cardinality estimation problem which is an important subproblem in query optimization. Query optimization is a part of every relational DBMS responsible for finding the best way of the execution for the given query. These ways are called plans. The execution time of different plans may differ by several orders, so query optimizer has a great influence on the whole DBMS performance. We consider cost-based query optimization approach as the most popular one. It was observed that cost-based optimization quality depends much on cardinality estimation quality. Cardinality of the plan node is the number of tuples returned by it. In the paper we propose a novel cardinality estimation approach with the use of machine learning methods. The main point of the approach is using query execution statistics of the previously executed queries to improve cardinality estimations. We called this approach adaptive cardinality estimation to reflect this point. The approach is general, flexible, and easy to implement. The experimental evaluation shows that this approach significantly increases the quality of cardinality estimation, and therefore increases the DBMS performance for some queries by several times or even by several dozens of times.
Budget Allocation in Binary Opinion Dynamics
Venue: arXiv cs.GT 2017
Year: 2017

In this article we study the allocation of a budget to promote an opinion in a group of agents. We assume that their opinion dynamics are based on the well-known voter model. We are interested in finding the most efficient use of a budget over time in order to manipulate a social network. We address the problem using the theory of discounted Markov decision processes. Our contributions can be summarized as follows: (i) we introduce the discounted Markov decision process in our cases, (ii) we present the corresponding Bellman equations, and, (iii) we solve the Bellman equations via backward programming. This work is a step towards providing a solid formulation of the budget allocation in social networks.
Trailing the Snail: SDN Controller Security Evolution
Venue: arXiv cs.NI 2017
Year: 2017

The first OpenFlow Software-Defined Network (SDN) Controller, NOX, was developed by Nicira Networks and donated to the research community in 2008. Almost 10 years later, there are at least 29 open-source SDN Controllers and many more proprietary solutions. Two of the open-source SDN controllers stand out in terms of broad deployment and strong contributor base; Open Network Operating System (ONOS) and OpenDaylight (ODL). Both have been deployed in live networks. However, despite increasing adoption of SDN, the security of the SDN control plane has developed at a snail's pace. In this paper, the evolution of ONOS and ODL security is discussed. The reflection of this on secure SDN Controller design is analyzed.
Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
Venue: arXiv cs.CL 2017
Year: 2017

Word embeddings use vectors to represent words such that the geometry between vectors captures semantic relationship between the words. In this paper, we develop a framework to demonstrate how the temporal dynamics of the embedding can be leveraged to quantify changes in stereotypes and attitudes toward women and ethnic minorities in the 20th and 21st centuries in the United States. We integrate word embeddings trained on 100 years of text data with the U.S. Census to show that changes in the embedding track closely with demographic and occupation shifts over time. The embedding captures global social shifts -- e.g., the women's movement in the 1960s and Asian immigration into the U.S -- and also illuminates how specific adjectives and occupations became more closely associated with certain populations over time. Our framework for temporal analysis of word embedding opens up a powerful new intersection between machine learning and quantitative social science.
SolarisNet: A Deep Regression Network for Solar Radiation Prediction
Venue: arXiv cs.CV 2017
Year: 2017

Effective utilization of photovoltaic (PV) plants requires weather variability robust global solar radiation (GSR) forecasting models. Random weather turbulence phenomena coupled with assumptions of clear sky model as suggested by Hottel pose significant challenges to parametric & non-parametric models in GSR conversion rate estimation. Also, a decent GSR estimate requires costly high-tech radiometer and expert dependent instrument handling and measurements, which are subjective. As such, a computer aided monitoring (CAM) system to evaluate PV plant operation feasibility by employing smart grid past data analytics and deep learning is developed. Our algorithm, SolarisNet is a 6-layer deep neural network trained on data collected at two weather stations located near Kalyani metrological site, West Bengal, India. The daily GSR prediction performance using SolarisNet outperforms the existing state of art and its efficacy in inferring past GSR data insights to comprehend daily and seasonal GSR variability along with its competence for short term forecasting is discussed.
A Reliability Study of Parallelized VNF Chaining
Venue: arXiv cs.NI 2017
Year: 2017

In this paper, we study end-to-end service reliability in Data Center Networks (DCN) with flow and Service Function Chains (SFCs) parallelism. In our approach, we consider large flows to i) be split into multiple parallel smaller sub-flows; ii) SFC along with their VNFs are replicated into at least as many VNF instances as there are sub-flows, resulting in parallel sub-SFCs; and iii) all sub-flows are distributed over multiple shortest paths and processed in parallel by parallel sub-SFCs. We study service reliability as a function of flow and SFC parallelism and placement of parallel active and backup sub-SFCs within DCN. Based on the probability theory and by considering both server and VNF failures, we analytically derive for each studied VNF placement method the probability that all sub-flows can be successfully processed by the parallelized SFC without service interruption. We evaluate the amount of backup VNFs required to protect the parallelized SFC with a certain level of service reliability. The results show that the proposed flow and SFC parallelism in DCN can significantly increase end-to-end service reliability, while reducing the amount of backup VNFs required, as compared to traditional SFCs with serial traffic flows.
MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
Venue: arXiv cs.LG 2017
Year: 2017

MagNet and "Efficient Defenses..." were recently proposed as a defense to adversarial examples. We find that we can construct adversarial examples that defeat these defenses with only a slight increase in distortion.
W-Net: A Deep Model for Fully Unsupervised Image Segmentation
Venue: arXiv cs.CV 2017
Year: 2017

While significant attention has been recently focused on designing supervised deep semantic segmentation algorithms for vision tasks, there are many domains in which sufficient supervised pixel-level labels are difficult to obtain. In this paper, we revisit the problem of purely unsupervised image segmentation and propose a novel deep architecture for this problem. We borrow recent ideas from supervised semantic segmentation methods, in particular by concatenating two fully convolutional networks together into an autoencoder--one for encoding and one for decoding. The encoding layer produces a k-way pixelwise prediction, and both the reconstruction error of the autoencoder as well as the normalized cut produced by the encoder are jointly minimized during training. When combined with suitable postprocessing involving conditional random field smoothing and hierarchical segmentation, our resulting algorithm achieves impressive results on the benchmark Berkeley Segmentation Data Set, outperforming a number of competing methods.
$k$-NN Estimation of Directed Information
Venue: arXiv cs.IT 2017
Year: 2017

This report studies data-driven estimation of the directed information (DI) measure between two{em discrete-time and continuous-amplitude} random process, based on the $k$-nearest-neighbors ($k$-NN) estimation framework. Detailed derivations of two $k$-NN estimators are provided. The two estimators differ in the metric based on which the nearest-neighbors are found. To facilitate the estimation of the DI measure, it is assumed that the observed sequences are (jointly) Markovian of order $m$. As $m$ is generally not known, a data-driven method (that is also based on the $k$-NN principle) for estimating $m$ from the observed sequences is presented. An exhaustive numerical study shows that the discussed $k$-NN estimators perform well even for relatively small number of samples (few thousands). Moreover, it is shown that the discussed estimators are capable of accurately detecting linear as well as non-linear causal interactions.