OpenRaiser/Intern-Atlas huggingface dataset:
Distribution Matching Losses Can Hallucinate Features in Medical Image Translation
Venue: arXiv cs.CV 2018
Year: 2018

This paper discusses how distribution matching losses, such as those used in CycleGAN, when used to synthesize medical images can lead to mis-diagnosis of medical conditions. It seems appealing to use these new image synthesis methods for translating images from a source to a target domain because they can produce high quality images and some even do not require paired data. However, the basis of how these image translation models work is through matching the translation output to the distribution of the target domain. This can cause an issue when the data provided in the target domain has an over or under representation of some classes (e.g. healthy or sick). When the output of an algorithm is a transformed image there are uncertainties whether all known and unknown class labels have been preserved or changed. Therefore, we recommend that these translated images should not be used for direct interpretation (e.g. by doctors) because they may lead to misdiagnosis of patients based on hallucinated image features by an algorithm that matches a distribution. However there are many recent papers that seem as though this is the goal.
Community Detection with Side Information: Exact Recovery under the Stochastic Block Model
Venue: arXiv cs.IT 2018
Year: 2018

The community detection problem involves making inferences about node labels in a graph, based on observing the graph edges. This paper studies the effect of additional, non-graphical side information on the phase transition of exact recovery in the binary stochastic block model (SBM) with $n$ nodes. When side information consists of noisy labels with error probability $α$, it is shown that phase transition is improved if and only if $\log(\frac{1-α}α)=Ω(\log(n))$. When side information consists of revealing a fraction $1-ε$ of the labels, it is shown that phase transition is improved if and only if $\log(1/ε)=Ω(\log(n))$. For a more general side information consisting of $K$ features, two scenarios are studied: (1)~$K$ is fixed while the likelihood of each feature with respect to corresponding node label evolves with $n$, and (2)~The number of features $K$ varies with $n$ but the likelihood of each feature is fixed. In each case, we find when side information improves the exact recovery phase transition and by how much. The calculated necessary and sufficient conditions for exact recovery are tight except for one special case. In the process of deriving inner bounds, a variation of an efficient algorithm is proposed for community detection with side information that uses a partial recovery algorithm combined with a local improvement procedure.
Enhancing Chinese Intent Classification by Dynamically Integrating Character Features into Word Embeddings with Ensemble Techniques
Venue: arXiv cs.CL 2018
Year: 2018

Intent classification has been widely researched on English data with deep learning approaches that are based on neural networks and word embeddings. The challenge for Chinese intent classification stems from the fact that, unlike English where most words are made up of 26 phonologic alphabet letters, Chinese is logographic, where a Chinese character is a more basic semantic unit that can be informative and its meaning does not vary too much in contexts. Chinese word embeddings alone can be inadequate for representing words, and pre-trained embeddings can suffer from not aligning well with the task at hand. To account for the inadequacy and leverage Chinese character information, we propose a low-effort and generic way to dynamically integrate character embedding based feature maps with word embedding based inputs, whose resulting word-character embeddings are stacked with a contextual information extraction module to further incorporate context information for predictions. On top of the proposed model, we employ an ensemble method to combine single models and obtain the final result. The approach is data-independent without relying on external sources like pre-trained word embeddings. The proposed model outperforms baseline models and existing methods.
Determining the Number of Samples Required to Estimate Entropy in Natural Sequences
Venue: arXiv cs.IT 2018
Year: 2018

Calculating the Shannon entropy for symbolic sequences has been widely considered in many fields. For descriptive statistical problems such as estimating the N-gram entropy of English language text, a common approach is to use as much data as possible to obtain progressively more accurate estimates. However in some instances, only short sequences may be available. This gives rise to the question of how many samples are needed to compute entropy. In this paper, we examine this problem and propose a method for estimating the number of samples required to compute Shannon entropy for a set of ranked symbolic natural events. The result is developed using a modified Zipf-Mandelbrot law and the Dvoretzky-Kiefer-Wolfowitz inequality, and we propose an algorithm which yields an estimate for the minimum number of samples required to obtain an estimate of entropy with a given confidence level and degree of accuracy.
Approximate Random Dropout
Venue: arXiv cs.LG 2018
Year: 2018

The training phases of Deep neural network~(DNN) consumes enormous processing time and energy. Compression techniques utilizing the sparsity of DNNs can effectively accelerate the inference phase of DNNs. However, it can be hardly used in the training phase because the training phase involves dense matrix-multiplication using General Purpose Computation on Graphics Processors (GPGPU), which endorse regular and structural data layout. In this paper, we propose the Approximate Random Dropout that replaces the conventional random dropout of neurons and synapses with a regular and predefined patterns to eliminate the unnecessary computation and data access. To compensate the potential performance loss we develop a SGD-based Search Algorithm to produce the distribution of dropout patterns. We prove our approach is statistically equivalent to the previous dropout method. Experiments results on MLP and LSTM using well-known benchmarks show that the proposed Approximate Random Dropout can reduce the training time by $20\%$-$77\%$ ($19\%$-$60\%$) when dropout rate is $0.3$-$0.7$ on MLP (LSTM) with marginal accuracy drop.
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow
Venue: arXiv cs.CL 2018
Year: 2018

For tasks like code synthesis from natural language, code retrieval, and code summarization, data-driven models have shown great promise. However, creating these models require parallel data between natural language (NL) and code with fine-grained alignments. Stack Overflow (SO) is a promising source to create such a data set: the questions are diverse and most of them have corresponding answers with high-quality code snippets. However, existing heuristic methods (e.g., pairing the title of a post with the code in the accepted answer) are limited both in their coverage and the correctness of the NL-code pairs obtained. In this paper, we propose a novel method to mine high-quality aligned data from SO using two sets of features: hand-crafted features considering the structure of the extracted snippets, and correspondence features obtained by training a probabilistic model to capture the correlation between NL and code using neural networks. These features are fed into a classifier that determines the quality of mined NL-code pairs. Experiments using Python and Java as test beds show that the proposed method greatly expands coverage and accuracy over existing mining methods, even when using only a small number of labeled examples. Further, we find that reasonable results are achieved even when training the classifier on one language and testing on another, showing promise for scaling NL-code mining to a wide variety of programming languages beyond those for which we are able to annotate data.
Maximal and maximum transitive relation contained in a given binary relation
Venue: arXiv cs.DS 2018
Year: 2018

We study the problem of finding a \textit{maximal} transitive relation contained in a given binary relation. Given a binary relation of size $m$ defined on a set of size $n$, we present a polynomial time algorithm that finds a maximal transitive sub-relation in time $O(n^2 + nm)$. We also study the problem of finding a \textit{maximum} transitive relation contained in a binary relation. This is the problem of computing a maximum transitive subgraph in a given digraph. For the class of directed graphs with the underlying graph being triangle-free, we present a $0.874$-approximation algorithm. This is achieved via a simple connection to the problem of maximum directed cut. Further, we give an upper bound for the size of any maximum transitive relation to be $m/4 + cm^{4/5}$, where $c > 0$ and $m$ is the number of edges in the digraph.
3D Human Pose Estimation with Relational Networks
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose a novel 3D human pose estimation algorithm from a single image based on neural networks. We adopted the structure of the relational networks in order to capture the relations among different body parts. In our method, each pair of different body parts generates features, and the average of the features from all the pairs are used for 3D pose estimation. In addition, we propose a dropout method that can be used in relational modules, which inherently imposes robustness to the occlusions. The proposed network achieves state-of-the-art performance for 3D pose estimation in Human 3.6M dataset, and it effectively produces plausible results even in the existence of missing joints.
Discovering Blind Spots in Reinforcement Learning
Venue: arXiv cs.LG 2018
Year: 2018

Agents trained in simulation may make errors in the real world due to mismatches between training and execution environments. These mistakes can be dangerous and difficult to discover because the agent cannot predict them a priori. We propose using oracle feedback to learn a predictive model of these blind spots to reduce costly errors in real-world applications. We focus on blind spots in reinforcement learning (RL) that occur due to incomplete state representation: The agent does not have the appropriate features to represent the true state of the world and thus cannot distinguish among numerous states. We formalize the problem of discovering blind spots in RL as a noisy supervised learning problem with class imbalance. We learn models to predict blind spots in unseen regions of the state space by combining techniques for label aggregation, calibration, and supervised learning. The models take into consideration noise emerging from different forms of oracle feedback, including demonstrations and corrections. We evaluate our approach on two domains and show that it achieves higher predictive performance than baseline methods, and that the learned model can be used to selectively query an oracle at execution time to prevent errors. We also empirically analyze the biases of various feedback types and how they influence the discovery of blind spots.
Distributed Approximation Algorithms for the Combinatorial Motion Planning Problem
Venue: arXiv cs.DM 2018
Year: 2018

We present a new $4$-approximation algorithm for the Combinatorial Motion Planning problem which runs in $\mathcal{O}(n^2α(n^2,n))$ time, where $α$ is the functional inverse of the Ackermann function, and a fully distributed version for the same in asynchronous message passing systems, which runs in $\mathcal{O}(n\log_2n)$ time with a message complexity of $\mathcal{O}(n^2)$. This also includes the first fully distributed algorithm in asynchronous message passing systems to perform "shortcut" operations on paths, a procedure which is important in approximation algorithms for the vehicle routing problem and its variants. We also show that our algorithm gives feasible solutions to the $k$-TSP problem with an approximation factor of $2$ in both centralized and distributed environments. The broad idea of the algorithm is to distribute the set of vertices into two subsets and construct paths for each salesman over each of the two subsets. Finally we combine these pairwise disjoint paths for each salesman to obtain a set of paths that span the entire graph. This is similar to the algorithm by Yadlapalli et. al. \cite{3.66} but differs in respect to the fact that it does not require us to use minimum cost matching as a subroutine, and hence can be easily distributed.
Efficient Relaxations for Dense CRFs with Sparse Higher Order Potentials
Venue: arXiv cs.CV 2018
Year: 2018

Dense conditional random fields (CRFs) have become a popular framework for modelling several problems in computer vision such as stereo correspondence and multi-class semantic segmentation. By modelling long-range interactions, dense CRFs provide a labelling that captures finer detail than their sparse counterparts. Currently, the state-of-the-art algorithm performs mean-field inference using a filter-based method but fails to provide a strong theoretical guarantee on the quality of the solution. A question naturally arises as to whether it is possible to obtain a maximum a posteriori (MAP) estimate of a dense CRF using a principled method. Within this paper, we show that this is indeed possible. We will show that, by using a filter-based method, continuous relaxations of the MAP problem can be optimised efficiently using state-of-the-art algorithms. Specifically, we will solve a quadratic programming (QP) relaxation using the Frank-Wolfe algorithm and a linear programming (LP) relaxation by developing a proximal minimisation framework. By exploiting labelling consistency in the higher-order potentials and utilising the filter-based method, we are able to formulate the above algorithms such that each iteration has a complexity linear in the number of classes and random variables. The presented algorithms can be applied to any labelling problem using a dense CRF with sparse higher-order potentials. In this paper, we use semantic segmentation as an example application as it demonstrates the ability of the algorithm to scale to dense CRFs with large dimensions. We perform experiments on the Pascal dataset to indicate that the presented algorithms are able to attain lower energies than the mean-field inference method.
Saliency deep embedding for aurora image search
Venue: arXiv cs.CV 2018
Year: 2018

Deep neural networks have achieved remarkable success in the field of image search. However, the state-of-the-art algorithms are trained and tested for natural images captured with ordinary cameras. In this paper, we aim to explore a new search method for images captured with circular fisheye lens, especially the aurora images. To reduce the interference from uninformative regions and focus on the most interested regions, we propose a saliency proposal network (SPN) to replace the region proposal network (RPN) in the recent Mask R-CNN. In our SPN, the centers of the anchors are not distributed in a rectangular meshing manner, but exhibit spherical distortion. Additionally, the directions of the anchors are along the deformation lines perpendicular to the magnetic meridian, which perfectly accords with the imaging principle of circular fisheye lens. Extensive experiments are performed on the big aurora data, demonstrating the superiority of our method in both search accuracy and efficiency.
Active Inference for Adaptive BCI: application to the P300 Speller
Venue: arXiv cs.HC 2018
Year: 2018

Adaptive Brain-Computer interfaces (BCIs) have shown to improve performance, however a general and flexible framework to implement adaptive features is still lacking. We appeal to a generic Bayesian approach, called Active Inference (AI), to infer user's intentions or states and act in a way that optimizes performance. In realistic P300-speller simulations, AI outperforms traditional algorithms with an increase in bit rate between 18% and 59%, while offering a possibility of unifying various adaptive implementations within one generic framework.
Complex System Design with Design Languages: Method, Applications and Design Principles
Venue: arXiv cs.SE 2018
Year: 2018

Graph-based design languages in UML (Unified Modeling Language) are presented as a method to encode and automate the complete design process and the final optimization of the product or complex system. A design language consists of a vocabulary (digital building blocks) and a set of rules (digital composition knowledge) along with an executable sequence of the rules (digital encoding of the design process). The rule-based mechanism instantiates a central and consistent global product data structure (the so-called design graph). Upon the generation of the abstract central model, the domain-specific engineering models are automatically generated, remotely executed and their results are fed-back into the central design model for subsequent design decisions or optimizations. The design languages are manually modeled and automatically executed in a so-called design compiler. Up to now, a variety of product designs in the areas of aerospace, automotive, machinery and consumer products have been successfully accelerated and automated using graph-based design languages. Different design strategies and mechanisms have been identified and applied in the automation of the design processes. Approaches ranging from the automated and declarative processing of constraints, through fractal nested design patterns, to mathematical dimension-based derivation of the sequence of design actions are used. The existing knowledge for a design determines the global design strategy (top-down vs. bottom-up). Similarity-mechanics in the form of dimensionless invariants are used for evaluation to downsize the solution for an overall complexity reduction. Design patterns, design paradigms (form follows function) and design strategies (divide and conquer) from information science are heavily used to structure, manage and handle complexity.
Optimal Load Ensemble Control in Chance-Constrained Optimal Power Flow
Venue: arXiv cs.SY 2018
Year: 2018

Distribution system operators (DSO) world-wide foresee a rapid roll-out of distributed energy resources. From the system perspective, their reliable and cost effective integration requires accounting for their physical properties in operating tools used by the DSO. This paper describes an approach to leverage the dispatch flexibility of thermostatically controlled loads (TCLs) for operating distribution systems with a high penetration level of photovoltaic resources. Each TCL ensemble is modeled using the Markov Decision Process (MDP). The MDP model is then integrated with the chance-constrained optimal power flow that accounts for the uncertainty of PV resources. Since the integrated optimization model cannot be solved efficiently by existing dynamic programming methods or off-the-shelf solvers, this paper proposes an iterative Spatio-Temporal Dual Decomposition algorithm (ST-D2). We demonstrate the usefulness of the proposed integrated optimization and ST-D2 algorithm on the IEEE 33-bus test system.
A logical representation of Arabic questions toward automatic passage extraction from the Web
Venue: arXiv cs.IR 2018
Year: 2018

With the expanding growth of Arabic electronic data on the web, extracting information, which is actually one of the major challenges of the question-answering, is essentially used for building corpus of documents. In fact, building a corpus is a research topic that is currently referred to among some other major themes of conferences, in Natural Language Processing (NLP), such as, Information Retrieval (IR), Question-Answering (QA), Automatic Summary (AS), etc. Generally, a question-answering system provides various passages to answer the user questions. To make these passages truly informative, this system needs access to an underlying knowledge base; this requires the construction of a corpus. The aim of our research is to build an Arabic question-answering system. In addition, analyzing the question must be the first step. Next, it is essential to retrieve a passage from the web that can serve as an appropriate answer. In this paper, we propose a method to analysis the question and retrieve the passage answer in the Arabic language. For the question analysis, five factual question types are processed. Additionally, our purpose is to experiment with the generation of a logic representation from the declarative form of each question. Several studies, deal with the logic approaches in question-answering, are discussed in other languages than the Arabic language. This representation is very promising because it helps us later in the selection of a justifiable answer. The accuracy of questions that are correctly analyzed and translated into the logic form achieved 64%. And then, the results of passages of texts that are automatically generated achieved an 87% score for accuracy and a 98% score for c@1.
Recent Advances in DRAM and Flash Memory Architectures
Venue: arXiv cs.AR 2018
Year: 2018

This article features extended summaries and retrospectives of some of the recent research done by our group, SAFARI, on (1) understanding, characterizing, and modeling various critical properties of modern DRAM and NAND flash memory, the dominant memory and storage technologies, respectively; and (2) several new mechanisms we have proposed based on our observations from these analyses, characterization, and modeling, to tackle various key challenges in memory and storage scaling. In order to understand the sources of various bottlenecks of the dominant memory and storage technologies, these works perform rigorous studies of device-level and application-level behavior, using a combination of detailed simulation and experimental characterization of real memory and storage devices.
Analysis of Novel Annotations in the Gene Ontology for Boosting the Selection of Negative Examples
Venue: arXiv cs.CE 2018
Year: 2018

Public repositories for genome and proteome annotations, such as the Gene Ontology (GO), rarely stores negative annotations, i.e. proteins not possessing a given function. This leaves undefined or ill defined the set of negative examples, which is crucial for training the majority of machine learning methods inferring proteins functions. Automated techniques to choose reliable negative proteins are thereby required to train accurate function prediction models. This study proposes the first extensive analysis of the temporal evolution of protein annotations in the GO repository. Novel annotations registered through the years have been analyzed to verify the presence of annotation patterns in the GO hierarchy. Our research supplied fundamental clues about proteins likely to be unreliable as negative examples, that we verified into a novel algorithm of our own construction, validated on two organisms in a genome wide fashion against approaches proposed to choose negative examples in the context of functional prediction.
Image Captioning
Venue: arXiv cs.CV 2018
Year: 2018

This paper discusses and demonstrates the outcomes from our experimentation on Image Captioning. Image captioning is a much more involved task than image recognition or classification, because of the additional challenge of recognizing the interdependence between the objects/concepts in the image and the creation of a succinct sentential narration. Experiments on several labeled datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions. As a toy application, we apply image captioning to create video captions, and we advance a few hypotheses on the challenges we encountered.