OpenRaiser/Intern-Atlas huggingface dataset:
Optimizing the Dice Score and Jaccard Index for Medical Image Segmentation: Theory & Practice
Venue: arXiv cs.CV 2019
Year: 2019

The Dice score and Jaccard index are commonly used metrics for the evaluation of segmentation tasks in medical imaging. Convolutional neural networks trained for image segmentation tasks are usually optimized for (weighted) cross-entropy. This introduces an adverse discrepancy between the learning optimization objective (the loss) and the end target metric. Recent works in computer vision have proposed soft surrogates to alleviate this discrepancy and directly optimize the desired metric, either through relaxations (soft-Dice, soft-Jaccard) or submodular optimization (Lovász-softmax). The aim of this study is two-fold. First, we investigate the theoretical differences in a risk minimization framework and question the existence of a weighted cross-entropy loss with weights theoretically optimized to surrogate Dice or Jaccard. Second, we empirically investigate the behavior of the aforementioned loss functions w.r.t. evaluation with Dice score and Jaccard index on five medical segmentation tasks. Through the application of relative approximation bounds, we show that all surrogates are equivalent up to a multiplicative factor, and that no optimal weighting of cross-entropy exists to approximate Dice or Jaccard measures. We validate these findings empirically and show that, while it is important to opt for one of the target metric surrogates rather than a cross-entropy-based loss, the choice of the surrogate does not make a statistical difference on a wide range of medical segmentation tasks.
An Efficient Word Lookup System by using Improved Trie Algorithm
Venue: arXiv cs.DS 2019
Year: 2019

Efficiently word storing and searching is an important task in computer science. An application space complexity, time complexity, and overall performance depend on this string data. Many word searching data structures and algorithms exist in the current world but few of them have space compress ability. Trie is a popular data structure for word searching for its linear searching capability. It is the basic and important part of various computer applications such as information retrieval, natural language processing, database system, compiler, and computer network. But currently, the available version of trie tree cannot be used widely because of its high memory requirement. This paper proposes a new Radix trie based data structure for word storing and searching which can share not only just prefix but also infix and suffix and thus reduces memory requirement. We propose a new emptiness property to Radix trie. Proposed trie has character cell reduction capability and it can dramatically reduce any application runtime memory size. Using it as data tank to an operating system the overall main memory requirement of a device can be reduced to a large extent.
Distance-Based Network Partitioning
Venue: arXiv cs.SI 2019
Year: 2019

A new method for identifying communities in networks is proposed. Reference nodes, either selected using a priory information about the network or according to relevant node measurements, are obtained so as to indicate putative communities. Distance vectors between each network node and the reference nodes are then used for defining a coordinate system representing the community structure of the network at many different scales. For modular networks, the distribution of nodes in this space often results in a well-separated clustered structure, with each cluster corresponding to a community. One interesting feature of the reported methodology for community finding is that the coordinate system defined by the seeds allows an intuitive and direct interpretation of the situation of each node with respect to the considered communities. The potential of the method is illustrated with respect to a community detection benchmark, a spatial network model and to city streets networks.
Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game
Venue: arXiv cs.CL 2019
Year: 2019

Research in multi-agent cooperation has shown that artificial agents are able to learn to play a simple referential game while developing a shared lexicon. This lexicon is not easy to analyze, as it does not show many properties of a natural language. In a simple referential game with two neural network-based agents, we analyze the object-symbol mapping trying to understand what kind of strategy was used to develop the emergent language. We see that, when the environment is uniformly distributed, the agents rely on a random subset of features to describe the objects. When we modify the objects making one feature non-uniformly distributed,the agents realize it is less informative and start to ignore it, and, surprisingly, they make a better use of the remaining features. This interesting result suggests that more natural, less uniformly distributed environments might aid in spurring the emergence of better-behaved languages.
Joint Power Allocation and Network Slicing in an Open RAN System
Venue: arXiv cs.NI 2019
Year: 2019

Open radio access network (ORAN) alliance which has been formed recently establishes a flexible, open, and smart radio access network (RAN) by combing the ideas from xRAN and cloud RAN (C-RAN). ORAN divides the functions of the RAN into three parts, namely remote unit (RU), distributed unit (DU), and central unit (CU). While RU contains lower PHY functions, DU contains higher PHY, MAC, and RLC and CU contains RRC, PDCP, and SDAP. CU and DU are implemented as virtual network functions (VNFs) running on a cloud environment. Interface between RU, CU, and DU are open standard interfaces. Network slicing as a new concept in 5G systems is used to share the network resources between various services while the operation of one service does not affect another service. In this paper, we study the problem of RAN network slicing in an ORAN system. We formulate the problem of wireless link scheduling, mapping the slices to the services, and mapping the physical data centers resource to slices. The objective is to jointly maximize the energy efficiency and minimize power consumption of RUs and the cost of physical resources in a downlink channel. The problem is formulated as a mixed-integer optimization problem that can be decomposed into two independent sub-problems. Heuristic algorithms are proposed for each of the sub-problems.
Scenarios and Recommendations for Ethical Interpretive AI
Venue: arXiv cs.AI 2019
Year: 2019

Artificially intelligent systems, given a set of non-trivial ethical rules to follow, will inevitably be faced with scenarios which call into question the scope of those rules. In such cases, human reasoners typically will engage in interpretive reasoning, where interpretive arguments are used to support or attack claims that some rule should be understood a certain way. Artificially intelligent reasoners, however, currently lack the ability to carry out human-like interpretive reasoning, and we argue that bridging this gulf is of tremendous importance to human-centered AI. In order to better understand how future artificial reasoners capable of human-like interpretive reasoning must be developed, we have collected a dataset of ethical rules, scenarios designed to invoke interpretive reasoning, and interpretations of those scenarios. We perform a qualitative analysis of our dataset, and summarize our findings in the form of practical recommendations.
A Survey of Blockchain Applications in Different Domains
Venue: arXiv cs.CR 2019
Year: 2019

Blockchains have received much attention recently since they provide decentralized approaches to the creation and management of value. Many banks, Internet companies, car manufacturers, and even governments worldwide have incorporated or started considering blockchains to improve the security, scalability, and efficiency of their services. In this paper, we survey blockchain applications in different areas. These areas include cryptocurrency, healthcare, advertising, insurance, copyright protection, energy, and societal applications. Our work provides a timely summary for individuals and organizations interested in blockchains. We envision our study to motivate more blockchain applications.
Guided Layer-wise Learning for Deep Models using Side Information
Venue: arXiv cs.LG 2019
Year: 2019

Training of deep models for classification tasks is hindered by local minima problems and vanishing gradients, while unsupervised layer-wise pretraining does not exploit information from class labels. Here, we propose a new regularization technique, called diversifying regularization (DR), which applies a penalty on hidden units at any layer if they obtain similar features for different types of data. For generative models, DR is defined as divergence over the variational posteriori distributions and included in the maximum likelihood estimation as a prior. Thus, DR includes class label information for greedy pretraining of deep belief networks which result in a better weight initialization for fine-tuning methods. On the other hand, for discriminative training of deep neural networks, DR is defined as a distance over the features and included in the learning objective. With our experimental tests, we show that DR can help the backpropagation to cope with vanishing gradient problems and to provide faster convergence and smaller generalization errors.
Hierarchical Mixtures of Generators for Adversarial Learning
Venue: arXiv cs.LG 2019
Year: 2019

Generative adversarial networks (GANs) are deep neural networks that allow us to sample from an arbitrary probability distribution without explicitly estimating the distribution. There is a generator that takes a latent vector as input and transforms it into a valid sample from the distribution. There is also a discriminator that is trained to discriminate such fake samples from true samples of the distribution; at the same time, the generator is trained to generate fakes that the discriminator cannot tell apart from the true samples. Instead of learning a global generator, a recent approach involves training multiple generators each responsible from one part of the distribution. In this work, we review such approaches and propose the hierarchical mixture of generators, inspired from the hierarchical mixture of experts model, that learns a tree structure implementing a hierarchical clustering with soft splits in the decision nodes and local generators in the leaves. Since the generators are combined softly, the whole model is continuous and can be trained using gradient-based optimization, just like the original GAN model. Our experiments on five image data sets, namely, MNIST, FashionMNIST, UTZap50K, Oxford Flowers, and CelebA, show that our proposed model generates samples of high quality and diversity in terms of popular GAN evaluation metrics. The learned hierarchical structure also leads to knowledge extraction.
Post-Training 4-bit Quantization on Embedding Tables
Venue: arXiv cs.LG 2019
Year: 2019

Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entities increases, the embedding components can easily contain millions of parameters and become the bottleneck in both storage and inference due to large memory consumption. This work focuses on post-training 4-bit quantization on the continuous embeddings. We propose row-wise uniform quantization with greedy search and codebook-based quantization that consistently outperforms state-of-the-art quantization approaches on reducing accuracy degradation. We deploy our uniform quantization technique on a production model in Facebook and demonstrate that it can reduce the model size to only 13.89% of the single-precision version while the model quality stays neutral.
A Divergence Minimization Perspective on Imitation Learning Methods
Venue: arXiv cs.LG 2019
Year: 2019

In many settings, it is desirable to learn decision-making and control policies through learning or bootstrapping from expert demonstrations. The most common approaches under this Imitation Learning (IL) framework are Behavioural Cloning (BC), and Inverse Reinforcement Learning (IRL). Recent methods for IRL have demonstrated the capacity to learn effective policies with access to a very limited set of demonstrations, a scenario in which BC methods often fail. Unfortunately, due to multiple factors of variation, directly comparing these methods does not provide adequate intuition for understanding this difference in performance. In this work, we present a unified probabilistic perspective on IL algorithms based on divergence minimization. We present $f$-MAX, an $f$-divergence generalization of AIRL [Fu et al., 2018], a state-of-the-art IRL method. $f$-MAX enables us to relate prior IRL methods such as GAIL [Ho & Ermon, 2016] and AIRL [Fu et al., 2018], and understand their algorithmic properties. Through the lens of divergence minimization we tease apart the differences between BC and successful IRL approaches, and empirically evaluate these nuances on simulated high-dimensional continuous control domains. Our findings conclusively identify that IRL's state-marginal matching objective contributes most to its superior performance. Lastly, we apply our new understanding of IL methods to the problem of state-marginal matching, where we demonstrate that in simulated arm pushing environments we can teach agents a diverse range of behaviours using simply hand-specified state distributions and no reward functions or expert demonstrations. For datasets and reproducing results please refer to https://github.com/KamyarGh/rl_swiss/blob/master/reproducing/fmax_paper.md .
Boosting Object Recognition in Point Clouds by Saliency Detection
Venue: arXiv cs.CV 2019
Year: 2019

Object recognition in 3D point clouds is a challenging task, mainly when time is an important factor to deal with, such as in industrial applications. Local descriptors are an amenable choice whenever the 6 DoF pose of recognized objects should also be estimated. However, the pipeline for this kind of descriptors is highly time-consuming. In this work, we propose an update to the traditional pipeline, by adding a preliminary filtering stage referred to as saliency boost. We perform tests on a standard object recognition benchmark by considering four keypoint detectors and four local descriptors, in order to compare time and recognition performance between the traditional pipeline and the boosted one. Results on time show that the boosted pipeline could turn out up to 5 times faster, with the recognition rate improving in most of the cases and exhibiting only a slight decrease in the others. These results suggest that the boosted pipeline can speed-up processing time substantially with limited impacts or even benefits in recognition accuracy.
In Search of Dense Subgraphs: How Good is Greedy Peeling?
Venue: arXiv cs.DS 2019
Year: 2019

The problem of finding the densest subgraph in a given graph has several applications in graph mining, particularly in areas like social network analysis, protein and gene analyses etc. Depending on the application, finding dense subgraphs can be used to determine regions of high importance, similar characteristics or enhanced interaction. The densest subgraph extraction problem is a fundamentally a non-linear optimization problem. Nevertheless, it can be solved in polynomial time by an exact algorithm based on the iterative solution of a series of maximum flow sub-problems. Despite its polynomial time complexity, the computing time required by the exact algorithms on very large graphs could be prohibitive. Thus, to approach graphs with millions of vertices and edges, one has to resort to heuristic algorithms. We provide an efficient implementation of a greedy heuristic from the literature that is extremely fast and has some nice theoretical properties. We also introduce a new heurisitic algorithm that is built on top of the greedy and the exact methods. An extensive computational study is presented to evaluate the performance of various solution methods on a benchmark composed of 86 instances taken from the literature. This analysis shows that the proposed heuristic algorithm proved very effective on a large number of test instances, often providing either the optimal solution or near-optimal solution within short computing times.
A Fog Computing Based Architecture for IoT Services and Applications Development
Venue: arXiv cs.SE 2019
Year: 2019

IoT paradigm exploits the Cloud Computing platform to extend its scope and service provisioning capabilities. However, due to the location of the underlying IoT devices which is far away from the cloud, some services cannot tolerate the possible latency resulted from this issue. To overcome the latency consequences that might affect the functionality of IoT services and applications, the Fog Computing has been proposed. Fog Computing paradigm utilizes local computing resources locating at the network edge instead of those residing at the cloud for processing data collected from sensors linked to physical devices in an IoT platform. The major benefits of such paradigm include low latency, real-time decision making and an optimal utilization of available bandwidth. In this paper, we offer a review of the Fog computing paradigm and in particular its impact on the IoT application development process. We also propose an architecture for Fog Computing based IoT services and applications.
The non-capacitor model of leaky integrate-and-fire $VO_2$ neuron with the thermal mechanism of the membrane potential
Venue: arXiv cs.ET 2019
Year: 2019

The study presents a numerical model of leaky integrate-and-fire neuron created on the basis of $VO_2$ switch. The analogue of the membrane potential in the model is the temperature of the switch channel, and the action potential from neighbouring neurons propagates along the substrate in the form of thermal pulses. We simulated the operation of three neurons and demonstrated that the total effect happens due to interference of thermal waves in the region of the neuron switching channel. The thermal mechanism of the threshold function operates due to the effect of electrical switching, and the magnitude (temperature) of the threshold can vary by external voltage. The neuron circuit does not contain capacitor, making it possible to produce a network with a high density of components, and has the potential for 3D integration due to the thermal mechanism of neurons interaction.
Optimizing Millions of Hyperparameters by Implicit Differentiation
Venue: arXiv cs.LG 2019
Year: 2019

We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations. We present results about the relationship between the IFT and differentiating through optimization, motivating our algorithm. We use the proposed approach to train modern network architectures with millions of weights and millions of hyper-parameters. For example, we learn a data-augmentation network - where every weight is a hyperparameter tuned for validation performance - outputting augmented training examples. Jointly tuning weights and hyperparameters with our approach is only a few times more costly in memory and compute than standard training.
Coverage-based Outlier Explanation
Venue: arXiv cs.LG 2019
Year: 2019

Outlier detection is a core task in data mining with a plethora of algorithms that have enjoyed wide scale usage. Existing algorithms are primarily focused on detection, that is the identification of outliers in a given dataset. In this paper we explore the relatively under-studied problem of the outlier explanation problem. Our goal is, given a dataset that is already divided into outliers and normal instances, explain what characterizes the outliers. We explore the novel direction of a semantic explanation that a domain expert or policy maker is able to understand. We formulate this as an optimization problem to find explanations that are both interpretable and pure. Through experiments on real-world data sets, we quantitatively show that our method can efficiently generate better explanations compared with rule-based learners.
Unsupervised Domain Adaptation of Contextual Embeddings for Low-Resource Duplicate Question Detection
Venue: arXiv cs.CL 2019
Year: 2019

Answering questions is a primary goal of many conversational systems or search products. While most current systems have focused on answering questions against structured databases or curated knowledge graphs, on-line community forums or frequently asked questions (FAQ) lists offer an alternative source of information for question answering systems. Automatic duplicate question detection (DQD) is the key technology need for question answering systems to utilize existing online forums like StackExchange. Existing annotations of duplicate questions in such forums are community-driven, making them sparse or even completely missing for many domains. Therefore, it is important to transfer knowledge from related domains and tasks. Recently, contextual embedding models such as BERT have been outperforming many baselines by transferring self-supervised information to downstream tasks. In this paper, we apply BERT to DQD and advance it by unsupervised adaptation to StackExchange domains using self-supervised learning. We show the effectiveness of this adaptation for low-resource settings, where little or no training data is available from the target domain. Our analysis reveals that unsupervised BERT domain adaptation on even small amounts of data boosts the performance of BERT.
A Domain-Specific Language for Verifying Software Requirement Constraints
Venue: arXiv cs.SE 2019
Year: 2019

Software requirement analysis can certainly benefit from prevention and early detection of failures, in particular by some kind of automatic analysis. Formal methods offer means to represent and analyze requirements with rigorous tools, avoiding ambiguities and allowing automatic verification of requirement consistency. However, formalisms often clash in the culture or lack of skills of software analysts, making them challenging to apply. In this article, we propose a Domain-Specific Language (DSL) based on Set Theory for requirement analysts. The Graphical InvaRiant Language (GIRL) can be used to specify software requirement structural invariants, with entities and their relationships. Those invariants can then have their consistency evaluated by the Alloy Analyzer, based on a mapping semantics we provide for transforming GIRL models into Alloy specifications with no user intervention. With a prototypical language editor and transformations implemented into an Eclipse plugin, we carried out a qualitative study with requirement analysts working for a government software company in Brazil, to evaluate usability and effectiveness of the GIRL-based analysis of real software requirements. The participants were able to effectively use the underlying formal analysis, since 79 out of 80 assigned invariants were correctly modeled. While participants perceived as low the complexity of learning and using GIRL's simplest, set-based structures and relationships, the most complex logical structures, such as quantification and implication, were challenging. Furthermore, almost all post-study evaluations from the participants were positive, especially as a tool for discovering requirement inconsistencies.