OpenRaiser/Intern-Atlas huggingface dataset:
A Push-Pull Layer Improves Robustness of Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

We propose a new layer in Convolutional Neural Networks (CNNs) to increase their robustness to several types of noise perturbations of the input images. We call this a push-pull layer and compute its response as the combination of two half-wave rectified convolutions, with kernels of opposite polarity. It is based on a biologically-motivated non-linear model of certain neurons in the visual system that exhibit a response suppression phenomenon, known as push-pull inhibition. We validate our method by substituting the first convolutional layer of the LeNet-5 and WideResNet architectures with our push-pull layer. We train the networks on nonperturbed training images from the MNIST, CIFAR-10 and CIFAR-100 data sets, and test on images perturbed by noise that is unseen by the training process. We demonstrate that our push-pull layers contribute to a considerable improvement in robustness of classification of images perturbed by noise, while maintaining state-of-the-art performance on the original image classification task.
Revised JNLPBA Corpus: A Revised Version of Biomedical NER Corpus for Relation Extraction Task
Venue: arXiv cs.IR 2019
Year: 2019

The advancement of biomedical named entity recognition (BNER) and biomedical relation extraction (BRE) researches promotes the development of text mining in biological domains. As a cornerstone of BRE, robust BNER system is required to identify the mentioned NEs in plain texts for further relation extraction stage. However, the current BNER corpora, which play important roles in these tasks, paid less attention to achieve the criteria for BRE task. In this study, we present Revised JNLPBA corpus, the revision of JNLPBA corpus, to broaden the applicability of a NER corpus from BNER to BRE task. We preserve the original entity types including protein, DNA, RNA, cell line and cell type while all the abstracts in JNLPBA corpus are manually curated by domain experts again basis on the new annotation guideline focusing on the specific NEs instead of general terms. Simultaneously, several imperfection issues in JNLPBA are pointed out and made up in the new corpus. To compare the adaptability of different NER systems in Revised JNLPBA and JNLPBA corpora, the F1-measure was measured in three open sources NER systems including BANNER, Gimli and NERSuite. In the same circumstance, all the systems perform average 10% better in Revised JNLPBA than in JNLPBA. Moreover, the cross-validation test is carried out which we train the NER systems on JNLPBA/Revised JNLPBA corpora and access the performance in both protein-protein interaction extraction (PPIE) and biomedical event extraction (BEE) corpora to confirm that the newly refined Revised JNLPBA is a competent NER corpus in biomedical relation application. The revised JNLPBA corpus is freely available at iasl-btm.iis.sinica.edu.tw/BNER/Content/Revised_JNLPBA.zip.
Multikernel activation functions: formulation and a case study
Venue: arXiv cs.LG 2019
Year: 2019

The design of activation functions is a growing research area in the field of neural networks. In particular, instead of using fixed point-wise functions (e.g., the rectified linear unit), several authors have proposed ways of learning these functions directly from the data in a non-parametric fashion. In this paper we focus on the kernel activation function (KAF), a recently proposed framework wherein each function is modeled as a one-dimensional kernel model, whose weights are adapted through standard backpropagation-based optimization. One drawback of KAFs is the need to select a single kernel function and its eventual hyper-parameters. To partially overcome this problem, we motivate an extension of the KAF model, in which multiple kernels are linearly combined at every neuron, inspired by the literature on multiple kernel learning. We provide an application of the resulting multi-KAF on a realistic use case, specifically handwritten Latin OCR, on a large dataset collected in the context of the `In Codice Ratio' project. Results show that multi-KAFs can improve the accuracy of the convolutional networks previously developed for the task, with faster convergence, even with a smaller number of overall parameters.
TiFi: Taxonomy Induction for Fictional Domains [Extended version]
Venue: arXiv cs.CL 2019
Year: 2019

Taxonomies are important building blocks of structured knowledge bases, and their construction from text sources and Wikipedia has received much attention. In this paper we focus on the construction of taxonomies for fictional domains, using noisy category systems from fan wikis or text extraction as input. Such fictional domains are archetypes of entity universes that are poorly covered by Wikipedia, such as also enterprise-specific knowledge bases or highly specialized verticals. Our fiction-targeted approach, called TiFi, consists of three phases: (i) category cleaning, by identifying candidate categories that truly represent classes in the domain of interest, (ii) edge cleaning, by selecting subcategory relationships that correspond to class subsumption, and (iii) top-level construction, by mapping classes onto a subset of high-level WordNet categories. A comprehensive evaluation shows that TiFi is able to construct taxonomies for a diverse range of fictional domains such as Lord of the Rings, The Simpsons or Greek Mythology with very high precision and that it outperforms state-of-the-art baselines for taxonomy induction by a substantial margin.
Combined tract segmentation and orientation mapping for bundle-specific tractography
Venue: arXiv cs.CV 2019
Year: 2019

While the major white matter tracts are of great interest to numerous studies in neuroscience and medicine, their manual dissection in larger cohorts from diffusion MRI tractograms is time-consuming, requires expert knowledge and is hard to reproduce. In previous work we presented tract orientation mapping (TOM) as a novel concept for bundle-specific tractography. It is based on a learned mapping from the original fiber orientation distribution function (FOD) peaks to tract specific peaks, called tract orientation maps. Each tract orientation map represents the voxel-wise principal orientation of one tract. Here, we present an extension of this approach that combines TOM with accurate segmentations of the tract outline and its start and end region. We also introduce a custom probabilistic tracking algorithm that samples from a Gaussian distribution with fixed standard deviation centered on each peak thus enabling more complete trackings on the tract orientation maps than deterministic tracking. These extensions enable the automatic creation of bundle-specific tractograms with previously unseen accuracy. We show for 72 different bundles on high quality, low quality and phantom data that our approach runs faster and produces more accurate bundle-specific tractograms than 7 state of the art benchmark methods while avoiding cumbersome processing steps like whole brain tractography, non-linear registration, clustering or manual dissection. Moreover, we show on 17 datasets that our approach generalizes well to datasets acquired with different scanners and settings as well as with pathologies. The code of our method is openly available at https://github.com/MIC-DKFZ/TractSeg.
Physical Foundations of Landauer's Principle
Venue: arXiv cs.ET 2019
Year: 2019

We review the physical foundations of Landauer's Principle, which relates the loss of information from a computational process to an increase in thermodynamic entropy. Despite the long history of the Principle, its fundamental rationale and proper interpretation remain frequently misunderstood. Contrary to some misinterpretations of the Principle, the mere transfer of entropy between computational and non-computational subsystems can occur in a thermodynamically reversible way without increasing total entropy. However, Landauer's Principle is not about general entropy transfers; rather, it more specifically concerns the ejection of (all or part of) some correlated information from a controlled, digital form (e.g., a computed bit) to an uncontrolled, non-computational form, i.e., as part of a thermal environment. Any uncontrolled thermal system will, by definition, continually re-randomize the physical information in its thermal state, from our perspective as observers who cannot predict the exact dynamical evolution of the microstates of such environments. Thus, any correlations involving information that is ejected into and subsequently thermalized by the environment will be lost from our perspective, resulting directly in an irreversible increase in total entropy. Avoiding the ejection and thermalization of correlated computational information motivates the reversible computing paradigm, although the requirements for computations to be thermodynamically reversible are less restrictive than frequently described, particularly in the case of stochastic computational operations. There are interesting possibilities for the design of computational processes that utilize stochastic, many-to-one computational operations while nevertheless avoiding net entropy increase that remain to be fully explored.
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Venue: arXiv cs.ET 2019
Year: 2019

Memristor crossbars are circuits capable of performing analog matrix-vector multiplications, overcoming the fundamental energy efficiency limitations of digital logic. They have been shown to be effective in special-purpose accelerators for a limited set of neural network applications. We present the Programmable Ultra-efficient Memristor-based Accelerator (PUMA) which enhances memristor crossbars with general purpose execution units to enable the acceleration of a wide variety of Machine Learning (ML) inference workloads. PUMA's microarchitecture techniques exposed through a specialized Instruction Set Architecture (ISA) retain the efficiency of in-memory computing and analog circuitry, without compromising programmability. We also present the PUMA compiler which translates high-level code to PUMA ISA. The compiler partitions the computational graph and optimizes instruction scheduling and register allocation to generate code for large and complex workloads to run on thousands of spatial cores. We have developed a detailed architecture simulator that incorporates the functionality, timing, and power models of PUMA's components to evaluate performance and energy consumption. A PUMA accelerator running at 1 GHz can reach area and power efficiency of $577~GOPS/s/mm^2$ and $837~GOPS/s/W$, respectively. Our evaluation of diverse ML applications from image recognition, machine translation, and language modelling (5M-800M synapses) shows that PUMA achieves up to $2,446\times$ energy and $66\times$ latency improvement for inference compared to state-of-the-art GPUs. Compared to an application-specific memristor-based accelerator, PUMA incurs small energy overheads at similar inference latency and added programmability.
Effective certification of approximate solutions to systems of equations involving analytic functions
Venue: arXiv cs.SC 2019
Year: 2019

We develop algorithms for certifying an approximation to a nonsingular solution of a square system of equations built from univariate analytic functions. These algorithms are based on the existence of oracles for evaluating basic data about the input analytic functions. One approach for certification is based on alpha-theory while the other is based on the Krawczyk generalization of Newton's iteration. We show that the necessary oracles exist for D-finite functions and compare the two algorithmic approaches for this case using our software implementation in SageMath.
Interleaving Loidreau's Rank-Metric Cryptosystem
Venue: arXiv cs.IT 2019
Year: 2019

We propose and analyze an interleaved variant of Loidreau's rank-metric cryptosystem based on rank multipliers. We analyze and adapt several attacks on the system, propose design rules, and study weak keys. Finding secure instances requires near-MRD rank-metric codes which are not investigated in the literature. Thus, we propose a random code construction that makes use of the fact that short random codes over large fields are MRD with high probability. We derive an upper bound on the decryption failure rate and give example parameters for potential key size reduction.
Quantifying the Impact of User Attention on Fair Group Representation in Ranked Lists
Venue: arXiv cs.CY 2019
Year: 2019

In this work, we introduce a novel metric for auditing group fairness in ranked lists. Our approach offers two benefits compared to the state of the art. First, we offer a blueprint for modeling of user attention. Rather than assuming a logarithmic loss in importance as a function of the rank, we can account for varying user behaviors through parametrization. For example, we expect a user to see more items during a viewing of a social media feed than when they inspect the results list of a single web search query. Second, we allow non-binary protected attributes to enable investigating inherently continuous attributes (\eg political alignment on the liberal to conservative spectrum) as well as to facilitate measurements across aggregated sets of search results, rather than separately for each result list. By combining these two elements into our metric, we are able to better address the human factors inherent in this problem. We measure the whole sociotechnical system, consisting of a ranking algorithm and individuals using it, instead of exclusively focusing on the ranking algorithm. Finally, we use our metric to perform three simulated fairness audits. We show that determining fairness of a ranked output necessitates knowledge (or a model) of the end-users of the particular service. Depending on their attention distribution function, a fixed ranking of results can appear biased both in favor and against a protected group.
Health Behavior Change in HCI: Trends, Patterns, and Opportunities
Venue: arXiv cs.HC 2019
Year: 2019

Unhealthy lifestyles could cause many chronic diseases, which bring patients and their families much burden. Research has shown the potential of digital technologies for supporting health behavior change to help us prevent these chronic diseases. The HCI community has contributed to the research on health behavior change for more than a decade. In this paper, we aim to explore the research trends and patterns of health behavior change in HCI. Our systematic review showed that physical activity drew much more attention than other behaviors. Most of the participants in the reviewed studies were adults, while children and the elderly were much less addressed. Also, we found there is a lack of standardized approaches to evaluating the user experience of interventions for health behavior change in HCI. Based on the reviewed studies, we provide suggestions and research opportunities on six topics, e.g., game integration, social support, and relevant AI application.
Age of Information for Discrete Time Queues
Venue: arXiv cs.IT 2019
Year: 2019

Age of information (AoI) is a time-evolving measure of information freshness, that tracks the time since the last received fresh update was generated. Analyzing peak and average AoI, two time average metrics of AoI, for various continuous time queueing systems has received considerable attention. We analyze peak and average age for various discrete time queueing systems. We first consider first come first serve (FCFS) Ber/G/1 and Ber/G/1 queue with vacations, and derive explicit expressions for peak and average age. We also obtain age expressions for the last come first serve (LCFS) queue and the $G/G/\infty$ queue. We build upon proof techniques from earlier results, and also present new techniques that might be of independent interest in analyzing age in discrete time queuing systems.
Decoder-tailored Polar Code Design Using the Genetic Algorithm
Venue: arXiv cs.IT 2019
Year: 2019

We propose a new framework for constructing polar codes (i.e., selecting the frozen bit positions) for arbitrary channels, and tailored to a given decoding algorithm, rather than based on the (not necessarily optimal) assumption of successive cancellation (SC) decoding. The proposed framework is based on the Genetic Algorithm (GenAlg), where populations (i.e., collections) of information sets evolve successively via evolutionary transformations based on their individual error-rate performance. These populations converge towards an information set that fits both the decoding behavior and the defined channel. Using our proposed algorithm over the additive white Gaussian noise (AWGN) channel, we construct a polar code of length 2048 with code rate 0.5, without the CRC-aid, tailored to plain successive cancellation list (SCL) decoding, achieving the same error-rate performance as the CRC-aided SCL decoding, and leading to a coding gain of 1 dB at BER of $10^{-6}$. Further, a belief propagation (BP)-tailored construction approaches the SCL error-rate performance without any modifications in the decoding algorithm itself. The performance gains can be attributed to the significant reduction in the total number of low-weight codewords. To demonstrate the flexibility, coding gains for the Rayleigh channel are shown under SCL and BP decoding. Besides improvements in error-rate performance, we show that, when required, the GenAlg can be also set up to reduce the decoding complexity, e.g., the SCL list size or the number of BP iterations can be reduced, while maintaining the same error-rate performance.
Non Binary Polar Codes with Equidistant Transform for Transmission over the AWGN Channel
Venue: arXiv cs.IT 2019
Year: 2019

Finding fast polarizing transforms is an important problem as polar codes suffer from slow finitelength performance. This paper considers non binary polar codes for transmission over the AWGN channel and designs polarizing transforms with better distance characteristics using a simple procedure for signal sets. The main idea of the paper is to define Equidistant Polarizing Transforms and show that they achieve the best distance spectrum bound. We provides an example of such transform for q = 5. In this case PSK-type signal set is used. Finally, we show performance gains and some comparison with other methods.
The Effects of Using Business Intelligence Systems on an Excellence Management and Decision-Making Process by Start-Up Companies: A Case Study
Venue: arXiv cs.CY 2019
Year: 2019

The rapid increase in data volumes in companies has meant that momentous and comprehensive information gathering is barely possible by manual means. Business intelligence solutions can help here. They provide tools with appropriate technologies to assist with the collection, integration, storage, editing, and analysis of existing data. While almost only large companies were interested in this topic a few years ago, it has meanwhile also become necessary for start-up companies, and so the market for business intelligence has been growing for years. This article focuses on the general potentials of using BI in start-ups. First, will be examined which providers of BI solutions that are suitable for start-ups and what opportunities exist for implementing BI systems in start-ups. Then it will be shown to what extent BI has prevailed in start-ups, in which areas the techniques of BI are used in start-ups and what purpose BI has in start-ups. Finally, the success factors for BI projects in start-ups are considered.
Directed Formation Control of n Planar Agents with Distance and Area Constraints
Venue: arXiv cs.MA 2019
Year: 2019

In this paper, we take a first step towards generalizing a recently proposed method for dealing with the problem of convergence to incorrect equilibrium points of distance-based formation controllers. Specifically, we introduce a distance and area-based scheme for the formation control of $n$-agent systems in two dimensions using directed graphs and the single-integrator model. We show that under certain conditions on the edge lengths of the triangulated desired formation, the control ensures almost-global convergence to the correct formation.
Using Floating Gate Memory to Train Ideal Accuracy Neural Networks
Venue: arXiv cs.ET 2019
Year: 2019

Floating gate SONOS (Silicon-Oxygen-Nitrogen-Oxygen-Silicon) transistors can be used to train neural networks to ideal accuracies that match those of floating point digital weights on the MNIST dataset when using multiple devices to represent a weight or within 1% of ideal accuracy when using a single device. This is enabled by operating devices in the subthreshold regime, where they exhibit symmetric write nonlinearities. A neural training accelerator core based on SONOS with a single device per weight would increase energy efficiency by 120X, operate 2.1X faster and require 5X lower area than an optimized SRAM based ASIC.
Automatic end-to-end De-identification: Is high accuracy the only metric?
Venue: arXiv cs.CY 2019
Year: 2019

De-identification of electronic health records (EHR) is a vital step towards advancing health informatics research and maximising the use of available data. It is a two-step process where step one is the identification of protected health information (PHI), and step two is replacing such PHI with surrogates. Despite the recent advances in automatic de-identification of EHR, significant obstacles remain if the abundant health data available are to be used to the full potential. Accuracy in de-identification could be considered a necessary, but not sufficient condition for the use of EHR without individual patient consent. We present here a comprehensive review of the progress to date, both the impressive successes in achieving high accuracy and the significant risks and challenges that remain. To best of our knowledge, this is the first paper to present a complete picture of end-to-end automatic de-identification. We review 18 recently published automatic de-identification systems -designed to de-identify EHR in the form of free text- to show the advancements made in improving the overall accuracy of the system, and in identifying individual PHI. We argue that despite the improvements in accuracy there remain challenges in surrogate generation and replacements of identified PHIs, and the risks posed to patient protection and privacy.