OpenRaiser/Intern-Atlas huggingface dataset:
Generation of Tree-Child phylogenetic networks
Venue: arXiv cs.DS 2019
Year: 2019

Phylogenetic networks generalize phylogenetic trees by allowing the modelization of events of reticulate evolution. Among the different kinds of phylogenetic networks that have been proposed in the literature, the subclass of binary tree-child networks is one of the most studied ones. However, very little is known about the combinatorial structure of these networks. In this paper we address the problem of generating all possible binary tree-child networks with a given number of leaves in an efficient way via reduction/augmentation operations that extend and generalize analogous operations for phylogenetic trees and are biologically relevant. Since our solution is recursive, this also provides us with a recurrence relation giving an upper bound on the number of such networks.
Rapidly Adapting Moment Estimation
Venue: arXiv cs.LG 2019
Year: 2019

Adaptive gradient methods such as Adam have been shown to be very effective for training deep neural networks (DNNs) by tracking the second moment of gradients to compute the individual learning rates. Differently from existing methods, we make use of the most recent first moment of gradients to compute the individual learning rates per iteration. The motivation behind it is that the dynamic variation of the first moment of gradients may provide useful information to obtain the learning rates. We refer to the new method as the rapidly adapting moment estimation (RAME). The theoretical convergence of deterministic RAME is studied by using an analysis similar to the one used in [1] for Adam. Experimental results for training a number of DNNs show promising performance of RAME w.r.t. the convergence speed and generalization performance compared to the stochastic heavy-ball (SHB) method, Adam, and RMSprop.
DLedger: An IoT-Friendly Private Distributed Ledger System Based on DAG
Venue: arXiv cs.CR 2019
Year: 2019

With the ever growing Internet of Things (IoT) market, ledger systems are facing new challenges to efficiently store and secure enormous customer records collected by the IoT devices. The authenticity, availability, and integrity of these records are critically important for both business providers and customers. In this paper, we describe DLedger, a lightweight and resilient distributed ledger system. Instead of a single chain of blocks, DLedger builds the ledger over a directed acyclic graph (DAG), so that its operations can tolerate network partition and intermittent connectivity. Instead of compute-intensive Proof-of-Work (PoW), DLedger utilizes Proof-of-Authentication (PoA), whose light-weight operations are IoT-friendly, to achieve consensus. Furthermore, DLedger is built upon a data-centric network called Named Data Networking (NDN), which facilitates the peer-to-peer data dissemination in heterogeneous IoT networks.
Pure Strategy Best Responses to Mixed Strategies in Repeated Games
Venue: arXiv cs.GT 2019
Year: 2019

Repeated games are difficult to analyze, especially when agents play mixed strategies. We study one-memory strategies in iterated prisoner's dilemma, then generalize the result to k-memory strategies in repeated games. Our result shows that there always exists a pure strategy best response, which can be computed with SMT or MDP solvers. However, there may not exist such pure strategy best response in multi-agent tournaments. All source code is released for verification.
Optimal Online Transmission Policy for Energy-Constrained Wireless-Powered Communication Networks
Venue: arXiv cs.IT 2019
Year: 2019

This work considers the design of online transmission policy in a wireless-powered communication system with a given energy budget. The system design objective is to maximize the long-term throughput of the system exploiting the energy storage capability at the wireless-powered node. We formulate the design problem as a constrained Markov decision process (CMDP) problem and obtain the optimal policy of transmit power and time allocation in each fading block via the Lagrangian approach. To investigate the system performance in different scenarios, numerical simulations are conducted with various system parameters. Our simulation results show that the optimal policy significantly outperforms a myopic policy which only maximizes the throughput in the current fading block. Moreover, the optimal allocation of transmit power and time is shown to be insensitive to the change of modulation and coding schemes, which facilitates its practical implementation.
Privacy-Preserving Action Recognition using Coded Aperture Videos
Venue: arXiv cs.CV 2019
Year: 2019

The risk of unauthorized remote access of streaming video from networked cameras underlines the need for stronger privacy safeguards. We propose a lens-free coded aperture camera system for human action recognition that is privacy-preserving. While coded aperture systems exist, we believe ours is the first system designed for action recognition without the need for image restoration as an intermediate step. Action recognition is done using a deep network that takes in as input, non-invertible motion features between pairs of frames computed using phase correlation and log-polar transformation. Phase correlation encodes translation while the log polar transformation encodes in-plane rotation and scaling. We show that the translation features are independent of the coded aperture design, as long as its spectral response within the bandwidth has no zeros. Stacking motion features computed on frames at multiple different strides in the video can improve accuracy. Preliminary results on simulated data based on a subset of the UCF and NTU datasets are promising. We also describe our prototype lens-free coded aperture camera system, and results for real captured videos are mixed.
DRAMNet: Authentication based on Physical Unique Features of DRAM Using Deep Convolutional Neural Networks
Venue: arXiv cs.CR 2019
Year: 2019

Nowadays, there is an increasing interest in the development of Autonomous Vehicles (AV). However, there are two types of attack challenges that can affect AVs and are yet to be resolved, i.e., sensor attacks and vehicle access attacks. This paper, to the best of our knowledge, is the first work that proposes a novel authentication scheme involving DRAM power-up unique features using deep Convolutional Neural Network (CNN), which can be used to implement secure access control of autonomous vehicles. Our approach consists of two parts. First, we convert raw power-up sequence data from DRAM cells into a two-dimensional (2D) format to generate a DRAM image structure. Second, we apply deep CNN to DRAM images, in order to extract unique features from each memory to classify them for authentication. To evaluate our proposed approach, we utilize data from three Commercial-Off-The-Shelf (COTS) DRAMs taken under various environmental and other conditions (high/low temperature, high/low supply voltage and aging effects). Based on our results, our proposed authentication method ``DRAMNet'' achieves 98.63% accuracy and 98.49% precision. In comparison to other state-of-the-art CNN architectures, such as the AlexNet and VGGNet models, our DRAMNet approach fares equally well or better than them.
The MBPEP: a deep ensemble pruning algorithm providing high quality uncertainty prediction
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning algorithms have been effectively applied into various real world tasks. However, it is difficult to provide high-quality machine learning solutions to accommodate an unknown distribution of input datasets; this difficulty is called the uncertainty prediction problems. In this paper, a margin-based Pareto deep ensemble pruning (MBPEP) model is proposed. It achieves the high-quality uncertainty estimation with a small value of the prediction interval width (MPIW) and a high confidence of prediction interval coverage probability (PICP) by using deep ensemble networks. In addition to these networks, unique loss functions are proposed, and these functions make the sub-learners available for standard gradient descent learning. Furthermore, the margin criterion fine-tuning-based Pareto pruning method is introduced to optimize the ensembles. Several experiments including predicting uncertainties of classification and regression are conducted to analyze the performance of MBPEP. The experimental results show that MBPEP achieves a small interval width and a low learning error with an optimal number of ensembles. For the real-world problems, MBPEP performs well on input datasets with unknown distributions datasets incomings and improves learning performance on a multi task problem when compared to that of each single model.
Pretraining-Based Natural Language Generation for Text Summarization
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we propose a novel pretraining-based encoder-decoder framework, which can generate the output sequence based on the input sequence in a two-stage manner. For the encoder of our model, we encode the input sequence into context representations using BERT. For the decoder, there are two stages in our model, in the first stage, we use a Transformer-based decoder to generate a draft output sequence. In the second stage, we mask each word of the draft sequence and feed it to BERT, then by combining the input sequence and the draft representation generated by BERT, we use a Transformer-based decoder to predict the refined word for each masked position. To the best of our knowledge, our approach is the first method which applies the BERT into text generation tasks. As the first step in this direction, we evaluate our proposed method on the text summarization task. Experimental results show that our model achieves new state-of-the-art on both CNN/Daily Mail and New York Times datasets.
A Virtual Teaching Assistant for Personalized Learning
Venue: arXiv cs.HC 2019
Year: 2019

In this extended abstract, we propose an intelligent system that can be used as a Personalized Virtual Teaching Assistant (PVTA) to improve the students learning experience both for online and on-site courses. We show the architecture of such system, which is composed of an instance of IBM Watson Assistant and a server, and present an initial implementation, consisting in a chatbot that can be questioned about the content and the organization of the RecSys course, an introductory course on recommender systems.
MIRA: A Computational Neuro-Based Cognitive Architecture Applied to Movie Recommender Systems
Venue: arXiv cs.AI 2019
Year: 2019

The human mind is still an unknown process of neuroscience in many aspects. Nevertheless, for decades the scientific community has proposed computational models that try to simulate their parts, specific applications, or their behavior in different situations. The most complete model in this line is undoubtedly the LIDA model, proposed by Stan Franklin with the aim of serving as a generic computational architecture for several applications. The present project is inspired by the LIDA model to apply it to the process of movie recommendation, the model called MIRA (Movie Intelligent Recommender Agent) presented percentages of precision similar to a traditional model when submitted to the same assay conditions. Moreover, the proposed model reinforced the precision indexes when submitted to tests with volunteers, proving once again its performance as a cognitive model, when executed with small data volumes. Considering that the proposed model achieved a similar behavior to the traditional models under conditions expected to be similar for natural systems, it can be said that MIRA reinforces the applicability of LIDA as a path to be followed for the study and generation of computational agents inspired by neural behaviors.
Attentional Encoder Network for Targeted Sentiment Classification
Venue: arXiv cs.CL 2019
Year: 2019

Targeted sentiment classification aims at determining the sentimental tendency towards specific targets. Most of the previous approaches model context and target words with RNN and attention. However, RNNs are difficult to parallelize and truncated backpropagation through time brings difficulty in remembering long-term patterns. To address this issue, this paper proposes an Attentional Encoder Network (AEN) which eschews recurrence and employs attention based encoders for the modeling between context and target. We raise the label unreliability issue and introduce label smoothing regularization. We also apply pre-trained BERT to this task and obtain new state-of-the-art results. Experiments and analysis demonstrate the effectiveness and lightweight of our model.
Sparse Elasticity Reconstruction and Clustering using Local Displacement Fields
Venue: arXiv cs.LG 2019
Year: 2019

This paper introduces an elasticity reconstruction method based on local displacement observations of elastic bodies. Sparse reconstruction theory is applied to formulate the underdetermined inverse problems of elasticity reconstruction including unobserved areas. An online local clustering scheme called a superelement is proposed to reduce the number of dimensions of the optimization parameters. Alternating the optimization of element boundaries and elasticity parameters enables the elasticity distribution to be estimated with a higher spatial resolution. The simulation experiments show that elasticity distribution is reconstructed based on observations of approximately 10% of the total body. The estimation error was improved when considering the sparseness of the elasticity distribution.
Similarity Measures based on Local Game Trees
Venue: arXiv cs.AI 2019
Year: 2019

We study strategic similarity of game positions in two-player extensive games of perfect information, by looking at the structure of their local game trees, with the aim of improving the performance of game playing agents in detecting forcing continuations. We present a range of measures over the induced game trees and compare them against benchmark problems in chess, observing a promising level of accuracy in matching up trap states.
Ultra-low Energy charge trap flash based synapse enabled by parasitic leakage mitigation
Venue: arXiv cs.ET 2019
Year: 2019

Brain-inspired computation promises complex cognitive tasks at biological energy efficiencies. The brain contains $10^4$ synapses per neuron. Hence, ultra-low energy, high-density synapses are needed for spiking neural networks (SNN). In this paper, we use tunneling enabled CTF (Charge Trap Flash) stack for ultra-low-energy operation (1F); Further, CTF on an SOI platform and back-to-back connected pn diode and Zener diode (2D) prevent parasitic leakage to preserve energy advantage in array operation. A bulk $100 μm $ x $100 μm$ CTF operation offers tunable, gradual conductance change $(ΔG) i.e. 10^4 $levels, which gives $100$x improvement over literature. SPICE simulations of 1F2D synapse shows ultra-low energy $(\leqslant 3 fJ/pulse)$ at 180 nm node for long-term potentiation (LTP) and depression (LTD), at 180nm node for long-term potentiation (LTP) and depression (LTD), which is comparable to energy estimate in biological synapses (10 fJ). A record low learning rate (i.e., maximum $ΔG< 1%$ of G-range) is observed - which is tunable. Excellent reliability ($>10^6 endurance cycles at full conductance swing) is observed. Such a highly energy efficient synapse with tunable learning rate on the CMOS platform is a key enabler for the human-brain-scale systems. Keywords: Spiking Neural Network; Charge trap flash, SONAS, Fowler-Nordheim Tunneling, Synapse
Rock, Rap, or Reggaeton?: Assessing Mexican Immigrants' Cultural Assimilation Using Facebook Data
Venue: arXiv cs.SI 2019
Year: 2019

The degree to which Mexican immigrants in the U.S. are assimilating culturally has been widely debated. To examine this question, we focus on musical taste, a key symbolic resource that signals the social positions of individuals. We adapt an assimilation metric from earlier work to analyze self-reported musical interests among immigrants in Facebook. We use the relative levels of interest in musical genres, where a similarity to the host population in musical preferences is treated as evidence of cultural assimilation. Contrary to skeptics of Mexican assimilation, we find significant cultural convergence even among first-generation immigrants, which problematizes their use as assimilative "benchmarks" in the literature. Further, 2nd generation Mexican Americans show high cultural convergence vis-à-vis both Anglos and African-Americans, with the exception of those who speak Spanish. Rather than conforming to a single assimilation path, our findings reveal how Mexican immigrants defy simple unilinear theoretical expectations and illuminate their uniquely heterogeneous character.
MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts
Venue: arXiv cs.CL 2019
Year: 2019

This paper presents the formal release of MedMentions, a new manually annotated resource for the recognition of biomedical concepts. What distinguishes MedMentions from other annotated biomedical corpora is its size (over 4,000 abstracts and over 350,000 linked mentions), as well as the size of the concept ontology (over 3 million concepts from UMLS 2017) and its broad coverage of biomedical disciplines. In addition to the full corpus, a sub-corpus of MedMentions is also presented, comprising annotations for a subset of UMLS 2017 targeted towards document retrieval. To encourage research in Biomedical Named Entity Recognition and Linking, data splits for training and testing are included in the release, and a baseline model and its metrics for entity linking are also described.
ERSFQ 8-bit Parallel Arithmetic Logic Unit
Venue: arXiv cs.AR 2019
Year: 2019

We have designed and tested a parallel 8-bit ERSFQ arithmetic logic unit (ALU). The ALU design employs wave-pipelined instruction execution and features modular bit-slice architecture that is easily extendable to any number of bits and adaptable to current recycling. A carry signal synchronized with an asynchronous instruction propagation provides the wave-pipeline operation of the ALU. The ALU instruction set consists of 14 arithmetical and logical instructions. It has been designed and simulated for operation up to a 10 GHz clock rate at the 10-kA/cm2 fabrication process. The ALU is embedded into a shift-register-based high-frequency testbed with on-chip clock generator to allow for comprehensive high frequency testing for all possible operands. The 8-bit ERSFQ ALU, comprising 6840 Josephson junctions, has been fabricated with MIT Lincoln Lab 10-kA/cm2 SFQ5ee fabrication process featuring eight Nb wiring layers and a high-kinetic inductance layer needed for ERSFQ technology. We evaluated the bias margins for all instructions and various operands at both low and high frequency clock. At low frequency, clock and all instruction propagation through ALU were observed with bias margins of +/-11% and +/-9%, respectively. Also at low speed, the ALU exhibited correct functionality for all arithmetical and logical instructions with +/-6% bias margins. We tested the 8-bit ALU for all instructions up to 2.8 GHz clock frequency.
Reliable State Machines: A Framework for Programming Reliable Cloud Services
Venue: arXiv cs.PL 2019
Year: 2019

Building reliable applications for the cloud is challenging because of unpredictable failures during a program's execution. This paper presents a programming framework called Reliable State Machines (RSMs), that offers fault-tolerance by construction. Using our framework, a programmer can build an application as several (possibly distributed) RSMs that communicate with each other via messages, much in the style of actor-based programming. Each RSM is additionally fault-tolerant by design and offers the illusion of being "always-alive". An RSM is guaranteed to process each input request exactly once, as one would expect in a failure-free environment. The RSM runtime automatically takes care of persisting state and rehydrating it on a failover. We present the core syntax and semantics of RSMs, along with a formal proof of failure-transparency. We provide an implementation of the RSM framework and runtime on the .NET platform for deploying services to Microsoft Azure. We carried out an extensive performance evaluation on micro-benchmarks to show that one can build high-throughput applications with RSMs. We also present a case study where we rewrote a significant part of a production cloud service using RSMs. The resulting service has simpler code and exhibits production-grade performance.
Dynamic Maintenance of the Lower Envelope of Pseudo-Lines
Venue: arXiv cs.CG 2019
Year: 2019

We present a fully dynamic data structure for the maintenance of lower envelopes of pseudo-lines. The structure has $O(\log^2 n)$ update time and $O(\log n)$ vertical ray shooting query time. To achieve this performance, we devise a new algorithm for finding the intersection between two lower envelopes of pseudo-lines in $O(\log n)$ time, using \emph{tentative} binary search; the lower envelopes are special in that at $x=-\infty$ any pseudo-line contributing to the first envelope lies below every pseudo-line contributing to the second envelope. The structure requires $O(n)$ storage space.
The State of Sparsity in Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet. Across thousands of experiments, we demonstrate that complex techniques (Molchanov et al., 2017; Louizos et al., 2017b) shown to yield high compression rates on smaller datasets perform inconsistently, and that simple magnitude pruning approaches achieve comparable or better results. Additionally, we replicate the experiments performed by (Frankle & Carbin, 2018) and (Liu et al., 2018) at scale and show that unstructured sparse architectures learned through pruning cannot be trained from scratch to the same test set performance as a model trained with joint sparsification and optimization. Together, these results highlight the need for large-scale benchmarks in the field of model compression. We open-source our code, top performing model checkpoints, and results of all hyperparameter configurations to establish rigorous baselines for future work on compression and sparsification.