OpenRaiser/Intern-Atlas huggingface dataset:
Variation-aware Binarized Memristive Networks
Venue: arXiv cs.ET 2019
Year: 2019

The quantization of weights to binary states in Deep Neural Networks (DNNs) can replace resource-hungry multiply accumulate operations with simple accumulations. Such Binarized Neural Networks (BNNs) exhibit greatly reduced resource and power requirements. In addition, memristors have been shown as promising synaptic weight elements in DNNs. In this paper, we propose and simulate novel Binarized Memristive Convolutional Neural Network (BMCNN) architectures employing hybrid weight and parameter representations. We train the proposed architectures offline and then map the trained parameters to our binarized memristive devices for inference. To take into account the variations in memristive devices, and to study their effect on the performance, we introduce variations in $R_{ON}$ and $R_{OFF}$. Moreover, we introduce means to mitigate the adverse effect of memristive variations in our proposed networks. Finally, we benchmark our BMCNNs and variation-aware BMCNNs using the MNIST dataset.
Preimplantation Blastomere Boundary Identification in HMC Microscopic Images of Early Stage Human Embryos
Venue: arXiv cs.CV 2019
Year: 2019

We present a novel method for identification of the boundary of embryonic cells (blastomeres) in Hoffman Modulation Contrast (HMC) microscopic images that are taken between day one to day three. Identification of boundaries of blastomeres is a challenging task, especially in the cases containing four or more cells. This is because these cells are bundled up tightly inside an embryo's membrane and any 2D image projection of such 3D embryo includes cell overlaps, occlusions, and projection ambiguities. Moreover, human embryos include fragmentation, which does not conform to any specific patterns or shape. Here we developed a model-based iterative approach, in which blastomeres are modeled as ellipses that conform to the local image features, such as edges and normals. In an iterative process, each image feature contributes only to one candidate and is removed upon being associated to a model candidate. We have tested the proposed algorithm on an image dataset comprising of 468 human embryos obtained from different sources. An overall Precision, Sensitivity and Overall Quality (OQ) of 92%, 88% and 83% are achieved.
Federated Transfer Reinforcement Learning for Autonomous Driving
Venue: arXiv cs.LG 2019
Year: 2019

Reinforcement learning (RL) is widely used in autonomous driving tasks and training RL models typically involves in a multi-step process: pre-training RL models on simulators, uploading the pre-trained model to real-life robots, and fine-tuning the weight parameters on robot vehicles. This sequential process is extremely time-consuming and more importantly, knowledge from the fine-tuned model stays local and can not be re-used or leveraged collaboratively. To tackle this problem, we present an online federated RL transfer process for real-time knowledge extraction where all the participant agents make corresponding actions with the knowledge learned by others, even when they are acting in very different environments. To validate the effectiveness of the proposed approach, we constructed a real-life collision avoidance system with Microsoft Airsim simulator and NVIDIA JetsonTX2 car agents, which cooperatively learn from scratch to avoid collisions in indoor environment with obstacle objects. We demonstrate that with the proposed framework, the simulator car agents can transfer knowledge to the RC cars in real-time, with 27% increase in the average distance with obstacles and 42% decrease in the collision counts.
Imitating by generating: deep generative models for imitation of interactive tasks
Venue: arXiv cs.LG 2019
Year: 2019

To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals. Humans acquire these skills in infancy and early childhood mostly by imitation learning and active engagement with a skilled partner. They require the ability to predict and adapt to one's partner during an interaction. In this work we want to explore these ideas in a human-robot interaction setting in which a robot is required to learn interactive tasks from a combination of observational and kinesthetic learning. To this end, we propose a deep learning framework consisting of a number of components for (1) human and robot motion embedding, (2) motion prediction of the human partner and (3) generation of robot joint trajectories matching the human motion. To test these ideas, we collect human-human interaction data and human-robot interaction data of four interactive tasks "hand-shake", "hand-wave", "parachute fist-bump" and "rocket fist-bump". We demonstrate experimentally the importance of predictive and adaptive components as well as low-level abstractions to successfully learn to imitate human behavior in interactive social tasks.
Scalability of TTool's AMS extensions: a case study
Venue: arXiv cs.OH 2019
Year: 2019

Embedded cyber-physical systems (CPS) are commonly built upon heterogeneous digital and analog integrated circuits, including sensors and actuators. Less common is their deployment on parallel, NoC based designs based on general purpose processor cores of a Multi-processor System-on-chip (MPSoC). Application code has to be run on the MPSoC for the digital part, and interact with the analog sensors. We recently proposed a major extension to the design and exploration tool named TTool, now allowing the design of CPS on a high level of abstraction and the generation of cycle-bit accurate simulations. We explore the scalability of our approach with an automotive case study.
ReActNet: Temporal Localization of Repetitive Activities in Real-World Videos
Venue: arXiv cs.CV 2019
Year: 2019

We address the problem of temporal localization of repetitive activities in a video, i.e., the problem of identifying all segments of a video that contain some sort of repetitive or periodic motion. To do so, the proposed method represents a video by the matrix of pairwise frame distances. These distances are computed on frame representations obtained with a convolutional neural network. On top of this representation, we design, implement and evaluate ReActNet, a lightweight convolutional neural network that classifies a given frame as belonging (or not) to a repetitive video segment. An important property of the employed representation is that it can handle repetitive segments of arbitrary number and duration. Furthermore, the proposed training process requires a relatively small number of annotated videos. Our method raises several of the limiting assumptions of existing approaches regarding the contents of the video and the types of the observed repetitive activities. Experimental results on recent, publicly available datasets validate our design choices, verify the generalization potential of ReActNet and demonstrate its superior performance in comparison to the current state of the art.
Some remarks on the performance of Matlab, Python and Octave in simulating dynamical systems
Venue: arXiv cs.MS 2019
Year: 2019

Matlab has been considered as a leader computational platform for many engineering fields. Well documented and reliable, Matlab presents as a great advantage its ability to increase the user productivity. However, Python and Octave are among some of the languages that have challenged Matlab. Octave and Python are well known examples of high-level scripting languages, with a great advantage of being open source software. The novelty of this paper is devoted to offer a comparison among these tree languages in the simulation of dynamical systems. We have applied the lower bound error to estimate the error of simulation. The comparison was performed with the chaotic systems Duffing-Ueda oscillator and the Chua's circuit, both identified with polynomial NARMAX. Octave presents the best reliable outcome. Nevertheless, Matlab needs the lowest time to undertake the same activity. Python has presented the worse result for the stop simulation criterion.
Estimating post-editing effort: a study on human judgements, task-based and reference-based metrics of MT quality
Venue: arXiv cs.CL 2019
Year: 2019

Devising metrics to assess translation quality has always been at the core of machine translation (MT) research. Traditional automatic reference-based metrics, such as BLEU, have shown correlations with human judgements of adequacy and fluency and have been paramount for the advancement of MT system development. Crowd-sourcing has popularised and enabled the scalability of metrics based on human judgements, such as subjective direct assessments (DA) of adequacy, that are believed to be more reliable than reference-based automatic metrics. Finally, task-based measurements, such as post-editing time, are expected to provide a more detailed evaluation of the usefulness of translations for a specific task. Therefore, while DA averages adequacy judgements to obtain an appraisal of (perceived) quality independently of the task, and reference-based automatic metrics try to objectively estimate quality also in a task-independent way, task-based metrics are measurements obtained either during or after performing a specific task. In this paper we argue that, although expensive, task-based measurements are the most reliable when estimating MT quality in a specific task; in our case, this task is post-editing. To that end, we report experiments on a dataset with newly-collected post-editing indicators and show their usefulness when estimating post-editing effort. Our results show that task-based metrics comparing machine-translated and post-edited versions are the best at tracking post-editing effort, as expected. These metrics are followed by DA, and then by metrics comparing the machine-translated version and independent references. We suggest that MT practitioners should be aware of these differences and acknowledge their implications when deciding how to evaluate MT for post-editing purposes.
Understanding The Characteristics, Benefits And Challenges Of Agile IT Project Management: A Literature Based Perspective
Venue: arXiv cs.SE 2019
Year: 2019

The objectives of this study was to bring out the understanding of the concept of agile IT project management; what it is and what it is not. It was also aimed at comparing the pros and cons of both agile and traditional methods of IT project management in a typical industry setting; the challenges of going purely agile, and so on. It is purely a review of literature of peer reviewed papers sourced mainly from Google Scholar. It was revealed that agile outweigh the traditional methods in terms of benefits, but its implementation poses a lot of challenges due to a number of issues, paramount among them being organizational culture and empowerment of the project team. This has resulted in a number of industries sticking to the traditional methods despite the overwhelming benefits of agile. In another school of thought, the combination of the two paradigms is the way forward.
Monkey Optimization System with Active Membranes: A New Meta-heuristic Optimization System
Venue: arXiv cs.NE 2019
Year: 2019

Optimization techniques, used to get the optimal solution in search spaces, have not solved the time-consuming problem. The objective of this study is to tackle the sequential processing problem in Monkey Algorithm and simulating the natural parallel behavior of monkeys. Therefore, a P system with active membranes is constructed by providing a codification for Monkey Algorithm within the context of a cell-like P system, defining accordingly the elements of the model - membrane structure, objects, rules and the behavior of it. The proposed algorithm has modeled the natural behavior of climb process using separate membranes, rather than the original algorithm. Moreover, it introduced the membrane migration process to select the best solution and the time stamp was added as an additional stopping criterion to control the timing of the algorithm. The results indicate a substantial solution for the time consumption problem, significant representation of the natural behavior of monkeys, and considerable chance to reach the best solution in the context of meta-heuristics purpose. In addition, experiments use the commonly used benchmark functions to test the performance of the algorithm as well as the expected time of the proposed P Monkey optimization algorithm and the traditional Monkey Algorithm running on population size. The unit times are calculated based on the complexity of algorithms, where P Monkey takes a time unit to fire rule(s) over a population size n; as soon as, Monkey Algorithm takes a time unit to run a step every mathematical equation over a population size.
Training Compact Models for Low Resource Entity Tagging using Pre-trained Language Models
Venue: arXiv cs.CL 2019
Year: 2019

Training models on low-resource named entity recognition tasks has been shown to be a challenge, especially in industrial applications where deploying updated models is a continuous effort and crucial for business operations. In such cases there is often an abundance of unlabeled data, while labeled data is scarce or unavailable. Pre-trained language models trained to extract contextual features from text were shown to improve many natural language processing (NLP) tasks, including scarcely labeled tasks, by leveraging transfer learning. However, such models impose a heavy memory and computational burden, making it a challenge to train and deploy such models for inference use. In this work-in-progress we combined the effectiveness of transfer learning provided by pre-trained masked language models with a semi-supervised approach to train a fast and compact model using labeled and unlabeled examples. Preliminary evaluations show that the compact models can achieve competitive accuracy with 36x compression rate when compared with a state-of-the-art pre-trained language model, and run significantly faster in inference, allowing deployment of such models in production environments or on edge devices.
Decentralized On-line Task Reallocation on Parallel Computing Architectures with Safety-Critical Applications
Venue: arXiv cs.DC 2019
Year: 2019

This work presents a decentralized allocation algorithm of safety-critical application on parallel computing architectures, where individual Computational Units can be affected by faults. The described method consists in representing the architecture by an abstract graph where each node represents a Computational Unit. Applications are also represented by the graph of Computational Units they require for execution. The problem is then to decide how to allocate Computational Units to applications to guarantee execution of the safety-critical application. The problem is formulated as an optimization problem, with the form of an Integer Linear Program. A state-of-the-art solver is then used to solve the problem. Decentralizing the allocation process is achieved through redundancy of the allocator executed on the architecture. No centralized element decides on the allocation of the entire architecture, thus improving the reliability of the system. Experimental reproduction of a multi-core architecture is also presented. It is used to demonstrate the capabilities of the proposed allocation process to maintain the operation of a physical system in a decentralized way while individual component fails.
BACKUS: Comprehensive High-Performance Research Software Engineering Approach for Simulations in Supercomputing Systems
Venue: arXiv cs.DC 2019
Year: 2019

High-Performance Computing (HPC) platforms enable scientific software to achieve breakthroughs in many research fields such as physics, biology, and chemistry, by employing Research Software Engineering (RSE) techniques. These include 1) novel parallelism paradigms such as Shared Memory Parallelism (with e.g. OpenMP 4.5); Distributed Memory Parallelism (with e.g. MPI 4); Hybrid Parallelism which combines them; and Heterogeneous Parallelism (for CPUs, co-processors and accelerators), 2) introducing advanced Software Engineering concepts such as Object Oriented Parallel Programming (OOPP); Parallel Unit testing; Parallel I/O Formats; Hybrid Parallel Visualization; and 3) Selecting the Best Practices in other necessary areas such as User Interface; Automatic Documentation; Version Control and Project Management. In this work we present BACKUS: Comprehensive High-Performance Research Software Engineering Approach for Simulations in Supercomputing Systems, which we found to fit best for long-lived parallel scientific codes.
Restoration of marker occluded hematoxylin and eosin stained whole slide histology images using generative adversarial networks
Venue: arXiv cs.CV 2019
Year: 2019

It is common for pathologists to annotate specific regions of the tissue, such as tumor, directly on the glass slide with markers. Although this practice was helpful prior to the advent of histology whole slide digitization, it often occludes important details which are increasingly relevant to immuno-oncology due to recent advancements in digital pathology imaging techniques. The current work uses a generative adversarial network with cycle loss to remove these annotations while still maintaining the underlying structure of the tissue by solving an image-to-image translation problem. We train our network on up to 300 whole slide images with marker inks and show that 70% of the corrected image patches are indistinguishable from originally uncontaminated image tissue to a human expert. This portion increases 97% when we replace the human expert with a deep residual network. We demonstrated the fidelity of the method to the original image by calculating the correlation between image gradient magnitudes. We observed a revival of up to 94,000 nuclei per slide in our dataset, the majority of which were located on tissue border.
End-to-End Adversarial Shape Learning for Abdomen Organ Deep Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Automatic segmentation of abdomen organs using medical imaging has many potential applications in clinical workflows. Recently, the state-of-the-art performance for organ segmentation has been achieved by deep learning models, i.e., convolutional neural network (CNN). However, it is challenging to train the conventional CNN-based segmentation models that aware of the shape and topology of organs. In this work, we tackle this problem by introducing a novel end-to-end shape learning architecture -- organ point-network. It takes deep learning features as inputs and generates organ shape representations as points that located on organ surface. We later present a novel adversarial shape learning objective function to optimize the point-network to capture shape information better. We train the point-network together with a CNN-based segmentation model in a multi-task fashion so that the shared network parameters can benefit from both shape learning and segmentation tasks. We demonstrate our method with three challenging abdomen organs including liver, spleen, and pancreas. The point-network generates surface points with fine-grained details and it is found critical for improving organ segmentation. Consequently, the deep segmentation model is improved by the introduced shape learning as significantly better Dice scores are observed for spleen and pancreas segmentation.
Spatial Data Science: Closing the human-spatial computing-environment loop
Venue: arXiv cs.SI 2019
Year: 2019

Over the last decade, the term spatial computing has grown to have two different, though not entirely unrelated, definitions. The first definition of spatial computing stems from industry, where it refers primarily to new kinds of augmented, virtual, mixed-reality, and natural user interface technologies. A second definition coming out of academia takes a broader perspective that includes active research in geographic information science as well as the aforementioned novel UI technologies. Both senses reflect an ongoing shift toward increased interaction with computing interfaces and sensors embedded in the environment and how the use of these technologies influence how we behave and make sense of and even change the world we live in. Regardless of the definition, research in spatial computing is humming along nicely without the need to identify new research agendas or new labels for communities of researchers. However, as a field of research, it could be helpful to view spatial data science as the glue that coheres spatial computing with problem-solving and learning in the real world into a more holistic discipline.
Optimal Codes Correcting a Single Indel / Edit for DNA-Based Data Storage
Venue: arXiv cs.IT 2019
Year: 2019

An indel refers to a single insertion or deletion, while an edit refers to a single insertion, deletion or substitution. In this paper, we investigate codes that combat either a single indel or a single edit and provide linear-time algorithms that encode binary messages into these codes of length n. Over the quaternary alphabet, we provide two linear-time encoders. One corrects a single edit with log n + O(log log n) redundancy bits, while the other corrects a single indel with log n + 2 redundant bits. These two encoders are order-optimal. The former encoder is the first known order-optimal encoder that corrects a single edit, while the latter encoder (that corrects a single indel) reduces the redundancy of the best known encoder of Tenengolts (1984) by at least four bits. Over the DNA alphabet, we impose an additional constraint: the GC-balanced constraint and require that exactly half of the symbols of any DNA codeword to be either C or G. In particular, via a modification of Knuth's balancing technique, we provide a linear-time map that translates binary messages into GC-balanced codewords and the resulting codebook is able to correct a single indel or a single edit. These are the first known constructions of GC-balanced codes that correct a single indel or a single edit.
Shapley Homology: Topological Analysis of Sample Influence for Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Data samples collected for training machine learning models are typically assumed to be independent and identically distributed (iid). Recent research has demonstrated that this assumption can be problematic as it simplifies the manifold of structured data. This has motivated different research areas such as data poisoning, model improvement, and explanation of machine learning models. In this work, we study the influence of a sample on determining the intrinsic topological features of its underlying manifold. We propose the Shapley Homology framework, which provides a quantitative metric for the influence of a sample of the homology of a simplicial complex. By interpreting the influence as a probability measure, we further define an entropy which reflects the complexity of the data manifold. Our empirical studies show that when using the 0-dimensional homology, on neighboring graphs, samples with higher influence scores have more impact on the accuracy of neural networks for determining the graph connectivity and on several regular grammars whose higher entropy values imply more difficulty in being learned.