OpenRaiser/Intern-Atlas huggingface dataset:
Channel Impulse Analysis of Light Propagation for Point-to-point Nano Communications through Cortical Neurons
Venue: arXiv cs.ET 2019
Year: 2019

Recent Brain-Machine Interfaces have shifted towards miniature devices that are constructed from nanoscale components. While these devices can be implanted into the brain, their functionalities can be limited, and will require communication and networking to enable cooperation. One form of communication for neuron stimulation is the use of light. A number of considerations needs to be taken into account for the propagation and this includes diffraction, scattering, absorption, as well as attenuation. These properties are not only affected by the medium, but also by the cell's geometric shape. These factor affects both the direction and amplitude of the light wave. This paper analyzes the propagation path loss and geometrical gain, channel impulse and frequency response for light propagation along the neural tissue. The total attenuation depends on the propagation medium loss and geometrical gain, and the channel response is highly dependent on the quantity of cells along the path. Additionally, the optical properties of the medium also impacts on the time delay at the receiver and the width the location of the detectors. Based on the numerical analysis, spherical cells attenuate approximately 20% of the transmitted power, which is less than the fusiform and pyramidal cells (35% and 65%, respectively).
Transformer-based Cascaded Multimodal Speech Translation
Venue: arXiv cs.CL 2019
Year: 2019

This paper describes the cascaded multimodal speech translation systems developed by Imperial College London for the IWSLT 2019 evaluation campaign. The architecture consists of an automatic speech recognition (ASR) system followed by a Transformer-based multimodal machine translation (MMT) system. While the ASR component is identical across the experiments, the MMT model varies in terms of the way of integrating the visual context (simple conditioning vs. attention), the type of visual features exploited (pooled, convolutional, action categories) and the underlying architecture. For the latter, we explore both the canonical transformer and its deliberation version with additive and cascade variants which differ in how they integrate the textual attention. Upon conducting extensive experiments, we found that (i) the explored visual integration schemes often harm the translation performance for the transformer and additive deliberation, but considerably improve the cascade deliberation; (ii) the transformer and cascade deliberation integrate the visual modality better than the additive deliberation, as shown by the incongruence analysis.
Region-based Convolution Neural Network Approach for Accurate Segmentation of Pelvic Radiograph
Venue: arXiv cs.CV 2019
Year: 2019

With the increasing usage of radiograph images as a most common medical imaging system for diagnosis, treatment planning, and clinical studies, it is increasingly becoming a vital factor to use machine learning-based systems to provide reliable information for surgical pre-planning. Segmentation of pelvic bone in radiograph images is a critical preprocessing step for some applications such as automatic pose estimation and disease detection. However, the encoder-decoder style network known as U-Net has demonstrated limited results due to the challenging complexity of the pelvic shapes, especially in severe patients. In this paper, we propose a novel multi-task segmentation method based on Mask R-CNN architecture. For training, the network weights were initialized by large non-medical dataset and fine-tuned with radiograph images. Furthermore, in the training process, augmented data was generated to improve network performance. Our experiments show that Mask R-CNN utilizing multi-task learning, transfer learning, and data augmentation techniques achieve 0.96 DICE coefficient, which significantly outperforms the U-Net. Notably, for a fair comparison, the same transfer learning and data augmentation techniques have been used for U-net training.
Detecting motorcycle helmet use with deep learning
Venue: arXiv cs.CV 2019
Year: 2019

The continuous motorization of traffic has led to a sustained increase in the global number of road related fatalities and injuries. To counter this, governments are focusing on enforcing safe and law-abiding behavior in traffic. However, especially in developing countries where the motorcycle is the main form of transportation, there is a lack of comprehensive data on the safety-critical behavioral metric of motorcycle helmet use. This lack of data prohibits targeted enforcement and education campaigns which are crucial for injury prevention. Hence, we have developed an algorithm for the automated registration of motorcycle helmet usage from video data, using a deep learning approach. Based on 91,000 annotated frames of video data, collected at multiple observation sites in 7 cities across the country of Myanmar, we trained our algorithm to detect active motorcycles, the number and position of riders on the motorcycle, as well as their helmet use. An analysis of the algorithm's accuracy on an annotated test data set, and a comparison to available human-registered helmet use data reveals a high accuracy of our approach. Our algorithm registers motorcycle helmet use rates with an accuracy of -4.4% and +2.1% in comparison to a human observer, with minimal training for individual observation sites. Without observation site specific training, the accuracy of helmet use detection decreases slightly, depending on a number of factors. Our approach can be implemented in existing roadside traffic surveillance infrastructure and can facilitate targeted data-driven injury prevention campaigns with real-time speed. Implications of the proposed method, as well as measures that can further improve detection accuracy are discussed.
Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets
Venue: arXiv cs.CV 2019
Year: 2019

Recent advances in computer vision and deep learning have led to breakthroughs in the development of automated skin image analysis. In particular, skin cancer classification models have achieved performance higher than trained expert dermatologists. However, no attempt has been made to evaluate the consistency in performance of machine learning models across populations with varying skin tones. In this paper, we present an approach to estimate skin tone in benchmark skin disease datasets, and investigate whether model performance is dependent on this measure. Specifically, we use individual typology angle (ITA) to approximate skin tone in dermatology datasets. We look at the distribution of ITA values to better understand skin color representation in two benchmark datasets: 1) the ISIC 2018 Challenge dataset, a collection of dermoscopic images of skin lesions for the detection of skin cancer, and 2) the SD-198 dataset, a collection of clinical images capturing a wide variety of skin diseases. To estimate ITA, we first develop segmentation models to isolate non-diseased areas of skin. We find that the majority of the data in the the two datasets have ITA values between 34.5° and 48°, which are associated with lighter skin, and is consistent with under-representation of darker skinned populations in these datasets. We also find no measurable correlation between performance of machine learning model and ITA values, though more comprehensive data is needed for further validation.
Secure and Efficient Federated Transfer Learning
Venue: arXiv cs.CR 2019
Year: 2019

Machine Learning models require a vast amount of data for accurate training. In reality, most data is scattered across different organizations and cannot be easily integrated under many legal and practical constraints. Federated Transfer Learning (FTL) was introduced in [1] to improve statistical models under a data federation that allow knowledge to be shared without compromising user privacy, and enable complementary knowledge to be transferred in the network. As a result, a target-domain party can build more flexible and powerful models by leveraging rich labels from a source-domain party. However, the excessive computational overhead of the security protocol involved in this model rendered it impractical. In this work, we aim towards enhancing the efficiency and security of existing models for practical collaborative training under a data federation by incorporating Secret Sharing (SS). In literature, only the semi-honest model for Federated Transfer Learning has been considered. In this paper, we improve upon the previous solution, and also allow malicious players who can arbitrarily deviate from the protocol in our FTL model. This is much stronger than the semi-honest model where we assume that parties follow the protocol precisely. We do so using the one of the practical MPC protocol called SPDZ, thus our model can be efficiently extended to any number of parties even in the case of a dishonest majority. In addition, the models evaluated in our setting significantly outperform the previous work, in terms of both runtime and communication cost. A single iteration in our model executes in 0.8 seconds for the semi-honest case and 1.4 seconds for the malicious case for 500 samples, as compared to 35 seconds taken by the previous implementation.
Symbolic Graph Embedding using Frequent Pattern Mining
Venue: arXiv cs.LG 2019
Year: 2019

Relational data mining is becoming ubiquitous in many fields of study. It offers insights into behaviour of complex, real-world systems which cannot be modeled directly using propositional learning. We propose Symbolic Graph Embedding (SGE), an algorithm aimed to learn symbolic node representations. Built on the ideas from the field of inductive logic programming, SGE first samples a given node's neighborhood and interprets it as a transaction database, which is used for frequent pattern mining to identify logical conjuncts of items that co-occur frequently in a given context. Such patterns are in this work used as features to represent individual nodes, yielding interpretable, symbolic node embeddings. The proposed SGE approach on a venue classification task outperforms shallow node embedding methods such as DeepWalk, and performs similarly to metapath2vec, a black-box representation learner that can exploit node and edge types in a given graph. The proposed SGE approach performs especially well when small amounts of data are used for learning, scales to graphs with millions of nodes and edges, and can be run on an of-the-shelf laptop.
Towards Deep Physical Reservoir Computing Through Automatic Task Decomposition And Mapping
Venue: arXiv cs.LG 2019
Year: 2019

Photonic reservoir computing is a promising candidate for low-energy computing at high bandwidths. Despite recent successes, there are bounds to what one can achieve simply by making photonic reservoirs larger. Therefore, a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that backpropagation can not be used directly to train multi-reservoir systems in our targeted setting, we propose an alternative approach that still uses its power to derive intermediate targets. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach by training a network of 3 Echo State Networks to perform the well-known NARMA-10 task using targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in a efficient way.
Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization
Venue: arXiv cs.RO 2019
Year: 2019

In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment. In real-world applications, test conditions may differ substantially from the training scenario and, therefore, focusing on pure reward maximization during training may lead to poor results at test time. In these cases, it is important to trade-off between performance and robustness while learning a policy. While several results exist for robust, model-based RL, the model-free case has not been widely investigated. In this paper, we cast the robust, model-free RL problem as a multi-objective optimization problem. To quantify the robustness of a policy, we use delay margin and gain margin, two robustness indicators that are common in control theory. We show how these metrics can be estimated from data in the model-free setting. We use multi-objective Bayesian optimization (MOBO) to solve efficiently this expensive-to-evaluate, multi-objective optimization problem. We show the benefits of our robust formulation both in sim-to-real and pure hardware experiments to balance a Furuta pendulum.
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications
Venue: arXiv cs.LG 2019
Year: 2019

We develop techniques to quantify the degree to which a given (training or testing) example is an outlier in the underlying distribution. We evaluate five methods to score examples in a dataset by how well-represented the examples are, for different plausible definitions of "well-represented", and apply these to four common datasets: MNIST, Fashion-MNIST, CIFAR-10, and ImageNet. Despite being independent approaches, we find all five are highly correlated, suggesting that the notion of being well-represented can be quantified. Among other uses, we find these methods can be combined to identify (a) prototypical examples (that match human expectations); (b) memorized training examples; and, (c) uncommon submodes of the dataset. Further, we show how we can utilize our metrics to determine an improved ordering for curriculum learning, and impact adversarial robustness. We release all metric values on training and test sets we studied.
Weight of Evidence as a Basis for Human-Oriented Explanations
Venue: arXiv cs.LG 2019
Year: 2019

Interpretability is an elusive but highly sought-after characteristic of modern machine learning methods. Recent work has focused on interpretability via $\textit{explanations}$, which justify individual model predictions. In this work, we take a step towards reconciling machine explanations with those that humans produce and prefer by taking inspiration from the study of explanation in philosophy, cognitive science, and the social sciences. We identify key aspects in which these human explanations differ from current machine explanations, distill them into a list of desiderata, and formalize them into a framework via the notion of $\textit{weight of evidence}$ from information theory. Finally, we instantiate this framework in two simple applications and show it produces intuitive and comprehensible explanations.
A Tunably Compliant Origami Mechanism for Dynamically Dexterous Robots
Venue: arXiv cs.RO 2019
Year: 2019

We present an approach to overcoming challenges in dynamical dexterity for robots through tunable origami structures. Our work leverages a one-parameter family of flat sheet crease patterns that folds into origami bellows, whose axial compliance can be tuned to select desired stiffness. Concentrically arranged cylinder pairs reliably manifest additive stiffness, extending the tunable range by nearly an order of magnitude and achieving bulk axial stiffness spanning 200-1500 N/m using 8 mil thick polyester-coated paper. Accordingly, we design origami energy-storing springs with a stiffness of 1035 N/m each and incorporate them into a three degree-of-freedom (DOF) tendon-driven spatial pointing mechanism that exhibits trajectory tracking accuracy less than 15% rms error within a ~2 cm^3 volume. The origami springs can sustain high power throughput, enabling the robot to achieve asymptotically stable juggling for both highly elastic (1~kg resilient shot put ball) and highly damped ("medicine ball") collisions in the vertical direction with apex heights approaching 10 cm. The results demonstrate that "soft" robotic mechanisms are able to perform a controlled, dynamically actuated task.
LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese
Venue: arXiv cs.CL 2019
Year: 2019

In Vietnamese dependency parsing, several methods have been proposed. Dependency parser which uses deep neural network model has been reported that achieved state-of-the-art results. In this paper, we proposed a new method which applies LSTM easy-first dependency parsing with pre-trained word embeddings and character-level word embeddings. Our method achieves an accuracy of 80.91% of unlabeled attachment score and 72.98% of labeled attachment score on the Vietnamese Dependency Treebank (VnDT).
A CNN-based methodology for breast cancer diagnosis using thermal images
Venue: arXiv cs.CV 2019
Year: 2019

Micro Abstract: A recent study from GLOBOCAN disclosed that during 2018 two million women worldwide had been diagnosed from breast cancer. This study presents a computer-aided diagnosis system based on convolutional neural networks as an alternative diagnosis methodology for breast cancer diagnosis with thermal images. Experimental results showed that lower false-positives and false-negatives classification rates are obtained when data pre-processing and data augmentation techniques are implemented in these thermal images. Background: There are many types of breast cancer screening techniques such as, mammography, magnetic resonance imaging, ultrasound and blood sample tests, which require either, expensive devices or personal qualified. Currently, some countries still lack access to these main screening techniques due to economic, social or cultural issues. The objective of this study is to demonstrate that computer-aided diagnosis(CAD) systems based on convolutional neural networks (CNN) are faster, reliable and robust than other techniques. Methods: We performed a study of the influence of data pre-processing, data augmentation and database size versus a proposed set of CNN models. Furthermore, we developed a CNN hyper-parameters fine-tuning optimization algorithm using a tree parzen estimator. Results: Among the 57 patients database, our CNN models obtained a higher accuracy (92\%) and F1-score (92\%) that outperforms several state-of-the-art architectures such as ResNet50, SeResNet50 and Inception. Also, we demonstrated that a CNN model that implements data-augmentation techniques reach identical performance metrics in comparison with a CNN that uses a database up to 50\% bigger. Conclusion: This study highlights the benefits of data augmentation and CNNs in thermal breast images. Also, it measures the influence of the database size in the performance of CNNs.
Rational Kernels: A survey
Venue: arXiv cs.LG 2019
Year: 2019

Many kinds of data are naturally amenable to being treated as sequences. An example is text data, where a text may be seen as a sequence of words. Another example is clickstream data, where a data instance is a sequence of clicks made by a visitor to a website. This is also common for data originating in the domains of speech processing and computational biology. Using such data with statistical learning techniques can often prove to be cumbersome since most of them only allow fixed-length feature vectors as input. In casting the data to fixed-length feature vectors to suit these techniques, we lose the convenience, and possibly information, a good sequence-based representation can offer. The framework of rational kernels partly addresses this problem by providing an elegant representation for sequences, for algorithms that use kernel functions. In this report, we take a comprehensive look at this framework, its various extensions and applications. We start with an overview of the core ideas, where we look at the characterization of rational kernels, and then extend our discussion to extensions, applications and use at scale. Rational kernels represent a family of kernels, and thus, learning an appropriate rational kernel instead of picking one, suggests a convenient way to use them; we explore this idea in our concluding section. Rational kernels are not as popular as the many other learning techniques in use today; however, we hope that this summary effectively shows that not only is their theory well-developed, but also that various practical aspects have been carefully studied over time.
How Coupled are Mass Spectrometry and Capillary Electrophoresis?
Venue: arXiv cs.DL 2019
Year: 2019

The understanding of how science works can contribute to making scientific development more effective. In this paper, we report an analysis of the organization and interconnection between two important issues in chemistry, namely mass spectrometry (MS) and capillary electrophoresis (CE). For that purpose, we employed science of science techniques based on complex networks. More specifically, we considered a citation network in which the nodes and connections represent papers and citations, respectively. Interesting results were found, including a good separation between some clusters of articles devoted to instrumentation techniques and applications. However, the papers that describe CE-MS did not lead to a well-defined cluster. In order to better understand the organization of the citation network, we considered a multi-scale analysis, in which we used the information regarding sub-clusters. Firstly, we analyzed the sub-cluster of the first article devoted to the coupling between CE and MS, which was found to be a good representation of its sub-cluster. The second analysis was about the sub-cluster of a seminal paper known to be the first that dealt with proteins by using CE-MS. By considering the proposed methodologies, our paper paves the way for researchers working with both techniques, since it elucidates the knowledge organization and can therefore lead to better literature reviews.
Predicting Rainfall using Machine Learning Techniques
Venue: arXiv cs.LG 2019
Year: 2019

Rainfall prediction is one of the challenging and uncertain tasks which has a significant impact on human society. Timely and accurate predictions can help to proactively reduce human and financial loss. This study presents a set of experiments which involve the use of prevalent machine learning techniques to build models to predict whether it is going to rain tomorrow or not based on weather data for that particular day in major cities of Australia. This comparative study is conducted concentrating on three aspects: modeling inputs, modeling methods, and pre-processing techniques. The results provide a comparison of various evaluation metrics of these machine learning techniques and their reliability to predict the rainfall by analyzing the weather data.
Rate Analysis of Ultra-Reliable Low-Latency Communications in Random Wireless Networks
Venue: arXiv cs.IT 2019
Year: 2019

In this letter, we analyze the achievable rate of ultra-reliable low-latency communications (URLLC) in a randomly modeled wireless network. We use two mathematical tools to properly characterize the considered system: i) stochastic geometry to model spatial locations of the transmitters in a network, and ii) finite block-length analysis to reflect the features of the short-packets. Exploiting these tools, we derive an integral-form expression of the decoding error probability as a function of the target rate, the path-loss exponent, the communication range, the density, and the channel coding length. We also obtain a tight approximation as a closed-form. The main finding from the analytical results is that, in URLLC, increasing the signal-to-interference ratio (SIR) brings significant improvement of the rate performance compared to increasing the channel coding length. Via simulations, we show that fractional frequency reuse improves the area spectral efficiency by reducing the amount of mutual interference.