OpenRaiser/Intern-Atlas huggingface dataset:
Reproducibility Evaluation of SLANT Whole Brain Segmentation Across Clinical Magnetic Resonance Imaging Protocols
Venue: arXiv cs.CV 2019
Year: 2019

Whole brain segmentation on structural magnetic resonance imaging (MRI) is essential for understanding neuroanatomical-functional relationships. Traditionally, multi-atlas segmentation has been regarded as the standard method for whole brain segmentation. In past few years, deep convolutional neural network (DCNN) segmentation methods have demonstrated their advantages in both accuracy and computational efficiency. Recently, we proposed the spatially localized atlas network tiles (SLANT) method, which is able to segment a 3D MRI brain scan into 132 anatomical regions. Commonly, DCNN segmentation methods yield inferior performance under external validations, especially when the testing patterns were not presented in the training cohorts. Recently, we obtained a clinically acquired, multi-sequence MRI brain cohort with 1480 clinically acquired, de-identified brain MRI scans on 395 patients using seven different MRI protocols. Moreover, each subject has at least two scans from different MRI protocols. Herein, we assess the SLANT method's intra- and inter-protocol reproducibility. SLANT achieved less than 0.05 coefficient of variation (CV) for intra-protocol experiments and less than 0.15 CV for inter-protocol experiments. The results show that the SLANT method achieved high intra- and inter- protocol reproducibility.
Sinusoidal wave generating network based on adversarial learning and its application: synthesizing frog sounds for data augmentation
Venue: arXiv cs.SD 2019
Year: 2019

Simulators that generate observations based on theoretical models can be important tools for development, prediction, and assessment of signal processing algorithms. In order to design these simulators, painstaking effort is required to construct mathematical models according to their application. Complex models are sometimes necessary to represent a variety of real phenomena. In contrast, obtaining synthetic observations from generative models developed from real observations often require much less effort. This paper proposes a generative model based on adversarial learning. Given that observations are typically signals composed of a linear combination of sinusoidal waves and random noises, sinusoidal wave generating networks are first designed based on an adversarial network. Audio waveform generation can then be performed using the proposed network. Several approaches to designing the objective function of the proposed network using adversarial learning are investigated experimentally. In addition, amphibian sound classification is performed using a convolutional neural network trained with real and synthetic sounds. Both qualitative and quantitative results show that the proposed generative model makes realistic signals and is very helpful for data augmentation and data analysis.
Team EP at TAC 2018: Automating data extraction in systematic reviews of environmental agents
Venue: arXiv cs.CL 2019
Year: 2019

We describe our entry for the Systematic Review Information Extraction track of the 2018 Text Analysis Conference. Our solution is an end-to-end, deep learning, sequence tagging model based on the BI-LSTM-CRF architecture. However, we use interleaved, alternating LSTM layers with highway connections instead of the more traditional approach, where last hidden states of both directions are concatenated to create an input to the next layer. We also make extensive use of pre-trained word embeddings, namely GloVe and ELMo. Thanks to a number of regularization techniques, we were able to achieve relatively large capacity of the model (31.3M+ of trainable parameters) for the size of training set (100 documents, less than 200K tokens). The system's official score was 60.9% (micro-F1) and it ranked first for the Task 1. Additionally, after rectifying an obvious mistake in the submission format, the system scored 67.35%.
Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning solutions are being increasingly used in mobile applications. Although there are many open-source software tools for the development of deep learning solutions, there are no guidelines in one place in a unified manner for using these tools towards real-time deployment of these solutions on smartphones. From the variety of available deep learning tools, the most suited ones are used in this paper to enable real-time deployment of deep learning inference networks on smartphones. A uniform flow of implementation is devised for both Android and iOS smartphones. The advantage of using multi-threading to achieve or improve real-time throughputs is also showcased. A benchmarking framework consisting of accuracy, CPU/GPU consumption and real-time throughput is considered for validation purposes. The developed deployment approach allows deep learning models to be turned into real-time smartphone apps with ease based on publicly available deep learning and smartphone software tools. This approach is applied to six popular or representative convolutional neural network models and the validation results based on the benchmarking metrics are reported.
Audio Captcha Recognition Using RastaPLP Features by SVM
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays, CAPTCHAs are computer generated tests that human can pass but current computer systems can not. They have common usage in various web services in order to be able to detect a human from computer programs autonomously. In this way, owners can protect their web services from bots. In addition to visual CAPTCHAs which consist of distorted images, mostly test images, that a user must write some description about that image, there are a significant amount of audio CAPTCHAs as well. Briefly, audio CAPTCHAs are sound files which consist of human sound under heavy noise where the speaker pronounces a bunch of digits consecutively. Generally, in those sound files, there are some periodic and non-periodic noises to get difficult to recognize them with a program but not for a human listener. We gathered numerous randomly collected audio file to train and then test them using our SVM algorithm to be able to extract digits out of each conversation.
Ensembles of feedforward-designed convolutional neural networks
Venue: arXiv cs.CV 2019
Year: 2019

An ensemble method that fuses the output decision vectors of multiple feedforward-designed convolutional neural networks (FF-CNNs) to solve the image classification problem is proposed in this work. To enhance the performance of the ensemble system, it is critical to increasing the diversity of FF-CNN models. To achieve this objective, we introduce diversities by adopting three strategies: 1) different parameter settings in convolutional layers, 2) flexible feature subsets fed into the Fully-connected (FC) layers, and 3) multiple image embeddings of the same input source. Furthermore, we partition input samples into easy and hard ones based on their decision confidence scores. As a result, we can develop a new ensemble system tailored to hard samples to further boost classification accuracy. Experiments are conducted on the MNIST and CIFAR-10 datasets to demonstrate the effectiveness of the ensemble method.
Inversion-based Measurement of Data Consistency for Read/Write Registers
Venue: arXiv cs.DC 2019
Year: 2019

Both providers and consumers of distributed storage services benefit from the quantification of the severity of consistency violations. However, existing methods fail to capture a typical pattern of violation - the disorder among operations with reference to the ideally strong atomic execution. Such disorder are often seen in Internet applications based on distributed storage services, such as instant messaging. To this end, we use inversions in a permutation with reference to the totally-ordered permutation to measure the consistency of data. The $i$-atomicity model is proposed and a (pseudo-)polynomial verification algorithm for a restricted type of histories is proposed. The basic idea of the verification algorithm is brute-force search of the permutation with less than $i$ inversions, which requires exponential cost. Reasonable assumptions abstracted from applications scenarios can be leveraged to prune the search space, which yields an efficient polynomial verification algorithm.
Optimal Multi-Quality Multicast for 360 Virtual Reality Video
Venue: arXiv cs.IT 2019
Year: 2019

A 360 virtual reality (VR) video, recording a scene of interest in every direction, provides VR users with immersive viewing experience. However, transmission of a 360 VR video which is of a much larger size than a traditional video to mobile users brings a heavy burden to a wireless network. In this paper, we consider multi-quality multicast of a 360 VR video from a single server to multiple users using time division multiple access (TDMA). To improve transmission efficiency, tiling is adopted, and each tile is pre-encoded into multiple representations with different qualities. We optimize the quality level selection, transmission time allocation and transmission power allocation to maximize the total utility of all users under the transmission time and power allocation constraints as well as the quality smoothness constraints for mixed-quality tiles. The problem is a challenging mixed discrete-continuous opti-mization problem. We propose two low-complexity algorithms to obtain two suboptimal solutions, using continuous relaxation and DC programming, respectively. Finally, numerical results demonstrate the advantage of the proposed solutions.
Subset Feedback Vertex Set in Chordal and Split Graphs
Venue: arXiv cs.DS 2019
Year: 2019

In the \textsc{Subset Feedback Vertex Set (Subset-FVS)} problem the input is a graph $G$, a subset \(T\) of vertices of \(G\) called the `terminal' vertices, and an integer $k$. The task is to determine whether there exists a subset of vertices of cardinality at most $k$ which together intersect all cycles which pass through the terminals. \textsc{Subset-FVS} generalizes several well studied problems including \textsc{Feedback Vertex Set} and \textsc{Multiway Cut}. This problem is known to be \NP-Complete even in split graphs. Cygan et al. proved that \textsc{Subset-FVS} is fixed parameter tractable (\FPT) in general graphs when parameterized by $k$ [SIAM J. Discrete Math (2013)]. In split graphs a simple observation reduces the problem to an equivalent instance of the $3$-\textsc{Hitting Set} problem with same solution size. This directly implies, for \textsc{Subset-FVS} \emph{restricted to split graphs}, (i) an \FPT algorithm which solves the problem in $\OhStar(2.076^k)$ time \footnote{The \(\OhStar()\) notation hides polynomial factors.}% for \textsc{Subset-FVS} in Chordal % Graphs [Wahlström, Ph.D. Thesis], and (ii) a kernel of size $\mathcal{O}(k^3)$. We improve both these results for \textsc{Subset-FVS} on split graphs; we derive (i) a kernel of size $\mathcal{O}(k^2)$ which is the best possible unless $\NP \subseteq \coNP/{\sf poly}$, and (ii) an algorithm which solves the problem in time $\mathcal{O}^*(2^k)$. Our algorithm, in fact, solves \textsc{Subset-FVS} on the more general class of \emph{chordal graphs}, also in $\mathcal{O}^*(2^k)$ time.
Unpaired Pose Guided Human Image Generation
Venue: arXiv cs.CV 2019
Year: 2019

This paper studies the task of full generative modelling of realistic images of humans, guided only by coarse sketch of the pose, while providing control over the specific instance or type of outfit worn by the user. This is a difficult problem because input and output domain are very different and direct image-to-image translation becomes infeasible. We propose an end-to-end trainable network under the generative adversarial framework, that provides detailed control over the final appearance while not requiring paired training data and hence allows us to forgo the challenging problem of fitting 3D poses to 2D images. The model allows to generate novel samples conditioned on either an image taken from the target domain or a class label indicating the style of clothing (e.g., t-shirt). We thoroughly evaluate the architecture and the contributions of the individual components experimentally. Finally, we show in a large scale perceptual study that our approach can generate realistic looking images and that participants struggle in detecting fake images versus real samples, especially if faces are blurred.
Spectral Clustering via Ensemble Deep Autoencoder Learning (SC-EDAE)
Venue: arXiv cs.LG 2019
Year: 2019

Recently, a number of works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These approaches follow either a sequential way, where a deep representation is learned using a deep autoencoder before obtaining clusters with k-means, or a simultaneous way, where deep representation and clusters are learned jointly by optimizing a single objective function. Both strategies improve clustering performance, however the robustness of these approaches is impeded by several deep autoencoder setting issues, among which the weights initialization, the width and number of layers or the number of epochs. To alleviate the impact of such hyperparameters setting on the clustering performance, we propose a new model which combines the spectral clustering and deep autoencoder strengths in an ensemble learning framework. Extensive experiments on various benchmark datasets demonstrate the potential and robustness of our approach compared to state-of-the-art deep clustering methods.
Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions
Venue: arXiv cs.LG 2019
Year: 2019

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points via gradient-based random sampling. The proposed technique is used to perform an empirical study of the loss surfaces generated by two different error metrics: quadratic loss and entropic loss. The empirical observations confirm the theoretical hypothesis regarding the nature of neural network attraction basins. Entropic loss is shown to exhibit stronger gradients and fewer stationary points than quadratic loss, indicating that entropic loss has a more searchable landscape. Quadratic loss is shown to be more resilient to overfitting than entropic loss. Both losses are shown to exhibit local minima, but the number of local minima is shown to decrease with an increase in dimensionality. Thus, the proposed visualisation technique successfully captures the local minima properties exhibited by the neural network loss surfaces, and can be used for the purpose of fitness landscape analysis of neural networks.
On neighbourhood degree sequences of complex networks
Venue: arXiv cs.SI 2019
Year: 2019

Network topology is a fundamental aspect of network science that allows us to gather insights into the complicated relational architectures of the world we inhabit. We provide a first specific study of neighbourhood degree sequences in complex networks. We consider how to explicitly characterise important physical concepts such as similarity, heterogeneity and organisation in these sequences, as well as updating the notion of hierarchical complexity to reflect previously unnoticed organisational principles. We also point out that neighbourhood degree sequences are related to a powerful subtree kernel for unlabelled graph classification. We study these newly defined sequence properties in a comprehensive array of graph models and over 200 real-world networks. We find that these indices are neither highly correlated with each other nor with classical network indices. Importantly, the sequences of a wide variety of real world networks are found to have greater similarity and organisation than is expected for networks of their given degree distributions. Notably, while biological, social and technological networks all showed consistently large neighbourhood similarity and organisation, hierarchical complexity was not a consistent feature of real world networks. Neighbourhood degree sequences are an interesting tool for describing unique and important characteristics of complex networks.
Learning the optimal state-feedback via supervised imitation learning
Venue: arXiv cs.LG 2019
Year: 2019

Imitation learning is a control design paradigm that seeks to learn a control policy reproducing demonstrations from expert agents. By substituting expert demonstrations for optimal behaviours, the same paradigm leads to the design of control policies closely approximating the optimal state-feedback. This approach requires training a machine learning algorithm (in our case deep neural networks) directly on state-control pairs originating from optimal trajectories. We have shown in previous work that, when restricted to low-dimensional state and control spaces, this approach is very successful in several deterministic, non-linear problems in continuous-time. In this work, we refine our previous studies using as a test case a simple quadcopter model with quadratic and time-optimal objective functions. We describe in detail the best learning pipeline we have developed, that is able to approximate via deep neural networks the state-feedback map to a very high accuracy. We introduce the use of the softplus activation function in the hidden units of neural networks showing that it results in a smoother control profile whilst retaining the benefits of rectifiers. We show how to evaluate the optimality of the trained state-feedback, and find that already with two layers the objective function reached and its optimal value differ by less than one percent. We later consider also an additional metric linked to the system asymptotic behaviour - time taken to converge to the policy's fixed point. With respect to these metrics, we show that improvements in the mean absolute error do not necessarily correspond to better policies.
Designing Data Protection for GDPR Compliance into IoT Healthcare Systems
Venue: arXiv cs.SE 2019
Year: 2019

In this paper, we investigate the implications of the General Data Privacy Regulation (GDPR) on the design of an IoT healthcare system. On 25th May 2018, the GDPR has become mandatory within the European Union and hence also for all suppliers of IT products. Infringements on the regulation are now fined with penalties of up 20 Million EUR or 4\% of the annual turnover of a company whichever is higher. This is a clear motivation for system designers to guarantee compliance to the GDPR. We propose a data labeling model to support access control for privacy-critical patient data together with the Fusion/UML process to design GDPR compliant system. We illustrate this design process on the case study of IoT based monitoring of Alzheimer's patients that we work on in the CHIST-ERA project SUCCESS.
Stable Electromyographic Sequence Prediction During Movement Transitions using Temporal Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Transient muscle movements influence the temporal structure of myoelectric signal patterns, often leading to unstable prediction behavior from movement-pattern classification methods. We show that temporal convolutional network sequential models leverage the myoelectric signal's history to discover contextual temporal features that aid in correctly predicting movement intentions, especially during interclass transitions. We demonstrate myoelectric classification using temporal convolutional networks to effect 3 simultaneous hand and wrist degrees-of-freedom in an experiment involving nine human-subjects. Temporal convolutional networks yield significant $(p<0.001)$ performance improvements over other state-of-the-art methods in terms of both classification accuracy and stability.
Choosing the Right Word: Using Bidirectional LSTM Tagger for Writing Support Systems
Venue: arXiv cs.CL 2019
Year: 2019

Scientific writing is difficult. It is even harder for those for whom English is a second language (ESL learners). Scholars around the world spend a significant amount of time and resources proofreading their work before submitting it for review or publication. In this paper we present a novel machine learning based application for proper word choice task. Proper word choice is a generalization the lexical substitution (LS) and grammatical error correction (GEC) tasks. We demonstrate and evaluate the usefulness of applying bidirectional Long Short Term Memory (LSTM) tagger, for this task. While state-of-the-art grammatical error correction uses error-specific classifiers and machine translation methods, we demonstrate an unsupervised method that is based solely on a high quality text corpus and does not require manually annotated data. We use a bidirectional Recurrent Neural Network (RNN) with LSTM for learning the proper word choice based on a word's sentential context. We demonstrate and evaluate our application on both a domain-specific (scientific), writing task and a general-purpose writing task. We show that our domain-specific and general-purpose models outperform state-of-the-art general context learning. As an additional contribution of this research, we also share our code, pre-trained models, and a new ESL learner test set with the research community.
Grey matter sublayer thickness estimation in themouse cerebellum
Venue: arXiv cs.CV 2019
Year: 2019

The cerebellar grey matter morphology is an important feature to study neurodegenerative diseases such as Alzheimer's disease or Down's syndrome. Its volume or thickness is commonly used as a surrogate imaging biomarker for such diseases. Most studies about grey matter thickness estimation focused on the cortex, and little attention has been drawn on the morphology of the cerebellum. Using ex vivo high-resolution MRI, it is now possible to visualise the different cell layers in the mouse cerebellum. In this work, we introduce a framework to extract the Purkinje layer within the grey matter, enabling the estimation of the thickness of the cerebellar grey matter, the granular layer and molecular layer from gadolinium-enhanced ex vivo mouse brain MRI. Application to mouse model of Down's syndrome found reduced cortical and layer thicknesses in the transchromosomic group.
A Spatial-temporal 3D Human Pose Reconstruction Framework
Venue: arXiv cs.CV 2019
Year: 2019

3D human pose reconstruction from single-view camera is a difficult and challenging topic. Many approaches have been proposed, but almost focusing on frame-by-frame independently while inter-frames are highly correlated in a pose sequence. In contrast, we introduce a novel spatial-temporal 3D reconstruction framework that leverages both intra and inter frame relationships in consecutive 2D pose sequences. Orthogonal Matching Pursuit (OMP) algorithm, pre-trained Pose-angle Limits and Temporal Models have been implemented. We quantitatively compare our framework versus recent works on CMU motion capture dataset and Vietnamese traditional dance sequences. Our method outperforms others with 10 percent lower of Euclidean reconstruction error and robustness against Gaussian noise. Additionally, it is also important to mention that our reconstructed 3D pose sequences are smoother and more natural than others.