OpenRaiser/Intern-Atlas huggingface dataset:
Multi-modal segmentation with missing MR sequences using pre-trained fusion networks
Venue: arXiv cs.CV 2019
Year: 2019

Missing data is a common problem in machine learning and in retrospective imaging research it is often encountered in the form of missing imaging modalities. We propose to take into account missing modalities in the design and training of neural networks, to ensure that they are capable of providing the best possible prediction even when multiple images are not available. The proposed network combines three modifications to the standard 3D UNet architecture: a training scheme with dropout of modalities, a multi-pathway architecture with fusion layer in the final stage, and the separate pre-training of these pathways. These modifications are evaluated incrementally in terms of performance on full and missing data, using the BraTS multi-modal segmentation challenge. The final model shows significant improvement with respect to the state of the art on missing data and requires less memory during training.
CAT: Compression-Aware Training for bandwidth reduction
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving visual processing tasks. One of the major obstacles hindering the ubiquitous use of CNNs for inference is their relatively high memory bandwidth requirements, which can be a main energy consumer and throughput bottleneck in hardware accelerators. Accordingly, an efficient feature map compression method can result in substantial performance gains. Inspired by quantization-aware training approaches, we propose a compression-aware training (CAT) method that involves training the model in a way that allows better compression of feature maps during inference. Our method trains the model to achieve low-entropy feature maps, which enables efficient compression at inference time using classical transform coding methods. CAT significantly improves the state-of-the-art results reported for quantization. For example, on ResNet-34 we achieve 73.1% accuracy (0.2% degradation from the baseline) with an average representation of only 1.79 bits per value. Reference implementation accompanies the paper at https://github.com/CAT-teams/CAT
Efficient Pilot Allocation for URLLC Traffic in 5G Industrial IoT Networks
Venue: arXiv cs.NI 2019
Year: 2019

In this paper we address the problem of resource allocation for alarm traffic in industrial Internet of Things networks using massive MIMO. We formulate the general problem of how to allocate pilot signals to alarm traffic such that delivery is guaranteed, while also minimising the number of pilots reserved for alarms, thus maximising the channel resources available for other traffic, such as industrial control traffic. We present an algorithm that fulfils these requirements, and evaluate its performance both analytically and through a simulation study. For realistic alarm traffic characteristics, on average our algorithm can deliver alarms within two time slots (of duration equal to the 5G transmission time interval) using fewer than 1.5 pilots per slot, and even in the worst case it uses around 3.5 pilots in any given slot, with delivery guaranteed in an average of approximately four slots.
Dual Adaptive Pyramid Network for Cross-Stain Histopathology Image Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Supervised semantic segmentation normally assumes the test data being in a similar data domain as the training data. However, in practice, the domain mismatch between the training and unseen data could lead to a significant performance drop. Obtaining accurate pixel-wise label for images in different domains is tedious and labor intensive, especially for histopathology images. In this paper, we propose a dual adaptive pyramid network (DAPNet) for histopathological gland segmentation adapting from one stain domain to another. We tackle the domain adaptation problem on two levels: 1) the image-level considers the differences of image color and style; 2) the feature-level addresses the spatial inconsistency between two domains. The two components are implemented as domain classifiers with adversarial training. We evaluate our new approach using two gland segmentation datasets with H&E and DAB-H stains respectively. The extensive experiments and ablation study demonstrate the effectiveness of our approach on the domain adaptive segmentation task. We show that the proposed approach performs favorably against other state-of-the-art methods.
PyDEns: a Python Framework for Solving Differential Equations with Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Recently, a lot of papers proposed to use neural networks to approximately solve partial differential equations (PDEs). Yet, there has been a lack of flexible framework for convenient experimentation. In an attempt to fill the gap, we introduce a PyDEns-module open-sourced on GitHub. Coupled with capabilities of BatchFlow, open-source framework for convenient and reproducible deep learning, PyDEns-module allows to 1) solve partial differential equations from a large family, including heat equation and wave equation 2) easily search for the best neural-network architecture among the zoo, that includes ResNet and DenseNet 3) fully control the process of model-training by testing different point-sampling schemes. With that in mind, our main contribution goes as follows: implementation of a ready-to-use and open-source numerical solver of PDEs of a novel format, based on neural networks.
Data-Driven 3D Placement of UAV Base Stations for Arbitrarily Distributed Crowds
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, we consider an Unmanned Aerial Vehicle (UAV)-assisted cellular system which consists of multiple UAV base stations (BSs) cooperating the terrestrial BSs. In such a heterogeneous network, for cellular operators, the problem is how to determine the appropriate number, locations, and altitudes of UAV-BSs to improve the system sumrate as well as satisfy the demands of arbitrarily flash crowds on data rates. We propose a data-driven 3D placement of UAV-BSs for providing an effective placement result with a feasible computational cost. The proposed algorithm searches for the appropriate number, location, coverage, and altitude of each UAV-BS in the serving area with the maximized system sumrate in polynomial time so as to guarantee the minimum data rate requirement of UE. The simulation results show that the proposed approach can improve system sumrate in comparison with the case without UAV-BSs.
Off-Policy Actor-Critic with Shared Experience Replay
Venue: arXiv cs.LG 2019
Year: 2019

We investigate the combination of actor-critic reinforcement learning algorithms with uniform large-scale experience replay and propose solutions for two challenges: (a) efficient actor-critic learning with experience replay (b) stability of off-policy learning where agents learn from other agents behaviour. We employ those insights to accelerate hyper-parameter sweeps in which all participating agents run concurrently and share their experience via a common replay module. To this end we analyze the bias-variance tradeoffs in V-trace, a form of importance sampling for actor-critic methods. Based on our analysis, we then argue for mixing experience sampled from replay with on-policy experience, and propose a new trust region scheme that scales effectively to data distributions where V-trace becomes unstable. We provide extensive empirical validation of the proposed solution. We further show the benefits of this setup by demonstrating state-of-the-art data efficiency on Atari among agents trained up until 200M environment frames.
A New Deterministic Algorithm for Dynamic Set Cover
Venue: arXiv cs.DS 2019
Year: 2019

We present a deterministic dynamic algorithm for maintaining a $(1+ε)f$-approximate minimum cost set cover with $O(f\log(Cn)/ε^2)$ amortized update time, when the input set system is undergoing element insertions and deletions. Here, $n$ denotes the number of elements, each element appears in at most $f$ sets, and the cost of each set lies in the range $[1/C, 1]$. Our result, together with that of Gupta et al. [STOC`17], implies that there is a deterministic algorithm for this problem with $O(f\log(Cn))$ amortized update time and $O(\min(\log n, f))$-approximation ratio, which nearly matches the polynomial-time hardness of approximation for minimum set cover in the static setting. Our update time is only $O(\log (Cn))$ away from a trivial lower bound. Prior to our work, the previous best approximation ratio guaranteed by deterministic algorithms was $O(f^2)$, which was due to Bhattacharya et al. [ICALP`15]. In contrast, the only result that guaranteed $O(f)$-approximation was obtained very recently by Abboud et al. [STOC`19], who designed a dynamic algorithm with $(1+ε)f$-approximation ratio and $O(f^2 \log n/ε)$ amortized update time. Besides the extra $O(f)$ factor in the update time compared to our and Gupta et al.'s results, the Abboud et al. algorithm is randomized, and works only when the adversary is oblivious and the sets are unweighted (each set has the same cost). We achieve our result via the primal-dual approach, by maintaining a fractional packing solution as a dual certificate. Unlike previous primal-dual algorithms that try to satisfy some local constraints for individual sets at all time, our algorithm basically waits until the dual solution changes significantly globally, and fixes the solution only where the fix is needed.
On the Expressiveness of LARA: A Unified Language for Linear and Relational Algebra
Venue: arXiv cs.DB 2019
Year: 2019

We study the expressive power of the LARA language -- a recently proposed unified model for expressing relational and linear algebra operations -- both in terms of traditional database query languages and some analytic tasks often performed in machine learning pipelines. We start by showing LARA to be expressive complete with respect to first-order logic with aggregation. Since LARA is parameterized by a set of user-defined functions which allow to transform values in tables, the exact expressive power of the language depends on how these functions are defined. We distinguish two main cases depending on the level of genericity queries are enforced to satisfy. Under strong genericity assumptions the language cannot express matrix convolution, a very important operation in current machine learning operations. This language is also local, and thus cannot express operations such as matrix inverse that exhibit a recursive behavior. For expressing convolution, one can relax the genericity requirement by adding an underlying linear order on the domain. This, however, destroys locality and turns the expressive power of the language much more difficult to understand. In particular, although under complexity assumptions the resulting language can still not express matrix inverse, a proof of this fact without such assumptions seems challenging to obtain.
MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis
Venue: arXiv cs.DC 2019
Year: 2019

This paper reports on the design and implementation of the HPC performance monitoring system deployed to continuously monitor performance metrics of all jobs on the HPC systems at the Max Planck Computing and Data Facility (MPCDF). Thereby it reveals important information to various stakeholders, in particular to users, application support, system administrators, and management. On each compute node, hardware and software performance monitoring data is collected by our newly developed lightweight open-source hpcmd middleware which builds upon standard Linux tools. The data is transported via rsyslog, and aggregated and processed by a Splunk system, enabling detailed per-cluster and per-job interactive analysis in a web browser. Additionally, performance reports are provided to the users as PDF files. Finally, we report on practical experience and benefits from large-scale deployments on MPCDF HPC systems, demonstrating how our solution can be useful to any HPC center.
Beyond Beaconing: Emerging Applications and Challenges of BLE
Venue: arXiv cs.NI 2019
Year: 2019

As an emerging technology with exceptional low energy consumption and low-latency data transmissions, Bluetooth Low Energy (BLE) has gained significant momentum in various application domains, such as Indoor Positioning, Home Automation, and Wireless Personal Area Network (WPAN) communications. With various novel protocol stack features, BLE is finding use on resource-constrained sensor nodes as well as more powerful gateway devices. Particularly proximity detection using BLE beacons has been a popular usage scenario ever since the release of Bluetooth 4.0, primarily due to the beacons' energy efficiency and ease of deployment. However, with the rapid rise of the Internet of Things (IoT), BLE is likely to be a significant component in many other applications with widely varying performance and Quality-of-Service (QoS) requirements and there is a need for a consolidated view of the role that BLE will play in applications beyond beaconing. This paper comprehensively surveys state-of-the-art applications built with BLE, obstacles to adoption of BLE in new application areas, and current solutions from academia and industry that further expand the capabilities of BLE.
A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we propose a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and a series of moderately-sized DNNs are constructed and trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function's higher frequency content over a specific range at the the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. The PhaseDNN will then be applied to find the solution of high frequency wave equations in inhomogeneous media through both differential and integral equation formulations with least square residual loss functions. Numerical results have demonstrated the capability of the PhaseDNN in learning high frequency functions and oscillatory solutions of interior and exterior Helmholtz equations.
A Multimodal Alerting System for Online Class Quality Assurance
Venue: arXiv cs.HC 2019
Year: 2019

Online 1 on 1 class is created for more personalized learning experience. It demands a large number of teaching resources, which are scarce in China. To alleviate this problem, we build a platform (marketplace), i.e., \emph{Dahai} to allow college students from top Chinese universities to register as part-time instructors for the online 1 on 1 classes. To warn the unqualified instructors and ensure the overall education quality, we build a monitoring and alerting system by utilizing multimodal information from the online environment. Our system mainly consists of two key components: banned word detector and class quality predictor. The system performance is demonstrated both offline and online. By conducting experimental evaluation of real-world online courses, we are able to achieve 74.3\% alerting accuracy in our production environment.
Adversarial Deep Embedded Clustering: on a better trade-off between Feature Randomness and Feature Drift
Venue: arXiv cs.LG 2019
Year: 2019

Clustering using deep autoencoders has been thoroughly investigated in recent years. Current approaches rely on simultaneously learning embedded features and clustering the data points in the latent space. Although numerous deep clustering approaches outperform the shallow models in achieving favorable results on several high-semantic datasets, a critical weakness of such models has been overlooked. In the absence of concrete supervisory signals, the embedded clustering objective function may distort the latent space by learning from unreliable pseudo-labels. Thus, the network can learn non-representative features, which in turn undermines the discriminative ability, yielding worse pseudo-labels. In order to alleviate the effect of random discriminative features, modern autoencoder-based clustering papers propose to use the reconstruction loss for pretraining and as a regularizer during the clustering phase. Nevertheless, a clustering-reconstruction trade-off can cause the \textit{Feature Drift} phenomena. In this paper, we propose ADEC (Adversarial Deep Embedded Clustering) a novel autoencoder-based clustering model, which addresses a dual problem, namely, \textit{Feature Randomness} and \textit{Feature Drift}, using adversarial training. We empirically demonstrate the suitability of our model on handling these problems using benchmark real datasets. Experimental results validate that our model outperforms state-of-the-art autoencoder-based clustering methods.
Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently
Venue: arXiv cs.LG 2019
Year: 2019

It has been observed \citep{zhang2016understanding} that deep neural networks can memorize: they achieve 100\% accuracy on training data. Recent theoretical results explained such behavior in highly overparametrized regimes, where the number of neurons in each layer is larger than the number of training samples. In this paper, we show that neural networks can be trained to memorize training data perfectly in a mildly overparametrized regime, where the number of parameters is just a constant factor more than the number of training samples, and the number of neurons is much smaller.
Stabilization of Exoskeletons through Active Ankle Compensation
Venue: arXiv cs.RO 2019
Year: 2019

This paper presents an active stabilization method for a fully actuated lower-limb exoskeleton. The method was tested on the exoskeleton ATALANTE, which was designed and built by the French start-up company Wandercraft. The main objective of this paper is to present a practical method of realizing more robust walking on hardware through active ankle compensation. The nominal gait was generated through the hybrid zero dynamic framework. The ankles are individually controlled to establish three main directives; (1) keeping the non-stance foot parallel to the ground, (2) maintaining rigid contact between the stance foot and the ground, and (3) closing the loop on pelvis orientation to achieve better tracking. Each individual component of this method was demonstrated separately to show each component's contribution to stability. The results showed that the ankle controller was able to experimentally maintain static balance in the sagittal plane while the exoskeleton was balanced on one leg, even when disturbed. The entire ankle controller was then also demonstrated on crutch-less dynamic walking. During testing, an anatomically correct manikin was placed in the exoskeleton, in lieu of a paraplegic patient. The pitch of the pelvis of the exoskeleton-manikin system was shown to track the gait trajectory better when ankle compensation was used. Overall, active ankle compensation was demonstrated experimentally to improve balance in the sagittal plane of the exoskeleton manikin system and points to an improved practical approach for stable walking.
Mathematical Reasoning in Latent Space
Venue: arXiv cs.LG 2019
Year: 2019

We design and conduct a simple experiment to study whether neural networks can perform several steps of approximate reasoning in a fixed dimensional latent space. The set of rewrites (i.e. transformations) that can be successfully performed on a statement represents essential semantic features of the statement. We can compress this information by embedding the formula in a vector space, such that the vector associated with a statement can be used to predict whether a statement can be rewritten by other theorems. Predicting the embedding of a formula generated by some rewrite rule is naturally viewed as approximate reasoning in the latent space. In order to measure the effectiveness of this reasoning, we perform approximate deduction sequences in the latent space and use the resulting embedding to inform the semantic features of the corresponding formal statement (which is obtained by performing the corresponding rewrite sequence using real formulas). Our experiments show that graph neural networks can make non-trivial predictions about the rewrite-success of statements, even when they propagate predicted latent representations for several steps. Since our corpus of mathematical formulas includes a wide variety of mathematical disciplines, this experiment is a strong indicator for the feasibility of deduction in latent space in general.
Fine-tune Bert for DocRED with Two-step Process
Venue: arXiv cs.CL 2019
Year: 2019

Modelling relations between multiple entities has attracted increasing attention recently, and a new dataset called DocRED has been collected in order to accelerate the research on the document-level relation extraction. Current baselines for this task uses BiLSTM to encode the whole document and are trained from scratch. We argue that such simple baselines are not strong enough to model to complex interaction between entities. In this paper, we further apply a pre-trained language model (BERT) to provide a stronger baseline for this task. We also find that solving this task in phases can further improve the performance. The first step is to predict whether or not two entities have a relation, the second step is to predict the specific relation.
Support Vector Machines on Noisy Intermediate Scale Quantum Computers
Venue: arXiv cs.ET 2019
Year: 2019

Support vector machine algorithms are considered essential for the implementation of automation in a radio access network. Specifically, they are critical in the prediction of the quality of user experience for video streaming based on device and network-level metrics. Quantum SVM is the quantum analogue of the classical SVM algorithm, which utilizes the properties of quantum computers to speed up the algorithm exponentially. In this work, we derive an optimized preprocessing unit for a quantum SVM that allows classifying any two-dimensional datasets that are linearly separable. We further provide a result readout method of the kernel matrix generation circuit to avoid quantum tomography that, in turn, reduces the quantum circuit depth. We also derive a quantum SVM system based on an optimized HHL quantum circuit with reduced circuit depth.