OpenRaiser/Intern-Atlas huggingface dataset:
Towards Enabling Critical mMTC: A Review of URLLC within mMTC
Venue: arXiv cs.NI 2020
Year: 2020

Massive machine-type communication (mMTC) and ultra-reliable and low-latency communication (URLLC) are two key service types in the fifth-generation (5G) communication systems, pursuing scalability and reliability with low-latency, respectively. These two extreme services are envisaged to agglomerate together into \emph{critical mMTC} shortly with emerging use cases (e.g., wide-area disaster monitoring, wireless factory automation), creating new challenges to designing wireless systems beyond 5G. While conventional network slicing is effective in supporting a simple mixture of mMTC and URLLC, it is difficult to simultaneously guarantee the reliability, latency, and scalability requirements of critical mMTC (e.g., < 4ms latency, $10^6$ devices/km$^2$ for factory automation) with limited radio resources. Furthermore, recently proposed solutions to scalable URLLC (e.g., machine learning aided URLLC for driverless vehicles) are ill-suited to critical mMTC whose machine type users have minimal energy budget and computing capability that should be (tightly) optimized for given tasks. To this end, our paper aims to characterize promising use cases of critical mMTC and search for their possible solutions. To this end, we first review the state-of-the-art (SOTA) technologies for separate mMTC and URLLC services and then identify key challenges from conflicting SOTA requirements, followed by potential approaches to prospective critical mMTC solutions at different layers.
Self-Supervised Bernoulli Autoencoders for Semi-Supervised Hashing
Venue: arXiv cs.LG 2020
Year: 2020

Semantic hashing is an emerging technique for large-scale similarity search based on representing high-dimensional data using similarity-preserving binary codes used for efficient indexing and search. It has recently been shown that variational autoencoders, with Bernoulli latent representations parametrized by neural nets, can be successfully trained to learn such codes in supervised and unsupervised scenarios, improving on more traditional methods thanks to their ability to handle the binary constraints architecturally. However, the scenario where labels are scarce has not been studied yet. This paper investigates the robustness of hashing methods based on variational autoencoders to the lack of supervision, focusing on two semi-supervised approaches currently in use. The first augments the variational autoencoder's training objective to jointly model the distribution over the data and the class labels. The second approach exploits the annotations to define an additional pairwise loss that enforces consistency between the similarity in the code (Hamming) space and the similarity in the label space. Our experiments show that both methods can significantly increase the hash codes' quality. The pairwise approach can exhibit an advantage when the number of labelled points is large. However, we found that this method degrades quickly and loses its advantage when labelled samples decrease. To circumvent this problem, we propose a novel supervision method in which the model uses its label distribution predictions to implement the pairwise objective. Compared to the best baseline, this procedure yields similar performance in fully supervised settings but improves the results significantly when labelled data is scarce. Our code is made publicly available at https://github.com/amacaluso/SSB-VAE.
Intrusion Detection in Binary Process Data: Introducing the Hamming-distance to Matrix Profiles
Venue: arXiv cs.CR 2020
Year: 2020

The digitisation of industry provides a plethora of novel applications that increase flexibility and reduce setup and maintenance time as well as cost. Furthermore, novel use cases are created by the digitisation of industry, commonly known as Industry 4.0 or the Industrial Internet of Things, applications make use of communication and computation technology that is becoming available. This enables novel business use cases, such as the digital twin, customer individual production, and data market places. However, the inter-connectivity such use cases rely on also significantly increases the attack surface of industrial enterprises. Sabotage and espionage are aimed at data, which is becoming the most crucial asset of an enterprise. Since the requirements on security solutions in industrial networks are inherently different from office networks, novel approaches for intrusion detection need to be developed. In this work, process data of a real water treatment process that contains attacks is analysed. Analysis is performed by an extension of Matrix Profiles, a motif discovery algorithm for time series. By extending Matrix Profiles with a Hammingdistance metric, binary and tertiary actuators can be integrated into the analysis in a meaningful fashion. This algorithm requires low training effort while providing accurate results. Furthermore, it can be employed in a real-time fashion. Selected actuators in the data set are analysed to highlight the applicability of the extended Matrix Profiles.
Overview of Security of Virtual Mobile Networks
Venue: arXiv cs.NI 2020
Year: 2020

5G is enabling different services over the same physical infrastructure through the concepts and technologies of virtualization, softwarization, slicing and cloud computing. Virtual Mobile Networks (VMNs), using these concepts, provide an opportunity to share the same physical infrastructure among multiple operators. Each VMN Operator (VMNO) can have own distinct operating and support systems. However, the technologies used to enable VMNs have their own explicit security challenges and solutions. The integrated environment built upon virtualization, softwarization, and cloudification, thus, will have complex security requirements and implications. In this vain, this article provides an overview of the security challenges and potential solutions for VMNs.
Superpixel-Guided Label Softening for Medical Image Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Segmentation of objects of interest is one of the central tasks in medical image analysis, which is indispensable for quantitative analysis. When developing machine-learning based methods for automated segmentation, manual annotations are usually used as the ground truth toward which the models learn to mimic. While the bulky parts of the segmentation targets are relatively easy to label, the peripheral areas are often difficult to handle due to ambiguous boundaries and the partial volume effect, etc., and are likely to be labeled with uncertainty. This uncertainty in labeling may, in turn, result in unsatisfactory performance of the trained models. In this paper, we propose superpixel-based label softening to tackle the above issue. Generated by unsupervised over-segmentation, each superpixel is expected to represent a locally homogeneous area. If a superpixel intersects with the annotation boundary, we consider a high probability of uncertain labeling within this area. Driven by this intuition, we soften labels in this area based on signed distances to the annotation boundary and assign probability values within [0, 1] to them, in comparison with the original "hard", binary labels of either 0 or 1. The softened labels are then used to train the segmentation models together with the hard labels. Experimental results on a brain MRI dataset and an optical coherence tomography dataset demonstrate that this conceptually simple and implementation-wise easy method achieves overall superior segmentation performances to baseline and comparison methods for both 3D and 2D medical images.
Identification of Tree Species in Japanese Forests based on Aerial Photography and Deep Learning
Venue: arXiv cs.CV 2020
Year: 2020

Natural forests are complex ecosystems whose tree species distribution and their ecosystem functions are still not well understood. Sustainable management of these forests is of high importance because of their significant role in climate regulation, biodiversity, soil erosion and disaster prevention among many other ecosystem services they provide. In Japan particularly, natural forests are mainly located in steep mountains, hence the use of aerial imagery in combination with computer vision are important modern tools that can be applied to forest research. Thus, this study constitutes a preliminary research in this field, aiming at classifying tree species in Japanese mixed forests using UAV images and deep learning in two different mixed forest types: a black pine (Pinus thunbergii)-black locust (Robinia pseudoacacia) and a larch (Larix kaempferi)-oak (Quercus mongolica) mixed forest. Our results indicate that it is possible to identify black locust trees with 62.6 % True Positives (TP) and 98.1% True Negatives (TN), while lower precision was reached for larch trees (37.4% TP and 97.7% TN).
Towards a Model of Testers' Cognitive Processes: Software Testing as a Problem Solving Approach
Venue: arXiv cs.SE 2020
Year: 2020

Software testing is a complex, intellectual activity based (at least) on analysis, reasoning, decision making, abstraction and collaboration performed in a highly demanding environment. Naturally, it uses and allocates multiple cognitive resources in software testers. However, while a cognitive psychology perspective is increasingly used in the general software engineering literature, it has yet to find its place in software testing. To the best of our knowledge, no theory of software testers' cognitive processes exists. Here, we take the first step towards such a theory by presenting a cognitive model of software testing based on how problem solving is conceptualized in cognitive psychology. Our approach is to instantiate a general problem solving process for the specific problem of creating test cases. We then propose an experiment for testing our cognitive test design model. The experiment makes use of verbal protocol analysis to understand the mechanisms by which human testers choose, design, implement and evaluate test cases. An initial evaluation was then performed with five software engineering master students as subjects. The results support a problem solving-based model of test design for capturing testers' cognitive processes.
Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
Venue: arXiv cs.SI 2020
Year: 2020

In recent years, substantial progress has been made on Graph Convolutional Networks (GCNs). However, the computing of GCN usually requires a large memory space for keeping the entire graph. In consequence, GCN is not flexible enough, especially for large scale graphs in complex real-world applications. Fortunately, methods based on Matrix Factorization (MF) naturally support constructing mini-batches, and thus are more friendly to distributed computing compared with GCN. Accordingly, in this paper, we analyze the connections between GCN and MF, and simplify GCN as matrix factorization with unitization and co-training. Furthermore, under the guidance of our analysis, we propose an alternative model to GCN named Unitized and Co-training Matrix Factorization (UCMF). Extensive experiments have been conducted on several real-world datasets. On the task of semi-supervised node classification, the experimental results illustrate that UCMF achieves similar or superior performances compared with GCN. Meanwhile, distributed UCMF significantly outperforms distributed GCN methods, which shows that UCMF can greatly benefit large scale and complex real-world applications. Moreover, we have also conducted experiments on a typical task of graph embedding, i.e., community detection, and the proposed UCMF model outperforms several representative graph embedding models.
Decoding up to 4 errors in Hyperbolic-like Abelian Codes by the Sakata Algorithm
Venue: arXiv cs.IT 2020
Year: 2020

We deal with two problems related with the use of the Sakata's algorithm in a specific class of bivariate codes. The first one is to improve the general framework of locator decoding in order to apply it on such abelian codes. The second one is to find a set of indexes oF the syndrome table such that no other syndrome contributes to implement the BMSa and, moreover, any of them may be ignored \textit{a priori}. In addition, the implementation on those indexes is sufficient to get the Groebner basis; that is, it is also a termination criterion.
Hybrid Discriminative-Generative Training via Contrastive Learning
Venue: arXiv cs.LG 2020
Year: 2020

Contrastive learning and supervised learning have both seen significant progress and success. However, thus far they have largely been treated as two separate objectives, brought together only by having a shared neural network. In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning. Beyond presenting this unified view, we show our specific choice of approximation of the energy-based loss outperforms the existing practice in terms of classification accuracy of WideResNet on CIFAR-10 and CIFAR-100. It also leads to improved performance on robustness, out-of-distribution detection, and calibration.
Constructing a Family Tree of Ten Indo-European Languages with Delexicalized Cross-linguistic Transfer Patterns
Venue: arXiv cs.CL 2020
Year: 2020

It is reasonable to hypothesize that the divergence patterns formulated by historical linguists and typologists reflect constraints on human languages, and are thus consistent with Second Language Acquisition (SLA) in a certain way. In this paper, we validate this hypothesis on ten Indo-European languages. We formalize the delexicalized transfer as interpretable tree-to-string and tree-to-tree patterns which can be automatically induced from web data by applying neural syntactic parsing and grammar induction technologies. This allows us to quantitatively probe cross-linguistic transfer and extend inquiries of SLA. We extend existing works which utilize mixed features and support the agreement between delexicalized cross-linguistic transfer and the phylogenetic structure resulting from the historical-comparative paradigm.
Reachability in Two-Dimensional Vector Addition Systems with States: One Test is for Free
Venue: arXiv cs.LO 2020
Year: 2020

Vector addition system with states is an ubiquitous model of computation with extensive applications in computer science. The reachability problem for vector addition systems is central since many other problems reduce to that question. The problem is decidable and it was recently proved that the dimension of the vector addition system is an important parameter of the complexity. In fixed dimensions larger than two, the complexity is not known (with huge complexity gaps). In dimension two, the reachability problem was shown to be PSPACE-complete by Blondin et al. in 2015. We consider an extension of this model, called 2-TVASS, where the first counter can be tested for zero. This model naturally extends the classical model of one counter automata (OCA). We show that reachability is still solvable in polynomial space for 2-TVASS. As in the work Blondin et al., our approach relies on the existence of small reachability certificates obtained by concatenating polynomially many cycles.
Trajectory Design and Power Allocation for Drone-Assisted NR-V2X Network with Dynamic NOMA/OMA
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we find trajectory planning and power allocation for a vehicular network in which an unmanned-aerial-vehicle (UAV) is considered as a relay to extend coverage for two disconnected far vehicles. We show that in a two-user network with an amplify-and-forward (AF) relay, non-orthogonal-multiple-access (NOMA) always has better or equal sum-rate in comparison to orthogonal-multiple-access (OMA) at high signal-to-noise-ratio (SNR) regime. However, for the cases where i) base station (BS)-to-relay link is weak, or ii) two users have similar links, or iii) BS-to-relay link is similar to relay-to-weak user link, applying NOMA has negligible sum-rate gain. Hence, due to the complexity of successive-interference-cancellation (SIC) decoding in NOMA, we propose a dynamic NOMA/OMA scheme in which OMA mode is selected for transmission when applying NOMA has only negligible gain. Also, we show that OMA always has better min-rate than NOMA at high SNR regime. Further, we formulate two optimization problems which maximize the sum-rate and min-rate of the two vehicles. These problems are non-convex, and hence we propose an iterative algorithm based on alternating-optimization (AO) method which solves trajectory and power allocation sub-problems by successive-convex-approximation (SCA) and difference-of-convex (DC) methods, respectively. Finally, the above-mentioned performance is confirmed by simulations.
SBI -- A toolkit for simulation-based inference
Venue: arXiv cs.LG 2020
Year: 2020

Scientists and engineers employ stochastic numerical simulators to model empirically observed phenomena. In contrast to purely statistical models, simulators express scientific principles that provide powerful inductive biases, improve generalization to new data or scenarios and allow for fewer, more interpretable and domain-relevant parameters. Despite these advantages, tuning a simulator's parameters so that its outputs match data is challenging. Simulation-based inference (SBI) seeks to identify parameter sets that a) are compatible with prior knowledge and b) match empirical observations. Importantly, SBI does not seek to recover a single 'best' data-compatible parameter set, but rather to identify all high probability regions of parameter space that explain observed data, and thereby to quantify parameter uncertainty. In Bayesian terminology, SBI aims to retrieve the posterior distribution over the parameters of interest. In contrast to conventional Bayesian inference, SBI is also applicable when one can run model simulations, but no formula or algorithm exists for evaluating the probability of data given parameters, i.e. the likelihood. We present $\texttt{sbi}$, a PyTorch-based package that implements SBI algorithms based on neural networks. $\texttt{sbi}$ facilitates inference on black-box simulators for practising scientists and engineers by providing a unified interface to state-of-the-art algorithms together with documentation and tutorials.
Analytical Derivation of Outage Correlation in Random Media Access with Application to Average Consensus in Wireless Networks: Extended Paper Version
Venue: arXiv cs.IT 2020
Year: 2020

We study a finite and fixed relative formation of possibly mobile wireless networked nodes. The nodes apply average consensus to agree on a common value like the formation's center. %Performance of consensus in terms of convergence speed is affected by message losses due path loss and interference. We assume framed slotted ALOHA based broadcast communication. Our work has two contributions. First, we analyze outage correlation of random media access in wireless networks under Nakagami fading. Second, the correlation terms are applied to the so called L2-joint spectral and numerical radii to analyze convergence speed of average consensus under wireless broadcast communication. This yields a unified framework for studying joint optimization of control and network parameters for consensus subject to message losses in wireless communications. Exemplary we show in this work how far outage correlation in wireless broadcast communication positively affects convergence speed of average consensus compared to consensus in the uncorrelated case.
Diversifying Anonymized Data with Diversity Constraints
Venue: arXiv cs.DB 2020
Year: 2020

Recently introduced privacy legislation has aimed to restrict and control the amount of personal data published by companies and shared to third parties. Much of this real data is not only sensitive requiring anonymization, but also contains characteristic details from a variety of individuals. This diversity is desirable in many applications ranging from Web search to drug and product development. Unfortunately, data anonymization techniques have largely ignored diversity in its published result. This inadvertently propagates underlying bias in subsequent data analysis. We study the problem of finding a diverse anonymized data instance where diversity is measured via a set of diversity constraints. We formalize diversity constraints and study their foundations such as implication and satisfiability. We show that determining the existence of a diverse, anonymized instance can be done in PTIME, and we present a clustering-based algorithm. We conduct extensive experiments using real and synthetic data showing the effectiveness of our techniques, and improvement over existing baselines. Our work aligns with recent trends towards responsible data science by coupling diversity with privacy-preserving data publishing.
Wearable vibrotactile stimulation for upper extremity rehabilitation in chronic stroke: clinical feasibility trial using the VTS Glove
Venue: arXiv cs.HC 2020
Year: 2020

Objective: Evaluate the feasibility and potential impacts on hand function using a wearable stimulation device (the VTS Glove) which provides mechanical, vibratory input to the affected limb of chronic stroke survivors. Methods: A double-blind, randomized, controlled feasibility study including sixteen chronic stroke survivors (mean age: 54; 1-13 years post-stroke) with diminished movement and tactile perception in their affected hand. Participants were given a wearable device to take home and asked to wear it for three hours daily over eight weeks. The device intervention was either (1) the VTS Glove, which provided vibrotactile stimulation to the hand, or (2) an identical glove with vibration disabled. Participants were equally randomly assigned to each condition. Hand and arm function were measured weekly at home and in local physical therapy clinics. Results: Participants using the VTS Glove showed significantly improved Semmes-Weinstein monofilament exam, reduction in Modified Ashworth measures in the fingers, and some increased voluntary finger flexion, elbow and shoulder range of motion. Conclusions: Vibrotactile stimulation applied to the disabled limb may impact tactile perception, tone and spasticity, and voluntary range of motion. Wearable devices allow extended application and study of stimulation methods outside of a clinical setting.
Towards Deeper Graph Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Graph neural networks have shown significant success in the field of graph representation learning. Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Nevertheless, one layer of these neighborhood aggregation methods only consider immediate neighbors, and the performance decreases when going deeper to enable larger receptive fields. Several recent studies attribute this performance deterioration to the over-smoothing issue, which states that repeated propagation makes node representations of different classes indistinguishable. In this work, we study this observation systematically and develop new insights towards deeper graph neural networks. First, we provide a systematical analysis on this issue and argue that the key factor compromising the performance significantly is the entanglement of representation transformation and propagation in current graph convolution operations. After decoupling these two operations, deeper graph neural networks can be used to learn graph node representations from larger receptive fields. We further provide a theoretical analysis of the above observation when building very deep models, which can serve as a rigorous and gentle description of the over-smoothing issue. Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields. A set of experiments on citation, co-authorship, and co-purchase datasets have confirmed our analysis and insights and demonstrated the superiority of our proposed methods.