OpenRaiser/Intern-Atlas huggingface dataset:
Reconsidering Design of Multi-Antenna NOMA Systems with Limited Feedback
Venue: arXiv cs.IT 2019
Year: 2019

We provide in this paper a comprehensive solution to the design, performance analysis, and optimization of a multi-antenna non-orthogonal multiple access (NOMA) system for multiuser downlink communications under a general limited channel state information (CSI) feedback framework for frequency division duplex mode. We design a general framework including user clustering, joint power and bits allocation, CSI quantization and feedback, signal superposition coding, transmit beamforming, and successive interference cancellation at receivers. Then, we conduct a mathematically strict performance analysis of the considered system, and obtain a closed-form lower bound on the ergodic rate of each user in terms of transmit power, CSI quantization accuracy and channel conditions. For exploiting the potentials of multiple-antenna techniques in NOMA systems, we jointly optimize two key parameters, i.e., transmit power and the number of feedback bits allocated to each user, and propose low-complexity closed-form solutions. Moreover, through asymptotic analysis, we reveal the interactions between the main system parameters and their impacts on the joint power and feedback bits allocation result, and hence show some guidelines on the system design. Finally, numerical results validate the correctness of our theoretical analysis and demonstrate the advantages of the proposed algorithms over the most related state of the art.
Semantic Enrichment of Streaming Healthcare Data
Venue: arXiv cs.IR 2019
Year: 2019

In the past decade, the healthcare industry has made significant advances in the digitization of patient information. However, a lack of interoperability among healthcare systems still imposes a high cost to patients, hospitals, and insurers. Currently, most systems pass messages using idiosyncratic messaging standards that require specialized knowledge to interpret. This increases the cost of systems integration and often puts more advanced uses of data out of reach. In this project, we demonstrate how two open standards, FHIR and RDF, can be combined both to integrate data from disparate sources in real-time and make that data queryable and susceptible to automated inference. To validate the effectiveness of the semantic engine, we perform simulations of real-time data feeds and demonstrate how they can be combined and used by client-side applications with no knowledge of the underlying sources.
RST-MODNet: Real-time Spatio-temporal Moving Object Detection for Autonomous Driving
Venue: arXiv cs.CV 2019
Year: 2019

Moving Object Detection (MOD) is a critical task for autonomous vehicles as moving objects represent higher collision risk than static ones. The trajectory of the ego-vehicle is planned based on the future states of detected moving objects. It is quite challenging as the ego-motion has to be modelled and compensated to be able to understand the motion of the surrounding objects. In this work, we propose a real-time end-to-end CNN architecture for MOD utilizing spatio-temporal context to improve robustness. We construct a novel time-aware architecture exploiting temporal motion information embedded within sequential images in addition to explicit motion maps using optical flow images.We demonstrate the impact of our algorithm on KITTI dataset where we obtain an improvement of 8% relative to the baselines. We compare our algorithm with state-of-the-art methods and achieve competitive results on KITTI-Motion dataset in terms of accuracy at three times better run-time. The proposed algorithm runs at 23 fps on a standard desktop GPU targeting deployment on embedded platforms.
Purpose-based access policy on provenance and data algebra
Venue: arXiv cs.CR 2019
Year: 2019

It is a crucial mechanism of access control to determine that data can only be accessed for allowed purposes. To achieve this mechanism, we propose purpose-based access policies in this paper. Different from provenance-based policies that determine if a piece of data can be accessed or not, purpose-based access policies determines for what purposes can data be accessed. Particularly, the purposes can be classified as different sensitivity levels. For the first time, We tailor policy algebras to include internal and external policy operators for hierarchical purposes, in order to merge purpose sets generated by individual policies. We also created external policy algebras to merge policies from multi-parties. With different types' testing experiments, our model is proved to be feasible and practical.
Generalizable prediction of academic performance from short texts on social media
Venue: arXiv cs.SI 2019
Year: 2019

It has already been established that digital traces can be used to predict various human attributes. In most cases, however, predictive models rely on features that are specific to a particular source of digital trace data. In contrast, short texts written by users $-$ tweets, posts, or comments $-$ are ubiquitous across multiple platforms. In this paper, we explore the predictive power of short texts with respect to the academic performance of their authors. We use data from a representative panel of Russian students that includes information about their educational outcomes and activity on a popular networking site, VK. We build a model to predict academic performance from users' posts on VK and then apply it to a different context. In particular, we show that the model could reproduce rankings of schools and universities from the posts of their students on social media. We also find that the same model could predict academic performance from tweets as well as from VK posts. The generalizability of a model trained on a relatively small data set could be explained by the use of continuous word representations trained on a much larger corpus of social media posts. This also allows for greater interpretability of model predictions.
Gaussian Embedding of Large-scale Attributed Graphs
Venue: arXiv cs.LG 2019
Year: 2019

Graph embedding methods transform high-dimensional and complex graph contents into low-dimensional representations. They are useful for a wide range of graph analysis tasks including link prediction, node classification, recommendation and visualization. Most existing approaches represent graph nodes as point vectors in a low-dimensional embedding space, ignoring the uncertainty present in the real-world graphs. Furthermore, many real-world graphs are large-scale and rich in content (e.g. node attributes). In this work, we propose GLACE, a novel, scalable graph embedding method that preserves both graph structure and node attributes effectively and efficiently in an end-to-end manner. GLACE effectively models uncertainty through Gaussian embeddings, and supports inductive inference of new nodes based on their attributes. In our comprehensive experiments, we evaluate GLACE on real-world graphs, and the results demonstrate that GLACE significantly outperforms state-of-the-art embedding methods on multiple graph analysis tasks.
Merging External Bilingual Pairs into Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

As neural machine translation (NMT) is not easily amenable to explicit correction of errors, incorporating pre-specified translations into NMT is widely regarded as a non-trivial challenge. In this paper, we propose and explore three methods to endow NMT with pre-specified bilingual pairs. Instead, for instance, of modifying the beam search algorithm during decoding or making complex modifications to the attention mechanism --- mainstream approaches to tackling this challenge ---, we experiment with the training data being appropriately pre-processed to add information about pre-specified translations. Extra embeddings are also used to distinguish pre-specified tokens from the other tokens. Extensive experimentation and analysis indicate that over 99% of the pre-specified phrases are successfully translated (given a 85% baseline) and that there is also a substantive improvement in translation quality with the methods explored here.
Multi-version Indexing in Flash-based Key-Value Stores
Venue: arXiv cs.DB 2019
Year: 2019

Maintaining multiple versions of data is popular in key-value stores since it increases concurrency and improves performance. However, designing a multi-version key-value store entails several challenges, such as additional capacity for storing extra versions and an indexing mechanism for mapping versions of a key to their values. We present SkimpyFTL, a FTL-integrated multi-version key-value store that exploits the remap-on-write property of flash-based SSDs for multi-versioning and provides a tradeoff between memory capacity and lookup latency for indexing.
Combining MixMatch and Active Learning for Better Accuracy with Fewer Labels
Venue: arXiv cs.LG 2019
Year: 2019

We propose using active learning based techniques to further improve the state-of-the-art semi-supervised learning MixMatch algorithm. We provide a thorough empirical evaluation of several active-learning and baseline methods, which successfully demonstrate a significant improvement on the benchmark CIFAR-10, CIFAR-100, and SVHN datasets (as much as 1.5% in absolute accuracy). We also provide an empirical analysis of the cost trade-off between incrementally gathering more labeled versus unlabeled data. This analysis can be used to measure the relative value of labeled/unlabeled data at different points of the learning curve, where we find that although the incremental value of labeled data can be as much as 20x that of unlabeled, it quickly diminishes to less than 3x once more than 2,000 labeled example are observed. Code can be found at https://github.com/google-research/mma.
Industrial Internet of Things (IIoT) Applications of Edge and Fog Computing: A Review and Future Directions
Venue: arXiv cs.NI 2019
Year: 2019

With rapid technological advancements within the domain of Internet of Things (IoT), strong trends have emerged which indicate a rapid growth in the number of smart devices connected to IoT networks and this growth cannot be supported by traditional cloud computing platforms. In response to the increased capacity of data being transferred over networks, the edge and fog computing paradigms have emerged as extremely viable frameworks that shift computational and storage resources towards the edge of the network, thereby migrating processing power from centralized cloud servers to distributed LAN resources and powerful embedded devices within the network. These computing paradigms, therefore, have the potential to support massive IoT networks of the future and have also fueled the advancement of IoT systems within industrial settings, leading to the creation of the Industrial Internet of Things (IIoT) technology that is revolutionizing industrial processes in a variety of domains. In this paper, we elaborate on the impact of edge and fog computing paradigms on IIoT. We also highlight the how edge and fog computing are poised to bring about a turnaround in several industrial applications through a use-case approach. Finally, we conclude with the current issues and challenges faced by these paradigms in IIoT and suggest some research directions that should be followed to solve these problems and accelerate the adaptation of edge and fog computing in IIoT.
A Fast Matrix-Completion-Based Approach for Recommendation Systems
Venue: arXiv cs.IR 2019
Year: 2019

Matrix completion is widely used in machine learning, engineering control, image processing, and recommendation systems. Currently, a popular algorithm for matrix completion is Singular Value Threshold (SVT). In this algorithm, the singular value threshold should be set first. However, in a recommendation system, the dimension of the preference matrix keeps changing. Therefore, it is difficult to directly apply SVT. In addition, what the users of a recommendation system need is a sequence of personalized recommended results rather than the estimation of their scores. According to the above ideas, this paper proposes a novel approach named probability completion model~(PCM). By reducing the data dimension, the transitivity of the similar matrix, and singular value decomposition, this approach quickly obtains a completion matrix with the same probability distribution as the original matrix. The approach greatly reduces the computation time based on the accuracy of the sacrifice part, and can quickly obtain a low-rank similarity matrix with data trend approximation properties. The experimental results show that PCM can quickly generate a complementary matrix with similar data trends as the original matrix. The LCS score and efficiency of PCM are both higher than SVT.
On a generalization of the Jensen-Shannon divergence
Venue: arXiv cs.IT 2019
Year: 2019

The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar $α$-Jensen-Bregman divergences and derive thereof the vector-skew $α$-Jensen-Shannon divergences. We study the properties of these novel divergences and show how to build parametric families of symmetric Jensen-Shannon-type divergences. Finally, we report an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probability densities belonging to a mixture family: This includes the case of the Jensen-Shannon centroid of a set of categorical distributions or normalized histograms.
Meta-Path Constrained Random Walk Inference for Large-Scale Heterogeneous Information Networks
Venue: arXiv cs.SI 2019
Year: 2019

Heterogeneous information network (HIN) has shown its power of modeling real world data as a multi-typed entity-relation graph. Meta-path is the key contributor to this power since it enables inference by capturing the proximities between entities via rich semantic links. Previous HIN studies ask users to provide either 1) the meta-path(s) directly or 2) biased examples to generate the meta-path(s). However, lots of HINs (e.g., YAGO2 and Freebase) have rich schema consisting of a sophisticated and large number of types of entities and relations. It is impractical for users to provide the meta-path(s) to support the large scale inference, and biased examples will result in incorrect meta-path based inference, thus limit the power of the meta-path. In this paper, we propose a meta-path constrained inference framework to further release the ability of the meta-path, by efficiently learning the HIN inference patterns via a carefully designed tree structure; and performing unbiased random walk inference with little user guidance. The experiment results on YAGO2 and DBLP datasets show the state-of-the-art performance of the meta-path constrained inference framework.
Identifying the number of clusters for K-Means: A hypersphere density based approach
Venue: arXiv cs.LG 2019
Year: 2019

Application of K-Means algorithm is restricted by the fact that the number of clusters should be known beforehand. Previously suggested methods to solve this problem are either ad hoc or require parametric assumptions and complicated calculations. The proposed method aims to solve this conundrum by considering cluster hypersphere density as the factor to determine the number of clusters in the given dataset. The density is calculated by assuming a hypersphere around the cluster centroid for n-different number of clusters. The calculated values are plotted against their corresponding number of clusters and then the optimum number of clusters is obtained after assaying the elbow region of the graph. The method is simple, easy to comprehend, and provides robust and reliable results.
Stochastic Variational Inference via Upper Bound
Venue: arXiv cs.LG 2019
Year: 2019

Stochastic variational inference (SVI) plays a key role in Bayesian deep learning. Recently various divergences have been proposed to design the surrogate loss for variational inference. We present a simple upper bound of the evidence as the surrogate loss. This evidence upper bound (EUBO) equals to the log marginal likelihood plus the KL-divergence between the posterior and the proposal. We show that the proposed EUBO is tighter than previous upper bounds introduced by $χ$-divergence or $α$-divergence. To facilitate scalable inference, we present the numerical approximation of the gradient of the EUBO and apply the SGD algorithm to optimize the variational parameters iteratively. Simulation study with Bayesian logistic regression shows that the upper and lower bounds well sandwich the evidence and the proposed upper bound is favorably tight. For Bayesian neural network, the proposed EUBO-VI algorithm outperforms state-of-the-art results for various examples.
A Human-AI Loop Approach for Joint Keyword Discovery and Expectation Estimation in Micropost Event Detection
Venue: arXiv cs.IR 2019
Year: 2019

Microblogging platforms such as Twitter are increasingly being used in event detection. Existing approaches mainly use machine learning models and rely on event-related keywords to collect the data for model training. These approaches make strong assumptions on the distribution of the relevant micro-posts containing the keyword -- referred to as the expectation of the distribution -- and use it as a posterior regularization parameter during model training. Such approaches are, however, limited as they fail to reliably estimate the informativeness of a keyword and its expectation for model training. This paper introduces a Human-AI loop approach to jointly discover informative keywords for model training while estimating their expectation. Our approach iteratively leverages the crowd to estimate both keyword specific expectation and the disagreement between the crowd and the model in order to discover new keywords that are most beneficial for model training. These keywords and their expectation not only improve the resulting performance but also make the model training process more transparent. We empirically demonstrate the merits of our approach, both in terms of accuracy and interpretability, on multiple real-world datasets and show that our approach improves the state of the art by 24.3%.
KRM-based Dialogue Management
Venue: arXiv cs.HC 2019
Year: 2019

A KRM-based dialogue management (DM) is proposed using to implement human-computer dialogue system in complex scenarios. KRM-based DM has a well description ability and it can ensure the logic of the dialogue process. Then a complex application scenario in the Internet of Things (IOT) industry and a dialogue system implemented based on the KRM-based DM will be introduced, where the system allows enterprise customers to customize topics and adapts corresponding topics in the interaction process with users. The experimental results show that the system can complete the interactive tasks well, and can effectively solve the problems of topic switching, information inheritance between topics, change of dominance.
DLTPy: Deep Learning Type Inference of Python Function Signatures using Natural Language Context
Venue: arXiv cs.SE 2019
Year: 2019

Due to the rise of machine learning, Python is an increasingly popular programming language. Python, however, is dynamically typed. Dynamic typing has shown to have drawbacks when a project grows, while at the same time it improves developer productivity. To have the benefits of static typing, combined with high developer productivity, types need to be inferred. In this paper, we present DLTPy: a deep learning type inference solution for the prediction of types in function signatures based on the natural language context (identifier names, comments and return expressions) of a function. We found that DLTPy is effective and has a top-3 F1-score of 91.6%. This means that in most of the cases the correct type is within the top-3 predictions. We conclude that natural language contained in comments and return expressions are beneficial to predicting types more accurately. DLTPy does not significantly outperform or underperform the previous work NL2Type for Javascript, but does show that similar prediction is possible for Python.
EduBERT: Pretrained Deep Language Models for Learning Analytics
Venue: arXiv cs.CY 2019
Year: 2019

The use of large pretrained neural networks to create contextualized word embeddings has drastically improved performance on several natural language processing (NLP) tasks. These computationally expensive models have begun to be applied to domain-specific NLP tasks such as re-hospitalization prediction from clinical notes. This paper demonstrates that using large pretrained models produces excellent results on common learning analytics tasks. Pre-training deep language models using student forum data from a wide array of online courses improves performance beyond the state of the art on three text classification tasks. We also show that a smaller, distilled version of our model produces the best results on two of the three tasks while limiting computational cost. We make both models available to the research community at large.
GPU Support for Automatic Generation of Finite-Differences Stencil Kernels
Venue: arXiv cs.DC 2019
Year: 2019

The growth of data to be processed in the Oil & Gas industry matches the requirements imposed by evolving algorithms based on stencil computations, such as Full Waveform Inversion and Reverse Time Migration. Graphical processing units (GPUs) are an attractive architectural target for stencil computations because of its high degree of data parallelism. However, the rapid architectural and technological progression makes it difficult for even the most proficient programmers to remain up-to-date with the technological advances at a micro-architectural level. In this work, we present an extension for an open source compiler designed to produce highly optimized finite difference kernels for use in inversion methods named Devito. We embed it with the Oxford Parallel Domain Specific Language (OP-DSL) in order to enable automatic code generation for GPU architectures from a high-level representation. We aim to enable users coding in a symbolic representation level to effortlessly get their implementations leveraged by the processing capacities of GPU architectures. The implemented backend is evaluated on a NVIDIA GTX Titan Z, and on a NVIDIA Tesla V100 in terms of operational intensity through the roof-line model for varying space-order discretization levels of 3D acoustic isotropic wave propagation stencil kernels with and without symbolic optimizations. It achieves approximately 63% of V100's peak performance and 24% of Titan Z's peak performance for stencil kernels over grids with 256 points. Our study reveals that improving memory usage should be the most efficient strategy for leveraging the performance of the implemented solution on the evaluated architectures.
Short-Packet Transmission over a Bidirectional Massive MIMO link
Venue: arXiv cs.IT 2019
Year: 2019

We consider the transmission of short packets over a bidirectional communication link where multiple devices, e.g., sensors and actuators, exchange small-data payloads with a base station equipped with a large antenna array. Using results from finite-blocklength information theory, we characterize the minimum SNR required to achieve a target error probability for a fixed packet length and a fixed payload size. Our nonasymptotic analysis, which applies to the scenario in which the bidirectional communication is device-initiated, and also to the more challenging case when it is base-station initiated, provides guidelines on the design of massive multiple-input multiple-output links that need to support sporadic ultra-reliable low-latency transmissions. Specifically, it allows us to determine the optimal amount of resources that need to be dedicated to the acquisition of channel state information.