OpenRaiser/Intern-Atlas huggingface dataset:
Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis
Venue: arXiv cs.MS 2019
Year: 2019

Hierarchical Matrix (H-matrix) is an approximation technique which splits a target dense matrix into multiple submatrices, and where a selected portion of submatrices are low-rank approximated. The technique substantially reduces both time and space complexity of dense matrix vector multiplication, and hence has been applied to numerous practical problems. In this paper, we aim to accelerate the H-matrix vector multiplication by introducing mixed precision computing, where we employ both binary64 (FP64) and binary32 (FP32) arithmetic operations. We propose three methods to introduce mixed precision computing to H-matrix vector multiplication, and then evaluate them in a boundary element method (BEM) analysis. The numerical tests examine the effects of mixed precision computing, particularly on the required simulation time and rate of convergence of the iterative (BiCG-STAB) linear solver. We confirm the effectiveness of the proposed methods.
Link-Layer Capacity of Downlink NOMA with Generalized Selection Combining Receivers
Venue: arXiv cs.IT 2019
Year: 2019

Non-orthogonal multiple access (NOMA) has drawn tremendous attention, being a potential candidate for the spectrum access technology for the fifth-generation (5G) and beyond 5G (B5G) wireless communications standards. Most research related to NOMA focuses on the system performance from Shannon's capacity perspective, which, although a critical system design criterion, fails to quantify the effect of delay constraints imposed by future wireless applications. In this paper, we analyze the performance of a single-input multiple-output (SIMO) two-user downlink NOMA system, in terms of the link-layer achievable rate, known as effective capacity (EC), which captures the performance of the system under a delay-limited quality-of-service (QoS) constraint. For signal combining at the receiver side, we use generalized selection combining (GSC), which bridges the performance gap between the two conventional diversity combining schemes, namely selection combining (SC) and maximal-ratio combining (MRC). We also derive two approximate expressions for the EC of NOMA-GSC which are accurate at low-SNR and at high-SNR, respectively. The analysis reveals a tradeoff between the number of implemented receiver radio-frequency (RF) chains and the achieved performance, and can be used to determine the appropriate number of paths to combine in a practical receiver design.
Deep Learning of Subsurface Flow via Theory-guided Neural Network
Venue: arXiv cs.LG 2019
Year: 2019

Active researches are currently being performed to incorporate the wealth of scientific knowledge into data-driven approaches (e.g., neural networks) in order to improve the latter's effectiveness. In this study, the Theory-guided Neural Network (TgNN) is proposed for deep learning of subsurface flow. In the TgNN, as supervised learning, the neural network is trained with available observations or simulation data while being simultaneously guided by theory (e.g., governing equations, other physical constraints, engineering controls, and expert knowledge) of the underlying problem. The TgNN can achieve higher accuracy than the ordinary Artificial Neural Network (ANN) because the former provides physically feasible predictions and can be more readily generalized beyond the regimes covered with the training data. Furthermore, the TgNN model is proposed for subsurface flow with heterogeneous model parameters. Several numerical cases of two-dimensional transient saturated flow are introduced to test the performance of the TgNN. In the learning process, the loss function contains data mismatch, as well as PDE constraint, engineering control, and expert knowledge. After obtaining the parameters of the neural network by minimizing the loss function, a TgNN model is built that not only fits the data, but also adheres to physical/engineering constraints. Predicting the future response can be easily realized by the TgNN model. In addition, the TgNN model is tested in more complicated scenarios, such as prediction with changed boundary conditions, learning from noisy data or outliers, transfer learning, and engineering controls. Numerical results demonstrate that the TgNN model achieves much better predictability, reliability, and generalizability than ANN models due to the physical/engineering constraints in the former.
Construction of Const Dimension Codes from Serval Parallel Lift MRD Code
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we generalize the method of using two parallel versions of the lifted MRD code from the existing work [1]. The Delsarte theorem of the rank distribution of MRD codes is an important part to count codewords in our construction. We give a new generalize construction to the following bounds: if n>=k>=d, then $Aq(n + k,k,d)>=q^{n(k-\frac{d}{2}+1)}+\sum_{r=\frac{d}{2}}^{k-\frac{d}{2}} A_r(Q_q(n,k,\frac{d}{2})).$ On this basis, we also give a construction of constant-dimension subspace codes from several parallel versions of lifted MRD codes. This construction contributes to a new lower bounds for Aq((s+1)k+n,d,k).
Energy-Aware Analog Aggregation for Federated Learning with Redundant Data
Venue: arXiv cs.IT 2019
Year: 2019

Federated learning (FL) enables workers to learn a model collaboratively by using their local data, with the help of a parameter server (PS) for global model aggregation. The high communication cost for periodic model updates and the non-independent and identically distributed (i.i.d.) data become major bottlenecks for FL. In this work, we consider analog aggregation to scale down the communication cost with respect to the number of workers, and introduce data redundancy to the system to deal with non-i.i.d. data. We propose an online energy-aware dynamic worker scheduling policy, which maximizes the average number of workers scheduled for gradient update at each iteration under a long-term energy constraint, and analyze its performance based on Lyapunov optimization. Experiments using MNIST dataset show that, for non-i.i.d. data, doubling data storage can improve the accuracy by 9.8% under a stringent energy budget, while the proposed policy can achieve close-to-optimal accuracy without violating the energy constraint.
Federated Learning with Differential Privacy: Algorithms and Performance Analysis
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a $K$-random scheduling strategy, where $K$ ($1<K<N$) clients are randomly selected from the $N$ overall clients to participate in each aggregation. We also develop the corresponding convergence bound of the loss function in this case and the $K$-random scheduling strategy can also retain the above three properties. Moreover, we find that there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the designs on various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.
A Simple and Efficient Method to Compute a Single Linkage Dendrogram
Venue: arXiv cs.DS 2019
Year: 2019

We address the problem of computing a single linkage dendrogram. A possible approach is to: (i) Form an edge weighted graph $G$ over the data, with edge weights reflecting dissimilarities. (ii) Calculate the MST $T$ of $G$. (iii) Break the longest edge of $T$ thereby splitting it into subtrees $T_L$, $T_R$. (iv) Apply the splitting process recursively to the subtrees. This approach has the attractive feature that Prim's algorithm for MST construction calculates distances as needed, and hence there is no need to ever store the inter-point distance matrix. The recursive partitioning algorithm requires us to determine the vertices (and edges) of $T_L$ and $T_R$. We show how this can be done easily and efficiently using information generated by Prim's algorithm without any additional computational cost.
Privacy-Preserving Machine Learning Using EtC Images
Venue: arXiv cs.CR 2019
Year: 2019

In this paper, we propose a novel privacy-preserving machine learning scheme with encrypted images, called EtC (Encryption-then-Compression) images. Using machine learning algorithms in cloud environments has been spreading in many fields. However, there are serious issues with it for end users, due to semi-trusted cloud providers. Accordingly, we propose using EtC images, which have been proposed for EtC systems with JPEG compression. In this paper, a novel property of EtC images is considered under the use of z-score normalization. It is demonstrated that the use of EtC images allows us not only to protect visual information of images, but also to preserve both the Euclidean distance and the inner product between vectors. In addition, dimensionality reduction is shown to can be applied to EtC images for fast and accurate matching. In an experiment, the proposed scheme is applied to a facial recognition algorithm with classifiers for confirming the effectiveness of the scheme under the use of support vector machine (SVM) with the kernel trick.
Engaging in Dialogue about an Agent's Norms and Behaviors
Venue: arXiv cs.CL 2019
Year: 2019

We present a set of capabilities allowing an agent planning with moral and social norms represented in temporal logic to respond to queries about its norms and behaviors in natural language, and for the human user to add and remove norms directly in natural language. The user may also pose hypothetical modifications to the agent's norms and inquire about their effects.
Extending Relational Query Processing with ML Inference
Venue: arXiv cs.DB 2019
Year: 2019

The broadening adoption of machine learning in the enterprise is increasing the pressure for strict governance and cost-effective performance, in particular for the common and consequential steps of model storage and inference. The RDBMS provides a natural starting point, given its mature infrastructure for fast data access and processing, along with support for enterprise features (e.g., encryption, auditing, high-availability). To take advantage of all of the above, we need to address a key concern: Can in-RDBMS scoring of ML models match (outperform?) the performance of dedicated frameworks? We answer the above positively by building Raven, a system that leverages native integration of ML runtimes (i.e., ONNX Runtime) deep within SQL Server, and a unified intermediate representation (IR) to enable advanced cross-optimizations between ML and DB operators. In this optimization space, we discover the most exciting research opportunities that combine DB/Compiler/ML thinking. Our initial evaluation on real data demonstrates performance gains of up to 5.5x from the native integration of ML in SQL Server, and up to 24x from cross-optimizations--we will demonstrate Raven live during the conference talk.
Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning
Venue: arXiv cs.LG 2019
Year: 2019

Generative adversarial imitation learning (GAIL) has attracted increasing attention in the field of robot learning. It enables robots to learn a policy to achieve a task demonstrated by an expert while simultaneously estimating the reward function behind the expert's behaviors. However, this framework is limited to learning a single task with a single reward function. This study proposes an extended framework called situated GAIL (S-GAIL), in which a task variable is introduced to both the discriminator and generator of the GAIL framework. The task variable has the roles of discriminating different contexts and making the framework learn different reward functions and policies for multiple tasks. To achieve the early convergence of learning and robustness during reward estimation, we introduce a term to adjust the entropy regularization coefficient in the generator's objective function. Our experiments using two setups (navigation in a discrete grid world and arm reaching in a continuous space) demonstrate that the proposed framework can acquire multiple reward functions and policies more effectively than existing frameworks. The task variable enables our framework to differentiate contexts while sharing common knowledge among multiple tasks.
Finding the most similar textual documents using Case-Based Reasoning
Venue: arXiv cs.LG 2019
Year: 2019

In recent years, huge amounts of unstructured textual data on the Internet are a big difficulty for AI algorithms to provide the best recommendations for users and their search queries. Since the Internet became widespread, a lot of research has been done in the field of Natural Language Processing (NLP) and machine learning. Almost every solution transforms documents into Vector Space Models (VSM) in order to apply AI algorithms over them. One such approach is based on Case-Based Reasoning (CBR). Therefore, the most important part of those systems is to compute the similarity between numerical data points. In 2016, the new similarity TS-SS metric is proposed, which showed state-of-the-art results in the field of textual mining for unsupervised learning. However, no one before has investigated its performances for supervised learning (classification task). In this work, we devised a CBR system capable of finding the most similar documents for a given query aiming to investigate performances of the new state-of-the-art metric, TS-SS, in addition to the two other geometrical similarity measures --- Euclidean distance and Cosine similarity --- that showed the best predictive results over several benchmark corpora. The results show surprising inappropriateness of TS-SS measure for high dimensional features.
Capacity improvement of non-orthogonal multiple access downlink transmission by orbital angular momentum based mode division multiple access
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, non-orthogonal multiple access (NOMA) downlink transmission is integrated with orbital angular momentum (OAM) based mode division multiple access (MDMA), called NOMA-OAM-MDMA. Different OAM modes can generate different OAM waves for different superimposed signals. So, every OAM wave will be transmitted a superimposed signal from the base station to the cell center user (CCU) and cell edge user (CEU). In this way, multiple OAM waves with multiple superimposed signals will transmit from BS to CCU and CEU simultaneously to enhance the capacity of NOMA downlink transmission. Finally, the effectiveness of the proposed schemes over the existing scheme and conventional orthogonal multiple access based scheme are demonstrated through the result analysis.
Practice meets Principle: Tracking Software and Data Citations to Zenodo DOIs
Venue: arXiv cs.DL 2019
Year: 2019

Data and software citations are crucial for the transparency of research results and for the transmission of credit. But they are hard to track, because of the absence of a common citation standard. As a consequence, the FORCE11 recently proposed data and software citation principles as guidance for authors. Zenodo is recognized for the implementation of DOIs for software on a large scale. The minting of complementary DOIs for the version and concept allows measuring the impact of dynamic software. This article investigates characteristics of 5,456 citations to Zenodo data and software that were captured by the Asclepias Broker in January 2019. We analyzed the current state of data and software citation practices and the quality of software citation recommendations with regard to the impact of recent standardization efforts. Our findings prove that current citation practices and recommendations do not match proposed citation standards. We consequently suggest practical first steps towards the implementation of the software citation principles.
Operator and Manufacturer Independent D2D Private Link for Future 5G Networks
Venue: arXiv cs.CR 2019
Year: 2019

Direct Mobile-to-Mobile communication mode known also as Device-to-Device (D2D) communication is expected to be supported in the 5G mobile system. D2D communication aims to improve system spectrum efficiency, overall system throughput, energy efficiency and reduce the connection delay between devices. However, new security threats and challenges need to be considered regarding device and user authentication to avoid unauthorized access, abuse and attacks on the whole system. In this paper, a strong standalone authentication technique therefore is proposed. It is based on combining users biometric identities and a new clone-resistant device identity. The novel property of the proposal is that it is fully independent on both device manufacturer and mobile system operator. The biometric identity deploys user keystroke dynamics and accelerometer to generate user biometric identity by deploying a machine learning technique. The proposed mobile device clone-resistant identity is based on deploying a new concept of a pure digital clone-resistant structure which is both manufacturer and mobile operator-independent. When combining both identities, a mutually authenticated D2D secured link between any two devices can be established in addition to a strong user-device authentication. Furthermore, the concept does not allow the managing trusted authority to intercept users private links. Being an independent and standalone system, the technique would offer a broad spectrum of attractive future smart applications over the 5G mobile system infrastructure.
Picking groups instead of samples: A close look at Static Pool-based Meta-Active Learning
Venue: arXiv cs.LG 2019
Year: 2019

Active Learning techniques are used to tackle learning problems where obtaining training labels is costly. In this work we use Meta-Active Learning to learn to select a subset of samples from a pool of unsupervised input for further annotation. This scenario is called Static Pool-based Meta- Active Learning. We propose to extend existing approaches by performing the selection in a manner that, unlike previous works, can handle the selection of each sample based on the whole selected subset.
Topology Optimization Design of Stretchable Metamaterials with Bezier Skeleton Explicit Density (BSED) Representation Algorithm
Venue: arXiv cs.CE 2019
Year: 2019

A new density field representation technique called the Bezier skeleton explicit density (BSED) representation scheme for topology optimization of stretchable metamaterials under finite deformation is proposed for the first time. The proposed approach overcomes a key deficiency in existing density-based optimization methods that typically yield designs that do not have smooth surfaces but have large number of small intricate features, which are difficult to manufacture even by additive manufacturing. In the proposed approach, Bezier curves are utilized to describe the skeleton of the design being optimized where the description of the entire design is realized by assigning thickness along the curves. This geometric representation technique ensures that the optimized design is smooth and concise and can easily be tuned to be manufacturable by additive manufacturing. In the optimization method, the density field is described by the Heaviside function defined on the Bezier curves. Compared to NURBS or B-spline based models, Bezier curves have fewer control parameters and hence are easier to manipulate for sensitivity derivation, especially for distance sensitivities. Due to its powerful curve fitting ability, using Bezier curve to represent density field allows exploring design space effectively and generating concise structures without any intricate small features at the borders. Furthermore, this density representation method is mesh independent and design variables are reduced significantly so that optimization problem can be solved efficiently using small-scale optimization algorithms such as sequential quadratic programming.
THAAD: Efficient Matching Queries under Temporal Abstraction for Anomaly Detection
Venue: arXiv cs.DS 2019
Year: 2019

In this paper we present a novel algorithm and efficient data structure for anomaly detection based on temporal data. Time-series data are represented by a sequence of symbolic time intervals, describing increasing and decreasing trends, in a compact way using gradient temporal abstraction technique. Then we identify unusual subsequences in the resulting sequence using dynamic data structure based on the geometric observations supporting polylogarithmic update and query times. Moreover, we introduce a new parameter to control the pairwise difference between the corresponding symbols in addition to a distance metric between the subsequences. Experimental results on a public DNS network traffic dataset show the superiority of our approach compared to the baselines.
3D hierarchical optimization for Multi-view depth map coding
Venue: arXiv cs.CV 2019
Year: 2019

Depth data has a widespread use since the popularity of high-resolution 3D sensors. In multi-view sequences, depth information is used to supplement the color data of each view. This article proposes a joint encoding of multiple depth maps with a unique representation. Color and depth images of each view are segmented independently and combined in an optimal Rate-Distortion fashion. The resulting partitions are projected to a reference view where a coherent hierarchy for the multiple views is built. A Rate-Distortionoptimization is applied to obtain the final segmentation choosing nodes of the hierarchy. The consistent segmentation is used to robustly encode depth maps of multiple views obtaining competitive results with HEVC coding standards. Available at: http://link.springer.com/article/10.1007/s11042-017-5409-z
Long-distance Detection of Bioacoustic Events with Per-channel Energy Normalization
Venue: arXiv cs.SD 2019
Year: 2019

This paper proposes to perform unsupervised detection of bioacoustic events by pooling the magnitudes of spectrogram frames after per-channel energy normalization (PCEN). Although PCEN was originally developed for speech recognition, it also has beneficial effects in enhancing animal vocalizations, despite the presence of atmospheric absorption and intermittent noise. We prove that PCEN generalizes logarithm-based spectral flux, yet with a tunable time scale for background noise estimation. In comparison with pointwise logarithm, PCEN reduces false alarm rate by 50x in the near field and 5x in the far field, both on avian and marine bioacoustic datasets. Such improvements come at moderate computational cost and require no human intervention, thus heralding a promising future for PCEN in bioacoustics.