OpenRaiser/Intern-Atlas huggingface dataset:
NDE 4.0 From Design Thinking to Strategy
Venue: arXiv cs.OH 2020
Year: 2020

Cyber technologies are offering new horizons for quality control in manufacturing and safety assurance in-service of physical assets. The line between non-destructive evaluation (NDE) and Industry 4.0 is getting blurred since both are sensory data-driven domains. This multidisciplinary approach has led to the emergence of a new capability: NDE 4.0. The NDT community is coming together once again to define the purpose, chart the process, and address the adoption of emerging technologies. In this paper, the authors have taken a design thinking approach to spotlight proper objectives for research on this subject. It begins with qualitative research on twenty different perceptions of stakeholders and misconceptions around the current state of NDE. The interpretation is used to define ten value propositions or use cases under "NDE for Industry 4.0" and "Industry 4.0 for NDE" leading up to the clarity of purpose for NDE 4.0, enhanced safety and economic value for stakeholders. To pursue this worthy cause, the paper delves into some of the top adoption challenges, and proposes a journey of managed innovation, conscious skills development, and a new form of leadership required to succeed in the cyber-physical world.
A Spectral Approach to Network Design
Venue: arXiv cs.DS 2020
Year: 2020

We present a spectral approach to design approximation algorithms for network design problems. We observe that the underlying mathematical questions are the spectral rounding problems, which were studied in spectral sparsification and in discrepancy theory. We extend these results to incorporate additional non-negative linear constraints, and show that they can be used to significantly extend the scope of network design problems that can be solved. Our algorithm for spectral rounding is an iterative randomized rounding algorithm based on the regret minimization framework. In some settings, this provides an alternative spectral algorithm to achieve constant factor approximation for the classical survivable network design problem, and partially answers a question of Bansal about survivable network design with concentration property. We also show many other applications of the spectral rounding results, including weighted experimental design and additive spectral sparsification.
Self-Supervised Log Parsing
Venue: arXiv cs.LG 2020
Year: 2020

Logs are extensively used during the development and maintenance of software systems. They collect runtime events and allow tracking of code execution, which enables a variety of critical tasks such as troubleshooting and fault detection. However, large-scale software systems generate massive volumes of semi-structured log records, posing a major challenge for automated analysis. Parsing semi-structured records with free-form text log messages into structured templates is the first and crucial step that enables further analysis. Existing approaches rely on log-specific heuristics or manual rule extraction. These are often specialized in parsing certain log types, and thus, limit performance scores and generalization. We propose a novel parsing technique called NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling (MLM). In the process of parsing, the model extracts summarizations from the logs in the form of a vector embedding. This allows the coupling of the MLM as pre-training with a downstream anomaly detection task. We evaluate the parsing performance of NuLog on 10 real-world log datasets and compare the results with 12 parsing techniques. The results show that NuLog outperforms existing methods in parsing accuracy with an average of 99% and achieves the lowest edit distance to the ground truth templates. Additionally, two case studies are conducted to demonstrate the ability of the approach for log-based anomaly detection in both supervised and unsupervised scenario. The results show that NuLog can be successfully used to support troubleshooting tasks. The implementation is available at https://github.com/nulog/nulog.
AMIL: Adversarial Multi Instance Learning for Human Pose Estimation
Venue: arXiv cs.CV 2020
Year: 2020

Human pose estimation has an important impact on a wide range of applications from human-computer interface to surveillance and content-based video retrieval. For human pose estimation, joint obstructions and overlapping upon human bodies result in departed pose estimation. To address these problems, by integrating priors of the structure of human bodies, we present a novel structure-aware network to discreetly consider such priors during the training of the network. Typically, learning such constraints is a challenging task. Instead, we propose generative adversarial networks as our learning model in which we design two residual multiple instance learning (MIL) models with the identical architecture, one is used as the generator and the other one is used as the discriminator. The discriminator task is to distinguish the actual poses from the fake ones. If the pose generator generates the results that the discriminator is not able to distinguish from the real ones, the model has successfully learnt the priors. In the proposed model, the discriminator differentiates the ground-truth heatmaps from the generated ones, and later the adversarial loss back-propagates to the generator. Such procedure assists the generator to learn reasonable body configurations and is proved to be advantageous to improve the pose estimation accuracy. Meanwhile, we propose a novel function for MIL. It is an adjustable structure for both instance selection and modeling to appropriately pass the information between instances in a single bag. In the proposed residual MIL neural network, the pooling action adequately updates the instance contribution to its bag. The proposed adversarial residual multi-instance neural network that is based on pooling has been validated on two datasets for the human pose estimation task and successfully outperforms the other state-of-arts models.
ScanSSD: Scanning Single Shot Detector for Mathematical Formulas in PDF Document Images
Venue: arXiv cs.CV 2020
Year: 2020

We introduce the Scanning Single Shot Detector (ScanSSD) for locating math formulas offset from text and embedded in textlines. ScanSSD uses only visual features for detection: no formatting or typesetting information such as layout, font, or character labels are employed. Given a 600 dpi document page image, a Single Shot Detector (SSD) locates formulas at multiple scales using sliding windows, after which candidate detections are pooled to obtain page-level results. For our experiments we use the TFD-ICDAR2019v2 dataset, a modification of the GTDB scanned math article collection. ScanSSD detects characters in formulas with high accuracy, obtaining a 0.926 f-score, and detects formulas with high recall overall. Detection errors are largely minor, such as splitting formulas at large whitespace gaps (e.g., for variable constraints) and merging formulas on adjacent textlines. Formula detection f-scores of 0.796 (IOU $\geq0.5$) and 0.733 (IOU $\ge 0.75$) are obtained. Our data, evaluation tools, and code are publicly available.
Stable Neural Flows
Venue: arXiv cs.LG 2020
Year: 2020

We introduce a provably stable variant of neural ordinary differential equations (neural ODEs) whose trajectories evolve on an energy functional parametrised by a neural network. Stable neural flows provide an implicit guarantee on asymptotic stability of the depth-flows, leading to robustness against input perturbations and low computational burden for the numerical solver. The learning procedure is cast as an optimal control problem, and an approximate solution is proposed based on adjoint sensivity analysis. We further introduce novel regularizers designed to ease the optimization process and speed up convergence. The proposed model class is evaluated on non-linear classification and function approximation tasks.
Discovering Business Area Effects to Process Mining Analysis Using Clustering and Influence Analysis
Venue: arXiv cs.DB 2020
Year: 2020

A common challenge for improving business processes in large organizations is that business people in charge of the operations are lacking a fact-based understanding of the execution details, process variants, and exceptions taking place in business operations. While existing process mining methodologies can discover these details based on event logs, it is challenging to communicate the process mining findings to business people. In this paper, we present a novel methodology for discovering business areas that have a significant effect on the process execution details. Our method uses clustering to group similar cases based on process flow characteristics and then influence analysis for detecting those business areas that correlate most with the discovered clusters. Our analysis serves as a bridge between BPM people and business, people facilitating the knowledge sharing between these groups. We also present an example analysis based on publicly available real-life purchase order process data.
An Efficient Implementation of Manacher's Algorithm
Venue: arXiv cs.DS 2020
Year: 2020

Manacher's algorithm has been shown to be optimal to the longest palindromic substring problem. Many of the existing implementations of this algorithm, however, unanimously required in-memory construction of an augmented string that is twice as long as the original string. Although it has found widespread use, we found that this preprocessing is neither economic nor necessary. We present a more efficient implementation of Manacher's algorithm based on index mapping that makes the string augmentation process obsolete.
Patient-centric HetNets Powered by Machine Learning and Big Data Analytics for 6G Networks
Venue: arXiv cs.NI 2020
Year: 2020

Having a cognitive and self-optimizing network that proactively adapts not only to channel conditions, but also according to its users needs can be one of the highest forthcoming priorities of future 6G Heterogeneous Networks (HetNets). In this paper, we introduce an interdisciplinary approach linking the concepts of e-healthcare, priority, big data analytics (BDA) and radio resource optimization in a multi-tier 5G network. We employ three machine learning (ML) algorithms, namely, naive Bayesian (NB) classifier, logistic regression (LR), and decision tree (DT), working as an ensemble system to analyze historical medical records of stroke out-patients (OPs) and readings from body-attached internet-of-things (IoT) sensors to predict the likelihood of an imminent stroke. We convert the stroke likelihood into a risk factor functioning as a priority in a mixed integer linear programming (MILP) optimization model. Hence, the task is to optimally allocate physical resource blocks (PRBs) to HetNet users while prioritizing OPs by granting them high gain PRBs according to the severity of their medical state. Thus, empowering the OPs to send their critical data to their healthcare provider with minimized delay. To that end, two optimization approaches are proposed, a weighted sum rate maximization (WSRMax) approach and a proportional fairness (PF) approach. The proposed approaches increased the OPs average signal to interference plus noise (SINR) by 57% and 95%, respectively. The WSRMax approach increased the system total SINR to a level higher than that of the PF approach, nevertheless, the PF approach yielded higher SINRs for the OPs, better fairness and a lower margin of error.
Dynamic Distribution-Sensitive Point Location
Venue: arXiv cs.CG 2020
Year: 2020

We propose a dynamic data structure for the distribution-sensitive point location problem. Suppose that there is a fixed query distribution in $\mathbb{R}^2$, and we are given an oracle that can return in $O(1)$ time the probability of a query point falling into a polygonal region of constant complexity. We can maintain a convex subdivision $\cal S$ with $n$ vertices such that each query is answered in $O(\mathrm{OPT})$ expected time, where OPT is the minimum expected time of the best linear decision tree for point location in $\cal S$. The space and construction time are $O(n\log^2 n)$. An update of $\cal S$ as a mixed sequence of $k$ edge insertions and deletions takes $O(k\log^5 n)$ amortized time. As a corollary, the randomized incremental construction of the Voronoi diagram of $n$ sites can be performed in $O(n\log^5 n)$ expected time so that, during the incremental construction, a nearest neighbor query at any time can be answered optimally with respect to the intermediate Voronoi diagram at that time.
Random Access based Reliable Uplink Communication and Power Transfer using Dynamic Power Splitting
Venue: arXiv cs.IT 2020
Year: 2020

Large communication networks, e.g. Internet of Things (IoT), are known to be vulnerable to co-channel interference. One possibility to address this issue is the use of orthogonal multiple access (OMA) techniques. However, due to a potentially very long duty cycle, OMA is not well suited for such schemes. Instead, random medium access (RMA) appears more promising. An RMA scheme is based on transmission of short data packets with random scheduling, which is typically unknown to the receiver. The received signal, which consists of the overlapping packets, can be used for energy harvesting and powering of a relay device. Such an energy harvesting relay may utilize the energy for further information processing and uplink transmission. In this paper, we address the design of a simultaneous information and power transfer scheme based on randomly scheduled packet transmissions and reliable symbol detection. We formulate a prediction problem with the goal to maximize the harvested power for an RMA scenario. In order to solve this problem, we propose a new prediction method, which shows a significant performance improvement compared to the straightforward baseline scheme. Furthermore, we investigate the complexity of the proposed method and its vulnerability to imperfect channel state information.
Deep Quaternion Features for Privacy Protection
Venue: arXiv cs.LG 2020
Year: 2020

We propose a method to revise the neural network to construct the quaternion-valued neural network (QNN), in order to prevent intermediate-layer features from leaking input information. The QNN uses quaternion-valued features, where each element is a quaternion. The QNN hides input information into a random phase of quaternion-valued features. Even if attackers have obtained network parameters and intermediate-layer features, they cannot extract input information without knowing the target phase. In this way, the QNN can effectively protect the input privacy. Besides, the output accuracy of QNNs only degrades mildly compared to traditional neural networks, and the computational cost is much less than other privacy-preserving methods.
Placement Optimization with Deep Reinforcement Learning
Venue: arXiv cs.AI 2020
Year: 2020

Placement Optimization is an important problem in systems and chip design, which consists of mapping the nodes of a graph onto a limited set of resources to optimize for an objective, subject to constraints. In this paper, we start by motivating reinforcement learning as a solution to the placement problem. We then give an overview of what deep reinforcement learning is. We next formulate the placement problem as a reinforcement learning problem and show how this problem can be solved with policy gradient optimization. Finally, we describe lessons we have learned from training deep reinforcement learning policies across a variety of placement optimization problems.
MINT: Deep Network Compression via Mutual Information-based Neuron Trimming
Venue: arXiv cs.LG 2020
Year: 2020

Most approaches to deep neural network compression via pruning either evaluate a filter's importance using its weights or optimize an alternative objective function with sparsity constraints. While these methods offer a useful way to approximate contributions from similar filters, they often either ignore the dependency between layers or solve a more difficult optimization objective than standard cross-entropy. Our method, Mutual Information-based Neuron Trimming (MINT), approaches deep compression via pruning by enforcing sparsity based on the strength of the relationship between filters of adjacent layers, across every pair of layers. The relationship is calculated using conditional geometric mutual information which evaluates the amount of similar information exchanged between the filters using a graph-based criterion. When pruning a network, we ensure that retained filters contribute the majority of the information towards succeeding layers which ensures high performance. Our novel approach outperforms existing state-of-the-art compression-via-pruning methods on the standard benchmarks for this task: MNIST, CIFAR-10, and ILSVRC2012, across a variety of network architectures. In addition, we discuss our observations of a common denominator between our pruning methodology's response to adversarial attacks and calibration statistics when compared to the original network.
An Analysis on the Learning Rules of the Skip-Gram Model
Venue: arXiv cs.CL 2020
Year: 2020

To improve the generalization of the representations for natural language processing tasks, words are commonly represented using vectors, where distances among the vectors are related to the similarity of the words. While word2vec, the state-of-the-art implementation of the skip-gram model, is widely used and improves the performance of many natural language processing tasks, its mechanism is not yet well understood. In this work, we derive the learning rules for the skip-gram model and establish their close relationship to competitive learning. In addition, we provide the global optimal solution constraints for the skip-gram model and validate them by experimental results.
The Cost of Privacy in Asynchronous Differentially-Private Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

We consider training machine learning models using Training data located on multiple private and geographically-scattered servers with different privacy settings. Due to the distributed nature of the data, communicating with all collaborating private data owners simultaneously may prove challenging or altogether impossible. In this paper, we develop differentially-private asynchronous algorithms for collaboratively training machine-learning models on multiple private datasets. The asynchronous nature of the algorithms implies that a central learner interacts with the private data owners one-on-one whenever they are available for communication without needing to aggregate query responses to construct gradients of the entire fitness function. Therefore, the algorithm efficiently scales to many data owners. We define the cost of privacy as the difference between the fitness of a privacy-preserving machine-learning model and the fitness of trained machine-learning model in the absence of privacy concerns. We prove that we can forecast the performance of the proposed privacy-preserving asynchronous algorithms. We demonstrate that the cost of privacy has an upper bound that is inversely proportional to the combined size of the training datasets squared and the sum of the privacy budgets squared. We validate the theoretical results with experiments on financial and medical datasets. The experiments illustrate that collaboration among more than 10 data owners with at least 10,000 records with privacy budgets greater than or equal to 1 results in a superior machine-learning model in comparison to a model trained in isolation on only one of the datasets, illustrating the value of collaboration and the cost of the privacy. The number of the collaborating datasets can be lowered if the privacy budget is higher.
Super Low Resolution RF Powered Accelerometers for Alerting on Hospitalized Patient Bed Exits
Venue: arXiv cs.CY 2020
Year: 2020

Falls have serious consequences and are prevalent in acute hospitals and nursing homes caring for older people. Most falls occur in bedrooms and near the bed. Technological interventions to mitigate the risk of falling aim to automatically monitor bed-exit events and subsequently alert healthcare personnel to provide timely supervisions. We observe that frequency-domain information related to patient activities exist predominantly in very low frequencies. Therefore, we recognise the potential to employ a low resolution acceleration sensing modality in contrast to powering and sensing with a conventional MEMS (Micro Electro Mechanical System) accelerometer. Consequently, we investigate a batteryless sensing modality with low cost wirelessly powered Radio Frequency Identification (RFID) technology with the potential for convenient integration into clothing, such as hospital gowns. We design and build a passive accelerometer-based RFID sensor embodiment---ID-Sensor---for our study. The sensor design allows deriving ultra low resolution acceleration data from the rate of change of unique RFID tag identifiers in accordance with the movement of a patient's upper body. We investigate two convolutional neural network architectures for learning from raw RFID-only data streams and compare performance with a traditional shallow classifier with engineered features. We evaluate performance with 23 hospitalized older patients. We demonstrate, for the first time and to the best of knowledge, that: i) the low resolution acceleration data embedded in the RF powered ID-Sensor data stream can provide a practicable method for activity recognition; and ii) highly discriminative features can be efficiently learned from the raw RFID-only data stream using a fully convolutional network architecture.
Detecting Lane and Road Markings at A Distance with Perspective Transformer Layers
Venue: arXiv cs.CV 2020
Year: 2020

Accurate detection of lane and road markings is a task of great importance for intelligent vehicles. In existing approaches, the detection accuracy often degrades with the increasing distance. This is due to the fact that distant lane and road markings occupy a small number of pixels in the image, and scales of lane and road markings are inconsistent at various distances and perspectives. The Inverse Perspective Mapping (IPM) can be used to eliminate the perspective distortion, but the inherent interpolation can lead to artifacts especially around distant lane and road markings and thus has a negative impact on the accuracy of lane marking detection and segmentation. To solve this problem, we adopt the Encoder-Decoder architecture in Fully Convolutional Networks and leverage the idea of Spatial Transformer Networks to introduce a novel semantic segmentation neural network. This approach decomposes the IPM process into multiple consecutive differentiable homographic transform layers, which are called "Perspective Transformer Layers". Furthermore, the interpolated feature map is refined by subsequent convolutional layers thus reducing the artifacts and improving the accuracy. The effectiveness of the proposed method in lane marking detection is validated on two public datasets: TuSimple and ApolloScape