Reviewerly/exHarmony huggingface dataset:
Accelerating Deep Convolutional Neural Network Inference Based on OpenCL

In recent years, in order to facilitate the efficient application of deep convolutional neural networks, it has become increasingly important to accelerate the inference stage of deep convolutional neural networks. But with the development of numerous heterogeneous computing devices, today’s popular deep learning inference tools only support specific devices, so they cannot effectively utilize different GPU devices to accelerate DNN inference. To address this issue, we propose an OpenCL-based parallel deep convolutional neural network inference algorithms. Firstly, we design and implement parallel kernel code using OpenCL to accelerate depthwise separable convolution, and implement parallel matrix multiplication combined with clBLAS to accelerate traditional convolution. Meanwhile, we design OpenCL parallel kernel codes for other operations in the inference stage of deep convolutional neural networks. Secondly, we further improve the inference performance by means of kernel fusion and increasing the workload per core. Finally, MobileNet v1 network and the 21-layer residual network based on OpenCL are run on AMD Radeon Vega Frontier GPU and Nvidia GeForce GTX 1070 GPU. Compared to the Caffe implementation, 40.16x, 1.67x speedups are achieved on the AMD GPU and 14.95x, 1.11x speedups are achieved on the Nvidia GPU.
C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication-Efficient Split Learning

Most existing studies improve the efficiency of Split learning (SL) by compressing the transmitted features. However, most works focus on dimension-wise compression that transforms high-dimensional features into a low-dimensional space. In this paper, we propose circular convolution-based batch-wise compression for SL (C3-SL) to compress multiple features into one single feature. To avoid information loss while merging multiple features, we exploit the quasi-orthogonality of features in high-dimensional space with circular convolution and superposition. To the best of our knowledge, we are the first to explore the potential of batch-wise compression under the SL scenario. Based on the simulation results on CIFAR-10 and CIFAR-100, our method achieves a 16x compression ratio with negligible accuracy drops compared with the vanilla SL. Moreover, C3-SL significantly reduces 1152x memory and 2.25x computation overhead compared to the state-of-the-art dimension-wise compression method.
Scene-Text Oriented Referring Expression Comprehension

Referring expression comprehension (REC) aims to identify and locate a specific object in visual scenes referred to by a natural language expression. Existing studies of REC only focus on basic visual attributes and neglect scene text. Since scene text has the functions of object identification and disambiguation, it is naturally and frequently used to refer to objects. However, existing methods do not explicitly recognize text in images and fail to align scene text mentioned in expressions with the text shown in images, resulting in object localization errors. This article takes the first step toward addressing these limitations. First, we introduce a new task called scene-text oriented referring expression comprehension, which aims to align visual cues and textual semantics of scene text with referring expressions and visual contents. Second, we propose a scene text awareness network that can bridge the gap between texts from two modalities by grounding visual representations of expression-correlated scene texts. Specifically, we propose a correlated text extraction module to solve the problem of lacking semantic understanding, and a correlated region activation module to address the fixed alignment problem and absent alignment problem. These modules ensure that the proposed method focuses on local regions that are most relevant to scene text, thus mitigating the misalignment of scene text with irrelevant regions. Third, to conduct quantitative evaluations, we establish a new benchmark dataset called RefText. Experimental results demonstrate that the proposed method can effectively comprehend scene-text oriented referring expressions and achieves excellent performance.
An Attention-Based Network for Single Image HDR Reconstruction

High dynamic range (HDR) imaging can represent a great range of real-world luminosity. In contrast, the traditional low dynamic range (LDR) imaging fails to represent a wide range of luminance since most digital cameras can capture a limited range of light intensity in a natural scene. Recent advances in deep learning allow reconstructing an HDR image from a single LDR image and surpass conventional methods performance. In this work, we propose a novel CNN for HDR image reconstruction based on residual learning and attention mechanism. The proposed network adopts an autoencoder structure with residual blocks trained in a fully end-to-end manner. Residual learning boosts the performance by optimizing the network to converge faster. Moreover, the attention mechanism allows the network to select and enhance meaningful features that will contribute to the reconstruction of the HDR image. In addition, we employ a contextual attention module to perform patch replacement on deep feature maps to help recover information in over-exposed areas. Extensive quantitative and qualitative experiments on public HDR datasets demonstrate the ability of our proposed method to effectively reconstruct a visually pleasing HDR image from a single LDR image and outperform existing approaches.
BERT-Based Vulnerability Type Identification with Effective Program Representation

Detecting vulnerabilities is essential to maintaining software security. At present, vulnerability detection based on deep learning has achieved remarkable results. The type of vulnerability could tell the vulnerability principles and help the programmer quickly pinpoint the precise location of vulnerabilities. Moreover, the type of vulnerability is very valuable for remediating it. Therefore, it is essential to identify vulnerability types. This paper proposes a new vulnerability type identification framework based on deep learning. The framework is based on syntax and semantics, and the detection granularity is fine to the slice level. To include comprehensive vulnerability types, we use four slicing methods to represent the program. In addition, we model four kinds of code slice features based on BERT. For evaluation, we used 64 three-level CWE-IDs vulnerability types in National Vulnerability Database (NVD) and Software Assurance Reference Dataset (SARD) for vulnerability type identification. The experimental results show that it has significant performance in vulnerability type identification.
Explainable Knowledge Graph Embedding: Inference Reconciliation for Knowledge Inferences Supporting Robot Actions

Learned knowledge graph representations supporting robots contain a wealth of domain knowledge that drives robot behavior. However, there does not exist an inference reconciliation framework that expresses how a knowledge graph representation affects a robot's sequential decision making. We use a pedagogical approach to explain the inferences of a learned, black-box knowledge graph representation, a knowledge graph embedding. Our interpretable model uses a decision tree classifier to locally approximate the predictions of the black-box model and provides natural language explanations interpretable by non-experts. Results from our algorithmic evaluation affirm our model design choices, and the results of our user studies with non-experts support the need for the proposed inference reconciliation framework. Critically, results from our simulated robot evaluation indicate that our explanations enable non-experts to correct erratic robot behaviors due to nonsensical beliefs within the black-box.
Scheduling Quantum Teleportation with Noisy Memories

Quantum teleportation channels can overcome the effects of photonic loss, a major challenge in the implementation of a quantum network over fiber. Teleportation channels are created by distributing an entangled state between two nodes, which is a probabilistic process requiring classical communication. This causes critical delays that can cause information loss as quantum data suffers from decoherence when stored in memory. In this work, we quantify the effect of decoherence on fidelity at a node in a quantum network due to the storage of qubits in noisy memory platforms. We model a memory platform as a buffer that stores incoming qubits waiting for the creation of a teleportation channel. Memory platforms are parameterized with decoherence rate and buffer size. We show that fidelity at a node is a linear sum of terms, exponentially decaying with time, where the decay rate depends on the decoherence rate of the memory platform. This allows us to utilize Laplace transforms to derive computable functions of average fidelity with respect to the load, buffer size, and decoherence rate of the memory platform. We prove that serving qubits in a Last In First Out order with pushout for buffer overflow management maximizes average fidelity. Last, we apply this framework to model a single repeater node to calculate the average fidelity of the end-to-end entanglement created by this repeater assuming perfect gate operations.
Structural causal models as boundary objects in AI system development

Artificial Intelligence (AI), and especially machine learning can be used to find statistical patterns in datasets with thousands of variables with ease. But an understanding of causality is difficult to learn for a machine. For humans however, realising causal relations is often not a difficult process, as we can refer to experience or scientific knowledge. Here we propose the use of structural causal models, represented through direct acyclic graphs, to design, determine, and communicate causal relations hidden beyond the statistical models of an AI. The idea is to make human insight in causal relations explicit and use this knowledge during AI system development. In a joint-industry project we discovered that structural causal models can serve as living boundary objects that facilitate coordination of domain experts, data scientists, systems engineers, and AI experts in AI system development.
Self-supervised Indoor 360-Degree Depth Estimation via Structural Regularization

Estimating $$360^{\circ }$$ depth information has attracted considerable attention due to the fast development of emerging $$360^{\circ }$$ cameras. However, most researches only focus on dealing with the distortion of $$360^{\circ }$$ images without considering the geometric information of $$360^{\circ }$$ images, leading to poor performance. In this paper, we conduct to apply indoor structure regularities for self-supervised $$360^{\circ }$$ image depth estimation. Specifically, we carefully design two geometric constraints for efficient model optimization including dominant direction normal constraint and planar consistency depth constraint. The dominant direction normal constraint enables to align the normal of indoor $$360^{\circ }$$ images with the direction of vanishing points. The planar consistency depth constraint is utilized to fit the estimated depth of each pixel by its 3D plane. Hence, incorporating these two geometric constraints can further facilitate the generation of accurate depth results for $$360^{\circ }$$ images. Extensive experiments illustrate that our designed method improves $$\delta _1$$ by an average of 4.82% compared to state-of-the-art methods on Matterport3D and Stanford2D3D datasets within 3D60.
GEOTHERMALCLOUD: MACHINE LEARNING FOR GEOTHERMAL RESOURCE EXPLORATION

Geothermal is a renewable energy source that can provide reliable and flexible electricity generation for the world. In the past decade, play fairway analysis (PFA) studies identified that geothermal resources without surface expression (e.g., blind/hidden hydrothermal systems) have vast potential. However, a comprehensive search for these blind systems can be time-consuming, expensive, and resource-intensive, with a low probability of success. Accelerated discovery of these blind resources is needed with growing energy needs and higher chances of exploration success. Recent advances in machine learning (ML) have shown promise in shortening the timeline for this discovery. This paper presents a novel ML-based methodology for geothermal exploration towards PFA applications. Our methodology is provided through our open-source ML framework, GeoThermalCloud https://github.com/SmartTensors/GeoThermalCloud.jl. The GeoThermalCloud uses a series of un-supervised, supervised, and physics-informed ML methods available in SmartTensors AI platform https://github.com/SmartTensors. Through GeoThermalCloud, we can identify hidden patterns in the geothermal field data needed to discover blind systems efficiently. Crucial geothermal signatures often overlooked in traditional PFA are extracted using the GeoThermalCloud and analyzed by the subject matter experts to provide ML-enhanced PFA (ePFA), which is informative for efficient exploration. We applied our ML methodology to various open-source geothermal datasets within the U.S. (some of these are collected by past PFA work). The results provide valuable insights into resource types within those regions. This ML-enhanced workflow makes the GeoThermalCloud attractive for the geothermal community to improve existing datasets and extract valuable information often unnoticed during geothermal exploration.
OWL ontology evolution: understanding and unifying the complex changes

Abstract Knowledge-based systems and their ontologies evolve due to different reasons. Ontology evolution is the adaptation of an ontology and the propagation of these changes to dependent artifacts such as queries and other ontologies. Besides identifying basic/simple changes, it is imperative to identify complex changes between two versions of the same ontology to make this adaptation possible. There are many definitions of complex changes applied to ontologies in the literature. However, their specifications across works vary both in formalization and textual description. Some works also use different terminologies to refer to a change, while others use the same vocabulary to refer to distinct changes. Therefore, there is a lack of a unified list of complex changes. The main goals of this paper are: (i) present the primary documents that identify complex changes; (ii) provide critical analyses about the set of the complex changes proposed in the literature and the documents mentioning them; (iii) provide a unified list of complex changes mapping different sets of complex changes proposed by several authors; (iv) present a classification for those complex changes; and (v) describe some open directions of the area. The mappings between the complex changes provide a mechanism to relate and compare different proposals. The unified list is thus a reference for the complex changes published in the literature. It may assist the development of tools to identify changes between two versions of the same ontology and enable the adaptation of artifacts that depend on the evolved ontology.
Detecting Insulator Strings as Linked Chain Structure in Smart Grid Inspection

In high-voltage power systems, insulators are essential components in transmission lines for increasing shooting distance and securing wires. Unmanned aerial vehicle imaging becomes a common way of inspecting the state of the insulators. However, the automatic detection of insulators with complex backgrounds is still a challenging task. Most of the existing object detection methods are based on anchors, which do not have sufficient ability to describe objects that have a string-like structure. To tackle it, inspired by the keypoints-based object detection method, we propose a novel chain structure framework to detect insulators. First, we model the string-like object as a chain structure consisting of keypoints and their linkage, by which the insulator strings features are efficiently encoded and trained in the proposed ChainNet. Then, an assembling algorithm is proposed to assemble the estimated keypoints and linkages into chains, by which the insulator strings can be tightly enclosed in rotational bounding boxes. We evaluate the proposed approach on our collected large-scale rigorous dataset under the Directed Intersection over Union metric. The extensive experimental results show that the proposed method achieves 75.7% mAP, which yields 3.4, 15, and 10.6 improvements to the state-of-the-art rotational anchor, axial-aligned anchor, and anchor-free detection methods, respectively. Moreover, the proposed framework can easily be extended to detect other string-like manmade objects in the industrial area.
Finding Badly Drawn Bunnies

As lovely as bunnies are, your sketched version would probably not do it justice (Fig. 1). This paper recognises this very problem and studies sketch quality measurement for the first time - letting you find these badly drawn ones. Our key discovery lies in exploiting the magnitude ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L$</tex> <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> norm) of a sketch feature as a quantitative quality metric. We propose Geometry-Aware Classification Layer (GACL), a generic method that makes feature-magnitude-as-quality-metric possible and importantly does it without the need for specific quality annotations from humans. GACL sees feature magnitude and recognisability learning as a dual task, which can be simultaneously optimised under a neat crossentropy classification loss. GACL is lightweight with theoretic guarantees and enjoys a nice geometric interpretation to reason its success. We confirm consistent quality agreements between our GACL-induced metric and human perception through a carefully designed human study. Notably, we demonstrate three practical sketch applications enabled for the first time using our quantitative quality metric.
Anomaly Detection for CPS via Memory-Augmented Reconstruction and Time Series Prediction

With the advent of the industrial information age, the scale and complexity of cyber-physical systems (CPS) are increasing. In this environment, cyber attacks against CPS are on the rise. To ensure the proper operation of CPS, unsupervised anomaly detection methods for CPS become very important. Current unsupervised anomaly detection work usually uses autoencoder model. However, autoencoder may suffer from over-generalization problems resulting in abnormal missed detection. Also, some works do not consideration of the temporal information about the data and will ignore contextual anomalies. In this paper, we propose a memory-augmented deep Autoencoder with predictive analytics (MemAE-P) for CPS unsupervised anomaly detection method, which simultaneously predicts and reconstructs the input data, overcoming the drawback of using each one alone. Specifically, we use a memory module to help the autoencoder reconstruction converge to normal samples to enhance the anomalous reconstruction error and avoid the generalization problem of anomalous samples. The attention-based Bi-LSTM is explored, which captures temporal dependence of CPS multi-sensor data, to predict the next moment sample given multiple previous moment samples. We consider reconstruction error and prediction error of input data to distinguish abnormal data from normal data. In addition, we use the error obtained from the model to infer which devices in the anomaly sample are most likely to be attacked. We demonstrated the effectiveness of MemAE-P through various experiments on three real CPS datasets.
A Blockchain-based Scalable Electronic Contract Signing System

As the COVID-19 continues to spread globally, more and more companies are transforming into remote online offices, leading to the expansion of electronic signatures. However, the existing electronic signatures platform has the problem of data-centered management. The system is subject to data loss, tampering, and leakage when an attack from outside or inside occurs. In response to the above problems, this paper designs an electronic signature solution and implements a prototype system based on the consortium blockchain. The solution divides the contract signing process into four states: contract upload, initiation signing, verification signing, and confirm signing. The signing process is mapped with the blockchain-linked data. Users initiate the signature transaction by signing the uploaded contract's hash. The sign state transition is triggered when the transaction is uploaded to the blockchain under the consensus mechanism and the smart contract control, which effectively ensures the integrity of the electronic contract and the non-repudiation of the electronic signature. Finally, the blockchain performance test shows that the system can be applied to the business scenario of contract signing.
Shape-analysis driven memory graph visualization

Analyzing heap dumps containing complex dynamic data structures is essential when debugging modern software systems. However, existing tools for visualizing memory graphs can neither deal with corrupt structures such as binary trees exhibiting cycles, nor do they offer adequate abstractions when being confronted with large heaps. This paper presents MGE (Memory Graph Explorer), a memory analyzer and visualizer that combines a novel memory graph abstraction with an interactive visualization. MGE borrows ideas from separation logic and shape analysis to reveal relationships between memory nodes, name recognized structures such as doubly-linked lists and binary trees, and summarize complex structures. This summarization works for corrupt data structures, too, and is particularly powerful for large, nested structures due to its support for interactive (un)folding. MGE's utility for aiding program comprehension is illustrated by real-world and textbook examples and contrasted with existing debuggers.
C<sup>2</sup>-YOLO: Rotating Object Detection Network for Remote Sensing Images with Complex Backgrounds

In remote sensing images, the background is complex, the distribution is dense, the target scale varies widely, there are many tiny targets, and the target directions are diverse, which is a challenging detection task. This paper proposes a remote sensing image rotating target detection network (C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -YOLO) that integrates the upsampling feature enhancement module and attention mechanism in response to these problems. The network is based on the YOLOV5 target detection algorithm, and the prediction head is added to enhance the ability of small target detection. The content-Aware ReAssembly of Features(CARAFE) module is introduced to design a new feature fusion module. We also integrate the Coordinate Attention (CA) module to focus on object locations in complex scenes. According to the rotation characteristics, we add an angle loss to the loss function to detect the rotation angle of the object. We conduct experiments on two public datasets, DOTA and HRSC2016. Compared with the original YOLOv5 algorithm, the detection accuracy of our algorithm is improved by 2.99% and 3.52%, which can achieve comparable performance to the state-of-the-art detection methods.