Reviewerly/exHarmony huggingface dataset:
Trouble-Shooting at GAN Point: Improving Functional Safety in Deep Learning Accelerators

The proliferation of Deep Neural Networks (DNNs) in real-time mission critical applications has promoted the implementation of custom-built DNN inference accelerators. These accelerators require a considerable amount of on-chip memory to store millions of trained DNN parameters for executing inference at the edge. Drastic technology scaling in recent years have made these memory circuits highly vulnerable to faults due to various reasons like aging, latent defects, single event upsets, etc. Such faults are highly detrimental to the classification accuracy of the DNN accelerator, leading to the crucial Functional Safety (FuSa) violation. This can eventuate to catastrophic circumstances, when used in mission-critical applications. In order to detect such violations in mission mode, we propose to generate a set of functional test patterns by leveraging the concept of Generative Adversarial Networks (GANs), that are independent of the DNN model and the accelerator characteristics. Our experimental results demonstrate that, the generated test patterns significantly improve FuSa violation detection coverage by up to 130.28%, compared to existing techniques. To the best of our knowledge, this is the first work that generates GAN-based test patterns in order to perform FuSa violation detection in mission-critical DNN accelerators.
A Blockchain-Based Copyright Protection Scheme With Proactive Defense

Copyright protection, including copyright registration, copyright transfer and infringement penalty, plays a critical role in preventing illegal usage of original works. The mainstream traditional copyright protection schemes need an authority online all the time to handle copyright issues and face some problems such as intricate copyright transfer, single point of failure and so on. To alleviate the burden of the authority, a few blockchain-based copyright protection schemes are proposed. However, most of them do not consider copyright transfer, and their infringement penalty may only happen after copyright owners discover the infringement behavior (i.e., “ex-post penalty”). In this article, we propose a new security strategy, called “Proactive Defense” in copyright protection which can prevent infringement before it occurs. With our proposed proactive defense strategy, we design a secure copyright protection scheme which provides advantages of compact copyright transfer and prior infringement penalty. More concrete, both copyright registration and transfer are regarded as transactions and recorded to the blockchain. Based on the double-authentication-prevention signature and non-interactive zero-knowledge proof techniques, illegal copyright transfer can be detected and the infringement penalty can be done automatically with a tailored smart contract before the completion of the transfer. Our security analysis shows that the proposed scheme can achieve all desirable security properties. Moreover, we implement our scheme in Java and evaluate the performance experimentally. Experimental results show that the proposed scheme has good security and efficiency, which can be applied for the copyright protection.
Unsupervised Radio Frequency Fingerprint Identification Based on Curriculum Learning

Radio frequency fingerprint (RFF) identification aims to identify emitters by extracting the inherent physical-layer features. Despite the significant accuracy achieved in a supervised learning manner, few works have focused on unsupervised RFF identification. To this end, we propose an unsupervised training paradigm based on the curriculum learning framework. Specifically, the unlabeled samples are first sorted by the signal-to-noise (SNR) of signals. Then, the network is pretrained with contrastive learning on the high SNR subset. To further improve identification performance, the model is gradually optimized from the easy (high SNR) to the hard (low SNR) subsets with pseudo label learning. Experimental results indicate that the proposed scheme achieves comparable performance against schemes based on supervised learning.
HRNeXt: High-Resolution Context Network for Crowd Pose Estimation

Occlusion handling in crowded scenes is an intractable challenge for human pose estimation. To address this problem, we propose two novel feed-forward network structures named Global Feed-Forward Network (GFFN) and Dynamic Feed-Forward Network (DFFN), which are specifically designed for image-based tasks to capture both local and global contextual information within intermediate features and update feature representations with high adaptability for occlusions. By exploiting the context modeling ability of the proposed GFFN and DFFN, we present a novel backbone network, namely High-Resolution Context Network (HRNeXt), which learns high-resolution representations with abundant contextual information to better estimate poses of occluded human bodies. Compared to state-of-the-art pose estimation networks, our HRNeXt absorbs advantages of convolution operation and attention mechanism, and it is more efficient in terms of training data sizes, network parameters and computational costs. Experimental results show that our HRNeXt significantly outperforms state-of-the-art backbone networks on challenging pose estimation datasets with high occurrence of crowds and occlusions.
Physics Guided Remote Sensing Image Synthesis Network for Ship Detection

Automatic detection and localization of objects in remote sensing images are of great significance for remote sensing systems. Existing frameworks usually train an object detection network using collected remote sensing images. However, these models usually perform poorly due to the lack of large-scale training datasets, which is often the case for special remote sensing scenarios, e.g., the detection of ships in the open sea. Although image synthesis is a common strategy to alleviate the issue of data insufficiency, the trained model still performs poorly when being tested on real-world scenes. Aimed at this, a novel sensor-related image synthesis framework, dubbed as remote sensing-image synthesis pipeline (RS-ISP), is developed to address the lack of on-orbit remote sensing images. Specifically, our RS-ISP introduces two novel designs to ensure the distribution consistency between the generated images and the real images: 1) the first is a novel pipeline for modeling the physical process of noise production during image capture using specific sensors and 2) the second is the design of a detection-oriented image harmonization model. Similar to the existing design, our model first produces coarse synthetic images by copy–paste operation, on which the proposed harmonization process is used to reduce the variation in the pasted foreground and background. By incorporating these two designs into a unified framework, our RS-ISP is designed and used to produce large-scale synthetic images used to train the object detection model for detecting ships in remote sensing images. Comparative experiments demonstrated that RS-ISP increased the AP@.50 from 0.148 to 0.498 for the ship detection task. Code will be publicly available.
Hybrid RSFQ-QFP Superconducting Neuron

Neuromorphic computing is the implementation of neural networks in hardware, in a manner mimicking the human brain. We present a superconducting synaptic circuit for use in such a system. The proposed system makes use of quantum flux parametron architecture to attain both excitatory and inhibitory weightings, and rapid single flux quantum architecture to adjust the magnitude of the synapse weighting. We show that the proposed synaptic circuit is capable of being tuned to eight different weightings, and these adjustments can all be done on the fly. We also demonstrate the synapse's capabilities in full neuron architecture using simple perceptron logic. The result is successfully training the system to emulate simple Boolean logic gates, from scratch.
2D Convolutional LSTM-Based Approach for Human Action Recognition on Various Sensor Data

Human activity recognition (HAR) has appeared as a prominent player in the video surveillance. The conventional pattern recognition systems have made significant improvements in recent years. On the other hand, these approaches depend heavily on manual feature extraction, which may hamper the performance of the generalized model. In this paper, a deep neural network comprising a one-dimensional convolutional neural network (Covn1D) and a two-dimensional convolutional layer with long short-term memory (ConvLSTM2D) is proposed for recognizing human activity. With a very small number of model parameters, this model could automatically extract relevant action features and recognize them. To handle temporal features and long-term dependencies properly, an LSTM is used in this scheme. The raw data acquired by wearable sensors has been fed into the proposed architecture of Conv1D and ConvLSTM2D. Furthermore, to optimize model parameters, a pooling layer has been used. In addition, a Batch Normalization layer (BN) has been introduced to speed up convergence with noticeable outcomes, just after the pooling layer. The performance of the proposed system is evaluated on the publicly available UCI HAR dataset which achieves accuracy of 91.25% and 93.15%, respectively. The results of the proposed system outperform the existing related methods.
2D-CTM and DNA-Based Computing for Medical Image Encryption

Medical imaging plays an important role in the proper treatment of a patient’s disease. Nowadays, the virtual diagnosis of patients is becoming popular, so these images are sent from one place to another via the network for diagnosis purposes. So, the medical images need to be secured from any illegal access and modification. To handle this situation, a novel key generation with medical image encryption and decryption algorithms is proposed based on chaos, DNA computing, and Mersenne Twister (MT). The key is generated using a proposed new 2D-Chaotic Tan Map (2D-TCM). Then, the image is passed through two levels of confusion and diffusion based on DNA computing, chaos, and MT to get a final, highly secured encrypted image. Extensive comparative evaluation of the results is performed on X-ray images and MRI images using different security metrics, such as key space, key sensitivity, histogram analysis, entropy, and correlation analysis. The results show that the proposed model achieved a better result in comparison with other related techniques.
Fast Measurement of Surface Topographies Using a Phase-Measuring Deflectometric Microscopy

With the fast development in the fields of photovoltaics and integrated circuits, the measurement of surface topographies and micro-scaled defects has attracted intensive interests. However, existing measurement methods are time-consuming and unsuitable for in-situ detection. Therefore, a fast topographic measurement method is developed based on the deflectometric microscope system. Microscopic deflectometry is an attractive tool due to its high sensibility to surface slopes and large dynamic range of measurement. For the quantitative reconstruction of surface topographies from slopes, an integration method is developed based on the minimum spanning tree, and the integration path is designed based on a sparse representation and a curl map. The robustness and accuracy of the integration method are validated by numerical simulation and experiments. This method can achieve a high measurement accuracy with a high lateral resolution and a depth of field of 90 μm, hereby improving the manufacturing efficiency and quality of opto-electronics functional devices.
Offloading Deep Learning Powered Vision Tasks From UAV to 5G Edge Server With Denoising

Offloading computationally heavy tasks from an unmanned aerial vehicle (UAV) to a remote server helps improve battery life and can help reduce resource requirements. Deep learning based state-of-the-art computer vision tasks, such as object segmentation and detection, are computationally heavy algorithms, requiring large memory and computing power. Many UAVs are using (pretrained) off-the-shelf versions of such algorithms. Offloading such power-hungry algorithms to a remote server could help UAVs save power significantly. However, deep learning based algorithms are susceptible to noise, and a wireless communication system, by its nature, introduces noise to the original signal. When the signal represents an image, noise affects the image. There has not been much work studying the effect of the noise introduced by the communication system on pretrained deep networks. In this work, we first analyze how reliable it is to offload deep learning based computer vision tasks (including both object segmentation and detection) by focusing on the effect of various parameters of a 5G wireless communication system on the transmitted image and demonstrate how the introduced noise of the used 5G system reduces the performance of the offloaded deep learning task. Then solutions are introduced to eliminate (or reduce) the negative effect of the noise. Proposed framework starts with introducing many classical techniques as alternative solutions, and then introduces a novel deep learning based solution to denoise the given noisy input image. The performance of various denoising algorithms on offloading both object segmentation and object detection tasks are compared. Our proposed deep transformer-based denoiser algorithm (NR-Net) yields state-of-the-art results in our experiments.
A new method for writer identification based on historical documents

Abstract Identifying the writer of a handwritten document has remained an interesting pattern classification problem for document examiners, forensic experts, and paleographers. While mature identification systems have been developed for handwriting in contemporary documents, the problem remains challenging from the viewpoint of historical manuscripts. Design and development of expert systems that can identify the writer of a questioned manuscript or retrieve samples belonging to a given writer can greatly help the paleographers in their practices. In this context, the current study exploits the textural information in handwriting to characterize writer from historical documents. More specifically, we employ oBIF(oriented Basic Image Features) and hinge features and introduce a novel moment-based matching method to compare the feature vectors extracted from writing samples. Classification is based on minimization of a similarity criterion using the proposed moment distance. A comprehensive series of experiments using the International Conference on Document Analysis and Recognition 2017 historical writer identification dataset reported promising results and validated the ideas put forward in this study.
Learning Weight Signed Network Embedding with Graph Neural Networks

Abstract Network embedding aims to map nodes in a network to low-dimensional vector representations. Graph neural networks (GNNs) have received much attention and have achieved state-of-the-art performance in learning node representation. Using fundamental sociological theories (status theory and balance theory) to model signed networks, basing GNN on learning node embedding has become a hot topic in signed network embedding. However, most GNNs fail to use edge weight information in signed networks, and most models cannot be directly used in weighted signed networks. We propose a novel signed directed graph neural networks model named WSNN to learn node embedding for Weighted signed networks. The proposed model reconstructs link signs, link directions, and signed directed triangles simultaneously. Based on the network representations learned by the proposed model, we conduct link sign prediction in signed networks. Extensive experimental results in real-world datasets demonstrate the superiority of the proposed model over the state-of-the-art network embedding algorithms for graph representation learning in signed networks.
Leveraging machine learning to identify acute myeloid leukemia patients and their chemotherapy regimens in an administrative database

Administrative datasets are useful for identifying rare disease cohorts such as pediatric acute myeloid leukemia (AML). Previously, cohorts were assembled using labor-intensive, manual reviews of patients' longitudinal chemotherapy data.We utilized a two-step machine learning (ML) method to (i) identify pediatric patients with newly diagnosed AML, and (ii) among the identified AML patients, their chemotherapy courses, in an administrative/billing database. Using 2558 patients previously manually reviewed, multiple ML algorithms were derived from 75% of the study sample, and the selected model was tested in the remaining hold-out sample. The selected model was also applied to assemble a new pediatric AML cohort and further assessed in an external validation, using a standalone cohort established by manual chart abstraction.For patient identification, the selected Support Vector Machine model yielded a sensitivity of 0.97 and a positive predictive value (PPV) of 0.97 in the hold-out test sample. For course-specific chemotherapy regimen and start date identification, the selected Random Forest model yielded overall PPV greater than or equal to 0.88 and sensitivity greater than or equal to 0.86 across all courses in the test sample. When applied to new cohort assembly, ML identified 3016 AML patients with 10,588 treatment courses. In the external validation subset, PPV was greater than or equal to 0.75 and sensitivity was greater than or equal to 0.82 for patient identification, and PPV was greater than or equal to 0.93 and sensitivity was greater than or equal to 0.94 for regimen identifications.A carefully designed ML model can accurately identify pediatric AML patients and their chemotherapy courses from administrative databases. This approach may be generalizable to other diseases and databases.
Towards Measuring Fairness for Local Differential Privacy

Local differential privacy (LDP) approaches provide data subjects with the strong privacy guarantees of Differential Privacy under the scenario of untrusted data curators. They are used by companies (e.g., Google’s RAPPOR) to collect potentially sensitive data from clients through randomized response. Randomized response was proposed as a method to allow respondents to surveys to answer questions about sensitive issues such as illegal behavior or private preferences. By randomizing their answers, the respondents are provided by plausible deniability, their answers about a sensitive issue may be “yes” either because it is true or by chance. We study how randomized response mechanisms that provide LDP for a fixed $$\varepsilon $$ , may provide different privacy guarantees to respondents depending on their sensitive attribute value, i.e., they have disparate impact regarding the privacy protection. We propose measures for fair privacy when applying LDP and show that the parameters on the randomized response matrix can be tuned to generate fairer-LDP mechanisms with the same global privacy guarantee $$\varepsilon $$ . We show the effectiveness of our approach through an experimental evaluation in Machine Learning Classification tasks on three commonly used benchmark datasets: Adult Income, COMPAS and German Credit.
Preserving Buyer-Privacy in Decentralized Supply Chain Marketplaces

Technology is being used increasingly for lowering the trust barrier in domains where collaboration and cooperation are necessary, but reliability and efficiency are critical due to high stakes. An example is an industrial marketplace where many suppliers must participate in production while ensuring reliable outcomes; hence, partnerships must be pursued with care. Online marketplaces like Xometry facilitate partnership formation by vetting suppliers and mediating the marketplace. However, such an approach requires that all trust be vested in the middleman. This centralizes control, making the system vulnerable to being biased towards specific providers. The use of blockchains is now being explored to bridge the trust gap needed to support decentralizing marketplaces, allowing suppliers and customers to interact more directly by using the information on the blockchain. A typical scenario is the need to preserve privacy in certain interactions initiated by the buyer (e.g., protecting a buyer's intellectual property during outsourcing negotiations). In this work, we initiate the formal study of matching between suppliers and buyers when buyer-privacy is required for some marketplace interactions and make the following contributions. First, we devise a formal security definition for private interactive matching in the Universally Composable (UC) Model that captures the privacy and correctness properties expected in specific supply chain marketplace interactions. Second, we provide a lean protocol based on any programmable blockchain, anonymous group signatures, and public-key encryption. Finally, we implement the protocol by instantiating some of the blockchain logic by extending the BigChainDB blockchain platform.
An accurate shared bicycle detection network based on faster R‐CNN

Abstract Detecting shared bicycles is an essential and challenging task. Deep learning has been widely used in object detection tasks in urban scenes, such as vehicle detection. However, deep learning algorithms still face many difficulties and challenges in shared bicycle detection. For example, the problem of large deformation of shared bicycles and the problem of small targets because the camera is far away from the shared bicycles. In order to solve these problems, this study introduces the feature fusion module and deformable convolution into the object detection network, which improves the efficiency of shared bicycle detection. This study proposes an enhanced faster R‐CNN network (A classic two‐stage object detection network) for shared bicycle detection and a shared bicycle dataset (SBD) is constructed for model training and testing. Compared with the original faster R‐CNN, the mean average precision (mAP) of the enhanced method on SBD is improved by 13%, which indicates that the method provided in this study is more suitable for detecting shared bicycles. This study also conducts experiments on the Microsoft Common Objects (COCO) dataset, where this method achieves 40.2% of the mAP, which is 5.8% higher than faster R‐CNN before improvement.
A video drowning detection device based on underwater computer vision

Abstract Drowning is a significant public health concern. A video drowning detection algorithm is a helpful tool for finding drowning victims. However, there are three challenges that drowning detection research typically encounters: a lack of actual drowning video data, subtle early drowning traits, and a lack of real time. In this paper, the authors propose an underwater computer vision based drowning detection device composed of embedded AI devices, camera, and waterproof case to solve the above problems. The detection device utilizes the high‐performance computing of Jetson Nano to realize real‐time detection of drowning events through the proposed drowning detection algorithm on the acquired underwater video stream. The proposed drowning detection algorithm primarily consists of two stages: in the first step, to successfully solve the interference of the surroundings and to give a trustworthy basis for video drowning detection, the YOLOv5n network is used to detect the near‐vertical human body based on the characteristics of the drowning person. In the second stage, the authors propose a lightweight drowning detection network (DDN) based on a deep Gaussian model for fast feature vector detection. The lightweight DDN is combined with the Gaussian model to detect anomaly in the high‐level semantic features, which has higher robustness and solves the lack of drowning videos. The experimental results show that the proposed drowning detection algorithm has good comprehensive performance and practical application value.