Reviewerly/exHarmony huggingface dataset:
Development of hybrid optical sensor based on deep learning to detect and classify the micro-size defects in printed circuit board

Testing with naked eye contact on large-sized printed circuit boards (PCBs) mass produced is exhausting and time consuming. Since the eyes will get tired during the long-term test phase, there is a possibility of erroneous measurement. In addition, for a quality PCB inspection, identifying the defective location on the PCBs and classifying the types of these defects is an important step in the production process. In this study, we propose a new hybrid optical sensor (HOS) based on deep learning (DL) to sense the micro-size defects on PCBs. In this context, the combination of the lateral shearing digital holographic microscopy (LSDHM) and the microscopic fringe projection profilometry (MFPP) is proposed. Another aim of ours is to classify the PCBs defects by using deep learning classifier based on Convolution Neural Network (CCN) algorithm. Contrast to the complex systems used in the literature for sensing the PCB defects, an optical microscopic sensor with minimum components has been proposed to use for the first time. Thanks to proposed HOS based on DL, we reveal the successful results obtained with high accuracy (99%) that the defects on PCBs can be detected and classified in very short time with non-contact and real-time imaging. Hence, the problem of time consuming has been overcome. It is thought that the proposed hybrid sensor will lead to future studies in terms of the early detection of the micro-size defects on PCBs with high performance before mass production process.
Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing

Testing deep neural networks (DNNs) has garnered great interest in the recent years due to their use in many applications. Black-box test adequacy measures are useful for guiding the testing process in covering the input domain. However, the absence of input specifications makes it challenging to apply black-box test adequacy measures in DNN testing. The Input Distribution Coverage (IDC) framework addresses this challenge by using a variational autoencoder to learn a low dimensional latent representation of the input distribution, and then using that latent space as a coverage domain for testing. IDC applies combinatorial interaction testing on a partitioning of the latent space to measure test adequacy. Empirical evaluation demonstrates that IDC is cost-effective, capable of detecting feature diversity in test inputs, and more sensitive than prior work to test inputs generated using different DNN test generation methods. The findings demonstrate that IDC overcomes several limitations of white-box DNN coverage approaches by discounting coverage from unrealistic inputs and enabling the calculation of test adequacy metrics that capture the feature diversity present in the input space of DNNs.
Accountable and Fine-Grained Controllable Rewriting in Blockchains

Most blockchains are designed to be immutable such that an object, e.g., a block or a transaction, is persisted once it has been registered. However, blockchain immutability hinders blockchain development due to the increasing abuse of blockchain storage and legal obligations. To break immutability in a controlled way, Derler et al. (NDSS’19) proposed a redactable blockchain with fine-grained controllable rewriting by introducing the notion of policy-based chameleon hash (PCH). Given a PCH-based object associated with an access policy, a trapdoor holder whose rewriting privileges satisfy the access policy can alter the object. Although this work offers an elegant approach to blockchain rewriting, it lacks accountability. In practice, the trapdoor holders may abuse their rewriting privileges, and even use their chameleon trapdoor to build a device in a blackbox manner to gain illegal profits while avoiding being caught. In this paper, we introduce a new design of PCH with blackbox accountability (PCHA). Blackbox accountability offers not only linkability between any modified object and its modifier, but also traceability that enables a central authority to identify responsible trapdoor holders whose secret keys have contributed to the blackbox device. Besides modeling PCHAs, we present a generic construction of PCHAs with rigorous security proofs. We instantiate a concrete construction of PCHA by introducing a practical attribute-based traitor tracing (ABTT) with adaptive security on prime-order pairing groups. The experimental analysis demonstrates that our PCHA and ABTT schemes have modest overheads and superior functionality to the state-of-the-art solutions. In particular, the price of accountability in key generation, hash, and adaption is almost negligible compared to the state-of-the-art solution.
Curiosity-Driven and Victim-Aware Adversarial Policies

Recent years have witnessed great potential in applying Deep Reinforcement Learning (DRL) in various challenging applications, such as autonomous driving, nuclear fusion control, complex game playing, etc. However, recently researchers have revealed that deep reinforcement learning models are vulnerable to adversarial attacks: malicious attackers can train adversarial policies to tamper with the observations of a well-trained victim agent, the latter of which fails dramatically when faced with such an attack. Understanding and improving the adversarial robustness of deep reinforcement learning is of great importance in enhancing the quality and reliability of a wide range of DRL-enabled systems.
Are There Wireless Hidden Cameras Spying on Me?

The proliferation of IoT devices has created risks of their abuse for unauthorized sensing/monitoring of our daily activities. Especially, the leakage of images taken by wireless spy cameras in sensitive spaces, such as hotel rooms, Airbnb rentals, public restrooms, and shower rooms, has become a serious privacy concern/threat. To mitigate/address this pressing concern, we propose a Spy Camera Finder (SCamF) that uses ubiquitous smartphones to detect and locate wireless spy cameras by analyzing encrypted Wi-Fi network traffic. Not only by characterizing the network traffic patterns of wireless cameras but also by reconstructing encoded video frame sizes from encrypted traffic, SCamF effectively determines the existence of wireless cameras on the Wi-Fi networks, and accurately verifies whether the thus-detected cameras are indeed recording users’ activities. SCamF also accurately locates spy cameras by analyzing reconstructed video frame sizes. We have implemented SCamF on Android smartphones and evaluated its performance on a real testbed across 20 types of wireless cameras. Our experimental results show SCamF to: (1) classify wireless cameras with an accuracy of 0.98; (2) detect spy cameras among the classified wireless cameras with a true positive rate (TPR) of 0.97; (3) incur low false positive rates (FPRs) of 0 and 0.031 for non-camera devices and cameras not recording the users’ activities, respectively; (4) locate spy cameras with centimeter-level distance errors.
M3S: Scene Graph Driven Multi-Granularity Multi-Task Learning for Multi-Modal NER

Multi-modal Named Entity Recognition (MNER), which mainly focuses on enhancing text-only NER with visual information, has recently attracted considerable attention. Most current MNER models have made significant progress by jointly understanding visual and language modalities through layers of cross-modality attention. However, these approaches largely ignore the visual bias brought by the image contents and barely consider exploiting the multi-granularity representations and the interactions between visual objects, which are essential in recognizing ambiguous entities. In this paper, we propose a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</b> cene graph driven <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti-modal <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti-granularity <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> ulti-task learning ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M3S</b> ) framework to better exploit visual and textual information in MNER. Specifically, to explicitly alleviate visual bias, we present a novel multi-task approach by employing the task of Named Entity Segmentation (NES) cascade with Named Entity Categorization (NEC). To obtain detailed visual semantics by explicitly modeling objects and relationships between paired objects, we construct scene graphs as a structured representation of the visual contents. Furthermore, a well-designed Multi-granularity Gated Aggregation (MGA) mechanism is introduced to capture inter-modality interactions and extract critical features for named entity recognition. Extensive experiments on two real public datasets demonstrate the effectiveness of our proposed M3S.
Autonomous Driving for Natural Paths Using an Improved Deep Reinforcement Learning Algorithm

The purpose of this article is aimed to solve the problem associated with autonomous driving on the natural paths of planets. The contribution of this work is to propose an improved deep deterministic policy gradient (DDPG) framework for the autonomous driving on natural roads requires handling uneven surface of different throttle and braking reaction speeds. Our new finding is to design an adapted DDPG algorithm by double critic and excellent experience replay as DCEER-DDPG to reduce the overestimation of state action values. In addition, we created a virtual reality environment with TORCS simulator for fair evaluation. In the experiments, the proposed DCEER-DDPG has a better performance than previous algorithms, which can improve the utilization of driving experience on a natural path and increase the learning efficiency of the strategy. For the future applications, the proposed DCEER-DDPG is used not only on Earth, but also in lunar exploration.
Secure information processing for multimedia forensics using zero-trust security model for large scale data analytics in SaaS cloud computing environment

Trust is an essential security component and plays a crucial role in building secure information processing systems. Many organizations have adopted zero-trust principles in their security mechanisms, especially in Cloud Computing environments, where everything is presumed to be semi-honest and untrustworthy until proven otherwise. This is in blunt contrast to the traditional perimeter-security model, where it is presumed that the malicious actors are always outside the perimeter of the trusted network, and trustworthy users are always inside the network. One of the most significant challenges with the notion of trust is that till now more focus was put on the subjective aspect of trust which was challenging to evaluate and verify. There is an urgent need to address the objective and verifiable notion of trust, especially in the systems involving Federated Learning as in Cloud environments. We propose a Rich model based zero-trust security framework for trust verification of SaaS and employ machine learning functionalities to perform multimedia data analytics over the service processing behaviors for improved visibility into service operations and risks. Our proposed model relies heavily on the features extracted form Federated learning on the data of different cloud service users that is subjected to AI processing. We utilized the Rich models for feature extraction of large scale multimedia data for monitoring cloud service behavior and employed Ensemble classifier for analyzing the data features and employed majority voting of individual learners for decision making. The experiments performed on the standard dataset proved that the proposed model successfully monitored and analyzed the behavior of cloud services for validating the legitimacy of trusted SaaS and detecting any trust violations.
Blockchain and Internet of Things (IoT): A Disruptive Integration

The combination of internet of things (IoT) and blockchain can propose innovative applications in different smart networks. Due to the various applications in IoT ecosystem providing a secure network with high quality of privacy is so important. Blockchain, which is a decentralized and distributed ledger technology, can be a proper solution. In addition, various terms and conditions of different services in IoT need to be guaranteed, where a smart contract feature in blockchain can be used. In this paper, we studied the integration of blockchain in IoT networks applications. This integration is discussed in different use cases such as smart city, smart energy, smart home, smart industry, healthcare, agriculture, unmanned aerial vehicles (UAVs). The advantages of blockchain application in each of these areas is proposed in the paper. The combination and integration of blockchain and IoT can provide significant development in smart networks.
Audible Crime Scenes: ShotSpotter as Diagnostic, Policing, and Space-making Infrastructure

Acoustic gunshot detection systems (AGDS) have been emerging as a technological solution to the growing problem of gun violence around the world. We examine a particularly prominent AGDS technology called RespondTM developed by publicly traded US company ShotSpotter Inc. (NASDAQ: SSTI) to better understand the sociotechnical logics that inform its operation. Drawing from frameworks provided by science and technology studies and sound studies, we ask, “What are the broader conditions that allow for a successful AGDS as it is imagined by ShotSpotter?” At a time in which the accuracy and reliability of ShotSpotter’s AGDS are being seriously questioned through the numerous reports of false positives that reached as high as 99 percent in certain cities it was deployed in, it is imperative to interrogate what exactly is “false” in these false positive reports and how the company operates despite them. In this paper, we trace ShotSpotter and its artificial intelligence/machine learning AGDS technologies as they exist across various patents, promotional materials, financial documents, and public statements to not only better understand the ways in which they translate sound into “crime,” or space into “crime scene,” but also to bring attention to how ShotSpotter translates itself across its different audiences.
Multi-level GAN based enhanced CT scans for liver cancer diagnosis

Liver cancer diagnosis requires preprocessing of images with preserved structural details. In this study, a multi-level generative adversarial network (GAN) is proposed to enhance computed tomographic (CT) images. The generated enhanced images are used to perform computer-aided diagnosis between malignant and normal liver. Three publicly available datasets, Ircadb, Sliver07, and LiTS, are used to investigate the performance of the proposed method with qualitative and quantitative analysis, namely performance metrics and computer-aided diagnosis. The mean structure similarity index of 0.45 and peak signal-to-noise ratio of 16.20 dB is achieved for the metric analysis. The AlexNet is adopted to perform binary classification with the testing accuracy of 90.37% and 85.90% for enhanced and non-enhanced images, respectively, which demonstrates the effectiveness of the proposed multi-level GAN in producing enhanced biomedical images with preserved structural details and favorable reduction in artifacts. Moreover, the consistently better performance among three datasets confirms the merits of the proposed multi-level GAN for computer-aided diagnosis.
Colorization of infrared images based on feature fusion and contrastive learning

Converting infrared images to RGB images that match human eye perception is a challenging task. Current infrared image coloring techniques can bring visual improvements, but still suffer from texture distortion, blurred details, and poor image quality. In this paper, we work on solving the above problems. First, we design an improved generator structure. On the basis of Unet, we add dense convolutional blocks and skip connections to integrate low-level detail information with high-level semantic information. The developed generator can capture features at different levels and integrate them by feature fusion. It ensures that the captured features are not lost. Second, we design a new contrastive loss function. Based on the contrastive learning framework, this function focuses on learning common features between similar instances and distinguishing differences between non-similar instances. This ensures the consistency of the content and structure of the images. Finally, an in-depth contrast analysis is conducted based on commonly used datasets to demonstrate the superior colorization performance of our method against the state-of-the-art approaches.
Tamper-proof multitenant data storage using blockchain

Technologies like Internet of Things (IoT), cloud, artificial intelligence, blockchain etc. have become a perceptible part of our lives resulting in the generation of enormous amounts of data. Consequently, the systems used for storage and processing of this data are required to be scalable for handling the huge volumes of data. A shared, multitenant system such as a cloud-based storage-as-a-service provides scalability of storage as well as economics of sharing. However, there is a risk of data tampering when multiple tenants work in a shared environment. The benefits of a multitenant solution can be leveraged only if tenants' data is isolated from each other. Further, prevention of data tampering from malicious tenant nodes is also required. Therefore, the paper proposes the use of a private blockchain for an implementation of a multi-tenant-based storage system. The objective is to develop a scalable system where tenants' data is not at a risk of tampering. The efficacy of the proposed system has been demonstrated with synthetic data of multiple tenants using a Software as a Service (SaaS) healthcare application.
Quantum annealing and computation: challenges and perspectives

In the introductory article of this theme issue, we provide an overview of quantum annealing and computation with a very brief summary of the individual contributions to this issue made by experts as well as a few young researchers. We hope the readers will get the touch of the excitement as well as the perspectives in this unusually active field and important developments there. This article is part of the theme issue 'Quantum annealing and computation: challenges and perspectives'.
Anisotropic Weighted Total Variation Feature Fusion Network for Remote Sensing Image Denoising

Remote sensing images are widely applied in instance segmentation and objetive recognition; however, they often suffer from noise, influencing the performance of subsequent applications. Previous image denoising works have only obtained restored images without preserving detailed texture. To address this issue, we proposed a novel model for remote sensing image denoising, called the anisotropic weighted total variation feature fusion network (AWTVF2Net), consisting of four novel modules (WTV-Net, SOSB, AuEncoder, and FB). AWTVF2Net combines traditional total variation with a deep neural network, improving the denoising ability of the proposed approach. Our proposed method is evaluated by PSNR and SSIM metrics on three benchmark datasets (NWPU, PatternNet, UCL), and the experimental results show that AWTVF2Net can obtain 0.12∼19.39 dB/0.0237∼0.5362 higher on PSNR/SSIM values in the Gaussian noise removal and mixed noise removal tasks than State-of-The-Art (SoTA) algorithms. Meanwhile, our model can preserve more detailed texture features. The SSEQ, BLIINDS-II, and BRISQUE values of AWTVF2Net on the three real-world datasets (AVRIS Indian Pines, ROSIS University of Pavia, HYDICE Urban) are 3.94∼12.92 higher, 8.33∼27.5 higher, and 2.2∼5.55 lower than those of the compared methods, respectively. The proposed framework can guide subsequent remote sensing image applications, regarding the pre-processing of input images.
Implementation of a Biometric-Based Blockchain System for Preserving Privacy, Security, and Access Control in Healthcare Records

The use of Electronic Health Record (EHR) systems has emerged with the continuous advancement of the Internet of Things (IoT) and smart devices. This is driven by the various advantages for both patients and healthcare providers, including timely and distant alerts, continuous control, and reduced cost, to name a few. However, while providing these advantages, various challenges involving heterogeneity, scalability, and network complexity are still open. Patient security, data privacy, and trust are also among the main challenges that need more research effort. To this end, this paper presents an implementation of a biometric-based blockchain EHR system (BBEHR), a prototype that uniquely identifies patients, enables them to control access to their EHRs, and ensures recoverable access to their EHRs. This approach overcomes the dependency on the private/public key approach used by most blockchain technologies to identify patients, which becomes more crucial in situations where a loss of the private key permanently hinders the ability to access patients’ EHRs. Our solution covers component selection, high-level implementation, and integration of subsystems, was well as the coding of a prototype to validate the mitigation of the risk of permanent loss of access to EHRs by using patients’ fingerprints. A performance analysis of BBEHR showed our system’s robustness and effectiveness in identifying patients and ensuring access control for their EHRs by using blockchain smart contracts with no additional overhead.
XAI-FR: Explainable AI-Based Face Recognition Using Deep Neural Networks

Face Recognition aims at identifying or confirming an individual's identity in a still image or video. Towards this end, machine learning and deep learning techniques have been successfully employed for face recognition. However, the response of the face recognition system often remains mysterious to the end-user. This paper aims to fill this gap by letting an end user know which features of the face has the model relied upon in recognizing a subject's face. In this context, we evaluate the interpretability of several face recognizers employing deep neural networks namely, LeNet-5, AlexNet, Inception-V3, and VGG16. For this purpose, a recently proposed explainable AI tool-Local Interpretable Model-Agnostic Explanations (LIME) is used. Benchmark datasets such as Yale, AT &T dataset, and Labeled Faces in the Wild (LFW) are utilized for this purpose. We are able to demonstrate that LIME indeed marks the features that are visually significant features for face recognition.