Reviewerly/exHarmony huggingface dataset:
Thermal Defect Detection for Substation Equipment Based on Infrared Image Using Convolutional Neural Network

Thermal defects of substation equipment have a great impact on the stability of power systems. Temperature is crucial for thermal defect detection in infrared images. The traditional detection methods, which have low efficiency and poor accuracy, record the temperature of infrared images manually. In this study, a thermal defect detection method based on infrared images using a convolutional neural network (CNN) is proposed. Firstly, the improved pre-processing method is applied to reduce background information, and the region of interest is located according to the contour and position information, hence improving the quality of images. Then, the temperature values are segmented to establish the dataset (T-IR11), which contains 11 labels. Finally, the CNN model is constructed to extract features, and the support vector machine is trained for classification. To verify the effectiveness of the proposed method, precision, recall, and F1 score are adopted and 10-fold cross-validation is employed on the T-IR11 dataset. The results demonstrate that the accuracy of the proposed method is 99.50%, and the performance is superior to that of previous methods in terms of infrared images. The proposed method can realize automatic temperature recognition and equipment with thermal defects can be recorded systematically, which has significant practical value for defect detection in substation equipment.
Investigating Blockchain Technology Effects on Online Platforms Transactions: Do Risk Aversion and Technophilia Matter?

Despite its immense potential for industries and services as a promising solution for trust and transparency problems, Blockchain related studies in the field of Marketing are rare. This research mobilizes the technology acceptance Model (TAM) theory to investigate the added value of blockchain technology to online platforms while taking into consideration consumers characteristics. Data from 212 users of online platforms endowed with blockchain technology were collected. Findings demonstrate that blockchain features in terms of security and privacy enhance the perceived usefulness and the perceived ease of use of the online platform, leading to stronger trust and more actual transactions. In addition, results demonstrate that risk aversion weakens only the relationship between usefulness and trust. It has indeed no significant effect on the impact of ease of use on trust. Results also show that technophilia fosters the relationship between intents and actual transactions. The research provides heuristic implications for academia and industry.
Facial Expressions as a Vulnerability in Face Recognition

This work explores facial expression bias as a security vulnerability of face recognition systems. Despite the great performance achieved by state-of-the-art face recognition systems, the algorithms are still sensitive to a large range of covariates. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies. Our study analyzes: i) facial expression biases in the most popular face recognition databases; and ii) the impact of facial expression in face recognition performances. Our experimental framework includes two face detectors, three face recognition models, and three different databases. Our results demonstrate a huge facial expression bias in the most widely used databases, as well as a related impact of face expression in the performance of state-of-the-art algorithms. This work opens the door to new research lines focused on mitigating the observed vulnerability.
An Accuracy-Lossless Perturbation Method for Defending Privacy Attacks in Federated Learning

Although federated learning improves privacy of training data by exchanging local gradients or parameters rather than raw data, the adversary still can leverage local gradients and parameters to obtain local training data by launching reconstruction and membership inference attacks. To defend against such privacy attacks, many noises perturbed methods (like differential privacy or CountSketch matrix) have been widely designed. However, the strong defence ability and high learning accuracy of these schemes cannot be ensured at the same time, which will impede the wide application of FL in practice (especially for medical or financial institutions that require both high accuracy and strong privacy guarantee). To overcome this issue, we propose an efficient model perturbation method for federated learning to defend against reconstruction and membership inference attacks launched by curious clients. On the one hand, similar to the differential privacy, our method also selects random numbers as perturbed noises added to the global model parameters, and thus it is very efficient and easy to be integrated in practice. Meanwhile, the random selected noises are positive real numbers and the corresponding value can be arbitrarily large, and thus the strong defence ability can be ensured. On the other hand, unlike differential privacy or other perturbation methods that cannot eliminate added noises, our method allows the server to recover the true aggregated gradients by eliminating the added noises. Therefore, our method does not hinder learning accuracy at all. Extensive experiments demonstrate that for both regression and classification tasks, our method achieves the same accuracy as non-private approaches and outperforms the state-of-the-art defence schemes. Besides, the defence ability of our method against reconstruction and membership inference attack is significantly better than the state-of-the-art related defence schemes.
WinoCNN: Kernel Sharing Winograd Systolic Array for Efficient Convolutional Neural Network Acceleration on FPGAs

The combination of Winograd’s algorithm and systolic array architecture has demonstrated the capability of improving DSP efficiency in accelerating convolutional neural networks (CNNs) on FPGA platforms. However, handling arbitrary convolution kernel sizes in FPGA-based Winograd processing elements and supporting efficient data access remain underexplored. In this work, we are the first to propose an optimized Winograd processing element (WinoPE), which can naturally support multiple convolution kernel sizes with the same amount of computing resources and maintains high runtime DSP efficiency. Using the proposed WinoPE, we construct a highly efficient systolic array accelerator, termed WinoCNN. We also propose a dedicated memory subsystem to optimize the data access. Based on the accelerator architecture, we build accurate resource and performance modeling to explore optimal accelerator configurations under different resource constraints. We implement our proposed accelerator on multiple FPGAs, which outperforms the state-of-the-art designs in terms of both throughput and DSP efficiency. Our implementation achieves DSP efficiency up to 1.33 GOPS/DSP and throughput up to 3.1 TOPS with the Xilinx ZCU102 FPGA. These are 29.1% and 20.0% better than the best solutions reported previously, respectively.
ArUco marker-based displacement measurement technique: uncertainty analysis

The displacement response of structures can provide important information regarding their behaviours. Since image-based measurements have been considerably improved in recent years, in this research, a fiducial marker-based approach is exploited and a displacement measurement technique is proposed. With this method, an industrial camera can be used to measure the displacement of a structure through the detection of an ArUco marker. A comparison with an established reference technique has been carried out. The uncertainty due to repeatability measurement and the relevance of the acquisition and environmental parameters on the measurement results were researched.
Extracting Causal Visual Features For Limited Label Classification

Neural networks trained to classify images do so by identifying features that allow them to distinguish between classes. These sets of features are either causal or context dependent. Grad-CAM is a popular method of visualizing both sets of features. In this paper, we formalize this feature divide and provide a methodology to extract causal features from Grad-CAM. We do so by defining context features as those features that allow contrast between predicted class and any contrast class. We then apply a set theoretic approach to separate causal from contrast features for COVID-19 CT scans. We show that on average, the image regions with the proposed causal features require 15% less bits when encoded using Huffman encoding, compared to Grad-CAM, for an average increase of 3% classification accuracy, over Grad-CAM. Moreover, we validate the transfer-ability of causal features between networks and comment on the non-human interpretable causal nature of current networks.
VR as a 3D Modelling Tool in Engineering Design Applications

The study aims to explore the usefulness of existing VR 3D modelling tools for use in mechanical engineering. Previous studies have investigated the use of VR 3D modelling tools in conceptual phases of the product development process. Our objective was to find out if VR tools are useful in creating advanced freeform CAD models that are part of the embodiment design phase in the context of mechanical design science. Two studies were conducted. In the preliminary study, the group of participants modelled a 3D part in a standard desktop CAD application, which provided information about the key characteristics that must be satisfied to obtain a solid model from a surface model. In the research study conducted with a focus group of participants, who were firstly trained in the use of VR, the same part was modelled using a VR headset. The results were analysed and the fulfilment of key characteristics in the use of VR was evaluated. It was found that using VR tools provides a fast way to create complex part geometries, however, it has certain drawbacks. Finally, the ease of use and specific features of the VR technology were discussed.
Characterizing Transaction-Reverting Statements in Ethereum Smart Contracts

Smart contracts are programs stored on blockchains to execute transactions. When input constraints or security properties are violated at runtime, the transaction being executed by a smart contract needs to be reverted to avoid undesirable consequences. On Ethereum, the most popular blockchain that supports smart contracts, developers can choose among three transaction-reverting statements (i.e., require, if…revert, and if…throw) to handle anomalous transactions. While these transaction-reverting statements are vital for preventing smart contracts from exhibiting abnormal behaviors or suffering malicious attacks, there is limited understanding of how they are used in practice. In this work, we perform the first empirical study to characterize transaction-reverting statements in Ethereum smart contracts. We measured the prevalence of these statements in 3,866 verified smart contracts from popular dapps and built a taxonomy of their purposes via manually analyzing 557 transaction-reverting statements. We also compared template contracts and their corresponding custom contracts to understand how developers customize the use of transaction-reverting statements. Finally, we analyzed the security impact of transaction-reverting statements by removing them from smart contracts and comparing the mutated contracts against the original ones. Our study led to important findings. For example, we found that transaction-reverting statements are commonly used to perform seven types of authority verifications or validity checks, and missing such statements may compromise the security of smart contracts. We also found that current smart contract security analyzers cannot effectively handle transaction-reverting statements when detecting security vulnerabilities. Our findings can shed light on further research in the broad area of smart contract quality assurance and provide practical guidance to smart contract developers on the appropriate use of transaction-reverting statements.
Anomaly Detection Via Self-Organizing Map

Anomaly detection plays a key role in industrial manufacturing for product quality control. Traditional methods for anomaly detection are rule-based with limited generalization ability. Recent methods based on supervised deep learning are more powerful but require large-scale annotated datasets for training. In practice, abnormal products are rare thus it is very difficult to train a deep model in a fully supervised way. In this paper, we propose a novel unsupervised anomaly detection approach based on Self-organizing Map (SOM). Our method, Self-organizing Map for Anomaly Detection (SOMAD) maintains normal characteristics by using topological memory based on multi-scale features. SOMAD achieves state-of-the-art performance on unsupervised anomaly detection and localization on the MVTec dataset.
Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance

Virtual reality (VR) head-mounted displays (HMD) have recently been used to provide an immersive, first-person vision/view in real-time for manipulating remotely-controlled unmanned ground vehicles (UGV). The teleoperation of UGV can be challenging for operators when it is done in real time. One big challenge is for operators to perceive quickly and rapidly the distance of objects that are around the UGV while it is moving. In this research, we explore the use of monoscopic and stereoscopic views and display types (immersive and non-immersive VR) for operating vehicles remotely. We conducted two user studies to explore their feasibility and advantages. Results show a significantly better performance when using an immersive display with stereoscopic view for dynamic, real-time navigation tasks that require avoiding both moving and static obstacles. The use of stereoscopic view in an immersive display in particular improved user performance and led to better usability.
Light-Field View Synthesis Using A Convolutional Block Attention Module

Consumer light-field (LF) cameras suffer from a low or limited resolution because of the angular-spatial trade-off. To alleviate this drawback, we propose a novel learning-based approach utilizing attention mechanism to synthesize novel views of a light-field image using a sparse set of input views (i.e., 4 corner views) from a camera array. In the proposed method, we divide the process into three stages, stereo-feature extraction, disparity estimation, and final image refinement. We use three sequential convolutional neural networks for each stage. A residual convolutional block attention module (CBAM) is employed for final adaptive image refinement. Attention modules are helpful in learning and focusing more on the important features of the image and are thus sequentially applied in the channel and spatial dimensions. Experimental results show the robustness of the proposed method. Our proposed network outperforms the state-of-the-art learning-based light-field view synthesis methods on two challenging real-world datasets by 0.5 dB on average. Furthermore, we provide an ablation study to substantiate our findings.
Emergent Tangled Program Graphs in Partially Observable Recursive Forecasting and ViZDoom Navigation Tasks

Modularity represents a recurring theme in the attempt to scale evolution to the design of complex systems. However, modularity rarely forms the central theme of an artificial approach to evolution. In this work, we report on progress with the recently proposed Tangled Program Graph (TPG) framework in which programs are modules. The combination of the TPG representation and its variation operators enable both teams of programs and graphs of teams of programs to appear in an emergent process. The original development of TPG was limited to tasks with, for the most part, complete information. This work details two recent approaches for scaling TPG to tasks that are dominated by partially observable sources of information using different formulations of indexed memory. One formulation emphasizes the incremental construction of memory, again as an emergent process, resulting in a distributed view of state. The second formulation assumes a single global instance of memory and develops it as a communication medium, thus a single global view of state. The resulting empirical evaluation demonstrates that TPG equipped with memory is able to solve multi-task recursive time-series forecasting problems and visual navigation tasks expressed in two levels of a commercial first-person shooter environment.
Open research data repositories: Practices, norms, and metadata for sharing images

Abstract Open research data repositories are promoted as one of the cornerstones in the open research paradigm, promoting collaboration, interoperability, and large‐scale sharing and reuse. There is, however, a lack of research investigating what these sharing platforms actually share and a more critical interface analysis of the norms and practices embedded in this datafication of academic practice is needed. This article takes image data sharing in the humanities as a case study for investigating the possibilities and constraints in 5 open research data repositories. By analyzing the visual and textual content of the interface along with the technical means for metadata, the study shows how the platforms are differentiated in terms of signifiers of research paradigms, but that beneath the rhetoric of the interface, they are designed in a similar way, which does not correspond well with the image researchers' need for detailed metadata. Combined with the problem of copyright limitations, these data‐sharing tools are simply not sophisticated enough when it comes to sharing and reusing images. The result also corresponds with previous research showing that these tools are used not so much for sharing research data, but more for promoting researcher personas.
Communication-Efficient and Byzantine-Robust Distributed Learning With Error Feedback

We develop a communication-efficient distributed learning algorithm that is robust against Byzantine worker machines. We propose and analyze a distributed gradient-descent algorithm that performs a simple thresholding based on gradient norms to mitigate Byzantine failures. We show the (statistical) error-rate of our algorithm matches that of Yin <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> (2018), which uses more complicated schemes (coordinate-wise median, trimmed mean). Furthermore, for communication efficiency, we consider a generic class of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\delta $ </tex-math></inline-formula> -approximate compressors from Karimireddi <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> (2019) that encompasses sign-based compressors and top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> sparsification. Our algorithm uses compressed gradients and gradient norms for aggregation and Byzantine removal respectively. We establish the statistical error rate for non-convex smooth loss functions. We show that, in certain range of the compression factor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\delta $ </tex-math></inline-formula> , the (order-wise) rate of convergence is not affected by the compression operation. Moreover, we analyze the compressed gradient descent algorithm with error feedback (proposed in Karimireddi <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> 2019) in a distributed setting and in the presence of Byzantine worker machines. We show that exploiting error feedback improves the statistical error rate. Finally, we experimentally validate our results and show good performance in convergence for convex (least-square regression) and non-convex (neural network training) problems.
Generating and Restoring Private Face Images for Internet of Vehicles Based on Semantic Features and Adversarial Examples

The disclosure of face image features can seriously threaten the security of user information, which limits the application of face recognition technology in Internet of Vehicles. This paper proposes a new method of generating and restoring private face images based on semantic features and adversarial examples. The Segnet network first segments the face images semantically, then a generative adversarial network generates adversarial examples and perturbs the semantic features of the face image. The perturbation position can be accurately controlled through a coefficient matrix as the identity tag of the face image is concealed steganographically. A restoration network is trained to extract the real identity tag from the private face image using a discriminator against the generation network, then it restores the private face image to its original state. Compared to other state-of-the-art methods, private face images generated by the proposed method experimentally show high detection resistance, better quality, and stronger median filtering defense.
Text2video: Text-Driven Talking-Head Video Synthesis with Personalized Phoneme - Pose Dictionary

With the advance of deep learning technology, automatic video generation from audio or text has become an emerging and promising research topic. In this paper, we present a novel approach to synthesize video from the text. The method builds a phoneme-pose dictionary and trains a generative adversarial network (GAN) to generate video from interpolated phoneme poses. Compared to audio-driven video generation algorithms, our approach has a number of advantages: 1) It only needs about 1 min of the training data, which is significantly less than audio-driven approaches; 2) It is more flexible and not subject to vulnerability due to speaker variation; 3) It significantly reduces the preprocessing and training time from several days for audio-based methods to 4 hours, which is 10 times faster. We perform extensive experiments to compare the proposed method with state-of-the-art talking face generation methods on a benchmark dataset and datasets of our own. The results demonstrate the effectiveness and superiority of our approach.
Part-based annotation-free fine-grained classification of images of retail products

We propose a novel solution that classifies very similar images (fine-grained classification) of variants of retail products displayed on the racks of supermarkets. The proposed scheme simultaneously captures object-level and part-level cues of the product images. The object-level cues of the product images are captured with our novel reconstruction-classification network (RC-Net). For annotation-free modeling of part-level cues, the discriminatory parts of the product images are identified around the keypoints. The ordered sequences of these discriminatory parts, encoded using convolutional LSTM, describe the products uniquely. Finally, the part-level and object-level models jointly determine the products explicitly explaining coarse to finer descriptions of the products. This bi-level architecture is embedded in R-CNN for recognizing variants of retail products on the rack. We perform extensive experiments on one In-house and three benchmark datasets. The proposed scheme outperforms competing methods in almost all the evaluations.