Reviewerly/exHarmony huggingface dataset:
Federated Transfer Learning for Bearing Fault Diagnosis With Discrepancy-Based Weighted Federated Averaging

Generally, high performance of deep learning (DL)-based machinery fault diagnosis methods relies on abundant labeled fault samples under various working conditions, while they are usually stored by different users and not communicated with each other due to data privacy protection. Federated learning (FL) is a possible solution, but the traditional federated averaging (FedAvg) algorithm in FL ignores the potential domain shift of different FL participants, which limits its further application. Therefore, a federated transfer learning framework with discrepancy-based weighted federated averaging (D-WFA) is proposed to train the good global diagnosis model collaboratively as well as protect data privacy. First, local labeled source domain data and unlabeled target domain data are utilized to update multiple local models with generalization ability. Then, a maximum mean discrepancy (MMD)-based dynamic weighted averaging algorithm is designed to aggregate the updated local models with automatically learned weight. The proposed D-WFA overcomes the disadvantage of the traditional FedAvg algorithm which assumes all clients have the same contribution in constructing the global model during FL training. Experiment results on a bearing dataset show that the proposed D-WFA outperforms the traditional FedAvg and relative federated transfer learning method, which offers a promising solution in privacy-preserving machine learning for fault diagnosis.
Lightweight Deep Learning: An Overview

With the recent success of the deep neural networks (DNNs) in the field of artificial intelligence, the urge of deploying DNNs has drawn tremendous attention because it can benefit a wide range of applications on edge or embedded devices. Lightweight deep learning (DL) indicates the procedures of compressing DNN models into more compact ones, which are suitable to be executed on edge devices due to their limited resources and computational capabilities while maintaining comparable performance as the original. Currently, the approaches of model compression include but not limited to network pruning, quantization, knowledge distillation, neural architecture search. In this work, we present a fresh overview to summarize recent development and challenges for model compression.
Joint Representation Learning and Keypoint Detection for Cross-View Geo-Localization

In this paper, we study the cross-view geo-localization problem to match images from different viewpoints. The key motivation underpinning this task is to learn a discriminative viewpoint-invariant visual representation. Inspired by the human visual system for mining local patterns, we propose a new framework called RK-Net to jointly learn the discriminative Representation and detect salient Keypoints with a single Network. Specifically, we introduce a Unit Subtraction Attention Module (USAM) that can automatically discover representative keypoints from feature maps and draw attention to the salient regions. USAM contains very few learning parameters but yields significant performance improvement and can be easily plugged into different networks. We demonstrate through extensive experiments that (1) by incorporating USAM, RK-Net facilitates end-to-end joint learning without the prerequisite of extra annotations. Representation learning and keypoint detection are two highly-related tasks. Representation learning aids keypoint detection. Keypoint detection, in turn, enriches the model capability against large appearance changes caused by viewpoint variants. (2) USAM is easy to implement and can be integrated with existing methods, further improving the state-of-the-art performance. We achieve competitive geo-localization accuracy on three challenging datasets, i. e., University-1652, CVUSA and CVACT. Our code is available at https://github.com/AggMan96/RK-Net.
Breast Cancer Diagnosis Using Support Vector Machines Optimized by Whale Optimization and Dragonfly Algorithms

Breast Cancer (BC) has become a critical illness with a high mortality rate during the previous decade. It is considered the women’s most common cancer. In this paper, we propose two optimum automated BC classification approaches based on a hybridization of the Whale Optimization Algorithm (WOA) and Dragonfly Algorithm (DA), with Radial Basis Function Kernel Support Vector Machines (RBF-SVM), to increase the accuracy of BC classification (CA) by determining the optimum SVM parameters. The effectiveness of the proposed WOA-SVM and DA-SVM algorithms is tested on the Wisconsin Diagnosis Breast Cancer (WDBC) databases and the Wisconsin Breast Cancer Database (WBCD). Various metric parameters such as CA, confusion matrix, the area under the ROC curve (AUC), sensitivity, and specificity are utilized to assess and consider the effectiveness of the proposed approaches. The results are compared not only to the most common optimizers, Particle Swarm Optimization (PSO) and Genetic Algorithm (GA), that are used for training SVM and artificial neural networks (ANN) classifiers, but also to other classification models. The WOA-SVM and DA-SVM are also explored for feature selection, and their findings are compared to the offered models. According to the experimental results, the proposed WOA-SVM method outperforms previous classification approaches on the WBCD dataset. On the WDBC dataset, however, the proposed DA-SVM algorithm outperforms the previous classification algorithms. Using typical datasets’ partition, the resultant CA is as high as 99.65% and 100% for WDBC and WBCD, respectively. However, using a 10-fold cross-validation datasets’ partition, the mean resultant CA are 97.89% and 99.27%, respectively.
PupilRec: Leveraging Pupil Morphology for Recommending on Smartphones

As mobile shopping has gradually become the mainstream shopping mode, recommendation systems are gaining an increasingly wide adoption. Existing recommendation systems are mainly based on explicit and implicit user behaviors. However, these user behaviors may not directly indicate users’ inner feelings, causing erroneous user preference estimation and thus leading to inaccurate recommendations. Inspired by our key observation on the correlation between pupil size and users’ inner feelings, we consider using the change of pupil size when browsing to model users’ preferences, so as to achieve targeted recommendations. To this end, we propose PupilRec as a computer-vision-based recommendation framework involving a mobile terminal and a server side. On the mobile terminal, PupilRec collects users’ pupil size change information through the front camera of smartphones; it then preprocesses the raw pupil size data before transmitting them to the server. On the server side, PupilRec utilizes the Tsfresh package and Random Forest algorithm to figure out the key time-series features directly implying user preferences. PupilRec then trains a neural network to fit a user preference model. Using this model, PupilRec predicts user preference to obtain a user–product matrix and further simplifies it by singular value decomposition. Finally, the real-time recommendation is achieved by a collaborative filtering module that retrieves recommended contents to users smartphones. We prototype PupilRec and conduct both experiments and field studies to comprehensively evaluate the effectiveness of PupilRec by recruiting 67 volunteers. The overall results show that PupilRec can accurately estimate users’ preference and can recommend products users interested in.
Artificial intelligence in renewable systems for transformation towards intelligent buildings

Carbon-neutrality transition in building sectors requires combinations of renewable systems and artificial intelligence (AI) for robustness, reliability, automation, and flexibility. In this study, a comprehensive review on AI applications in renewable systems, have been conducted, to report the current progress, tendency and challenges. Underlying learning mechanisms of AI-based various applications have been studied, in terms of modelling techniques on solar power forecasting, multi-level stochastic uncertainty analysis, smart controls, fault detection and diagnosis, single and multi-objective optimisations and intelligent buildings. Furthermore, AI techniques in renewable energy utilisations have been clearly reviewed, with respect to solar potential evaluation, multi-level stochastic uncertainty analysis, smart controls, fault detection and diagnosis, single and multi-objective optimisations. Results showed that, compared to physical models, the data-driven models show superiority in modelling simplification and modification, high prediction accuracy, and computational efficiency. In order to improve the system operational robustness under multi-level scenario uncertainty, a generic approach with artificial intelligence was proposed, including nonlinear data-driven model development, uncertainty-based energy performance prediction, stochastic uncertainty and statistical analysis. Furthermore, artificial intelligence can also be applied in renewable systems for security, reliability and stability. This study provides a clear roadmap on historical development, recent advances, cutting-edge techniques and progress, and future challenges of AI applications in renewable energy systems, paving path for developing smart and resilient renewable energy systems for decarbonization in buildings.
CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions

The success of StyleGAN has enabled unprecedented semantic editing capabilities, on both synthesized and real images. However, such editing operations are either trained with semantic supervision or annotated manually by users. In another development, the CLIP architecture has been trained with internet-scale loose image and text pairings, and has been shown to be useful in several zero-shot learning settings. In this work, we investigate how to effectively link the pretrained latent spaces of StyleGAN and CLIP, which in turn allows us to automatically extract semantically-labeled edit directions from StyleGAN, finding and naming meaningful edit operations, in a fully unsupervised setup, without additional human guidance. Technically, we propose two novel building blocks; one for discovering interesting CLIP directions and one for semantically labeling arbitrary directions in CLIP latent space. The setup does not assume any pre-determined labels and hence we do not require any additional supervised text/attributes to build the editing framework. We evaluate the effectiveness of the proposed method and demonstrate that extraction of disentangled labeled StyleGAN edit directions is indeed possible, revealing interesting and non-trivial edit directions.
An advanced soil organic carbon content prediction model via fused temporal-spatial-spectral (TSS) information based on machine learning and deep learning algorithms

Knowledge of the soil organic carbon (SOC) content is critical for environmental sustainability and carbon neutrality. With the development of remote sensing data and prediction models, the comprehensive utilization of multisource remote sensing data based on a fusion approach and testing its effectiveness in SOC content prediction is an interesting and challenging topic. However, there is no evidence showing the role of different data sources in the SOC content prediction process. In this study, a total of 796 topsoil samples (0–20 cm) were collected at Site 1, and 111 samples were collected at Site 2. The samples from Site 2 were used to verify the transferability of the prediction model established at Site 1. The discrete wavelet transform based on the regional energy weight (RW-DWT) and spectral band segmentation methods were used to fuse the temporal information of 10 scenes of Landsat multispectral image data from 2009 to 2019, the spatial information of topography data and the spectral information of GaoFen-5 hyperspectral images. Then, the SOC content prediction models were established by temporal-spatial-spectral (TSS) information using partial least squares regression (PLSR), random forest (RF) and convolutional neural network (CNN) algorithms. The results indicated that the optimal SOC content prediction model consisted of TSS information as input and the CNN as the prediction model, where the lowest root mean square error (RMSE) was 2.49 g kg −1 , the highest coefficient of determination (R 2 ) was 0.86 and the ratio of performance to interquartile distance (RPIQ) was 1.91. Next, the order of the effect was spectral > temporal > spatial information in terms of SOC content prediction, and their roles in improving the accuracy of the model were 26.79%, 19.64% and 14.29%, respectively, with the CNN model. In addition, the CNN yielded a higher prediction accuracy than PLSR and RF regardless of which group of input variables was used. The average RMSE of the CNN was 0.42 g kg −1 lower than that of the RF, and the average R 2 and RPIQ were 9.25% and 0.14 higher, respectively, than those of the RF. The above conclusions were confirmed in the verification area, namely, the optimal SOC content prediction model at Site 2 consisted of TSS information as input and the CNN as the prediction model (RMSE = 1.01 g kg −1 , R 2 = 0.76 and RPIQ = 1.41). Therefore, the novel method proposed in this study is robust. This work provides a new idea for predicting soil properties by the comprehensive use of multisource remote sensing images and deep learning algorithms in the future. • The ability of TSS information in SOC prediction is determined. • The SOC prediction model with high-accuracy and high-transferability is established. • The method of temporal, spatial, spectral information fusion is improved. • The role of temporal, spatial, spectral information for SOC prediction is revealed. • “Data fusion + deep learning” strategy provide a new paradigm for SOC prediction.
Towards Enhancing Virtual Museums by Contextualizing Art through Interactive Visualizations

In response to the COVID-19 pandemic, public spaces such as museums and art galleries are experiencing increased demands to offer virtual online access. While current solutions seek to replace or augment a real visit, online tours often suffer from being too passive and lack in-depth interactivity to keep virtual visitors meaningfully engaged with an exhibition. Museums and art galleries seeking to broaden and engage their audience more deeply should offer intriguing experiences that invite the visitor to explore, to be entertained, and to learn by interacting with the content. We propose a novel virtual museum experience that utilizes multiple visualizations to contextualize a gallery’s digitized artworks with related artworks from large image archives. We make use of the WikiArt dataset that includes more than 200,000 images and offers diverse metadata used for comparative visual exploration. In addition, we apply machine learning methods to extract multifaceted information about the objects detected in the images and to compute similarities across them. Visitors of our virtual museum can interactively explore the artworks using different search filters such as artist, style, or object classes detected within an image. The results are displayed through interactive visualizations offering different perspectives on artwork collections, leading to serendipitous discoveries and stimulating new insights. The utility of our concept was confirmed by an informal evaluation with virtual museum visitors.
Mining bitcoins with carbon capture and renewable energy for carbon neutrality across states in the USA

Bitcoin mining in the U.S. will be more sustainable with more renewables in the grid, lower electricity prices, and carbon capture and storage.
Deep Reinforcement Learning for energy-aware task offloading in join SDN-Blockchain 5G massive IoT edge network

The Internet-of-Things (IoT) edge allows cloud computing services for topology and location-sensitive distributed computing. As an immediate benefit, it improves network reliability and latency by enabling data access and processing rapidly and efficiently near IoT devices. However, it comes with several issues stemming from the complexity, the security, the energy consumption, and the instability due to the decentralization of service localization. Furthermore, the multi-resource allocation and task scheduling make this task the furthest from being straightforward. Blockchain has been envisioned to enforce trustworthiness in diverse IoT environments. However, high latency and high energy costs are incurred to process IoT transactions. This paper introduces a novel Blockchain-based Deep Reinforcement Learning (DRL) approach to enable energy-aware task scheduling and offloading in an Software Defined Networking (SDN)-enabled IoT network. The Asynchronous Actor-Critic Agent (A3C) DRL-based policy achieves efficient task scheduling and offloading. The latter is in symbiosis with Proof-of-Authority Blockchain consensus to validate IoT transactions and blocks. By doing so, we improve reliability and low latency and achieve energy efficiency for SDN-enabled IoT networks. The A3C policy combined with the Blockchain is proved theoretically. Carried out experiments put forth that our approach offers 50% better energy efficiency, which outperforms traditional consensus algorithms, i.e., Proof of Work and PBFT, in terms of throughput and network latency and offers better scheduling performance. • Blockchain-based Deep Reinforcement Learning applied for task scheduling and offloading in an SDN-enabled IoT network. • Optimization of consumable energy with improving QoS during tasks scheduling and offloading. • Enhance task scheduling and offloading compared to well established DRL models.
A Novel Decentralized Blockchain Architecture for the Preservation of Privacy and Data Security against Cyberattacks in Healthcare

Nowadays, in a world full of uncertainties and the threat of digital and cyber-attacks, blockchain technology is one of the major critical developments playing a vital role in the creative professional world. Along with energy, finance, governance, etc., the healthcare sector is one of the most prominent areas where blockchain technology is being used. We all are aware that data constitute our wealth and our currency; vulnerability and security become even more significant and a vital point of concern for healthcare. Recent cyberattacks have raised the questions of planning, requirement, and implementation to develop more cyber-secure models. This paper is based on a blockchain that classifies network participants into clusters and preserves a single copy of the blockchain for every cluster. The paper introduces a novel blockchain mechanism for secure healthcare sector data management, which reduces the communicational and computational overhead costs compared to the existing bitcoin network and the lightweight blockchain architecture. The paper also discusses how the proposed design can be utilized to address the recognized threats. The experimental results show that, as the number of nodes rises, the suggested architecture speeds up ledger updates by 63% and reduces network traffic by 10 times.
Graph Neural Networks: Foundation, Frontiers and Applications

The field of graph neural networks (GNNs) has seen rapid and incredible strides over the recent years. Graph neural networks, also known as deep learning on graphs, graph representation learning, or geometric deep learning, have become one of the fastest-growing research topics in machine learning, especially deep learning. This wave of research at the intersection of graph theory and deep learning has also influenced other fields of science, including recommendation systems, computer vision, natural language processing, inductive logic programming, program synthesis, software mining, automated planning, cybersecurity, and intelligent transportation. However, as the field rapidly grows, it has been extremely challenging to gain a global perspective of the developments of GNNs. Therefore, we feel the urgency to bridge the above gap and have a comprehensive tutorial on this fast-growing yet challenging topic. This tutorial of Graph Neural Networks (GNNs): Foundation, Frontiers and Applications will cover a broad range of topics in graph neural networks, by reviewing and introducing the fundamental concepts and algorithms of GNNs, new research frontiers of GNNs, and broad and emerging applications with GNNs. In addition, rich tutorial materials will be included and introduced to help the audience gain a systematic understanding by using our recently published book-Graph Neural Networks (GNN): Foundation, Frontiers, and Applications [12], which can easily be accessed at https://graph-neural-networks.github.io/index.html.
The 6th AI City Challenge

The 6th edition of the AI City Challenge specifically focuses on problems in two domains where there is tremendous unlocked potential at the intersection of computer vision and artificial intelligence: Intelligent Traffic Systems (ITS), and brick and mortar retail businesses. The four challenge tracks of the 2022 AI City Challenge received participation requests from 254 teams across 27 countries. Track 1 addressed city-scale multi-target multi-camera (MTMC) vehicle tracking. Track 2 addressed natural-language-based vehicle track retrieval. Track 3 was a brand new track for naturalistic driving analysis, where the data were captured by several cameras mounted inside the vehicle focusing on driver safety, and the task was to classify driver actions. Track 4 was another new track aiming to achieve retail store automated checkout using only a single view camera. We released two leader boards for submissions based on different methods, including a public leader board for the contest, where no use of external data is allowed, and a general leader board for all submitted results. The top performance of participating teams established strong baselines and even outperformed the state-of-the-art in the proposed challenge tracks.
Smart contract vulnerability detection combined with multi-objective detection

Blockchains have been booming in recent years. As a decentralized system architecture, smart contracts give blockchains a user-defined logic. A smart contract is an executable program that can automatically carry out transactions on the Ethereum blockchain. However, some security issues in smart contracts are difficult to fix, and smart contracts also lack quality assessment standards. Therefore, this study proposes a Multiple-Objective Detection Neural Network (MODNN), a more scalable smart contract vulnerability detection tool. MODNN can validate 12 types of vulnerabilities, including 10 recognized threats, and identify more unknown types without the need for specialist or predefined knowledge through implicit features and Multi-Objective detection (MOD) algorithms. It supports the parallel detection of multiple vulnerabilities and has high scalability, eliminating the need to train separate models for each type of vulnerability and reducing significant time and labor costs. This paper also developed a data processing tool called Smart Contract-Crawler (SCC) to address the lack of smart contract vulnerability datasets. MODNN was evaluated using more than 18,000 smart contracts from Ethereum. Experiments showed that MODNN could achieve an average F1 Score of 94.8%, the current highest compared to several standard machine learning (ML) classification models.