Reviewerly/exHarmony huggingface dataset:
A Survey of Deep Learning-Based Object Detection Methods and Datasets for Overhead Imagery

Significant advancements and progress made in recent computer vision research enable more effective processing of various objects in high-resolution overhead imagery obtained by various sources from drones, airplanes, and satellites. In particular, overhead images combined with computer vision allow many real-world uses for economic, commercial, and humanitarian purposes, including assessing economic impact from access crop yields, financial supply chain prediction for company’s revenue management, and rapid disaster surveillance system (wildfire alarms, rising sea levels, weather forecast). Likewise, object detection in overhead images provides insight for use in many real-world applications yet is still challenging because of substantial image volumes, inconsistent image resolution, small-sized objects, highly complex backgrounds, and nonuniform object classes. Although extensive studies in deep learning-based object detection have achieved remarkable performance and success, they are still ineffective yielding a low detection performance, due to the underlying difficulties in overhead images. Thus, high-performing object detection in overhead images is an active research field to overcome such difficulties. This survey paper provides a comprehensive overview and comparative reviews on the most up-to-date deep learning-based object detection in overhead images. Especially, our work can shed light on capturing the most recent advancements of object detection methods in overhead images and the introduction of overhead datasets that have not been comprehensively surveyed before.
Blockchain-Based Privacy-Preserving and Sustainable Data Query Service Over 5G-VANETs

Intelligent Transport Systems (ITSs) play an important role in future smart city design to improve traffic safety and traffic congestion by sharing data collected by vehicles. For sharing the traffic data with other vehicles, the vehicular sensory data are usually uploaded to the cloud server. However, existing data sharing systems for VANETs cannot provide selective data with sufficient privacy protection. Moreover, some schemes also cannot ensure stable data accessibility and the integrity of retrieved data. On the other hand, with the improvements such as lower latency, higher capacity, and increased bandwidth, 5G technology brings more possibilities to future applications. The join of the software-defined networks (SDNs) also offers efficient and effective network management. This paper proposes a primitive vehicular communication system named blockchain-based privacy-preserving and sustainable data query service. The proposed scheme is designed to realize stable data accessibility by leveraging smart contracts and blockchain oracle. With the help of 5G technology and P2P file-sharing system, InterPlanetary File System (IPFS), the proposed scheme aims to support video downloading files with searchable capability and fairness. An incentive token mechanism is also equipped. The merit of auditability is ensured by Ethereum blockchain platform to support the accountability. Besides, we also evaluate its networking performance via SUMO and NS-3 simulators. Our simulation results show that the request-response delay of BPSDQS is less than existing blockchain-based proxy re-encryption (PRE) scheme. Our simulation results also showed that the average request-response delay in our scheme can saving up to 98%.
Mobile Augmented Reality Laboratory for Learning Acid–Base Titration

Traditionally, laboratory practice aims to establish schemas learned by students in theoretical courses through concrete experiences. However, access to laboratories might not always be available to students. Therefore, it is advantageous to diversify the tools that students could use to train practical skills. This technology report describes the design, development, and first testing of a mobile augmented reality application that enables a hands-on learning experience of a titration experiment. Additionally, it presents the extension of the TrainAR framework for chemical education through the implementation of specific domain features, i.e., logbook, graph, and practical oriented hints. To test the application, 15 participants were recruited from five different high schools and two universities in Belgium. The findings reflect that the MAR Lab app was well-received by the users. In addition, they valued the design elements (e.g., logbook and multiple-choice questions), and the system has “good” usability (SUS score 72.8, SD = 14.0). Nevertheless, the usability and learners’ experience can be improved by tackling technical problems, providing more explicit instructions for subtasks, and modifying certain features. Therefore, future development will concentrate on improving upon these shortcomings, adding additional levels to target a larger audience, and evaluating the improvements’ effects with more participants.
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation

Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning.
Modality-Invariant Asymmetric Networks for Cross-Modal Hashing

Cross-modal hashing has garnered considerable attention and gained great success in many cross-media similarity search applications due to its prominent computational efficiency and low storage overhead. However, it still remains challenging how to effectively take multilevel advantages of semantics on the entire database to jointly bridge the semantic and heterogeneity gaps across different modalities. In this paper, we propose a novel Modality-Invariant Asymmetric Networks (MIAN) architecture, which explores the asymmetric intra- and inter-modal similarity preservation under a probabilistic modality alignment framework. Specifically, an intra-modal asymmetric network is conceived to capture the query-vs-all internal pairwise similarities for each modality in a probabilistic asymmetric learning manner. Moreover, an inter-modal asymmetric network is deployed to fully harness the cross-modal semantic similarities supported by the maximum inner product search formula between two distinct hash embeddings. Particularly, the pairwise, piecewise and transformed semantics are jointly considered into one unified semantic-preserving hash codes learning scheme. Furthermore, we construct a modality alignment network to distill the redundancy-free visual features and maximize the conditional bottleneck information between different modalities. Such a network could close the heterogeneity and domain shift across different modalities. Extensive experiments evidence that our MIAN approach can outperform the state-of-the-art cross-modal hashing methods.
Performance evaluation of CNN-based pedestrian detectors for autonomous vehicles

With the widespread application of deep learning methodologies, many fields including Intelligent Transportation Systems (ITS) have integrated neural network-based models. In return, the promising performance of neural network-based models attracts more research efforts being paid to the deep learning area. Autonomous vehicles, as the future ITS participants, have achieved tremendous development with the help of convolutional neural network (CNN)-based detectors (e.g., vehicle detectors, lane detectors, pedestrian detectors, traffic sign detectors, etc.) in recent years. However, we have noticed that researchers leveraged different computing power when publishing their experimental results, which could lead to unfair comparisons. In this paper, we focus on CNN-based pedestrian detectors. We conduct a comprehensive comparative study of the representative CNN-based pedestrian detectors, aiming to investigate the influence of experimental settings by eliminating the bias of different experimental environments.
A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images

Breast cancer is one of the most fatal diseases leading to the death of several women across the world. But early diagnosis of breast cancer can help to reduce the mortality rate. So an efficient multi-task learning approach is proposed in this work for the automatic segmentation and classification of breast tumors from ultrasound images. The proposed learning approach consists of an encoder, decoder, and bridge blocks for segmentation and a dense branch for the classification of tumors. For efficient classification, multi-scale features from different levels of the network are used. Experimental results show that the proposed approach is able to enhance the accuracy and recall of segmentation by 1.08%, 4.13%, and classification by 1.16%, 2.34%, respectively than the methods available in the literature.
Low-resolution human pose estimation

Human pose estimation has achieved significant progress on images with high imaging resolution. However, low-resolution imagery data bring nontrivial challenges which are still under-studied. To fill this gap, we start with investigating existing methods and reveal that the most dominant heatmap-based methods would suffer more severe model performance degradation from low-resolution, and offset learning is an effective strategy. Established on this observation, in this work we propose a novel Confidence-Aware Learning (CAL) method which further addresses two fundamental limitations of existing offset learning methods: inconsistent training and testing, decoupled heatmap and offset learning. Specifically, CAL selectively weighs the learning of heatmap and offset with respect to ground-truth and most confident prediction, whilst capturing the statistical importance of model output in mini-batch learning manner. Extensive experiments conducted on the COCO benchmark show that our method outperforms significantly the state-of-the-art methods for low-resolution human pose estimation.
Semi-Synchronous Federated Learning Protocol With Dynamic Aggregation in Internet of Vehicles

In an Internet of Vehicle (IoV) system, federated learning (FL) is a new approach to process real-time vehicle data in a distributed way, which can improve the driving experience and service quality. However, due to the high mobility and uncertainty of vehicles, the existing federated learning protocols are difficult to meet the full requirements from an IoV system, such as efficient resource allocation, high precision learning, and fast convergence of learning algorithm. To solve the above problems, in this paper, we propose a semi-synchronous federated learning (Semi-SynFed) protocol, to improve the performance of machine learning at Internet of Vehicles. In Semi-SynFed, we first select appropriate nodes to participate the aggregation by its computing capacity, network capacity and learning value of training samples. Meanwhile, the dynamic waiting time technique is designed to adjust the server waiting time at each round dynamically, which makes federated learning process work much more efficient. Finally, a dynamic aggregation scheme is designed to aggregate model parameters in an asynchronous way. To evaluate the reliability of Semi-SynFed, we establish a simulation of IoVs based on the real world data. In the comparative experiments, the results show that the Semi-SynFed outperforms the existing federated learning protocols in terms of convergence speed and resource consumption.
Deep Graph Learning for Anomalous Citation Detection

Anomaly detection is one of the most active research areas in various critical domains, such as healthcare, fintech, and public security. However, little attention has been paid to scholarly data, that is, anomaly detection in a citation network. Citation is considered as one of the most crucial metrics to evaluate the impact of scientific research, which may be gamed in multiple ways. Therefore, anomaly detection in citation networks is of significant importance to identify manipulation and inflation of citations. To address this open issue, we propose a novel deep graph learning model, namely graph learning for anomaly detection (GLAD), to identify anomalies in citation networks. GLAD incorporates text semantic mining to network representation learning by adding both node attributes and link attributes via graph neural networks (GNNs). It exploits not only the relevance of citation contents, but also hidden relationships between papers. Within the GLAD framework, we propose an algorithm called Citation PUrpose (CPU) to discover the purpose of citation based on citation context. The performance of GLAD is validated through a simulated anomalous citation dataset. Experimental results demonstrate the effectiveness of GLAD on the anomalous citation detection task.
An Outlier-Robust Kalman Filter With Adaptive Selection of Elliptically Contoured Distributions

In this paper, elliptically contoured (EC) distributions are used to model outlier-contaminated measurement noises. Exploiting a heuristic approach to introduce an unknown parameter, we present an analytical update form of the joint posterior probability density function of the state vector and auxiliary random variable, from which a novel robust EC distributions-based Kalman filtering framework is first derived. To illustrate the effectiveness of the proposed framework, the convergence, robustness, optimality and computational complexity analyses of the proposed method are then given. In addition, to cope with complex noise environments, the interaction multiple model is employed to achieve the adaptive selection of EC distributions such that well-behaved estimation performance can be obtained for different noise cases. Simulation results demonstrate the validity and superiority of the proposed algorithm.
ScanGAN360: A Generative Model of Realistic Scanpaths for 360° Images

Understanding and modeling the dynamics of human gaze behavior in 360° environments is crucial for creating, improving, and developing emerging virtual reality applications. However, recruiting human observers and acquiring enough data to analyze their behavior when exploring virtual environments requires complex hardware and software setups, and can be time-consuming. Being able to generate virtual observers can help overcome this limitation, and thus stands as an open problem in this medium. Particularly, generative adversarial approaches could alleviate this challenge by generating a large number of scanpaths that reproduce human behavior when observing new scenes, essentially mimicking virtual observers. However, existing methods for scanpath generation do not adequately predict realistic scanpaths for 360° images. We present ScanGAN360, a new generative adversarial approach to address this problem. We propose a novel loss function based on dynamic time warping and tailor our network to the specifics of 360° images. The quality of our generated scanpaths outperforms competing approaches by a large margin, and is almost on par with the human baseline. ScanGAN360 allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior, facilitating experimentation, and aiding novel applications in virtual reality and beyond.
Privacy-Preserved Credit Data Sharing Integrating Blockchain and Federated Learning for Industrial 4.0

In this article, we aim to design an architecture for privacy-preserved credit data and model sharing to guarantee the secure storage and sharing of credit information in a distributed environment. The proposed architecture optimizes the data privacy by sharing the data model instead of revealing the actual data. This article also proposes an efficient credit data storage mechanism combined with a deletable Bloom filter to guarantee a uniform consensus for the training and computation process. In addition, we propose authority control contract and credit verification contract for the secure certification of credit sharing model results under federated learning. Extensive experimental results and security analysis demonstrate that our proposed credit model sharing system based on federated learning and blockchain is of high accuracy, efficiency, as well as stability. In particular, the findings of this article could alleviate the potential credit crisis under financial pressure that assist to economic recovery after the global COVID-19 pandemic. Our approach has further boosted up the demand for efficient, secure credit models for Industry 4.0.
A Secure Blockchain Platform for Supporting AI-Enabled IoT Applications at the Edge Layer

In this study, a new blockchain protocol and a novel architecture that integrate the advantages offered by edge computing, artificial intelligence (AI), IoT end-devices, and blockchain were designed, developed, and validated. This new architecture has the ability to monitor the environment, collect data, analyze it, process it using an AI-expert engine, provide predictions and actionable outcomes, and finally share it on a public blockchain platform. For the use-case implementation, the pandemic caused by the wide and rapid spread of the novel coronavirus COVID-19 was used to test and evaluate the proposed system. Recently, various authors traced the spread of viruses in sewage water and studied how it can be used as a tracking system. Early warning notifications can allow governments and organizations to take appropriate actions at the earliest stages possible. The system was validated experimentally using 14 Raspberry Pis, and the results and analyses proved that the system is able to utilize low-cost and low-power flexible IoT hardware at the processing layer to detect COVID-19 and predict its spread using the AI engine, with an accuracy of 95%, and share the outcome over the blockchain platform. This is accomplished when the platform is secured by the honesty-based distributed proof of authority (HDPoA) and without any substantial impact on the devices’ power sources, as there was only a power consumption increase of 7% when the Raspberry Pi was used for blockchain mining and 14% when used to produce an AI prediction.
Modeling Aleatoric Uncertainty for Camouflaged Object Detection

Aleatoric uncertainty captures noise within the observations. For camouflaged object detection, due to similar appearance of the camouflaged foreground and the back-ground, it’s difficult to obtain highly accurate annotations, especially annotations around object boundaries. We argue that training directly with the "noisy" camouflage map may lead to a model of poor generalization ability. In this paper, we introduce an explicitly aleatoric uncertainty estimation technique to represent predictive uncertainty due to noisy labeling. Specifically, we present a confidence-aware camouflaged object detection (COD) framework using dynamic supervision to produce both an accurate camouflage map and a reliable "aleatoric uncertainty". Different from existing techniques that produce deterministic prediction following the point estimation pipeline, our framework formalises aleatoric uncertainty as probability distribution over model output and the input image. We claim that, once trained, our confidence estimation network can evaluate the pixel-wise accuracy of the prediction without relying on the ground truth camouflage map. Extensive results illustrate the superior performance of the proposed model in explaining the camouflage prediction. Our codes are available at https://github.com/Carlisle-Liu/OCENet
Feature fusion and kernel selective in Inception-v4 network

In recent years, deep learning has been developed very quickly, and related research has shown a blossoming scene. Inception-v4 is a wide and deep network with good classification performance. The network structure is very complex, with convolution operations of different sizes, but there are two limitations: the inability to adaptively select the convolution kernel according to the characteristics of the image and the feature extraction from the high-level layer is not strong. This paper focuses on the investigation on the Inception-v4 model and has made several improvements. The improved Inception-v4 model is named BeIn-v4, which integrates the ideas of the Selective Kernel Network (SKNet) into the Inception-v4 network, and adjusts the network structure to achieve improvements. A number of comparative experiments have been carried out on the network before and after the improvements. The experimental results show that BeIn-v4 can obtain better classification results on the tested image datasets than Inception-v4.
GraDual: Graph-based Dual-modal Representation for Image-Text Matching

Image-text retrieval task is a challenging task. It aims to measure the visual-semantic correspondence between an image and a text caption. This is tough mainly because the image lacks semantic context information as in its corresponding text caption, and the text representation is very limited to fully describe the details of an image. In this paper, we introduce Graph-based Dual-modal Representations (GraDual), including Vision-Integrated Text Embedding (VITE) and Context-Integrated Visual Embedding (CIVE), for image-text retrieval. The GraDual improves the coverage of each modality by exploiting textual context semantics for the image representation, and using visual features as a guidance for the text representation. To be specific, we design: 1) a dual-modal graph representation mechanism to solve the lack of coverage issue for each modality. 2) an intermediate graph embedding integration strategy to enhance the important pattern across other modality global features. 3) a dual-modal driven crossmodal matching network to generate a filtered representation of another modality. Extensive experiments on two benchmark datasets, MS-COCO and Flickr30K, demonstrates the superiority of the proposed GraDual in comparison to state-of-the-art methods.