Reviewerly/exHarmony huggingface dataset:
Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learning

The majority of existing few-shot learning methods describe image relations with binary labels. However, such binary relations are insufficient to teach the network complicated real-world relations, due to the lack of decision smoothness. Furthermore, current few-shot learning models capture only the similarity via relation labels, but they are not exposed to class concepts associated with objects, which is likely detrimental to the classification performance due to underutilization of the available class labels. For instance, children learn the concept of tiger from a few of actual examples as well as from comparisons of tiger to other animals. Thus, we hypothesize that both similarity and class concept learning must be occurring simultaneously. With these observations at hand, we study the fundamental problem of simplistic class modeling in current few-shot learning methods. We rethink the relations between class concepts, and propose a novel Absolute-relative Learning paradigm to fully take advantage of label information to refine the image an relation representations in both supervised and unsupervised scenarios. Our proposed paradigm improves the performance of several state-of-the-art models on publicly available datasets.
Data-Free Model Extraction

Current model extraction attacks assume that the adversary has access to a surrogate dataset with characteristics similar to the proprietary data used to train the victim model. This requirement precludes the use of existing model extraction techniques on valuable models, such as those trained on rare or hard to acquire datasets. In contrast, we propose data-free model extraction methods that do not require a surrogate dataset. Our approach adapts techniques from the area of data-free knowledge transfer for model extraction. As part of our study, we identify that the choice of loss is critical to ensuring that the extracted model is an accurate replica of the victim model. Further-more, we address difficulties arising from the adversary’s limited access to the victim model in a black-box setting. For example, we recover the model’s logits from its probability predictions to approximate gradients. We find that the proposed data-free model extraction approach achieves high-accuracy with reasonable query complexity – 0.99× and 0.92× the victim model accuracy on SVHN and CIFAR- 10 datasets given 2M and 20M queries respectively.
AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

As self-driving systems become better, simulating scenarios where the autonomy stack may fail becomes more important. Traditionally, those scenarios are generated for a few scenes with respect to the planning module that takes ground-truth actor states as input. This does not scale and cannot identify all possible autonomy failures, such as perception failures due to occlusion. In this paper, we propose AdvSim, an adversarial framework to generate safety-critical scenarios for any LiDAR-based autonomy system. Given an initial traffic scenario, AdvSim modifies the actors’ trajectories in a physically plausible manner and updates the LiDAR sensor data to match the perturbed world. Importantly, by simulating directly from sensor data, we obtain adversarial scenarios that are safety-critical for the full autonomy stack. Our experiments show that our approach is general and can identify thousands of semantically meaningful safety-critical scenarios for a wide range of modern self-driving systems. Furthermore, we show that the robustness and safety of these systems can be further improved by training them with scenarios generated by AdvSim.
F-RRT*: An improved path planning algorithm with improved initial solution and convergence rate

During the last decades, sampling-based algorithms have been used to solve the problem of motion planning. RRT*, as an optimal variant of RRT, provides asymptotic optimality. However, the slow convergence rate and costly initial solution make it inefficient. To overcome these limitations, this paper proposes a modified RRT* algorithm, F-RRT*, which generates a better initial solution and converges faster than RRT*. F-RRT* optimizes the cost of paths by creating a parent node for the random point, instead of selecting it among the existing vertices. The creation process can be divided into two steps, the FindReachest, and CreatNode procedures, which require few calculations, and the triangle inequality is used repeatedly throughout the process, thus, resulting in paths with higher performance than those of RRT*. Since the algorithm proposed in this paper is a tree extending algorithm, its performance can be further enhanced when combined with other sampling strategies. The advantages of the proposed algorithm in the initial solution and fast convergence rate are demonstrated by comparing with RRT*, RRT*-Smart, and Q-RRT* through numerical simulations in this paper.
Self-Supervised Graph Convolutional Network for Multi-View Clustering

Despite the promising preliminary results, existing graph convolutional network (GCN) based multi-view learning methods directly use the graph structure as view descriptor, which may inhibit the ability of multi-view learning for multimedia data. The major reason is that, in real multimedia applications, the graph structure may contain outliers. Moreover, they fail to take advantage of the information embedded in the inaccurate clustering labels obtained from their proposed methods, resulting in inferior clustering results. These observations motivate us to study whether there is a better alternative GCN based framework for multi-view clustering. To this end, in this paper, we propose an end-to-end self-supervised graph convolutional network for multi-view clustering (SGCMC). Specifically, SGCMC constructs a new view descriptor for graph-structured data by mapping the raw node content into the complex space via Euler transformation, which not only suppresses outliers but also reveals non-linear patterns embedded in data. Meanwhile, the proposed SGCMC uses the clustering labels to guide the learning of the latent representation and coefficient matrix, and the latter in turn is used to conduct the subsequent node clustering. By this way, clustering and representation learning are seamlessly connected, with the aim to achieve better clustering results. Extensive experiments indicate that the proposed SGCMC outperforms the state-of-the-art methods.
Path planning of mobile robot with PSO-based APF and fuzzy-based DWA subject to moving obstacles

This paper proposes a two-layer path-planning method, where an optimized artificial potential field (APF) method and an improved dynamic window approach (DWA) are used at the global and local layer, respectively. This method enables the robot to plan a better path under a multi-obstacle environment while avoiding the moving obstacles effectively. For the part of global path planning, a new repulsive field is proposed based on the APF method. The length and smoothness of the global path are taken as fitness functions of particle swarm optimization (PSO) to obtain the obstacle influence range, the coefficients of gravitation and repulsion in APF. At the level of local path planning, on the basis of DWA, a fuzzy control scheme is adopted to evaluate the danger level of moving obstacles via collision risk index and relative distance. In brief, compared with existing methods, the robot can reasonably plan a shorter and smoother path with the aid of PSO-based APF, meanwhile quickly react to the moving obstacles and avoid them by fuzzy-based DWA. Finally, a static multi-obstacle environment and two dynamic scenarios with moving obstacles are simulated to verify the effectiveness of the proposed path-planning method.
Educational applications of augmented reality: A bibliometric study

Augmented Reality (AR) has been used successfully in several industries; one of these is education. A systematic understanding of how AR contributes to education still lacks studies about the content type and its effects on learning outcomes. This article systematically analyzes the AR state-of-the-art in education, determines productivity and publication indicators in this field, and identifies research works that have studied how content type affects the learning outcomes. The methodology was performed through a bibliometric analysis using the Scopus database, focusing on AR's educational uses. Engineering education is the primary research trend, followed by simulation, tracking, and virtual reality. Education and e-learning also have leading roles within this analysis, along with gamification and human-computer interaction, whose impacts are further explored. There is no preferred design methodology for creating AR content. In its absence, most of the works suggest a design based on the developers' and researchers' experience.
Leveraging Line-point Consistence to Preserve Structures for Wide Parallax Image Stitching

Generating high-quality stitched images with natural structures is a challenging task in computer vision. In this paper, we succeed in preserving both local and global geometric structures for wide parallax images, while reducing artifacts and distortions. A projective invariant, Characteristic Number, is used to match co-planar local sub-regions for input images. The homography between these well-matched sub-regions produces consistent line and point pairs, suppressing artifacts in overlapping areas. We explore and introduce global collinear structures into an objective function to specify and balance the desired characters for image warping, which can preserve both local and global structures while alleviating distortions. We also develop comprehensive measures for stitching quality to quantify the collinearity of points and the discrepancy of matched line pairs by considering the sensitivity to linear structures for human vision. Extensive experiments demonstrate the superior performance of the proposed method over the state-of-the-art by presenting sharp textures and preserving prominent natural structures in stitched images. Especially, our method not only exhibits lower errors but also the least divergence across all test images. Code is available at https://github.com/dut-media-lab/Image-Stitching.
Hybrid Inception v3 XGBoost Model for Acute Lymphoblastic Leukemia Classification

Acute lymphoblastic leukemia (ALL) is the most common type of pediatric malignancy which accounts for 25% of all pediatric cancers. It is a life-threatening disease which if left untreated can cause death within a few weeks. Many computerized methods have been proposed for the detection of ALL from microscopic cell images. In this paper, we propose a hybrid Inception v3 XGBoost model for the classification of acute lymphoblastic leukemia (ALL) from microscopic white blood cell images. In the proposed model, Inception v3 acts as the image feature extractor and the XGBoost model acts as the classification head. Experiments indicate that the proposed model performs better than the other methods identified in literature. The proposed hybrid model achieves a weighted F1 score of 0.986. Through experiments, we demonstrate that using an XGBoost classification head instead of a softmax classification head improves classification performance for this dataset for several different CNN backbones (feature extractors). We also visualize the attention map of the features extracted by Inception v3 to interpret the features learnt by the proposed model.
Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies

Temporal electronic health records (EHRs) contain a wealth of information for secondary uses, such as clinical events prediction and chronic disease management. However, challenges exist for temporal data representation. We therefore sought to identify these challenges and evaluate novel methodologies for addressing them through a systematic examination of deep learning solutions.We searched five databases (PubMed, Embase, the Institute of Electrical and Electronics Engineers [IEEE] Xplore Digital Library, the Association for Computing Machinery [ACM] Digital Library, and Web of Science) complemented with hand-searching in several prestigious computer science conference proceedings. We sought articles that reported deep learning methodologies on temporal data representation in structured EHR data from January 1, 2010, to August 30, 2020. We summarized and analyzed the selected articles from three perspectives: nature of time series, methodology, and model implementation.We included 98 articles related to temporal data representation using deep learning. Four major challenges were identified, including data irregularity, heterogeneity, sparsity, and model opacity. We then studied how deep learning techniques were applied to address these challenges. Finally, we discuss some open challenges arising from deep learning.Temporal EHR data present several major challenges for clinical prediction modeling and data utilization. To some extent, current deep learning solutions can address these challenges. Future studies may consider designing comprehensive and integrated solutions. Moreover, researchers should incorporate clinical domain knowledge into study designs and enhance model interpretability to facilitate clinical implementation.
Survey on Visual Analysis of Event Sequence Data

Event sequence data record series of discrete events in the time order of occurrence. They are commonly observed in a variety of applications ranging from electronic health records to network logs, with the characteristics of large-scale, high-dimensional and heterogeneous. This high complexity of event sequence data makes it difficult for analysts to manually explore and find patterns, resulting in ever-increasing needs for computational and perceptual aids from visual analytics techniques to extract and communicate insights from event sequence datasets. In this paper, we review the state-of-the-art visual analytics approaches, characterize them with our proposed design space, and categorize them based on analytical tasks and applications. From our review of relevant literature, we have also identified several remaining research challenges and future research opportunities.
Breast cancer histopathological image classification using attention <scp>high‐order</scp> deep network

Abstract Computer‐aided classification of pathological images is of the great significance for breast cancer diagnosis. In recent years, deep learning methods for breast cancer pathological image classification have made breakthrough progress, becoming the mainstream in this field. To capture more discriminant deep features for breast cancer pathological images, this work introduces a novel attention high‐order deep network (AHoNet) by simultaneously embedding attention mechanism and high‐order statistical representation into a residual convolutional network. AHoNet firstly employs an efficient channel attention module with non‐dimensionality reduction and local cross‐channel interaction to achieve local salient deep features of breast cancer pathological images. Then, their second‐order covariance statistics are further estimated through matrix power normalization, which provides a more robust global feature presentation of breast cancer pathological images. We extensively evaluate AHoNet on the public BreakHis and BACH breast cancer pathology datasets. Experimental results illustrate that AHoNet gains the optimal patient‐level classification accuracies of 99.29% and 85% on the BreakHis and BACH database, respectively, demonstrating the competitive performance with state‐of‐the‐art single models on this medical image application.
Blockchain technology for the industrial Internet of Things: A comprehensive survey on security challenges, architectures, applications, and future research directions

Abstract The blockchain has emerged as an innovative and powerful technology that shows the tremendous potential to enhance the smart industrial frameworks by providing encryption, immutable storage, and decentralization. In the past few years, several applications in the industrial Internet of Things (IIoT) have emerged and blockchain technologies gained great attention from academia and industry. To discover the great potential of blockchain technology for the IIoT, we present a comprehensive survey on security issues, blockchain architectures, and applications from the industrial perspective. This article starts with a comparison of exiting state‐of‐the‐art surveys of blockchain technologies for IoT/IIoT applications. A four‐layer reference architecture of IIoT is presented along with the functionalities and security issues of each layer. To address these challenges, we assess the potential of blockchain technology by considering the key characteristics, architectures, consensus algorithms, and implementation platforms. Furthermore, we also discussed some use cases of blockchain for the IIoT frameworks. Finally, this survey is concluded by highlighting some open issues and future research directions.
RECALL: Replay-based Continual Learning in Semantic Segmentation

Deep networks allow to obtain outstanding results in semantic segmentation, however they need to be trained in a single shot with a large amount of data. Continual learning settings where new classes are learned in incremental steps and previous training data is no longer available are challenging due to the catastrophic forgetting phenomenon. Existing approaches typically fail when several incremental steps are performed or in presence of a distribution shift of the background class. We tackle these issues by recreating no longer available data for the old classes and outlining a content inpainting scheme on the background class. We propose two sources for replay data. The first resorts to a generative adversarial network to sample from the class space of past learning steps. The second relies on web-crawled data to retrieve images containing examples of old classes from online databases. In both scenarios no samples of past steps are stored, thus avoiding privacy concerns. Replay data are then blended with new samples during the incremental steps. Our approach, RECALL, outperforms state-of-the-art methods.
Ship Detection Algorithm based on Improved YOLO V5

With the rapid development of computer vision and machine vision, deep learning based methods have achieved good results in the field of target detection, recognition, and tracking. However, for ship detection and recognition on the sea surface, the detection and recognition rate is greatly affected by the uneven distribution of the horizontal and vertical features for ships and the different sizes of ships. In order to improve the ship detection accuracy and real-time performance, this paper proposed a ship detection algorithm based on YOLO V5, in which the feature extraction process was merged with the GhostbottleNet algorithm. Specifically, the algorithm consisted of two stacked GhostNet to refine and capture the image features, so as to overcome the incomprehensive feature capture problem in the original YOLO V5 network due to the inhomogeneous distribution of ship image features in transverse and vertical. Experimental results show that the proposed method not only improves the detection accuracy of YOLO V5 algorithm but also makes the GIoU decrease steadily.
A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost

Currently, face-swapping deepfake techniques are widely spread, generating a significant number of highly realistic fake videos that threaten the privacy of people and countries. Due to their devastating impacts on the world, distinguishing between real and deepfake videos has become a fundamental issue. This paper presents a new deepfake detection method: you only look once–convolutional neural network–extreme gradient boosting (YOLO-CNN-XGBoost). The YOLO face detector is employed to extract the face area from video frames, while the InceptionResNetV2 CNN is utilized to extract features from these faces. These features are fed into the XGBoost that works as a recognizer on the top level of the CNN network. The proposed method achieves 90.62% of an area under the receiver operating characteristic curve (AUC), 90.73% accuracy, 93.53% specificity, 85.39% sensitivity, 85.39% recall, 87.36% precision, and 86.36% F1-measure on the CelebDF-FaceForencics++ (c23) merged dataset. The experimental study confirms the superiority of the presented method as compared to the state-of-the-art methods.
Data science and AI in FinTech: an overview

Financial technology (FinTech) has been playing an increasingly critical role in driving modern economies, society, technology, and many other areas. Smart FinTech is the new-generation FinTech, largely inspired and empowered by data science and artificial intelligence (DSAI) techniques. Smart FinTech synthesizes broad DSAI and transforms finance and economies to drive intelligent, automated, whole-of-business and personalized economic and financial businesses, services and systems. The research on data science and AI in FinTech involves many latest progress made in smart FinTech for BankingTech, TradeTech, LendTech, InsurTech, WealthTech, PayTech, RiskTech, cryptocurrencies, and blockchain, and the DSAI techniques including complex system methods, quantitative methods, intelligent interactions, recognition and responses, data analytics, deep learning, federated learning, privacy-preserving processing, augmentation, optimization, and system intelligence enhancement. Here, we present a highly dense research overview of smart financial businesses and their challenges, the smart FinTech ecosystem, the DSAI techniques to enable smart FinTech, and some research directions of smart FinTech futures to the DSAI communities.
A Robustly Optimized BERT Pre-training Approach with Post-training

In the paper, we present a ‘ ’+‘ ’ +‘ ’ three-stage paradigm, which is a supplementary framework for the standard ‘ ’+‘ ’ language model approach. Furthermore, based on three-stage paradigm, we present a language model named PPBERT. Compared with original BERT architecture that is based on the standard two-stage paradigm, we do not fine-tune pre-trained model directly, but rather post-train it on the domain or task related dataset first, which helps to better incorporate task-awareness knowledge and domain-awareness knowledge within pre-trained model, also from the training dataset reduce bias. Extensive experimental results indicate that proposed model improves the performance of the baselines on 24 NLP tasks, which includes eight GLUE benchmarks, eight SuperGLUE benchmarks, six extractive question answering benchmarks. More remarkably, our proposed model is a more flexible and pluggable model, where post-training approach is able to be plugged into other PLMs that are based on BERT. Extensive ablations further validate the effectiveness and its state-of-the-art (SOTA) performance. The open source code, pre-trained models and post-trained models are available publicly.