Reviewerly/exHarmony huggingface dataset:
Evaluation of Nested U-Net models performance on MVTec AD dataset

Anomaly detection (AD) from image data using convolutional neural networks and deep learning has become a widespread topic among both scientists and engineers. In addition to the development of new methods and models, specialized datasets are created as well. The most cited dataset specialised on anomaly detection tasks and created for testing the most recent methods is MVTec AD. This dataset has been used in more than 40 articles which are mainly devoted to creating or modifying AD methods. Subsequently, their performance is usually tested on the MVTec AD dataset. However, despite a large number of different methods and models, there is a lack of performance evaluation of U-Net++ (Nested U-Net architecture), a robust model which is well-known in the field of segmentation tasks. This article is focused on the evaluation of two Nested U-Net architectures (U-Net++, ANU-Net) on the MVTec AD dataset. It is shown that the direct use of the Nested U-Net models to reconstruct anomaly-free input data together with their strong augmentation during training phase leads to inability to reconstruct image data with anomalies at inference time. Achieved results can compete with some of the state-of-the-art reconstruction-based methods. The average image-level AUROC performance of U-Net++ model is 97.9% and 96.2% for image size of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$64\times 64$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$128\times 128$</tex> pixels, respectively. Further, the average performance of ANU-Net on image-level detection is 96.5% and 96.8% for image size of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$64\times 64$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$128\times 128$</tex> pixels, respectively.
Anomaly detection for real-world industrial applications: benchmarking recent self-supervised and pretrained methods

Visual anomaly detection (AD) is currently a very active research area with great potential in many real-world applications, e.g. quality control in industry and manufacturing, where it can provide cost savings and overall better product quality. Recently, many new anomaly detection methods have been introduced and many of them are intended for industrial usage. These methods are usually evaluated on a narrow selection of datasets that may differ significantly from certain types of real-world applications. Due to this approach, some methods provide different performance when deployed in real use cases. In this paper, we perform evaluation of recent state of the art visual anomaly detection methods on the problem of defect detection in metal parts fabrication, an area not well covered in existing publications. We introduce a new dataset focused specifically on the problem of metal parts fabrication and use the dataset to perform the evaluation. For the evaluation, we selected methods that use two different feature extraction approaches for anomaly detection. One of the approaches is using feature extractors pretrained on the ImageNet dataset and the second approach is training the feature extractor from scratch using self-supervised learning. We show that, in contrast to one of the most widely used anomaly detection benchmark - the MVTec-AD dataset, self-supervised methods perform significantly better (in average by 17 % AUROC) on the proposed dataset.
Interval Type-2 Fuzzy Logic PID Controller Based on Phase and Margins Gains of the System Applied to a Non-Linear Control of a Conical Tank

This paper proposes a new algorithm for auto tuning of PID controllers whose gains are calculated through a self-tuning of the phase and gain margins for the closed-loop system using Interval Fuzzy Type-2 Logic with genetic auto-tuning of its membership functions. Simulations by using a nonlinear conical tank are presented with the objective of verifying the effectiveness of proposed control method. The results show that the presented strategy improves the controller performance for the conical tank.
A Novel Explainable Nature-inspired Metaheuristic: Jaguar Algorithm with Precision Hunting Behavior

Metaheuristics are crucial for solving complex optimization problems effectively in many fields. With the recent increased interest in explainable artificial intelligence (XAI), the issue of how metaheuristics interact with XAI has attracted much attention. Most metaheuristics have stochastic search processes, and the jaguar algorithm (JA) is a unique algorithm that has exact search paths. JA shows potential abilities both in exploration and exploitation, but still faces limitations. Therefore, this study proposes a new metaheuristic based on JA and invents precision hunting behavior (PH-JA), which inherits the concept of JA and significantly strengthens its search efficiency. PH-JA improves the solution quality by adaptively detecting the current environment’s trends during movement and precisely hunting prey. Then, PH-JA makes the best of the historical information to reduce the computational cost. PH-JA is an explainable metaheuristic and has systematic operational procedures in that each movement is performed according to the present situation rather than random elements. Therefore, the path to a solution is interpretable and can return the same result with the same input. The experimental results demonstrated the superiority of PH-JA, which is better than other classical and state-of-the-art metaheuristics in terms of solution quality and evaluation costs.
BSAM: Research on image-text matching method based on Bert and self-attention mechanism

Image-text matching plays a crucial role in connecting vision and language. The details of the objects in the image, the positional relationship, and the correspondence between the background and the text description are the keys to image-text matching. Previous studies either only extract the salient objects of the image, or only pay attention to the location of the object, ignoring the detailed features and background features of the object, and the extraction of the overall semantic information of the image is not comprehensive enough. Accordingly, this paper proposes a model based on Bert and Self-Attention Mechanism (BSAM), we segment the image area, use the self-attention mechanism to enhance the weight of the key area, pay attention to each object and their detailed features and background features, the image regions are mapped into original region features and new features with other region relationships, and the global information of the image is inferred based on the relationship between each region and background features. The text extracts word features and new features with other word relationships through the Bert model. We propose the Cross-Attention and Similarity -Attention Filtering (CA-SAF) module to align all relevant image regions and words, enhance matching pairs with high weights, and filter matching pairs with lower weights. Extensive experiments on two datasets, Flickr30K and MS COCO, show that the BSAM model significantly outperforms state-of-the-art methods.
Object Detection in Unmanned Aerial Vehicle Camera Stream Using Deep Neural Network

Nowadays, the world is experiencing an increasing boom in applications of artificial intelligence, especially deep learning. This is more and more used in many areas such as industry, medicine and security systems, etc. This article deals with object detection from Unmanned Aerial Vehicle (UAV) perspective. The whole system uses one camera, which is suitably positioned on the UAV to capture the scene. Image processing and subsequent object detection using the YOLOv4 model are performed on the Jetson Nano device. The device itself is relatively powerful, but to save the computing power of the device, the YOLOv4 neural network model was modified. The YOLOv4 model was trained on our own dataset. This training set was created specifically for UAV applications. The result of this work is a learned YOLOv4 neural network model designed for UAVs with regard to the used training set. The modified network model is also able to run in real-time and save computing power for possibly other UAV operations. All materials, dataset and scripts used in this work, are available at https://github.com/KicoSVK/object-detection-in-uav-using-yolov4.
Online Refinement of a Scene Recognition Model for Mobile Robots by Observing Human’s Interaction with Environments

This paper describes a method of online refinement of a scene recognition model for robot navigation considering traversable plants, flexible plant parts which a robot can push aside while moving. In scene recognition systems that consider traversable plants growing out to the paths, misclassification may lead the robot to getting stuck due to the traversable plants recognized as obstacles. Yet, misclassification is inevitable in any estimation methods. In this work, we propose a framework that allows for refining a semantic segmentation model on the fly during the robot’s operation. We introduce a few-shot segmentation based on weight imprinting for online model refinement without fine-tuning. Training data are collected via observation of a human’s interaction with the plant parts. We propose novel robust weight imprinting to mitigate the effect of noise included in the masks generated by the interaction. The proposed method was evaluated through experiments using real-world data and shown to outperform an ordinary weight imprinting and provide competitive results to fine-tuning with model distillation while requiring less computational cost.
View-Robust Neural Networks for Unseen Human Action Recognition in Videos

Data-driven deep learning achieved excellent performance for human action recognition. However, unseen action recognition remains a challenge for most existing neural networks. Because the action categories, collection perspectives, and scenarios considered during data collection are limited. Compared with class-unseen action recognition, view-unseen action recognition in videos is under-explored. This paper proposes view-robust neural networks (VR-Net) to recognize unseen actions in videos. The VR-Net consists of a 3D pose estimation module, skeleton adaptive transformation neural networks, and classification modules. We first extract 3D skeleton models from the video sequence based on existing pose estimation methods. Next, we propose a skeleton representation transformation scheme and achieve it based on Convolutional Neural Networks (VR-CNN) and Graph Neural Networks (VR-GCN), resulting in the optimal skeleton representations. Futhermore, we explore an associate optimization scheme and a fused output method. We evaluate the proposed neural networks on three challenging benchmarks, i.e., NTU RGB-D dataset (NTU), Kinetics-400 dataset, and Human3.6M dataset (H3.6M). The experimental results show that view robust neural networks achieve the top performance compared to state-of-the-art RGB-based and skeleton-based works, such as 93.6% on the NTU (CV) and 94.6% on the Kinetics-400 dataset (Top-5). The proposed neural networks significantly improve the recognition performance for unseen action recognition, such as 86.8% on the H3.6M (View 2).
Chaotic Evolution Using Deterministic Crowding Method for Multi-modal Optimization

This paper proposes a novel population-based optimization algorithm to solve the multi-modal optimization problem. We call it the chaotic evolution deterministic crowding (CEDC) algorithm. Since the genetic algorithm is difficult to find all optimal solutions and the accuracy is not high when searching for multi-modal optimization problems, we use the ergodicity of chaos to implement the exploration and fitness comparison of the deterministic crowding algorithm. Through the tests of several multi-modal benchmark functions, it is shown that the algorithm can effectively and accurately find the most optimal solutions to the multi-modal problem. It does not need to set the niche radius in advance, so it can better solve multimodal optimization problems. We test it with nine multi-modal benchmark functions ranging from one-dimension (1-D) to ten-dimension (10-D), and we compare it with a genetic algorithm and evaluate from peak ratio, max peak ratio, and running time. The experimental results show that the CEDC algorithm is better than conventional algorithms in both runtime and peak ratio.
Graph Instinctive Attention Convolutional Network for Skeleton-Based Action Recognition

Graph convolutional networks (GCNs) are widely used in skeleton-based action recognition and have achieved excellent results. However, it is evident that the convolution operation can lead to losing some original input information. The incomplete utilisation of original input data limits GCNs’ ability to obtain the skeleton’s correlation. This paper proposes a graph instinctive attention convolutional network (GIAN) to solve this problem. In particular, it contains an instinctive attention module that uses self-attention to obtain the correlation within the original input skeleton. Then, parameter attention is used to further refine the relationship between different skeleton joints. Experimental results on publicly available datasets demonstrate that the GIAN outperforms most of the state-of-the-art algorithms.
Interpretable Navigation Agents Using Attention-Augmented Memory

Deep reinforcement learning (DRL) has achieved remarkable success in various domains, from games to complex tasks. In several DRL applications, the agents must perform long-horizon tasks in partially observable environments. However, owing to the high performance of DRL, the decision-making process for the long-horizon task is unclear and difficult to interpret. Although memory-based models and the Transformer model have been proposed to overcome the interpretability issue, challenges of limited flexibility, lack of stability, and high computational cost remain. To address these concerns, we propose a low-computational-complexity and scalable DRL model that uses attention-augmented memory (AAM) to interpret the long-horizon decision-making process. AAM adds long short-term memory states of past observations to the memory and uses a soft attention bottleneck to combine them into a single contextual vector. The agent is then trained to make decisions based on this AAM together with the current observation. The AAM model is applied to the navigation problem of the Labyrinth [1], and attention and saliency maps are generated to show the areas highlighted by the agent in the current observation and the areas it attends to from memory. The resiliency of the model was evaluated using saliency and it was discovered that the proposed method is more resilient to the noise of visual observations compared with the baseline model. In summary, the proposed method demonstrates an interpretable and noise-robust DRL approach for long-horizon tasks.
A CNN Inference micro-benchmark for Performance Analysis and Optimization on GPUs

optimization of per-layer or/and total inference time without accuracy loss in convolutional neural networks (CNNs) is significantly crucial in resource-constrained Edge-AI devices and embedded systems. To do this, this work 1) introduces a CNN inference micro-benchmark (mbNet) for performance analysis and optimization and 2) proposes a simple yet effective performance model for adaptive kernel selection to optimize per-layer CNN inference time. Considering the convolutional layer is the core part of CNNs, the two mainstream convolutional strategies of unrolling based convolution (UNROLL) and direct convolution (DIRECT) are adopted/implemented, compared and analyzed in terms of per-layer convolutional time. Using the obtained data from our mbNet, we build an accurate and interpretable tree-based performance model, with which our adaptive kernel selection approach shows significant convolutional performance improvement up to $5.4\times$ speedup (on average, $2.7\times$ and $1.6\times$ speedup, compared to the default UNROLL and DIRECT respectively).
Reproducibility in Activity Recognition Based on Wearable Devices: a Focus on Used Datasets

Reproducibility of proposed approaches is a crucial element in scientific fields, in order to let other researchers trust published works. Moreover, in order to let authors compare the effectiveness of a novel method to the state of the art, benchmark datasets should be commonly used. Concentrating on the task of activity recognition using data coming from wearable devices with inertial sensors, we have analyzed the reproducibility of proposed approaches with a focus on used datasets. In this work, with a literature review, we have measured what percentage of works in the literature verified their approach using public datasets or sharing the ones created on purpose. At the same time, we have also examined the characteristics of considered datasets, with attention to the amount of data recorded, involved population, and studied activities. Starting from 1289 works retrieved on Scopus, we analyzed in detail 146 of them and found out that approximately one out of three (~33%) used public datasets and that less than one out of three (~28%) of the specially made datasets were shared with the public. Moreover, considering all the examined datasets, 13% of them had restricted access (e.g. requiring requests to authors or subscriptions to websites for a fee) or were offline.
Adaptive Metric-weight and Bias Network for Few-shot Learning

Few-shot learning is dedicated to dealing with the dependence of deep learning on a large amount of data. It learns some new concepts through a large number of training tasks instead of a large amount of data, which enables it learn new tasks quickly without a large amount of training data. Compared with other meta-learning methods, the effect of metric-based meta-learning methods is more advantageous. Despite the success of metric-based meta-learning approaches, the adaptability of metric methods to data distribution is still insufficient, which leads to classification results more dependent on the accuracy of encoder networks and are susceptible to noisy features. In this paper, we realize the filtering of noisy features and the correction of data distribution by adaptive learning of metric weights and data distribution biases, and propose corresponding loss functions to evaluate and update our adaptive module and encoding network. Experimental results on standard few-shot learning datasets demonstrate that our proposed method achieves good improvement.
Satellite video target tracking based on kernel correlation filtering and trajectory prediction

There are various problems in the current target tracking of satellite videos, such as the small size of moving targets, few textures, and easy occlusion. To solve the problem of tracking failure caused by occlusion of moving objects, a tracking algorithm based on kernel correlation filtering and trajectory prediction is proposed in this paper. The algorithm in this paper uses the motion characteristics of moving objects in satellite video to predict the trajectory and uses the trajectory prediction algorithm to predict the trajectory of the target when it is occluded. Combined with the kernel correlation filtering algorithm, the target can still be accurately tracked after it is occluded. Experiments on satellite video datasets show that the algorithm in this paper can effectively solve the problem of tracking failure caused by the occlusion of moving objects, and the speed is also very fast.
A new local binary pattern for smoke recognition

In order to improve the detection rate of smoke recognition and reduce the false positive and error rates, a new local binary pattern (Zigzag Local Binary Pattern, ZLBP) is proposed. In ZLBP, we first rearrange the pixels in the local area into four linear areas by four zigzags with four directions, and then design two coding methods for the linear areas. For four linear areas, we can get four feature vectors, each of which is computed based on two codes of the same linear area. Finally, we concatenate the four feature vectors to generate ZLBP feature. Experimental results show that the new proposed pattern is effective and suitable for smoke identification.
Aircraft swarm detection based on multi-scale-feature-fused attention

To improve the detection performance of aircraft swarms in remote sensing images with characteristics of small target, scale diversity and dense distribution, a multi-scale-feature-fused attention mechanism is proposed and used for deep learning networks in this paper. Based on the fundamental YOLOv5 network, an enhanced multi-scale CBAM attention module that combines the channel attention and the spatial attention is performed on the fused feature maps at various stages and scales. Consequently, more detailed attention information can be obtained. Experimental results demonstrate that the proposed method can effectively improve the detection accuracy of aircraft swarm targets compared with some traditional methods. In detail, the proposed method can reach 9.2% higher recall than the original YOLOv5 and 3.4% higher recall than the YOLOv5 integrated with traditional CBAM modules while the computational cost is similar
Super-resolution reconstruction of remote sensing images based on SRGAN

In the field of remote sensing images, due to the limitations of hardware equipment, image transmission, natural environment and other reasons, the resolution of the obtained remote sensing images cannot reach the desired resolution. The emergence of image super-resolution reconstruction technology can improve the resolution of remote sensing images without increasing the high cost. Image super-resolution reconstruction refers to the fact that low-resolution images can obtain high-resolution images through certain algorithmic techniques. With the rapid development of deep learning ideas, researchers have applied it to the field of image super-resolution reconstruction and achieved good results. Image super-resolution reconstruction also shifts from traditional reconstruction methods to deep learning-based methods. The emergence of the idea of Generative Adversarial Networks has further advanced the field of image super-resolution reconstruction. By using the idea of Generative Adversarial Network (GAN), researchers can obtain more realistic high-resolution images. This paper mainly uses the SRGAN model, the image dataset DIV2K for super-resolution reconstruction, and uses a dense residual structure in the generator network to obtain more image information, so that the effect of image reconstruction is more realistic. Through the experimental verification on the SIRI-WHU remote sensing test data set, the two evaluation indicators of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are compared, and the effect is improved. Better generation results can also be observed through subjective human vision.
3D human pose estimation using aided constraints of physiological and action feature

3D human pose estimation is a hot research topic at present, and it also has a wide application potential. The inherent uncertainty and multiple solutions of 2D to 3D mapping based on a single image limit the accuracy of 3D human pose estimation. Considering that human posture is affected by physiological features and motion states, the network design in this paper uses physiological and motion features to provide constraints for posture estimation, in order to achieve better accuracy. Specifically, in the network design of this paper, three auxiliary judgment networks, namely gender, motion type and true false judgment, are used to further constrain the generated posture. Moreover, experiments on Human3.6M dataset show that the accuracy of mapping 2D joint coordinates to 3D pose coordinates can be effectively improved by introducing constraints of physiological features and motion states.