Reviewerly/exHarmony huggingface dataset:
Re-identification framework for long term visual object tracking based on object detection and classification

In this paper, we address the problem of long-term visual object tracking and we present an efficient real-time single object tracking system suitable for integration in autonomous platforms that need to encompass intelligent capabilities. We propose a novel long-term tracking framework for classification based re-detection and tracking, that incorporates state estimation, object re-identification and automated management of tracking and detection results. Our method integrates a novel object re-identification technique which efficiently filters a number of detection candidates and systematically corrects the tracking results. Through extensive experimental validation on the UAV123, UAV20L and TLP datasets, we demonstrate the effectiveness of the proposed system and its advantage over several state-of-the art trackers. The results furthermore highlight the proposed tracker’s ability to handle challenges arising from real-world and long-term scenarios, such as variations in pose, scale, occlusions and out-of-view situations. Furthermore, we propose a variant that is suitable for deployment on autonomous robots, such as Unmanned Aerial Vehicles.
On the Design of Secure Communication Framework for Blockchain-Based Internet of Intelligent Battlefield Things Environment

On the basis of the Internet of Things (IoT) applications, several variants such as Internet of Energy (IoE), Internet of Vehicles (IoV), Internet of Drones (IoD), Internet of Medical Things (Internet of Health, IoMT) and Internet of Intelligent Battlefield Things (IoIBT) are developed. The smart devices involved in IoT can also utilize artificial intelligence algorithm (i.e., machine learning) to make the communication environment more effective and efficient. Intelligent Battlefield Things (IBT) communication environment is an example of that kind. It consists of smart and intelligent devices (i.e., drones), which monitor the activities of the enemies and retaliate accordingly. Hence, we need very less or negligible human involvement. IBT communication environment is very suitable for the purpose of battlefield communication (i.e., war zone, target tracking and hitting, border area surveillance and retaliation). However, IBT has certain drawbacks, because it may be vulnerable to different types of attacks, such as replay, man-in-the-middle, impersonation, data modification, data leakage attacks, etc. Therefore, it becomes essential to provide a security mechanism to secure the communication that happens in IBT environment. To achieve this purpose, we propose a secure communication framework for the blockchain of IBT environment. In addition, the benefits of applications of blockchain of IBT are discussed. The issues and challenges of blockchain of IBT environment are also highlighted. Finally, a security analysis of the proposed framework is conducted to depict its resilience against possible attacks.
A Camera Model for Line-Scan Cameras with Telecentric Lenses

Abstract We propose a camera model for line-scan cameras with telecentric lenses. The camera model assumes a linear relative motion with constant velocity between the camera and the object. It allows to model lens distortions, while supporting arbitrary positions of the line sensor with respect to the optical axis. We comprehensively examine the degeneracies of the camera model and propose methods to handle them. Furthermore, we examine the relation of the proposed camera model to affine cameras. In addition, we propose an algorithm to calibrate telecentric line-scan cameras using a planar calibration object. We perform an extensive evaluation of the proposed camera model that establishes the validity and accuracy of the proposed model. We also show that even for lenses with very small lens distortions, the distortions are statistically highly significant. Therefore, they cannot be omitted in real-world applications.
Android Based Expert System Application for Diagnose COVID-19 Disease: Cases Study of Banyumas Regency

Since being confirmed by WHO, the status of COVID-19 outbreak has become a global pandemic, the number of cases has been confirmed positive, cured, and even death worldwide. Artificial intelligence in the medical has given rise to expert systems that can replace the role of experts (doctors). Tools to detect someone affected by COVID-19 have not been widely applied in all regions. Banyumas Regency, Indonesia is included confirmed region of COVID-19 cases, and it’s difficult for someone to know the symptoms that are felt whether these symptoms include indications of someone ODP, PDP, positive, or negative COVID-19, and still at least a referral hospital handling COVID-19. Expert system with certainty factor can help someone make a self-diagnose whether including ODP, PDP, positive, or negative COVID-19. This expert system provides ODP diagnostic results with a confidence level of 99.96%, PDP 99.99790%, positive 99.9999997%, negative 99.760384%, and the application runs well on Android OS
Attention-Based Activation Pruning to Reduce Data Movement in Real-Time AI: A Case-Study on Local Motion Planning in Autonomous Vehicles

In state-of-the-art deep neural network (DNN), the layer-wise activation maps leads to significant data movement in hardware accelerators operating on real-time streaming inputs. We explore an architecture-aware algorithmic approach to reduce data movement and the resulting latency and power consumption. This article presents an attention-based feedback for controlling input data, referred to as the activation pruning, that reduces activation maps in early layers of a DNN network which are critical for reducing data movement in real-time AI processing. The proposed approach is demonstrated for coupling RGB and Lidar images to perform real-time perception and local motion planning in autonomous systems. Lidar data is used to determine “Pixels of Interest”(PoI) in an RGB image depending on their distance from sensor, prune the RGB image to perform object detection only within the PoI, and use the detected objects to perform local motion planning. Experiments on sequences from KITTI dataset shows the activation pruning maintains quality of motion planning while increasing the sparsity of the activation maps. The sparsity-aware computing architectures is considered to leverage activation sparsity for improved performance. The simulation results show that proposed activation pruning algorithm reduces data movement (38.5%), computational load (30.1%), and memory latency (76.3%) in sparsity-aware compute architecture, leading to faster perception and lower energy consumption.
Blockchain based secure IoT data sharing framework for SDN-enabled smart communities

Software-defined networks offer lower latency service and massive intelligent devices connectivity for the Internet of Things(IoT). As one of the pivotal applications in the IoT, smart community provides smart services for residents through the SDN technique. However, the centralized SDN suffers from single point of attacks such as DDoS from IoT devices and has the issue of data leakage. In this paper, we use blockchain and proxy re-encryption(PRE) technologies to tackle these challenges. The blockchain authorizes all devices in the network to improve their credibility and authenticity. What's more, a blockchain-based data sharing framework that combines a PRE scheme is introduced for secure device-to-device communication in smart communities. A series of smart contracts are designed for flexible operations of searching and updating records on the blockchain. The experiments reveal that our design is highly efficient and has high performance.
A Vulnerability Risk Assessment Method Based on Heterogeneous Information Network

Due to the increasing number of network security vulnerabilities, vulnerability risk assessment must be performed to prioritize the repair of high-risk vulnerabilities. Traditional vulnerability risk assessment is based primarily on the Common Vulnerability Scoring Systems (CVSS) and attack graphs. Nevertheless, the CVSS metrics ignore the impact of the vulnerability on the specific network, which accounts that the identical vulnerability exists in different network environments is assigned repeated values. Additionally, the attack graphs still suffer from scalability and readability issues. To solve the above problems, a ranking method based on the heterogeneous information network is innovatively proposed to assess the vulnerability risk in a specific network. It considers the exploitability of a vulnerability, the impact of a vulnerability on the network components, and the importance of the vulnerable components. First, a heterogeneous information network containing vulnerability and host and the relationships between host and host is constructed to compute the risk score for each vulnerability and implement the ranking process. Second, a model extension method is proposed to adapt to situations in which additional factors related to vulnerability risk assessment need to be considered. Finally, we explore two case studies to compare the proposed method with CVSS and attack graph-based methods. The simulation results show that the proposed method can accurately assess the risk of vulnerabilities in a specific network environment and that it has a lower computational complexity than other methods.
A Hybrid Neural Network RBERT-C Based on Pre-trained RoBERTa and CNN for User Intent Classification

User intent classification plays a critical role in identifying the interests of users in question-answering and spoken dialog systems. The question texts of these systems are usually short and their conveyed semantic information are frequently insufficient. Therefore, the accuracy of user intent classification related to user satisfaction may be affected. To address the problem, this paper proposes a hybrid neural network named RBERT-C for text classification to capture user intent. The network uses the Chinese pre-trained RoBERTa to initialize representation layer parameters. Then, it obtains question representations through a bidirectional transformer structure and extracts essential features using a Convolutional Neural Network after question representation modeling. The evaluation is based on the publicly available dataset ECDT containing 3736 labeled sentences. Experimental result indicates that our model RBERT-C achieves a F1 score of 0.96 and an accuracy of 0.96, outperforming a number of baseline methods.
A Sensors Based Deep Learning Model for Unseen Locomotion Mode Identification using Multiple Semantic Matrices

With the availability of various sensors in the smartphone, identifying a locomotion mode becomes convenient and effortless in recent years. Information about locomotion mode helps to improve journey planning, travel time estimation, and traffic management. Though there exists a significant amount of work towards locomotion mode recognition, the performance of these work is not pertinent and heavily depends on the labeled training instances. As it is impractical to gather a prior information (labeled instances) about all types of locomotion modes, the recognition model should be able to identify a new or unseen locomotion mode without having any corresponding training instance. This paper proposes a sensors based deep learning model to identify a locomotion mode by using labeled training instances. The approach also incorporates a concept of Zero-Shot learning to identify an unseen locomotion mode. The model obtains an attribute matrix based on the fusion of three semantic matrices. It also constructs a feature matrix by extracting the deep learning and hand-crafted features from the training instances. Later, the model builds a classifier by learning a mapping between attribute and feature matrices. Finally, this work evaluates the performance of the approach on collected and existing datasets using accuracy and F1 score.
Model-based random forests for ordinal regression

We study and compare several variants of random forests tailored to prognostic models for ordinal outcomes. Models of the conditional odds function are employed to understand the various random forest flavours. Existing random forest variants for ordinal outcomes, such as Ordinal Forests and Conditional Inference Forests, are evaluated in the presence of a non-proportional odds impact of prognostic variables. We propose two novel random forest variants in the model-based transformation forest family, only one of which explicitly assumes proportional odds. These two novel transformation forests differ in the specification of the split procedures for the underlying ordinal trees. One of these split criteria is able to detect changes in non-proportional odds situations and the other one focuses on finding proportional-odds signals. We empirically evaluate the performance of the existing and proposed methods using a simulation study and illustrate the practical aspects of the procedures by a re-analysis of the respiratory sub-item in functional rating scales of patients suffering from Amyotrophic Lateral Sclerosis (ALS).
Toward Anomaly Behavior Detection as an Edge Network Service Using a Dual-Task Interactive Guided Neural Network

How to use artificial intelligence technology to mine human abnormal behavior from considerable video data generated by the Internet-of-Things system has been intensively studied for a long time. Existing deep learning anomaly detection algorithms deployed in the cloud typically perform supervised learning based on constant kinds of abnormal behavior data. However, this supervised learning model with preset abnormal behavior categories ignores the diversity and unpredictability of abnormal occurrences in open scenarios. Thus, we propose an abnormal behavior detection algorithm as an edge network service by combining the advantages of cloud computing and the efficiency of edge networks. This method combines the double verification of global behavior detection and local fine-grained action cycle alignment to detect whether a behavior is abnormal. Moreover, to enable abnormal behavior detection models to predict test samples whose categories do not appear during the training stage, we propose an active label learning algorithm based on cycle clustering, which not only improves the efficiency of data transmission between the edge and the cloud but also makes model updates in the cloud more efficient. Extensive and quantitative experimental results show that our method can not only accurately detect abnormal human behavior at the edge of limited resources but also has strong robustness under the interference of test samples of unknown categories.
Layout Detection and Table Recognition – Recent Challenges in Digitizing Historical Documents and Handwritten Tabular Data

In this paper, we discuss the computer-aided processing of handwritten tabular records of historical weather data. The observationes meteorologicae, which are housed by the Regensburg University Library, are one of the oldest collections of weather data in Europe. Starting in 1771, meteorological data was consistently documented in a standardized form over almost 60 years by several writers. The tabular structure, as well as the unconstrained textual layout of comments and the use of historical characters, propose various challenges in layout and text recognition. We present a customized strategy to digitize tabular and handwritten data by combining various state-of-the-art methods for OCR processing to fit the collection. Since the recognition of historical documents still poses major challenges, we provide lessons learned from experimental testing during the first project stages. Our results show that deep learning methods can be used for text recognition and layout detection. However, they are less efficient for the recognition of tabular structures. Furthermore, a tailored approach had to be developed for the historical meteorological characters during the manual creation of ground truth data. The customized system achieved an accuracy rate of 82% for the text recognition of the heterogeneous handwriting and 87% accuracy for layout recognition of the tables.
An intelligent target detection method of UAV swarms based on improved KM algorithm

Complete and efficient detection of unknown targets is the most popular application of UAV swarms. Under most situations, targets have directional characteristics so that they can only be successfully detected within specific angles. In such cases, how to coordinate UAVs and allocate optimal paths for them to efficiently detect all the targets is the primary issue to be solved. In this paper, an intelligent target detection method is proposed for UAV swarms to achieve real-time detection requirements. First, a target-feature-information-based disintegration method is built up to divide the search space into a set of cubes. Theoretically, when the cubes are traversed, all the targets can be detected. Then, a Kuhn-Munkres (KM)-algorithm-based path planning method is proposed for UAVs to traverse the cubes. Finally, to further improve search efficiency, a 3D real-time probability map is established over the search space which estimates the possibility of detecting new targets at each point. This map is adopted to modify the weights in KM algorithm, thereby optimizing the UAVs' paths during the search process. Simulation results show that with the proposed method, all targets, with detection angle limitations, can be found by UAVs. Moreover, by implementing the 3D probability map, the search efficiency is improved by 23.4%–78.1%.
An Adaptive Deep Learning Framework for Shipping Container Code Localization and Recognition

Shipping containers play an important role in global transportation. As container codes are the unique identifiers for shipping containers, recognizing these codes is an essential step to manage the containers and logistics. The conventional code localization methods can easily be interfered by varied noises and cannot identify the best regions for code recognition. In this article, we propose an adaptive deep learning framework for shipping container code localization and recognition. In the framework, the noisy text regions will be removed by an adaptive score aggregation (ASA) algorithm. The code region boundaries are identified by the average-to-maximum suppression range (AMSR) algorithm. Thus, the predicted locations can be adjusted within this range to fit the code recognition model to achieve higher accuracy. The experimental results on the comparative study with the state-of-the-art models, including EAST, PSENet, GCRNN, and MaskTextSpotter, demonstrated that the proposed framework achieved better localization performance and obtained 93.33% recognition accuracy. The processing speed reaches 1.13 frames/s, which is sufficient to meet the operational requirements. Thus, the proposed solution will facilitate the digital transformation of shipping container management and logistics at ports.
Building Super-Resolution Image Generator for OCR Accuracy Improvement

Super-resolving a low resolution (LR) document image can not only enhance the visual quality and readability of the text, but improve the optical character recognition (OCR) accuracy. However, even despite the ill-posed nature of image super-resolution (SR) problem, how do we treat the finer details of text with large upscale factors and suppress noises and artifacts at the same time, especially for low quality document images is still a challenging task. Thus, in order to boost the OCR accuracy, we propose a generative adversarial network (GAN) based framework in this paper, where a SR image generator and a document image quality discriminator are constructed. To obtain high quality SR document image, multiple losses are designed to encourage the generator to learn the structural properties of texts. Meanwhile, the quality discriminator is trained based on a relativistic loss function. Based on the proposed framework, the obtained SR document images not only maintain the details of textures but remove the background noises, which achieve better OCR performance on the public databases. The source codes and pre-trained models are available at https://gitlab.com/xujun.peng/doc-super-resolution .
Application of Watershed Algorithm and Gray Level Co-Occurrence Matrix in Leukemia Cells Images

A long with the development of technology, the image of a sample of leukemia can be digitally processed to reduce the level of human error in diagnosing the disease. This research conducted by designing the image processing system on two types of leukemia, there are Acute Myelogenous Leukemia (AML) and Normal cell images by applying Watershed segmentation methods and feature extraction Gray Level Co-Occurrence Matrix (GLCM). The system was designed to find out how effective these methods to be continue into the classification process. The results of testing the application of these methods are the accuracy of the watershed segmentation method for this kind of Normal class was 90.4% with the average computing time of 0.89 seconds, and for the class of AML is 100% with the average computation time of 0.94 seconds. Application of the method GLCM has a significant difference the two types of leukemia were examine for each value extraction features with faster computing time, average 0.0060 seconds of computing time for this kind of Normal images. Whereas, for the AML class with average computation time was 0.0054 seconds.
Interactive facial animation with deep neural networks

Creating realistic animations of human faces is still a challenging task in computer graphics. While computer graphics (CG) models capture much variability in a small parameter vector, they usually do not meet the necessary visual quality. This is due to the fact, that geometry‐based animation often does not allow fine‐grained deformations and fails in difficult areas (mouth, eyes) to produce realistic renderings. Image‐based animation techniques avoid these problems by using dynamic textures that capture details and small movements that are not explained by geometry. This comes at the cost of high‐memory requirements and limited flexibility in terms of animation because dynamic texture sequences need to be concatenated seamlessly, which is not always possible and prone to visual artefacts. In this study, the authors present a new hybrid animation framework that exploits recent advances in deep learning to provide an interactive animation engine that can be used via a simple and intuitive visualisation for facial expression editing. The authors describe an automatic pipeline to generate training sequences that consist of dynamic textures plus sequences of consistent three‐dimensional face models. Based on this data, they train a variational autoencoder to learn a low‐dimensional latent space of facial expressions that is used for interactive facial animation.