Reviewerly/exHarmony huggingface dataset:
Path planning for spot welding robots based on improved ant colony algorithm

Abstract A welding path can be planned effectively for spot welding robots using the ant colony algorithm, but the initial parameters of the ant colony algorithm are usually selected through human experience, resulting in an unreasonable planned path. This paper combines the ant colony algorithm with the particle swarm algorithm and uses the particle swarm algorithm to train the initial parameters of the ant colony algorithm to plan an optimal path. Firstly, a mathematical model for spot welding path planning is established using the ant colony algorithm. Then, the particle swarm algorithm is introduced into the ant colony algorithm to find the optimal combination of parameters by treating the initial parameters $\alpha$ and $\beta$ of the ant colony algorithm and as two-dimensional coordinates in the particle swarm algorithm. Finally, the simulation analysis was carried out using MATLAB to obtain the paths of the improved ant colony algorithm for six different sets of parameters with an average path length of 10,357.7509 mm, but the average path length obtained by conventional algorithm was 10,830.8394 mm. Convergence analysis of the improved ant colony algorithm showed that the average number of iterations was 17. Therefore, the improved ant colony algorithm has higher solution quality and converges faster.
Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Although semantic segmentation models based on deep neural networks (DNNs) have achieved excellent results, generalizing well from one remote sensing dataset (source domain) to another dataset with different acquisition conditions (target domain) remains a major challenge. Many domain adaptation (DA) approaches have been proposed to address this problem. DA aims to help DNNs learn a generalizable representation space in which source and target domains have similar feature distributions, but most of the existing DA approaches have difficulty in aligning the high-dimensional image representations of two domains directly. In this study, we proposed a model integrating contrastive learning and adversarial learning in a unified framework for aligning two domains in both representation space and spatial layout. Specifically, the model consists of a semantic segmentation network for feature extraction and two branches for DA. The first branch is used for adaptation in representation space directly by a proposed pixelwise contrastive loss, while the second branch is used for adaptation in predicted results to help two domains have similar spatial layouts through a novel but simple entropy-based similarity discriminator. Additionally, a training strategy called category similarity matching sampling was proposed to provide source and target image pairs with similar category composition for each training iteration, which can help the two branches work better. Extensive experiments indicated that the two branches can benefit each other to gain a superior performance and DA pretraining by our methods can achieve impressive results with only a small number of target labeled samples.
On the Integration of Blockchain With IoT and the Role of Oracle in the Combined System: The Full Picture

The remarkable increase in the number of interconnected smart devices in today’s Internet of things networks introduces more challenges related to security, trust, and centralization, which require more effective solutions. Fortunately, blockchain technology has recently emerged as a potential rescuer for IoT-based solutions due to its decentralization and enhanced security features. It is usual for smart contracts to arise in handling and processing the generated data when IoT devices are combined with blockchain. However, blockchain and smart contracts need to interact with input data of the same level of trust to guarantee correct applications execution. This implies using oracles to provide trust compatibility between inserted information collected from IoT devices and blockchain and smart contracts. Therefore, this study adopts a methodology that was shaped based on current literature and design and experiments to provide a full narrative of the process of combining two of the most intriguing systems in today’s world of technology, namely, blockchain and IoT including a very important part of the comprehensive system, viz. blockchain oracle. Moreover, it was found that the literature lacks a complete view of the IoT-blockchain integration process that covers all its important and related aspects. Therefore, this work is an attempt to fill the gap in literature and contribute to the body of knowledge by surveying the literature about existing IoT-blockchain architectures and shed light on the role of blockchain in addressing IoT issues while demonstrating the concept of oracles as well as their functions in addition to the main operating blockchain oracles. Additionally, this work illustrates a CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> measuring use case where a smart contract is developed and tested as part of two proposed oracle-based designs. The obtained results demonstrate a full picture of a practical integrated IoT-blockchain system architecture.
Cooperative path planning for air–sea heterogeneous unmanned vehicles using search-and-tracking mission

The cooperative path planning problem for air–sea heterogeneous unmanned vehicles using search-and-tracking mission is studied in this paper. Due to the limitations of single unmanned vehicle, the air–sea heterogeneous system, composed of an unmanned aerial–aquatic vehicle (UAAV), an unmanned surface vehicle (USV) and an autonomous underwater vehicle (AUV), is introduced to compensate for the singleness of unmanned vehicle and complete more complicated marine missions. One search-and-tracking mission containing the search phase and tracking phase is presented to accomplish the underwater target tracking mission with goals of minimizing the search time and finding the shortest tracking path. In the search phase, according to different initial positions of three unmanned vehicles, the corresponding task allocation algorithms are proposed to determine the location of the target efficiently and minimize the search time by allocating different task for three unmanned vehicles. Meanwhile, the unmanned vehicles only know the approximate area containing the target before the task execution and move in the direction of the area for searching target. In the tracking phase, an improved particle swarm optimization algorithm is addressed to solve the path planning problem with obstacles. Simulation results show that the underwater target can be detected using search-and-tracking mission in an air–sea heterogeneous system efficiently and accurately.
Relation extraction for manufacturing knowledge graphs based on feature fusion of attention mechanism and graph convolution network

Relation extraction is a crucial step in the constructions of knowledge graphs (KGs). However, relation extraction is performed manually in the manufacturing field due to the sentence characteristics, which include weak correlation and high entity density. This approach has the disadvantages of low efficiency and high dependence on experts. At present, very few studies have been performed on relation extraction in the manufacturing field, so establishing a relation extraction model with high efficiency is an urgent need. Therefore, in this paper, a relation extraction model is proposed for manufacturing knowledge (MKREM), in which word embedding is obtained by the Bi-LSTM layer to improve robustness, and a Simplified Graph Convolution Network (SGC) layer is applied to quickly mine the entity information. Then, dependency and semantic features are extracted by the multi-head stacked GCN and relation attention mechanism, respectively. Finally, the dependency and semantic features are fused to generate the comprehensive features for relation extraction so that better performance on texts with weak correlation and high entity density can be obtained. The performance of MKREM is tested by experiments on the equipment maintenance dataset and the quality dataset from an automobile enterprise, and its effectiveness is verified in the automobile manufacture filed. The results show that the F1 scores obtained using MKREM are 2% higher than those of the commonly used models on both datasets, and the F1 scores when using Contextualized-MKREM are improved by 3%, so MKREM is very suitable for the automatic relation extraction during establishing manufacturing KGs.
Spatial-Net for Human-Object Interaction Detection

Human-object interaction (HOI) detection is the detection of a human's relationship with an object in still images and videos.The majority of HOI detection methods rely on appearance features as the primary feature for detecting the relationship between humans and objects.Furthermore, the model's performance is affected by the abundance of false-positive pairs generated by the image's non-interactive human-object pairs and human-object mis-grouping.In this paper, we propose "Spatial-Net", a new HOI detection approach in still images.In the proposed approach, the HOI problem is divided into two main tasks, namely pair-prediction and global-rejection.In the pair-prediction task, the spatial relationship is adopted to predict the human-object interaction for each human-object pair using spatial features that contains spatial map which is a single channel image that represents human-object pairs including body parts and object masks, relative geometry features such as relative size, relative distance, and intersection-overunion between body part and objects, and weighted distance that is used as body part attention deterministic model.In the global-rejection task, an augmented model is employed to reject false positive pairs.We use the Hungarian matching technique to assign human-object pairs for each action and human-centric model to reject the non-interaction human-object pairs according to semantic co-occurrence between human and object.The experimental results on the V-COCO dataset demonstrate that the proposed Spatial-Net outperforms many state-of-the-art HOI models with less inference time.
Serious Game for VR Road Crossing in Special Needs Education

Autism spectrum disorder (ASD) is a developmental disability that can impair communication, social skills, living skills, and learning capabilities. Learning approaches usually differ between mainstream schools and special needs schools, to cater for the different learning processes of children with ASD. Besides the traditional classroom-based education, alternative technology and methods are explored for special needs education. One method is to train children with ASD using Virtual Reality (VR) technologies. Many prior works show the effectiveness of VR-based learning with varying degrees of success. Some children with ASD may face challenges to gain independent living skills. Their parents or guardians have to expend a significant amount of effort in taking care of children with ASD. It will be very helpful if they can have a learning opportunity to gain such living skills. In this research, we develop a VR serious game to train children with ASD one of the basic living skills for road crossing safely. The VR serious game operates on multiple types of platforms, with various user interaction inputs including the Microsoft Kinect sensor, keyboard, mouse, and touch screen. The game design and methodology will be described in this paper. Experiments have been conducted to evaluate the learning effectiveness of the road crossing game, with very positive results achieved in the quiz and survey questionnaire after the gameplay.
Voice Conversion Based Augmentation and a Hybrid CNN-LSTM Model for Improving Speaker-Independent Keyword Recognition on Limited Datasets

Keyword recognition is the basis of speech recognition, and its application is rapidly increasing in keyword spotting, robotics, and smart home surveillance.Because of these advanced applications, improving the accuracy of keyword recognition is crucial.In this paper, we proposed voice conversion (VC) -based augmentation to increase the limited training dataset and a fusion of a convolutional neural network (CNN) and long-short term memory (LSTM) model for robust speaker-independent isolated keyword recognition.Collecting and preparing a sufficient amount of voice data for speaker-independent speech recognition is a tedious and bulky task.To overcome this, we generated new raw voices from the original voices using an auxiliary classifier conditional variational autoencoder (ACVAE) method.In this study, the main intention of voice conversion is to obtain numerous and various human-like keywords' voices that are not identical to the source and target speakers' pronunciation.Parallel VC was used to accurately maintain the linguistic content.We examined the performance of the proposed voice conversion augmentation techniques using robust deep neural network algorithms.Original training data, excluding generated voice using other data augmentation and regularization techniques, were considered as the baseline.The results showed that incorporating voice conversion augmentation into the baseline augmentation techniques and applying the CNN-LSTM model improved the accuracy of isolated keyword recognition.
Siamese Graph-Based Dynamic Matching for Collaborative Filtering

Behaviorally similar neighbors in the interaction graph have been actively explored to facilitate the collaboration between users and items and address the interaction sparsity issue. We investigate homogenous neighbors between users or items to mine collaborative signals for embedding learning. In the case of multiple and complex composition of user interests and item attributes, traditional uniform embedding is insufficient to depict matching between a specific user-item pair. Therefore, we propose a siamese graph-based dynamic matching (SGDM) model for collaborative filtering. A target-aware dual attention module is introduced to update neighbors with their varying contributions to unveil users’ interests, which explicitly encodes the clue of the candidate matching target. For a target user-item pair, homogeneous neighbors participate in the interest propagation through graph convolution, which learns user/item embeddings respectively on the siamese homogeneous graphs. The dual dynamic aggregation in graph convolution endows a specific user-item pair with dynamic matching, which is expected to meet the needs of fine-grained filtering and promote the performance of collaborative filtering. Extensive experimental results confirm the collaboration between siamese homogeneous graphs of users and items. It further illustrates the effectiveness of the proposed SGDM in mining homogeneous collaborative signals for embedding learning and collaborative filtering.
F-TPE: Flexible Thumbnail-Preserving Encryption Based on Multi-Pixel Sum-Preserving Encryption

People are accustomed to utilizing mobile phones to capture images and uploading them to the cloud due to various incomparable advantages such as saving the storage space on the device. In this context, privacy concerns are raised since images may contain some sensitive information. Encrypting images by traditional schemes can alleviate privacy leak, but the usability is often deprived since it is tough to browse them on the cloud. Recently, Tajik et al. proposed an ideal <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">thumbnail-preserving encryption</i> (TPE) scheme to achieve the balance of usability and privacy by leveraging a two-pixel substitution encryption method while the Markov chain is utilized to prove the security. However, the connectivity of Markov chain in this scheme is weak since the length of pixel groups in the encryption process is only two, leading to a long mixing time of achieving the stationary distribution of Markov chain. To this end, we firstly propose a method of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-pixel sum-preserving encryption</i> (MP-SPE) that realizes the encipherment of vectors of arbitrary length. Then, with the help of MP-SPE, a novel flexible ideal TPE scheme (F-TPE) is designed and the connection of Markov chain is improved. The experiments have demonstrated that the proposed scheme can effectively attain the balance between usability and privacy. In addition, F-TPE takes much less time in encrypting compared with the existing work.
Task Relatedness-Based Multitask Genetic Programming for Dynamic Flexible Job Shop Scheduling

Multitask learning has been successfully used in handling multiple related tasks simultaneously. In reality, there are often many tasks to be solved together, and the relatedness between them is unknown in advance. In this article, we focus on the multitask genetic programming (GP) for the dynamic flexible job shop scheduling (DFJSS) problems, and address two challenges. The first is how to measure the relatedness between tasks accurately. The second is how to select task pairs to transfer knowledge during the multitask learning process. To measure the relatedness between DFJSS tasks, we propose a new relatedness metric based on the behavior distributions of the variable-length GP individuals. In addition, for more effective knowledge transfer, we develop an adaptive strategy to choose the most suitable assisted task for the target task based on the relatedness information between tasks. The findings show that in all of the multitask scenarios studied, the proposed algorithm can substantially increase the effectiveness of the learned scheduling heuristics for all the desired tasks. The effectiveness of the proposed algorithm has also been verified by the analysis of task relatedness and structures of the evolved scheduling heuristics, and the discussions of population diversity and knowledge transfer.
Designing complex manufacturing systems by virtual reality: A novel approach and its application to the virtual commissioning of a production line

The design of complex manufacturing systems (CMSs) is challenging, because of the requirements of efficiency, safety, and ergonomics, and the need of optimizing resources, i.e., space, machines, operators, and data. Virtual reality (VR) – one of the promising technologies at the base of Industry 4.0 – is able to address the design issues of CMSs, and even decrease costs and time when employed from the initial conception to the final validation of production lines, since it facilitates their virtual commissioning, i.e., it enables the full verification of systems and related components by virtual inspection and tests. Despite the above advantages, VR is still rarely used in the design of CMSs, and there is no standard VR approach in industry yet. In addition, the related scientific literature is scarce and often limited to small or simplified cases. To fill this gap, this work presents a novel VR-based approach for designing CMSs, composed of four phases: Three-dimensional CAD Export, Model Import, Scene Creation, and VR Review. The proposed approach is applied to a real industrial use case related to the virtual commissioning of an electric axles production line and it is evaluated through a questionnaire from industry professionals. The case study shows that using the VR technology enhanced the technical communication between experts in the teamwork, and it was particularly effective in finding ergonomics flaws like issues in visibility, reach, and posture using a virtual golden zone. In addition, all users found the VR interaction enjoyable and easy to learn, and beginner users perceived a comparable workload as advanced users.
A collaborative workflow between pathologists and deep learning for the evaluation of tumour cellularity in lung adenocarcinoma

The reporting of tumour cellularity in cancer samples has become a mandatory task for pathologists. However, the estimation of tumour cellularity is often inaccurate. Therefore, we propose a collaborative workflow between pathologists and artificial intelligence (AI) models to evaluate tumour cellularity in lung cancer samples and propose a protocol to apply it to routine practice.We developed a quantitative model of lung adenocarcinoma that was validated and tested on 50 cases, and a collaborative workflow where pathologists could access the AI results and adjust their original tumour cellularity scores (adjusted-score) that we tested on 151 cases. The adjusted-score was validated by comparing them with a ground truth established by manual annotation of haematoxylin and eosin slides with reference to immunostains with thyroid transcription factor-1 and napsin A. For training, validation, testing the AI and testing the collaborative workflow, we used 40, 10, 50 and 151 whole slide images of lung adenocarcinoma, respectively. The sensitivity and specificity of tumour segmentation were 97 and 87%, respectively, and the accuracy of nuclei recognition was 99%. One pathologist's visually estimated scores were compared to the adjusted-score, and the pathologist's scores were altered in 87% of cases. Comparison with the ground truth revealed that the adjusted-score was more precise than the pathologists' scores (P < 0.05).We proposed a collaborative workflow between AI and pathologists as a model to improve daily practice and enhance the prediction of tumour cellularity for genetic tests.
Best fit of mixture for multi-sensor poisson multi-Bernoulli mixture filtering

We propose a computationally efficient, the first so far, multi-sensor extension of the Poisson multi-Bernoulli mixture (PMBM) filter that accommodates both centralized and distributed sensor networks. In the distributed case, a distributed flooding algorithm is needed for internode communication, which iteratively shares the relevant multi-target posterior among neighbor sensors, thereby making each local sensor serve like a fusion center in the centralized case. The PMBM posterior yielded at each sensor is decomposed into two components corresponding to undetected and detected targets which are modelled by Poisson and multi-Bernoulli mixture (MBM) distributions, respectively. Both communication and fusion are performed with regard to the latter only. A “best-fit-of-mixture” fusion principle is adopted at each sensor to find an MBM that best fits the mixture of MBMs from distinct sensors, which results precisely in the arithmetic average (AA) of these MBMs. The information divergence of the AA from the true density is analyzed. We also provide a Bayesian model averaging interpretation of the MBM-AA fusion. Simulations in scenarios of different target detection probabilities demonstrate the performance of the proposed PMBM filter in terms of localization error, false-alarm/misdetection errors, and communication and computation costs, in comparison with the AA-based multisensor multi-Bernoulli filter.