Reviewerly/exHarmony huggingface dataset:
Transformer-based local-global guidance for image captioning

Image captioning is a difficult problem for machine learning algorithms to compress huge amounts of images into descriptive languages. The recurrent models are popularly used as the decoder to extract the caption with significant performance, while these models have complicated and inherently sequential overtime issues. Recently, transformers provide modeling long dependencies and support parallel processing of sequences compared to recurrent models. However, recent transformer-based models assign attention weights to all candidate vectors based on the assumption that all vectors are relevant and ignore the intra-object relationships. Besides, the complex relationships between key and query vectors cannot be provided using a single attention mechanism. In this paper, a new transformer-based image captioning structure without recurrence and convolution is proposed to address these issues. To this end, a generator network and a selector network to generate textual descriptions collaboratively are designed. Our work contains three main steps: (1) Design a transformer-based generator network as word-level guidance to generate next words based on the current state. (2) Train a latent space to learn the mapping of captions and images into the same embedding space to learn the text-image relation. (3) Design a selector network as sentence-level guidance to evaluate next words by assigning fitness scores to the partial captions through the embedding space. Compared with the architecture of existing methods, the proposed approach contains an attention mechanism without the dependencies of time. It executes each state to select the next best word using local–global guidance. In addition, the proposed model maintains dependencies between the sequences, and can be trained in parallel. Several experiments on the COCO and Flickr datasets demonstrate that the proposed approach can outperform various state-of-the-art models over well-known evaluation measures.
Dynamic graph convolutional networks by semi-supervised contrastive learning

The traditional graph convolutional network(GCN) and its variants usually only propagate node information through the topology given by the dataset. However, the given topology can only represent a certain relationship and ignore some correlative feature information between nodes, which may make the graph convolutional networks unable to fully utilize the data information. To address the above issue, a novel model named Dynamic Graph Convolutional Networks by Semi-Supervised Contrastive Learning (DGSCL) is proposed in this paper. First, a feature graph is dynamically constructed from the input node features to exploit the potential correlative feature information between nodes. Then, to ensure a high-quality feature graph, a semi-supervised contrastive learning method is designed to learn discriminative node embeddings, which can iteratively refine the constructed feature graph with the learned node embeddings. Finally, we fuse the node embeddings obtained from the given topology and the dynamic feature graph by two co-attention modules to produce more informative embeddings for the classification task. Through a series of experiments, we demonstrate the competitive performance of our model on seven node classification benchmarks.
Optimization of microfluidic synthesis of silver nanoparticles: A generic approach using machine learning

The properties of silver nanoparticles (AgNPs) are affected by various parameters, making optimisation of their synthesis a laborious task. This optimisation is facilitated in this work by concurrent use of a T-junction microfluidic system and machine learning approach. The AgNPs are synthesized by reducing silver nitrate with tannic acid in the presence of trisodium citrate, which has a dual role in the reaction as reducing and stabilizing agent. The study uses a decision tree-guided design of experiment method for the size of AgNPs. The developed approach uses kinetic nucleation and growth constants derived from an independent set of experiments to account for chemistry of synthesis, the Reynolds number and the ratio of Dean number to Reynolds number to reveal effect of hydrodynamics and mixing within device and storage temperature to account for particle stability after collection. The obtained model was used to define a parameter space for additional experiments carried out to improve the model further. The numerical results illustrate that well-designed experiments can contribute more effectively to the development of different machine learning models (decision tree, random forest and XGBoost) as opposed to randomly added experiments.
A Novel Hybrid Deep Learning Model for Detecting and Classifying Non-Functional Requirements of Mobile Apps Issues

As a result of the speed and availability of the Internet, mobile devices and apps are in widespread usage throughout the world. Thus, they can be seen in the hands of nearly every person, helping us in our daily activities to accomplish many tasks with less effort and without wasting time. However, many issues occur while using mobile apps, which can be considered as issues of functional or non-functional requirements (NFRs). Users can add their comments as a review on the mobile app stores that provide for technical feedback, which can be used to improve the software quality and features of the mobile apps. Minimum attention has been given to such comments by scholars in addressing, detecting, and classifying issues related to NFRs, which are still considered challenging. The purpose of this paper is to propose a hybrid deep learning model to detect and classify NFRs (according to usability, reliability, performance, and supportability) of mobile apps using natural language processing methods. The hybrid model combines three deep learning (DL) architectures: a recurrent neural network (RNN) and two long short-term memory (LSTM) models. It starts with a dataset construction extracted from the user textual reviews that contain significant information in the Arabic language. Several experiments were conducted using machine learning classifiers (MCLs) and DL, such as ANN, LSTM, and bidirectional LSTM architecture to measure the performance of the proposed hybrid deep learning model. The experimental results show that the performance of the proposed hybrid deep learning model outperforms all other models in terms of the F1 score measure, which reached 96%. This model helps mobile developers improve the quality of their apps to meet user satisfaction and expectations by detecting and classifying issues relating to NFRs.
Language Models for Human-Robot Interaction

Recent advances in large scale language models have significantly changed the landscape of automatic dialogue systems and chatbots. We believe that these models also have a great potential for changing the way we interact with robots. Here, we present the first integration of the OpenAI GPT-3 language model for the Aldebaran Pepper and Nao robots. The present work transforms the text-based API of GPT-3 into an open verbal dialogue with the robots. The system will be presented live during the HRI2023 conference and the source code of this integration is shared with the hope that it will serve the community in designing and evaluating new dialogue systems for robots.
Goal distance-based UAV path planning approach, path optimization and learning-based path estimation: GDRRT*, PSO-GDRRT* and BiLSTM-PSO-GDRRT*

The basic conditions for mobile robots to be autonomous are that the mobile robot localizes itself in the environment and knows the geometric structure of the environment (map). After these conditions are met, this mobile robot is given a specific task, but how the robot will navigate for this task is an important issue. Especially for Unmanned Aerial Vehicles (UAV), whose application has increased recently, path planning in a three-dimensional (3D) environment is a common problem. This study performs three experimental applications to discover the most suitable path for UAV in 3D environments with large and many obstacles. Inspired by Rapidly Random-Exploring Tree Star (RRT*), the first implementation develops the Goal Distance-based RRT* (GDRRT*) approach, which performs intelligent sampling taking into account the goal distance. In the second implementation, the path discovered by GDRRT* is shortened using Particle Swarm Optimization (PSO) (PSO-GDRRT*). In the final application, a network with a Bidirectional Long/Short Term Memory (BiLSTM) layer is designed for fast estimation of optimal paths found by PSO-GDRRT* (BiLSTM-PSO-GDRRT*). As a result of these applications, this study provides important novelties: GDRRT* converges to the goal faster than RRT* in large and obstacle-containing 3D environments. To generate groundtruth paths for training the learning-based network, PSO-GDRRT* finds the shortest paths relatively quickly. Finally, BiLSTM-PSO-GDRRT* provides extremely fast path planning for real-time UAV applications. This work is valuable for real-time autonomous UAV applications in a complex and large environment, as the new methods it offers have fast path planning capability.
An Extended AI-Experience: Industry 5.0 in Creative Product Innovation

Creativity plays a significant role in competitive product ideation. With the increasing emergence of Virtual Reality (VR) and Artificial Intelligence (AI) technologies, the link between such technologies and product ideation is explored in this research to assist and augment creative scenarios in the engineering field. A bibliographic analysis is performed to review relevant fields and their relationships. This is followed by a review of current challenges in group ideation and state-of-the-art technologies with the aim of addressing them in this study. This knowledge is applied to the transformation of current ideation scenarios into a virtual environment using AI. The aim is to augment designers' creative experiences, a core value of Industry 5.0 that focuses on human-centricity, social and ecological benefits. For the first time, this research reclaims brainstorming as a challenging and inspiring activity where participants are fully engaged through a combination of AI and VR technologies. This activity is enhanced through three key areas: facilitation, stimulation, and immersion. These areas are integrated through intelligent team moderation, enhanced communication techniques, and access to multi-sensory stimuli during the collaborative creative process, therefore providing a platform for future research into Industry 5.0 and smart product development.
DFGNN: An interpretable and generalized graph neural network for deepfakes detection

Deepfakes are generated using sophisticated deep-learning models to create fake images or videos. As the techniques for creating deepfakes improve, issues like defamation, impersonation, fraud, and misinformation on social media are becoming more prevalent. Existing deep learning-based deepfakes detection models are not interpretable and don't generalize well when tested across diverse deepfakes generating techniques and datasets. Therefore, the creation of reliable and effective deepfakes detection algorithms is required which are not only generalizable but also interpretable. This paper introduces a novel graph neural network-based architecture to identify hyper-realist deepfake content. Currently, very limited efforts have been done to address the problem of deepfakes detection using graph neural networks. The proposed model is based on the pyramid structure that takes advantage of multi-scale images property by extracting features with progressively smaller spatial sizes as layer depth increases. The method first sliced the image into patches, which are referred to as nodes, and then constructed a graph by connecting the nearest neighbors. To transform and exchange information between all nodes, the proposed model has two basic modules: GraphNet, which uses graph convolution layers to aggregate and update graph information, and FFN, which has linear layers for the transformation of node features. The effectiveness of the method is assessed using the diverse Deepfake Detection Challenge dataset (DFDC), FaceForensics++ (FF++), World Leaders dataset (WLRD), and the Celeb-DF. To demonstrate the generalizability of the proposed method for accurate deepfakes detection, open/close set, cross-set, and cross-corpora evaluations were also performed. The AUC values of 0.98 on FF++, 0.95 on Celeb-DF, 0.92 on DFDC, and 1.00 on most of the sets of WLRD datasets demonstrate the efficacy of the method for identifying manipulated facial images produced using various deepfakes techniques.
SiamST: Siamese network with spatio-temporal awareness for object tracking

Recently, Siamese trackers have achieved remarkable tracking performance. However, challenges such as accurate feature representation of targets with different spatial regions and the utilization of diverse target temporal states still need to be addressed. Here, we proposed a Siamese network with spatio-temporal awareness called SiamST. The standard square convolutional kernels and the single feature matching operation hardly represent the targets with different shapes accurately. Therefore, we designed a refined region fusion module that combines multiple convolutional kernels to fit targets with different aspect ratios. Furthermore, we proposed a multi-granularity matching module to obtain more robust feature matching results by combining fine-grained and coarse-grained matching results. However, most existing Siamese trackers do not adequately employ target temporal states. They usually only update the templates, which automatically causes motion information loss. Therefore, we built dynamic templates by screening high-quality samples to describe the target appearance changes accurately. In addition, we designed a trend guidance module to adjust the location prior constraint appropriately to match the tracking results to the target's motion trajectory. Extensive experimental results on eight tracking benchmarks demonstrate the competitive performance of SiamST compared to many advanced trackers.
Intrinsic decoherence effects on correlated coherence and quantum discord in XXZ Heisenberg model

Spin qubits are at the heart of technological advances in quantum processors and offer an excellent framework for quantum information processing. This work characterizes the time evolution of coherence and nonclassical correlations in a two-spin XXZ Heisenberg model, from which a two-qubit system is realized. We study the effects of intrinsic decoherence on coherence (correlated coherence) and nonclassical correlations (quantum discord), taking into consideration the combined impact of an external magnetic field, Dzyaloshinsky-Moriya (DM) and Kaplan Shekhtman Entin-Wohlman-Aharony (KSEA) interactions. To fully understand the effects of intrinsic decoherence, we suppose that the system can be prepared in one of the two well-known extended Werner-like (EWL) states. The findings show that intrinsic decoherence leads the coherence and quantum correlations to decay and that the behavior of the aforementioned quantum resources relies strongly on the initial EWL state parameters. We, likewise, found that the two-spin correlated coherence and quantum discord; become more robust against intrinsic decoherence depending on the type of the initial state. These outcomes shed light on how a quantum system should be engineered to achieve quantum advantages.
Flatness-Based Quadcopter Trajectory Planning and Tracking With Continuous-Time Safety Guarantees

This work proposes a convex optimization-based framework for the trajectory planning and tracking of quadcopters that ensures continuous-time safety guarantees. Using the convexity property of B-spline curves and the differential flatness property of quadcopters, a second-order cone program is formulated to generate an optimal nominal trajectory that respects state and input constraints, including position, linear velocity, angle, angular velocity, thrust, waypoints, and obstacle avoidance constraints, rigorously in the continuous-time sense. To ensure safe trajectory tracking, a convex quadratic program is proposed based on control barrier functions, which guarantees that the actual trajectory of the quadcopter remains within a prescribed safe tube of the nominal trajectory in continuous time. Furthermore, conditions that ensure the safe tracking controller respects thrust, roll, and pitch constraints are also presented. Both the planning and control approaches are suitable for online implementation, and the effectiveness of the proposed framework is demonstrated through simulations and experiments with a Crazyflie2.1 nano quadcopter.
Bayesian learning for neural networks: an algorithmic survey

Abstract The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of the topic and the multitude of ingredients involved therein, besides the complexity of turning theory into practical implementations, limit the use of the Bayesian learning paradigm, preventing its widespread adoption across different fields and applications. This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective. Upon providing a general introduction to Bayesian Neural Networks, we discuss and present both standard and recent approaches for Bayesian inference, with an emphasis on solutions relying on Variational Inference and the use of Natural gradients. We also discuss the use of manifold optimization as a state-of-the-art approach to Bayesian learning. We examine the characteristic properties of all the discussed methods, and provide pseudo-codes for their implementation, paying attention to practical aspects, such as the computation of the gradients.
Face mask detection using deep convolutional neural network and multi-stage image processing

Face mask detection has several applications including real-time surveillance, biometrics, etc. Face mask detection is also useful for surveillance of the public to ensure face mask wearing in public places. Ensuring that people are wearing a face mask is not possible with monitoring staff; instead, automatic systems are a much better choice for face mask detection and monitoring to help manage public behaviour and contribute to restricting the outbreak of COVID-19. Despite the availability of several such systems, the lack of a real image dataset is a big hurdle to validating state-of-the-art face mask detection systems. In addition, using the simulated datasets lack the analysis needed for real-world scenarios. This study builds a new dataset namely RILFD by taking real pictures using a camera and annotating them with two labels (with mask, without mask) which are publicly available for future research. In addition, this study investigates various machine learning models and off-the-shelf deep learning models YOLOv3 and Faster R-CNN for the detection of face masks. The customized CNN models in combination with the 4 steps of image processing are proposed for face mask detection. The proposed approach outperforms other models and proved its robustness with a 97.5% of accuracy score in face mask detection on the RILFD dataset and two publicly available datasets (MAFA and MOXA).
Security of Cryptocurrencies: A View on the State-of-the-Art Research and Current Developments

[Context] The goal of security is to protect digital assets, devices, and services from being disrupted, exploited or stolen by unauthorized users. It is also about having reliable information available at the right time. [Motivation] Since the inception in 2009 of the first cryptocurrency, few studies have been undertaken to analyze and review the state-of-the-art research and current developments with respect to the security of cryptocurrencies. [Purpose] We aim to provide both theoretical and empirical insights into the security landscape, in particular focusing on both technical solutions and human-related facets. [Methodology] We used an integrative review which could help in building science and scholarly research, the basis for conceptual and empirical models. [Results] Successful defense against cyberattacks depends on technical measures on the one hand, as well as on self-education and training with the aim to develop competence, knowledge, skills and social abilities, on the other. [Contribution] Our findings provide a comprehensive review for the major achievements and developments of the recent progress on the security of cryptocurrencies. [Future research] Since there is increasing interest in adoption of the current solutions within the central bank digital currencies, the future research should explore the development and inception of effective measures against social engineering attacks, which still remain the main concern.
Using mixed reality (XR) immersive learning to enhance environmental education

Mixed reality (XR) environments combining real-to-virtual immersive experiences provide unprecedented potential for reframing educational pedagogy and practice. XR environments provide scaffolded learning points accommodating individual needs, while enhancing sensorial and embodied experiences. XR environments can facilitate self-determined (heutagogical) experience-based learning and esthetic visualization of wicked problems, making complex knowledge more accessible. Here we report on a study exploring the design of mobile learning with education outside the classroom (EOTC), heutagogy, and free-choice learning to enhance marine ecological literacy, based on a bring your own device (BYOD) XR intervention at a marine educational center in Aotearoa New Zealand. Findings indicate that XR affordances can enhance the understanding of complex marine conservation science, facilitating ecological literacy knowledge, attitudes and behavior change. Implications include pedagogical rethinking of EOTC with self-determined mobile learning; haptic, sensorial and embodied XR design considerations for environmental education; and epistemological speculation on learning phenomena in real-to-virtual immersive environments.