Reviewerly/exHarmony huggingface dataset:
GACF: Ground-Aerial Collaborative Framework for Large-Scale Emergency Rescue Scenarios

Large-scale emergency rescue tasks require search and rescue units to conduct autonomous exploration and to be able to respond quickly in complex and dangerous environments. This paper proposes a ground-aerial collaborative framework for search and rescue scenarios. The heterogeneous collaborative system is composed of a single unmanned aerial vehicle(UAV) and a single unmanned ground vehicle(UGV). The UAV is responsible for providing a global topological map of the airspace perspective and performing path planning, and the UGV is responsible for local obstacle avoidance. Collaborative exploration has improved the autonomous exploration capabilities of unmanned systems in large-scale emergency rescue scenarios. This paper proposes a road network extraction and topology construction algorithm based on a airspace perspective image, providing a global traversable map. A path planning algorithm based on the global topological map of airspace is proposed, which improves the efficiency of UGV's exploratory progress. In the real scene experiment, compared with the single UGV with a priori map, the time consumption of GACF is reduced by 59%, which greatly improves the exploration efficiency of the unmanned system in practical applications.
Dependency-Aware Resource Allocation for Serverless Functions at the Edge

Serverless computing allows developers to break their code into small components, known as functions. Being lightweight and modular, serverless functions have been increasingly employed in edge computing, where quick responses and adaptability are key to meeting strict latency requirements. In particular, edge nodes are intrinsically resource-constrained, and efficient resource allocation strategies are crucial for optimizing their usage. Different approaches exist in the literature, but they often overlook the dependencies among functions, that is, how and when functions invoke other functions, and obtain suboptimal results. This paper presents NEPTUNE $$^{+}$$ , a dependency-aware resource (CPU cores) allocation solution for serverless functions deployed on edge infrastructures. The approach extends NEPTUNE, an existing framework for managing edge infrastructures, with a new theoretical model and control algorithm that take function dependencies into account. We evaluated NEPTUNE $$^{+}$$ by using three applications and it is able to allocate up to 42% fewer cores compared to NEPTUNE.
FL2DP: Privacy-Preserving Federated Learning Via Differential Privacy for Artificial IoT

Federated learning (FL) is a promising paradigm for collaboratively training networks on distributed clients while retaining data locally. Recent work has shown that personal data can be recovered even though clients only send gradients to the server. To against the gradient leakage issue, differential privacy (DP)-based solutions are proposed to protect data privacy by adding noise to the gradient before sending it to the server. However, the introduced noise affects the training efficiency of local clients, resulting in low model accuracy. Moreover, the identity privacy of clients has not been seriously considered in FL. In this article, we propose FL2DP, a privacy-preserving scheme focusing on protecting the data privacy as well as the identity privacy of clients. Different from the current schemes that add noise sampled from the Gaussian or Laplace distribution, in our scheme the noise is added to the gradient based on the exponential mechanism to achieve high training efficiency. Then, clients upload the perturbed gradients to a shuffler, which reassigns these gradients with different identities. We give a formal privacy definition called gradient indistinguishability to provide strict unlinkability for gradients shuffle. We propose a new gradient shuffling mechanism by adapting the DP-based exponential mechanism to satisfy gradient indistinguishability using the designed utility function. In this case, an attacker cannot infer the real identity of the client via the shuffled gradient. We conduct extensive experiments on two real-world datasets, and the results demonstrate the effectiveness of the proposed scheme.
I-MSV 2022: Indic-Multilingual and Multi-sensor Speaker Verification Challenge

Speaker Verification (SV) is a task to verify the claimed identity of the claimant using his/her voice sample. Though there exists an ample amount of research in SV technologies, the development concerning a multilingual conversation is limited. In a country like India, almost all the speakers are polyglot in nature. Consequently, the development of a Multilingual SV (MSV) system on the data collected in the Indian scenario is more challenging. With this motivation, the Indic-Multilingual Speaker Verification (I-MSV) Challenge 2022 was designed to understand and compare the state-of-the-art SV techniques. An overview of the challenge and its outcomes is given here. For the challenge, approximately 100 h of data spoken by 100 speakers were collected using 5 different sensors in 13 Indian languages. The data is divided into development, training, and testing sets and has been made publicly available for further research. The goal of this challenge is to make the SV system robust to language and sensor variations between enrollment and testing. In the challenge, participants were asked to develop the SV system in two scenarios, viz. constrained and unconstrained. The best system in the constrained and unconstrained scenario achieved a performance of $$2.12\%$$ and $$0.26\%$$ in terms of Equal Error Rate (EER), respectively.
Does augmented reality help in industrial training? A comprehensive evaluation based on natural human behavior and knowledge retention

The complexity and diversity of products significantly increases the difficulty of training workers in modern industries. AR training, as a combination of industrial digital development and new interactive means, has attracted increasingly more attention. However, in human factor and education, research regarding AR based training is still in its infancy. To understand how AR could help industrial training from a human perspective, we evaluate a HoloLens-based AR training method for an assembly task on natural human behavior and the training effectiveness. Forty-four participants were equally divided into AR and ordinary training groups with the equivalent instructional materials, which included text information and 3D models. Results show that although AR did not significantly affect the overall workload and system usability assessment (SUS) recorded, several experimental values increased significantly compared with the ordinary training method, including task completion time, time required to interpret text-based information, amount of attention switching, and training effectiveness. Consequently, the use of AR did not result in a completion time and workload reduction; however, we found that training with the AR system was more effective than ordinary training in increasing the knowledge retention of participants without increasing their overall workload, especially for the month-term memory. Nonetheless, a longer time was required by the users to familiarize themselves with the AR environment and read the text information. The findings of this work could contribute to the design of future AR training systems in industry considering natural human behavior and enhance training effectiveness.
The Characteristics of Early Childhood Education in Spain

Spain is a key member of the European Union, which stands out with its high enrollment rates and comprehensive early childhood education system among EU countries. Early childhood education (Educación Infantil, 0-6 years old) in Spain is a non-compulsory stage in the country’s education system. Over 95% of Spanish children start school at the age of 3, and the enrollment rate of children aged 3 to 5 is close to 100%. The study explores the unique features of early childhood education in Spain, emphasizing its internationalized educational framework, comprehensive educational objectives and content, relatively well-developed welfare system, as well as the emphasis on children’s foreign language proficiency and digital literacy, provide valuable insights for the development of early childhood education in other countries.
Reward Shaping for Happier Autonomous Cyber Security Agents

As machine learning models become more capable, they have exhibited increased potential in solving complex tasks. One of the most promising directions uses deep reinforcement learning to train autonomous agents in computer network defense tasks. This work studies the impact of the reward signal that is provided to the agents when training for this task. Due to the nature of cybersecurity tasks, the reward signal is typically 1) in the form of penalties (e.g., when a compromise occurs), and 2) distributed sparsely across each defense episode. Such reward characteristics are atypical of classic reinforcement learning tasks where the agent is regularly rewarded for progress (cf. to getting occasionally penalized for failures). We investigate reward shaping techniques that could bridge this gap so as to enable agents to train more sample-efficiently and potentially converge to a better performance. We first show that deep reinforcement learning algorithms are sensitive to the magnitude of the penalties and their relative size. Then, we combine penalties with positive external rewards and study their effect compared to penalty-only training. Finally, we evaluate intrinsic curiosity as an internal positive reward mechanism and discuss why it might not be as advantageous for high-level network monitoring tasks.
Broken Promises: Measuring Confounding Effects in Learning-based Vulnerability Discovery

Several learning-based vulnerability detection methods have been proposed to assist developers during the secure software development life-cycle. In particular, recent learning-based large transformer networks have shown remarkably high performance in various vulnerability detection and localization benchmarks. However, these models have also been shown to have difficulties accurately locating the root cause of flaws and generalizing to out-of-distribution samples. In this work, we investigate this problem and identify spurious correlations as the main obstacle to transferability and generalization, resulting in performance losses of up to 30% for current models. We propose a method to measure the impact of these spurious correlations on learning models and estimate their true, unbiased performance. We present several strategies to counteract the underlying confounding bias, but ultimately our work highlights the limitations of evaluations in the laboratory for complex learning tasks such as vulnerability discovery.
Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition

Over the past decade, the machine learning security community has developed a myriad of defenses for evasion attacks. An understudied question in that community is: for whom do these defenses defend? This work considers common approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations. We outline appropriate parity metrics for analysis and begin to answer this question through empirical results of the fairness implications of machine learning security methods. We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training. The framework we propose for measuring defense equality can be applied to robustly trained models, preprocessing-based defenses, and rejection methods. We identify a set of datasets with a user-centered application and a reasonable computational cost suitable for case studies in measuring the equality of defenses. In our case study of speech command recognition, we show how such adversarial training and augmentation have non-equal but complex protections for social subgroups across gender, accent, and age in relation to user coverage. We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups. This represents the first work examining the disparity in the adversarial robustness in the speech domain and the fairness evaluation of rejection-based defenses.
Application of chatbot AI in the creation of web mining programs and their analysis

Artificial intelligence and ChatGPT became very popular in late November 2022. Their popularity has led to the development of other chat AI systems and AI tools. In order to monitor this progress and achieve key goals in the application of artificial intelligence, this paper has the task of presenting an analysis of the generated web mining program. One of the key aspects of this research is the presentation of advantages and disadvantages of this approach, including analysis of accuracy, speed and reliability. Also, potential challenges in the application of artificial intelligence (especially chat systems), such as the limitations of AI models, ethical and security aspects when processing data from the Internet, are explored. This research paper aims to contribute to a better understanding of the capabilities of AI in the development of web mining programs, identify potential areas of improvement and provide guidance for future development.
Probabilistic modelling of substorm occurrences with an echo state network

Abstract. The relationship between solar-wind conditions and substorm activity is modelled with an approach based on an echo state network. Substorms are a fundamental physical phenomenon in the magnetosphere–ionosphere system, but the deterministic prediction of substorm onset is very difficult because the physical processes that underlie substorm occurrences are complex. To model the relationship between substorm activity and solar-wind conditions, we treat substorm onset as a stochastic phenomenon and represent the stochastic occurrences of substorms with a non-stationary Poisson process. The occurrence rate of substorms is then described with an echo state network model. We apply this approach to two kinds of substorm onset proxies. One is a sequence of substorm onsets identified from auroral electrojet intensity, and the other is onset events identified from activity of Pi2 pulsations, which are irregular geomagnetic oscillations often associated with substorm onsets. We then analyse the response of substorm activity to solar-wind conditions by feeding synthetic solar-wind data into the echo state network. The results indicate that the effect of the solar-wind speed is important, especially for Pi2 substorms. A Pi2 pulsation can often occur even if the interplanetary magnetic field (IMF) is northward, while the activity of auroral electrojets is depressed during northward IMF conditions. We also observe spiky enhancements in the occurrence rate of substorms when the solar-wind density abruptly increases, which might suggest an external triggering due to a sudden impulse of solar-wind dynamic pressure. It seems that northward turning of the IMF also contributes to substorm occurrences, though the effect is likely to be minor.
Underwater Object Detection using Image Enhancement and Deep Learning Models

Autonomous underwater vehicles (AUVs) are efficient robotic tools, offering a wide range of applications in ocean exploration and research, such as oceanographic mapping, environmental monitoring, and archaeology. Incorporating an automatic object detection system with AUVs can substantially improve their ability to perceive and recognize objects in a complicated and often hazardous environment. Currently, detecting underwater objects relied on a man-in-the-loop approach, where AUVs captured vast amounts of data and saved them in memory for offline processing. This study investigates the use of deep learning for automatic image preprocessing and object detection, evaluating and comparing three state-of-the-art YOLO (You Only Look Once) models, including YOLOv8, YOLOv7, and YOLOv5. Extensive experiments were conducted using publicly available underwater image datasets, revealing that the pre-trained models attain superior performance on the Brackish dataset. YOLOv5 and YOLOv8 achieved the highest mean average precision (mAP) with a score of 99%, while YOLOv7 scored 89%. Furthermore, an underwater image enhancement algorithm is employed on the URPC2021 dataset, significantly improving the detection accuracy with a 3% increase in mAP across all three models. In terms of inference speed, YOLOv5 demonstrated the highest frames per second (FPS), while maintaining comparable performance in mAP and recall.
MAiVAR-T: Multimodal Audio-image and Video Action Recognizer using Transformers

In line with the human capacity to perceive the world by simultaneously processing and integrating high-dimensional inputs from multiple modalities like vision and audio, we propose a novel model, MAiVAR-T (Multimodal Audio-Image to Video Action Recognition Transformer). This model employs an intuitive approach for the combination of audio-image and video modalities, with a primary aim to escalate the effectiveness of multimodal human action recognition (MHAR). At the core of MAiVAR-T lies the significance of distilling substantial representations from the audio modality and transmuting these into the image domain. Subsequently, this audio-image depiction is fused with the video modality to formulate a unified representation. This concerted approach strives to exploit the contextual richness inherent in both audio and video modalities, thereby promoting action recognition. In contrast to existing state-of-the-art strategies that focus solely on audio or video modalities, MAiVAR-T demonstrates superior performance. Our extensive empirical evaluations conducted on a benchmark action recognition dataset corroborate the model’s remarkable performance. This underscores the potential enhancements derived from integrating audio and video modalities for action recognition purposes. To ensure transparency and reproducibility of our work, the source code is made publicly available at https://bit.ly/43do8DH.
Re-visited Privacy-Preserving Machine Learning

Local differential privacy is a quantitative privacy metric based on indistinguishability, and the increasing complexity of recent attacks on privacy information has fostered its adoption. WALDP is a recently proposed local differential privacy mechanism, which reduces the amount of noise using dimension reduction. We focus on this mechanism and propose another dimension reduction method DR.OR using odds ratios. It is possible to maintain the original concept, i.e., data generality, by using DR.OR as a subroutine of WALDP. Previous study focused only on support vector machines (SVMs), but we also evaluated WALDP for the logistic regression and the deep neural network, and showed that WALDP achieves high accuracy as well. In particular, the proposed DR.OR boasts high accuracy, especially for logistic regression, which has a strong relationship with odds ratios. This suggests that further performance improvements can be expected by selecting a dimension reduction method in consideration of the learning model.
Supervised Robustness-preserving Data-free Neural Network Pruning

When deploying pre-trained neural network models in real-world applications, model consumers often encounter resource-constraint platforms such as mobile and smart devices. They typically use the pruning technique to reduce the size and complexity of the model, generating a lighter one with less resource consumption. Nonetheless, most existing pruning methods are proposed with a premise that the model after being pruned has a chance to be fine-tuned or even retrained based on the original training data. This may be unrealistic in practice, as the data controllers are often reluctant to provide their model consumers with the original data. In this work, we study the neural network pruning in the data-free context, aiming to yield lightweight models that are not only accurate in prediction but also robust against undesired inputs in open-world deployments. Considering the absence of fine-tuning and retraining that can fix the mis-pruned units, we replace the traditional aggressive one-shot strategy with a conservative one that treats model pruning as a progressive process. We propose a pruning method based on stochastic optimization that uses robustness-related metrics to guide the pruning process. Our method is evaluated with a series of experiments on diverse neural network models. The experimental results show that it significantly outperforms existing one-shot data-free pruning approaches in terms of robustness preservation and accuracy.
Quantum teleportation and dynamics of quantum coherence and metrological non-classical correlations for open two-qubit systems

We investigate the dynamics of non-classical correlations and quantum coherence in open quantum systems by employing metrics like local quantum Fisher information, local quantum uncertainty, and quantum Jensen-Shannon divergence. Our focus here is on a system of two qubits in two distinct physical situations: the first one when the two qubits are coupled to a cavity field whether the system is closed or open, while the second consists of two qubits immersed in dephasing reservoirs. Our study places significant emphasis on how the evolution of these quantum criterion is influenced by the initial state's purity (whether pure or mixed) and the nature of the environment (whether Markovian or non-Markovian). We observe that a decrease in the initial state's purity corresponds to a reduction in both quantum correlations and quantum coherence, whereas higher purity enhances these quantumness. Furthermore, we establish a quantum teleportation strategy based on the two different physical scenarios. In this approach, the resulting state of the two qubits functions as a quantum channel integrated into a quantum teleportation protocol. We also analyze how the purity of the initial state and the Markovian or non-Markovian regimes impact the quantum teleportation process.
Exploring deep learning approaches for video captioning: A comprehensive review

While humans can easily describe visual data at varying levels of detail, the same task presents a significant challenge for machines. This challenge becomes even more complex when dealing with video data. The process of understanding a video and generating descriptive text for it is known as video captioning. Video captioning requires not only understanding the visual content but also producing human-like descriptions that accurately capture its semantics. Achieving this level of understanding requires the collaborative efforts of both the computer vision and natural language processing research communities. The captions produced through video captioning serve as valuable resources that can be further leveraged for various applications such as video search, accessibility for visually impaired people, and human-robot interaction. Deep learning strategies have emerged as powerful tools in addressing the complexities of video captioning. By leveraging large scale annotated video caption datasets and sophisticated neural network architectures, deep learning approaches have made significant advances in this challenging task. In the existing literature, numerous techniques, benchmark datasets, and evaluation metrics have been developed, emphasizing the necessity for a comprehensive examination to concentrate research efforts in this rapidly evolving field. This paper provides a survey of deep learning based methods for video captioning, highlighting their key components, challenges, and recent advancements.