Reviewerly/exHarmony huggingface dataset:
Multiagent Reinforcement Learning: Rollout and Policy Iteration

We discuss the solution of complex multistage decision problems using methods that are based on the idea of policy iteration (PI), i.e., start from some base policy and generate an improved policy. Rollout is the simplest method of this type, where just one improved policy is generated. We can view PI as repeated application of rollout, where the rollout policy at each iteration serves as the base policy for the next iteration. In contrast with PI, rollout has a robustness property: it can be applied on-line and is suitable for on-line replanning. Moreover, rollout can use as base policy one of the policies produced by PI, thereby improving on that policy. This is the type of scheme underlying the prominently successful AlphaZero chess program. In this paper we focus on rollout and PI-like methods for problems where the control consists of multiple components each selected (conceptually) by a separate agent. This is the class of multiagent problems where the agents have a shared objective function, and a shared and perfect state information. Based on a problem reformulation that trades off control space complexity with state space complexity, we develop an approach, whereby at every stage, the agents sequentially (one-at-a-time) execute a local rollout algorithm that uses a base policy, together with some coordinating information from the other agents. The amount of total computation required at every stage grows linearly with the number of agents. By contrast, in the standard rollout algorithm, the amount of total computation grows exponentially with the number of agents. Despite the dramatic reduction in required computation, we show that our multiagent rollout algorithm has the fundamental cost improvement property of standard rollout: it guarantees an improved performance relative to the base policy. We also discuss autonomous multiagent rollout schemes that allow the agents to make decisions autonomously through the use of precomputed signaling information, which is sufficient to maintain the cost improvement property, without any on-line coordination of control selection between the agents. For discounted and other infinite horizon problems, we also consider exact and approximate PI algorithms involving a new type of one-agent-at-a-time policy improvement operation. For one of our PI algorithms, we prove convergence to an agent-by-agent optimal policy, thus establishing a connection with the theory of teams. For another PI algorithm, which is executed over a more complex state space, we prove convergence to an optimal policy. Approximate forms of these algorithms are also given, based on the use of policy and value neural networks. These PI algorithms, in both their exact and their approximate form are strictly off-line methods, but they can be used to provide a base policy for use in an on-line multiagent rollout scheme.
Deep Learning for Security Problems in 5G Heterogeneous Networks

With increasingly complex network structure, requirements for heterogeneous 5G are also growing. The aim of this study is to meet the network security performance under the existing high-capacity and highly reliable transmission. In this context, deep learning technology is adopted to solve the security problem of the 5G heterogeneous network. First, the security problems existing in 5G heterogeneous networks are presented, mainly from two aspects of the physical layer security problems and application prospects of deep learning in communication technology. Then the combination of deep learning and 5G heterogeneous networks is analyzed. The combination of deep learning technology, modulation information recognition, and beam formation is introduced. The application of deep learning in communications technology is analyzed, and the modulation information recognition and beamforming based on deep learning are introduced. Finally, the challenges of solving security problems in 5G heterogeneous networks by deep learning are explored. The results show that the deep learning model can solve the modulation recognition problem well, and the modulation mode of the convolutional neural network can well identify the modulation signals involved in the experiment. Therefore, deep learning has a good advantage in solving modulation recognition. In addition, compared to the traditional algorithm, the unsupervised beamforming algorithm based on deep learning proposed in this research can effectively reduce the computational complexity under different numbers of transmitting antennas, which verifies the superiority of the unsupervised beamforming algorithm based on deep learning proposed in this research. Therefore, the present work provides a good idea for solving the security problem of 5G heterogeneous networks.
Explainable Active Learning (XAL)

The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term "machine teacher'' to refer to the role of people who inject domain knowledge into ML models. This "teaching'' perspective emphasizes supporting the productivity and mental wellbeing of machine teachers through efficient learning algorithms and thoughtful design of human-AI interfaces. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query a machine teacher for labels, so that the labeling workload could be largely reduced. However, in current AL settings, the human-AI interface remains minimal and opaque. A dearth of empirical studies further hinders us from developing teacher-friendly interfaces for AL algorithms. In this work, we begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the teacher ought to be able to understand the reasoning underlying its mistakes. When the model matures, the teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and additional cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and Need for Cognition. By reflecting on the results, we suggest future directions and design implications for XAL, and more broadly, machine teaching through AI explanations.
Light-Edge: A Lightweight Authentication Protocol for IoT Devices in an Edge-Cloud Environment

Due to the ever-growing use of active Internet devices, the Internet has achieved good popularity at present.The smart devices could connect to the Internet and communicate together that shape the Internet of Things (IoT).Such smart devices are generating data and are connecting to each other through edge-cloud infrastructure.Authentication of the IoT devices plays a critical role in the success of the integration of IoT, edge, and cloud computing technologies.The complexity and attack resistance of the authentication protocols are still the main challenges.Motivated by this, this paper introduces a lightweight authentication protocol for IoT devices named Light-Edge using a three-layer scheme, including IoT device layer, trust center at the edge layer, and cloud service providers.The results show the superiority of the proposed protocol against other approaches in terms of attack resistance, communication cost, and time cost.
Customized blockchain-based architecture for secure smart home for lightweight IoT

Safeguarding security and privacy remains a major challenge with regards to the Internet of Things (IoT) primarily due to the large scale and distribution of IoT networks. The information systems in Smart Homes are mainly based on sharing information through smart devices (IoT) and embedded sensors. Each sensor generates data to be processed or assembled by a central system. This data, while being transmitted over the internet to other users or servers, can be compromised for its privacy, user confidentiality and/or service availability. This paper proposes a novel Blockchain-based solution for secure smart home systems, using a combined hyperledger fabric and hyperledger composer. This solution is designed to overcome reported security limitations in commonly used permissioned blockchains approaches. The proposed architecture contains four layers: Cloud storage, Hyperledger fabric, Hyperledger composer, and a smart home layer. Another important aspect of the proposed solution is the mapping of the attributes of a smart home to those from the hyperledger composer. This mapping allows for a customized, designed-for-purpose solution which can meet the security requirements for IoT based smart homes. The proposed architecture was implemented and tested to improve the integrity, confidentiality, availability, authorization and privacy of smart homes as well as some inherited features, including transparency and interoperability.
Text Classification Based on Convolutional Neural Networks and Word Embedding for Low-Resource Languages: Tigrinya

This article studies convolutional neural networks for Tigrinya (also referred to as Tigrigna), which is a family of Semitic languages spoken in Eritrea and northern Ethiopia. Tigrinya is a “low-resource” language and is notable in terms of the absence of comprehensive and free data. Furthermore, it is characterized as one of the most semantically and syntactically complex languages in the world, similar to other Semitic languages. To the best of our knowledge, no previous research has been conducted on the state-of-the-art embedding technique that is shown here. We investigate which word representation methods perform better in terms of learning for single-label text classification problems, which are common when dealing with morphologically rich and complex languages. Manually annotated datasets are used here, where one contains 30,000 Tigrinya news texts from various sources with six categories of “sport”, “agriculture”, “politics”, “religion”, “education”, and “health” and one unannotated corpus that contains more than six million words. In this paper, we explore pretrained word embedding architectures using various convolutional neural networks (CNNs) to predict class labels. We construct a CNN with a continuous bag-of-words (CBOW) method, a CNN with a skip-gram method, and CNNs with and without word2vec and FastText to evaluate Tigrinya news articles. We also compare the CNN results with traditional machine learning models and evaluate the results in terms of the accuracy, precision, recall, and F1 scoring techniques. The CBOW CNN with word2vec achieves the best accuracy with 93.41%, significantly improving the accuracy for Tigrinya news classification.
A discrete bat algorithm based on Lévy flights for Euclidean traveling salesman problem

Bat algorithm is a swarm-intelligence-based metaheuristic proposed in 2010. This algorithm was inspired by echolocation behavior of bats when searching their prey in nature. Since it first introduction, it continues to be used extensively until today, owing to its simplicity, easy handling and applicability to a wide range of problems. However, sometimes the major challenge faced by this technique is can be trapped in a local optimum when facing large complex problems. In this research work, a new discrete bat algorithm is proposed to solve the famous traveling salesman problem as NP-hard combinatorial optimization problem. To enhance the searching strategy and to avoid getting stuck in local minima, random walks based on Lévy's flights are combined with bat’s movement. In addition, to improve the diversity and convergence of the swarm, a neutral crossover operator is embedded to the proposed algorithm. To evaluate the performance of our proposal, two experiments are conducted on 38 benchmark datasets and the obtained results are compared with eight different approaches. Furthermore, the student’s t-test, the Friedman’s test and the post hoc Wilcoxon's test are performed to check whether there are significant differences between the proposed optimizer and the alternative techniques. The experimental results under comparative studies have shown that, in most cases, the proposed discrete bat algorithm yields significantly better results compared with its competitors.
Solar Photovoltaic Forecasting of Power Output Using LSTM Networks

The penetration of renewable energies has increased during the last decades since it has become an effective solution to the world’s energy challenges. Among all renewable energy sources, photovoltaic (PV) technology is the most immediate way to convert solar radiation into electricity. Nevertheless, PV power output is affected by several factors, such as location, clouds, etc. As PV plants proliferate and represent significant contributors to grid electricity production, it becomes increasingly important to manage their inherent alterability. Therefore, solar PV forecasting is a pivotal factor to support reliable and cost-effective grid operation and control. In this paper, a stacked long short-term memory network, which is a significant component of the deep recurrent neural network, is considered for the prediction of PV power output for 1.5 h ahead. Historical data of PV power output from a PV plant in Nicosia, Cyprus, were used as input to the forecasting model. Once the model was defined and trained, the model performance was assessed qualitative (by graphical tools) and quantitative (by calculating the Root Mean Square Error (RMSE) and by applying the k-fold cross-validation method). The results showed that our model can predict well, since the RMSE gives a value of 0.11368, whereas when applying the k-fold cross-validation, the mean of the resulting RMSE values is 0.09394 with a standard deviation 0.01616.
Twitter for disaster relief through sentiment analysis for COVID-19 and natural hazard crises

In emergencies and disasters, large numbers of people require basic needs and medical attention. In such situations, online social media comes as a possible solution to aid the current disaster management methods. In this paper, supervised learning approaches are compared for the multi-class classification of Twitter data. A careful setting of Multilayer Perceptron (MLP) network layers and the optimizer has shown promising results for classification of tweets into three categories i.e. 'resource needs', 'resource availability', and 'others' being neutral and of no useful information. Public data of Nepal Earthquake (2015) and Italy Earthquake (2016) have been used for training and validation of the models, and original COVID-19 data is acquired, annotated, and used for testing. Detailed data analysis of tweets collected during different disasters has also been incorporated in the paper. The proposed model has been able to achieve 83% classification accuracy on the original COVID-19 dataset. Local Interpretable Model-Agnostic Explanations (LIME) is used to explain the behavior and shortcomings model on COVID-19 data. This paper provides a simple choice for real-world applications and a good starting point for future research.
Unmanned Aerial Vehicle Path Planning Algorithm Based on Deep Reinforcement Learning in Large-Scale and Dynamic Environments

Path planning is one of the key technologies for autonomous flight of Unmanned Aerial Vehicle. Traditional path planning algorithms have some limitations and deficiencies in the complex and dynamic environment. In this article, we propose a deep reinforcement learning approach for three-dimensional path planning by utilizing the local information and relative distance without global information. UAV can obtain the limited environmental information nearby in the actual scenario with limited sensor capabilities. Therefore, path planning can be formulated as a Partially Observable Markov Decision Process. The recurrent neural network with temporal memory is constructed to address the partial observability problem by extracting crucial information from historical state-action sequences. We develop an action selection strategy that combines the current reward value and the state-action value to reduce the meaningless exploration. In addition, we construct two sample memory pools and propose an adaptive experience replay mechanism based on the frequency of failure. The simulation experiment results show that our method has significant improvements over Deep Q-Network and Deep Recurrent Q-Network in terms of stability and learning efficiency. Our approach successfully plans a reasonable three-dimensional path in the large-scale and complex environment, and has the perfect ability to avoid obstacles.in the unknown environment.
Virtual reality in problem‐based learning contexts: Effects on the problem‐solving performance, vocabulary acquisition and motivation of English language learners

Abstract Learning a foreign language requires interaction with language input while involved in a task. Given that problem‐based learning (PBL) offers hands‐on application in realistic contexts, and that virtual reality (VR) enables learners to interact with multiple modalities of information, this study examines how the integration of VR technology into PBL contexts affects students' motivation for, problem‐solving during, and vocabulary acquisition in learning English as a foreign language (EFL). A total of 84 engineering majors who enrolled in a course of English for specific purposes were randomly assigned to either an experimental group or a control group. Students in the experimental group participated in a VR‐assisted PBL context, in which they were to view a PBL scenario using VR technology and then to create VR videos about solving the given problems. Those in the control group participated in a PBL context without the use of VR technology for viewing and solving the identical scenario. After the intervention, all the students wrote a problem‐solving analysis, took a vocabulary knowledge test, completed a learning motivation questionnaire, and participated in individual interviews. The results showed that the students in the experimental group significantly outperformed those in the control group in terms of vocabulary acquisition, and were more motivated to learn English related to their future careers, whereas there was no significant difference in the problem‐solving performance of the two groups. Implications of these findings highlight the value of engaging EFL learners in immersive environments for contextualized learning through the integrated use of VR and PBL.
Adaptive Channel Selection for Robust Visual Object Tracking with Discriminative Correlation Filters

Abstract Discriminative Correlation Filters (DCF) have been shown to achieve impressive performance in visual object tracking. However, existing DCF-based trackers rely heavily on learning regularised appearance models from invariant image feature representations. To further improve the performance of DCF in accuracy and provide a parsimonious model from the attribute perspective, we propose to gauge the relevance of multi-channel features for the purpose of channel selection. This is achieved by assessing the information conveyed by the features of each channel as a group, using an adaptive group elastic net inducing independent sparsity and temporal smoothness on the DCF solution. The robustness and stability of the learned appearance model are significantly enhanced by the proposed method as the process of channel selection performs implicit spatial regularisation. We use the augmented Lagrangian method to optimise the discriminative filters efficiently. The experimental results obtained on a number of well-known benchmarking datasets demonstrate the effectiveness and stability of the proposed method. A superior performance over the state-of-the-art trackers is achieved using less than $$10\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>10</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> deep feature channels.
Rethinking Dice Loss for Medical Image Segmentation

Deep learning has proved to be a powerful tool for medical image analysis in recent years. Data imbalance is a common problem in medical images. Dice Loss is widely used in medical image segmentation tasks to address the data imbalance problem. However, it only addresses the imbalance problem between foreground and background yet overlooks another imbalance between easy and hard examples that also severely affects the training process of a learning model. Empirically speaking, an easy example generally contributes less to the overall loss than a hard example. However, in practice, compared with hard examples, a large number of easy examples will be generated from a medical image and will dominate the training model, resulting in sub-optimal training or worse. To tackle this problem, we propose a novel Focal Dice Loss to alleviate the imbalance between hard examples and easy examples. Focal Dice Loss is able to reduce the contribution from easy examples and make the model focus on hard examples through our proposed novel balanced sampling strategy during the training process. Furthermore, to evaluate the effectiveness of our proposed loss functions, we conduct extensive experiments on two real-world medical image datasets with 2D and 3D convolutional neural networks. The experimental results show that our proposed Focal Dice Loss brings a significant improvement in segmentation performance compared to Dice Loss. Moreover, we find that our proposed Focal Dice Loss can effectively alleviate the over-fitting problem.
Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-Supervised Learning

This paper introduces a new concept called "transferable visual words" (TransVW), aiming to achieve annotation efficiency for deep learning in medical image analysis. Medical imaging-focusing on particular parts of the body for defined clinical purposes-generates images of great similarity in anatomy across patients and yields sophisticated anatomical patterns across images, which are associated with rich semantics about human anatomy and which are natural visual words. We show that these visual words can be automatically harvested according to anatomical consistency via self-discovery, and that the self-discovered visual words can serve as strong yet free supervision signals for deep models to learn semantics-enriched generic image representation via self-supervision (self-classification and self-restoration). Our extensive experiments demonstrate the annotation efficiency of TransVW by offering higher performance and faster convergence with reduced annotation cost in several applications. Our TransVW has several important advantages, including (1) TransVW is a fully autodidactic scheme, which exploits the semantics of visual words for self-supervised learning, requiring no expert annotation; (2) visual word learning is an add-on strategy, which complements existing self-supervised methods, boosting their performance; and (3) the learned image representation is semantics-enriched models, which have proven to be more robust and generalizable, saving annotation efforts for a variety of applications through transfer learning. Our code, pre-trained models, and curated visual words are available at https://github.com/JLiangLab/TransVW.