Reviewerly/exHarmony huggingface dataset:
Gesture-Radar: A Dual Doppler Radar Based System for Robust Recognition and Quantitative Profiling of Human Gestures

Gesture recognition is key to enabling natural human-computer interactions. Existing approaches based on wireless sensing focus on accurate identification of arm gesture types. It remains a challenge to recognize and profile the details of arm gestures for precise interaction control. In addition, current approaches have strict positioning requirements between radars and users, making them difficult for real-world deployment. In this article, we adopt the multisensor approach and present gesture-radar-a dual Doppler radar-based gesture recognition and profiling system, which can capture subtle arm gestures with less positioning or environmental dependence. Gesture-radar uses two vertically placed Doppler radars to collect complementary sensing data of gestures, based on which cross-analysis can be performed for gesture recognition and profiling. Specifically, we first propose a two-stage classification model and enhance the signal proximity matching method by applying constraint functions to the DTW algorithm, aiming to improve the accuracy of gesture type recognition. Afterward, we establish and analyze unique features from the time-frequency spectrogram, which can be used to characterize in-depth gesture details, e.g., the angle or range of an arm movement. Experimental results show that gesture-radar achieves up to 93.5% average accuracy for gesture type recognition, and over 80% precision for profiling gesture details. This proves that the proposed approach is viable and can work in real-world environments.
Detection and Semantic Segmentation of Disaster Damage in UAV Footage

In the aftermath of large-scale disasters, such as hurricanes, floods, or earthquakes, preliminarily damage assessment (PDA) is carried out to determine the impact and magnitude of damage and meet the needs of affected individuals, businesses, and communities. Traditionally, site evaluation and consensus-based assessment techniques are used to estimate the extent of the damage. More recently, given their low-cost, ease of operation, and ability to be deployed ondemand, unmanned aerial vehicles (UAVs) are increasingly used for disaster response and mitigation. However, the resulting large volume of visual data collected by and shared among first responders and volunteer groups is not used effectively because current practices of processing such data are heavily human-dependent, extremely resource-intensive, and significantly slow compared to the fast-evolving nature and progression of disaster impact. This paper contributes to the core body of knowledge by presenting a fully annotated dataset (with the object classes people, flooded area, and damaged and undamaged building roof, car, debris, vegetation, road, and boat) and a host of convolutional neural network (CNN) models for detecting and segmenting critical objects in the aerial footage of disaster sites. For best results, two CNN-based image segmentation architectures, namely, Mask-RCNN and Pyramid Scene Parsing Network (PSPNet), are adopted (through transfer learning), trained, validated, and tested on annotated videos to detect countable and bulk objects. The paper further introduces a targeted data augmentation technique to preserve data balance, as well as a data-driven approach to splitting highly mismatched classes for better model performance. Through these improvements, the best performing Mask-RCNN model generates pixel-level segmentations of countable objects with a 51.54% mean average precision (mAP). Additionally, the best performing PSPNet models can achieve mean intersection over union (mIoU) as high as 32.17% and accuracy as high as 77.01% on bulk objects.
Context-Aware Cross-Attention for Non-Autoregressive Translation

Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence.However, due to the lack of target dependency modelling in the decoder, the conditional generation process heavily depends on the cross-attention.In this paper, we reveal a localness perception problem in NAT cross-attention, for which it is difficult to adequately capture source context.To alleviate this problem, we propose to enhance signals of neighbour source tokens into conventional cross-attention.Experimental results on several representative datasets show that our approach can consistently improve translation quality over strong NAT baselines.Extensive analyses demonstrate that the enhanced cross-attention achieves better exploitation of source contexts by leveraging both local and global information.
CalibRCNN: Calibrating Camera and LiDAR by Recurrent Convolutional Neural Network and Geometric Constraints

In this paper, we present Calibration Recurrent Convolutional Neural Network (CalibRCNN) to infer a 6 degrees of freedom (DOF) rigid body transformation between 3D LiDAR and 2D camera. Different from the existing methods, our 3D-2D CalibRCNN not only uses the LSTM network to extract the temporal features between 3D point clouds and RGB images of consecutive frames, but also uses the geometric loss and photometric loss obtained by the interframe constraint to refine the calibration accuracy of the predicted transformation parameters. The CalibRCNN aims at inferring the correspondence between projected depth image and RGB image to learn the underlying geometry of 2D-3D calibration. Thus, the proposed calibration model achieves a good generalization ability to adapt to unknown initial calibration error ranges, and other 3D LiDAR and 2D camera pairs with different intrinsic parameters from the training dataset. Extensive experiments have demonstrated that our CalibRCNN can achieve state-of-the-art accuracy by comparison with other CNN based methods.
Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language

All over the world, deaf people use sign language as the only reliable source of communication with each other as well as with normal people. These communicating signs are made up of the shape of the hand and movement. In Pakistan, deaf people use Pakistan sign language (PSL) as a means of communication with people. In scientific literature, many studies have been done on PSL recognition and classification. Most of these work focused on colored-based hands while some others are sensors and Kinect-based approaches. These techniques are costly and also avoid user-friendliness. In this paper, a technique is proposed for the recognition of thirty-six static alphabets of PSL using bare hands. The dataset is obtained from the sign language videos. At a later step, four vision-based features are extracted i.e. local binary patterns, a histogram of oriented gradients, edge-oriented histogram, and speeded up robust features. The extracted features are individually classified using Multiple kernel learning (MKL) in support vector machine (SVM). We employed a one-to-all approach for the implementation of basic binary SVM into the multi-class SVM. A voting scheme is adopted for the final recognition of PSL. The performance of the proposed technique is measured in terms of accuracy, precision, recall, and F-score. The simulation results are promising as compared with existing approaches.
Communication-Efficient Federated Learning with Adaptive Parameter Freezing

Federated learning allows edge devices to collaboratively train a global model by synchronizing their local updates without sharing private data. Yet, with limited network bandwidth at the edge, communication often becomes a severe bottleneck. In this paper, we find that it is unnecessary to always synchronize the full model in the entire training process, because many parameters gradually stabilize prior to the ultimate model convergence, and can thus be excluded from being synchronized at an early stage. This allows us to reduce the communication overhead without compromising the model accuracy. However, challenges are that the local parameters excluded from global synchronization may diverge on different clients, and meanwhile some parameters may stabilize only temporally. To address these challenges, we propose a novel scheme called Adaptive Parameter Freezing (APF), which fixes (freezes) the non-synchronized stable parameters in intermittent periods. Specifically, the freezing periods are tentatively adjusted in an additively-increase and multiplicatively-decrease manner, depending on if the previously-frozen parameters remain stable in subsequent iterations. We implemented APF as a Python module in PyTorch. Our extensive array of experimental results show that APF can reduce data transfer by over 60%.
Public Auditing Scheme for Integrity Verification in Distributed Cloud Storage System

Cloud storage provides a potential solution replacing physical disk drives in terms of prominent outsourcing services. A threaten from an untrusted server affects the security and integrity of the data. However, the major problem between the data integrity and cost of communication and computation is directly proportional to each other. It is hence necessary to develop a model that provides the trade-off between the data integrity and cost metrics in cloud environment. In this paper, we develop an integrity verification mechanism that enables the utilisation of cryptographic solution with algebraic signature. The model utilises elliptic curve digital signature algorithm (ECDSA) to verify the data outsources. The study further resists the malicious attacks including forgery attacks, replacing attacks and replay attacks. The symmetric encryption guarantees the privacy of the data. The simulation is conducted to test the efficacy of the algorithm in maintaining the data integrity with reduced cost. The performance of the entire model is tested against the existing methods in terms of their communication cost, computation cost, and overhead cost. The results of simulation show that the proposed method obtains reduced computational of 0.25% and communication cost of 0.21% than other public auditing schemes.
Guidelines for volcano-observatory operations during crises: recommendations from the 2019 volcano observatory best practices meeting

Abstract In November 2019, the fourth Volcano Observatory Best Practices workshop was held in Mexico City as a series of talks, discussions, and panels. Volcanologists from around the world offered suggestions for ways to optimize volcano-observatory crisis operations. By crisis, we mean unrest that may or may not lead to eruption, the eruption itself, or its aftermath, all of which require analysis and communications by the observatory. During a crisis, the priority of the observatory should be to acquire, process, analyze, and interpret data in a timely manner. A primary goal is to communicate effectively with the authorities in charge of civil protection. Crisis operations should rely upon exhaustive planning in the years prior to any actual unrest or eruptions. Ideally, nearly everything that observatories do during a crisis should be envisioned, prepared, and practiced prior to the actual event. Pre-existing agreements and exercises with academic and government collaborators will minimize confusion about roles and responsibilities. In the situation where planning is unfinished, observatories should prioritize close ties and communications with the land and civil-defense authorities near the most threatening volcanoes. To a large extent, volcanic crises become social crises, and any volcano observatory should have a communication strategy, a lead communicator, regular status updates, and a network of colleagues outside the observatory who can provide similar messaging to a public that desires consistent and authoritative information. Checklists permit tired observatory staff to fulfill their duties without forgetting key communications, data streams, or protocols that need regular fulfilment (Bretton et al. Volcanic Unrest. Advances in Volcanology, 2018; Newhall et al. Bull Volcanol 64:3–20, 2020). Observatory leaders need to manage staff workload to prevent exhaustion and ensure that expertise is available as needed. Event trees and regular group discussions encourage multi-disciplinary thinking, consideration of disparate viewpoints, and documentation of all group decisions and consensus. Though regulations, roles and responsibilities differ around the world, scientists can justify their actions in the wake of an eruption if they document their work, are thoughtful and conscientious in their deliberations, and carry out protocols and procedures developed prior to volcanic unrest. This paper also contains six case studies of volcanic eruptions or observatory actions that illustrate some of the topics discussed herein. Specifically, we discuss Ambae (Vanuatu) in 2017–2018, Kīlauea (USA) in 2018, Etna (Italy) in 2018, Bárðarbunga (Iceland) in 2014, Cotopaxi (Ecuador) in 2015, and global data sharing to prepare for eruptions at Nyiragongo (Democratic Republic of Congo). A Spanish-language version of this manuscript is provided as Additional file 1.
FG-CF: Friends-aware graph collaborative filtering for POI recommendation

Collaborative filtering approach greatly promotes the development and application of personalized recommendation. In location-based social networks (LBSNs), the sparsity of check-in data is one of the main obstacles for traditional Point-of-Interest (POI) recommendation models. Graph convolutional network (GCN) is an efficient tool to overcome this kind of problems, which enhances the representational ability of embeddings by capture high-order connectivity of users and POIs. In real applications, social tie is a crucial factor for POI recommendation that ignored in most current graph-based methods. Moreover, most message aggregation functions fail to capture contextual information. To address these problems, a novel framework named Friends-aware Graph Collaborative Filtering (FG-CF) is proposed in this paper, which incorporates social information into a user-POI graph. Firstly, a user-POI correlation matrix is estimated by check-in data and social links, and then, user embedding is updated according to the user-POI correlation matrix. Secondly, interaction messages are constructed in a novel way by integrating nodes’ ego embeddings, neighbors’ embeddings and social embeddings. Thirdly, by aggregating previous state embeddings and non-linear combination of neighbor messages with interaction messages, a new message aggregation function is present to update user and POI embeddings. Fourthly, we concatenate embeddings from each additional interaction layer to get the final embeddings, and inner product is used to compute the preference score of a user to a targeted POI. Finally, extensive experiments on two large-scale LBSN datasets demonstrate the superiority of our model over several state-of-the-art approaches.
Personalized Adaptive Learning: An Emerging Pedagogical Approach Enabled by a Smart Learning Environment

Smart devices and intelligent technologies are enabling a smart learning environment to effectively promote the development of personalized learning and adaptive learning, in line with the trend of accelerating the integration of both. In this regard, we introduce a new teaching method enabled by a smart learning environment, which is a form of personalized adaptive learning. In order to clearly explain this approach, we have analyzed its two pillars: personalized learning and adaptive learning. From this analysis, we have also explored the core elements of personalized adaptive learning and its core concepts. On this basis, we further construct a framework of personalized adaptive learning. We hope this paper will provide readers with a clear understanding of personalized adaptive learning, and serve as an endeavor to contribute to future studies and practices.
Strength in Numbers: Improving Generalization with Ensembles in Machine Learning-based Profiled Side-channel Analysis

The adoption of deep neural networks for profiled side-channel attacks provides powerful options for leakage detection and key retrieval of secure products. When training a neural network for side-channel analysis, it is expected that the trained model can implement an approximation function that can detect leaking side-channel samples and, at the same time, be insensible to noisy (or non-leaking) samples. This outlines a generalization situation where the model can identify the main representations learned from the training set in a separate test set.This paper discusses how output class probabilities represent a strong metric when conducting the side-channel analysis. Further, we observe that these output probabilities are sensitive to small changes, like selecting specific test traces or weight initialization for a neural network. Next, we discuss the hyperparameter tuning, where one commonly uses only a single out of dozens of trained models, where each of those models will result in different output probabilities. We show how ensembles of machine learning models based on averaged class probabilities can improve generalization. Our results emphasize that ensembles increase a profiled side-channel attack’s performance and reduce the variance of results stemming from different hyperparameters, regardless of the selected dataset or leakage model.
Towards situational aware cyber-physical systems: A security-enhancing use case of blockchain-based digital twins

The complexity of cyberattacks in Cyber-Physical Systems (CPSs) calls for a mechanism that can evaluate critical infrastructures’ operational behaviour and security without affecting the operation of live systems. In this regard, Digital Twins (DTs) provide actionable insights through monitoring, simulating, predicting, and optimizing the state of CPSs. Through the use cases, including system testing and training, detecting system misconfigurations, and security testing, DTs strengthen the security of CPSs throughout the product lifecycle. However, such benefits of DTs depend on an assumption about data integrity and security. Data trustworthiness becomes more critical while integrating multiple components among different DTs owned by various stakeholders to provide an aggregated view of the complex physical system. This article envisions a blockchain-based DT framework as Trusted Twins for Securing Cyber-Physical Systems (TTS-CPS). With the automotive industry as a CPS use case, we demonstrate the viability of the TTS-CPS framework through a proof of concept. To utilize reliable system specification data for building the process knowledge of DTs, we ensure the trustworthiness of data-generating sources through Integrity Checking Mechanisms (ICMs). Additionally, Safety and Security (S&S) rules evaluated during simulation are stored and retrieved from the blockchain, thereby establishing more understanding and confidence in the decisions made by the underlying systems. Finally, we perform formal verification of the TTS-CPS.
Al-Biruni Earth Radius (BER) Metaheuristic Search Optimization Algorithm

Metaheuristic optimization algorithms present an effective method for solving several optimization problems from various types of applications and fields.Several metaheuristics and evolutionary optimization algorithms have been emerged recently in the literature and gained widespread attention, such as particle swarm optimization (PSO), whale optimization algorithm (WOA), grey wolf optimization algorithm (GWO), genetic algorithm (GA), and gravitational search algorithm (GSA).According to the literature, no one metaheuristic optimization algorithm can handle all present optimization problems.Hence novel optimization methodologies are still needed.The Al-Biruni earth radius (BER) search optimization algorithm is proposed in this paper.The proposed algorithm was motivated by the behavior of swarm members in achieving their global goals.The search space around local solutions to be explored is determined by Al-Biruni earth radius calculation method.A comparative analysis with existing state-of-the-art optimization algorithms corroborated the findings of BER's validation and testing against seven mathematical optimization problems.The results show that BER can both explore and avoid local optima.BER has also been tested on an engineering design optimization problem.The results reveal that, in terms of performance and capability, BER outperforms the performance of state-of-the-art metaheuristic optimization algorithms.
Memory-aware curriculum federated learning for breast cancer classification.

For early breast cancer detection, regular screening with mammography imaging is recommended. Routine examinations result in datasets with a predominant amount of negative samples. The limited representativeness of positive cases can be problematic for learning Computer-Aided Diagnosis (CAD) systems. Collecting data from multiple institutions is a potential solution to mitigate this problem. Recently, federated learning has emerged as an effective tool for collaborative learning. In this setting, local models perform computation on their private data to update the global model. The order and the frequency of local updates influence the final global model. In the context of federated adversarial learning to improve multi-site breast cancer classification, we investigate the role of the order in which samples are locally presented to the optimizers.We define a novel memory-aware curriculum learning method for the federated setting. We aim to improve the consistency of the local models penalizing inconsistent predictions, i.e., forgotten samples. Our curriculum controls the order of the training samples prioritizing those that are forgotten after the deployment of the global model. Our approach is combined with unsupervised domain adaptation to deal with domain shift while preserving data privacy.Two classification metrics: area under the receiver operating characteristic curve (ROC-AUC) and area under the curve for the precision-recall curve (PR-AUC) are used to evaluate the performance of the proposed method. Our method is evaluated with three clinical datasets from different vendors. An ablation study showed the improvement of each component of our method. The AUC and PR-AUC are improved on average by 5% and 6%, respectively, compared to the conventional federated setting.We demonstrated the benefits of curriculum learning for the first time in a federated setting. Our results verified the effectiveness of the memory-aware curriculum federated learning for the multi-site breast cancer classification. Our code is publicly available at: https://github.com/ameliajimenez/curriculum-federated-learning.
Factorizing Knowledge in Neural Networks

In this paper, we explore a novel and ambitious knowledge-transfer task, termed Knowledge Factorization (KF). The core idea of KF lies in the modularization and assemblability of knowledge: given a pretrained network model as input, KF aims to decompose it into several factor networks, each of which handles only a dedicated task and maintains task-specific knowledge factorized from the source network. Such factor networks are task-wise disentangled and can be directly assembled, without any fine-tuning, to produce the more competent combined-task networks. In other words, the factor networks serve as Lego-brick-like building blocks, allowing us to construct customized networks in a plug-and-play manner. Specifically, each factor network comprises two modules, a common-knowledge module that is task-agnostic and shared by all factor networks, alongside with a task-specific module dedicated to the factor network itself. We introduce an information-theoretic objective, InfoMax-Bottleneck (IMB), to carry out KF by optimizing the mutual information between the learned representations and input. Experiments across various benchmarks demonstrate that, the derived factor networks yield gratifying performances on not only the dedicated tasks but also disentanglement, while enjoying much better interpretability and modularity. Moreover, the learned common-knowledge representations give rise to impressive results on transfer learning. Our code is available at https://github.com/Adamdad/KnowledgeFactor .