Reviewerly/exHarmony huggingface dataset:
Integrated Learning of Robot Motion and Sentences: Real-Time Prediction of Grasping Motion and Attention based on Language Instructions

We propose a motion generation model that can achieve robust behavior against environmental changes based on language instructions at a low cost. Conventional robots that communicate with humans use a restricted environment and language to build up a mapping between language and motion, and thus need to prepare a huge training set in order to achieve versatility. Our method trains pairs of language, visual, and motor information of the robot, and generates motions in real-time based on the “attention” of the language instructions. Specifically, the robot generates motions while focusing on the indicated objects by the human when multiple objects are in the field of view. In addition, since position recognition and motion generation of the indicated object are performed in real-time, robust motion generation is possible in response to changes in the object position and lighting conditions. We clarified that features related to the object name and its location are self-organized in the latent (PB: Parametric Bias) space by end-to-end learning of robot motion and sentences. These observations may indicate the importance of integrated learning of robot motion and sentences since such feature representations cannot be obtained by learning motions alone.
RMS-FlowNet: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds

The proposed RMS-FlowNet is a novel end-to-end learning-based architecture for accurate and efficient scene flow estimation which can operate on point clouds of high density. For hierarchical scene flow estimation, the existing methods depend on either expensive Farthest-Point-Sampling (FPS) or structure-based scaling which decrease their ability to handle a large number of points. Unlike these methods, we base our fully supervised architecture on Random-Sampling (RS) for multiscale scene flow prediction. To this end, we propose a novel flow embedding design which can predict more robust scene flow in conjunction with RS. Exhibiting high accuracy, our RMS-FlowNet provides a faster prediction than state-of-the-art methods and works efficiently on consecutive dense point clouds of more than 250K points at once. Our comprehensive experiments verify the accuracy of RMS-FlowNet on the established FlyingThings3D data set with different point cloud densities and validate our design choices. Additionally, we show that our model presents a competitive ability to generalize towards the real-world scenes of KITTI data set without fine-tuning.
Coverage Path Planning in Large-scale Multi-floor Urban Environments with Applications to Autonomous Road Sweeping

Coverage Path Planning is the work horse of contemporary service task automation, powering autonomous floor cleaning robots and lawn mowers in households and office sites. While steady progress has been made on indoor cleaning and outdoor mowing, these environments are small and with simple geometry compared to general urban environments such as city parking garages, highway bridges or city crossings. To pave the way for autonomous road sweeping robots to operate in such difficult and complex environments, a benchmark suite with three large-scale 3D environments representative of this task is presented. On this benchmark we evaluate a new Coverage Path Planning method in comparison with previous well performing algorithms, and demonstrate state-of-the-art performance of the proposed method. Part of the success, for all evaluated algorithms, is the usage of automated domain adaptation by in-the-loop parameter optimization using Bayesian Optimization. Apart from improving the performance, tedious and bias-prone manual tuning is made obsolete, which makes the evaluation more robust and the results even stronger.
Secure Multi-Robot Information Sampling with Periodic and Opportunistic Connectivity

Multi-robot teams are becoming an increasingly popular approach for information gathering in large geographic areas, with applications in precision agriculture, surveying the aftermath of natural disasters or tracking pollution. These robot teams are often assembled from untrusted devices not owned by the user, making the maintenance of the integrity of the collected samples an important challenge. Furthermore, such robots often operate under conditions of opportunistic, or periodic connectivity and are limited in their energy budget and computational power. In this paper, we propose algorithms that build on blockchain technology to address the data integrity problem, but also take into account the limitations of the robots' resources and communication. We evaluate the proposed algorithms along the perspective of the tradeoffs between data integrity, model accuracy, and time consumption.
Human Navigational Intent Inference with Probabilistic and Optimal Approaches

Although human navigational intent inference has been studied in the literature, none have adequately considered both the dynamics that describe human motion and internal human parameters that may affect human navigational behaviour. In this paper, we propose a general probabilistic framework to infer the probability distribution over future navigational states of a human. Our framework incorporates an extended Dubins car dynamics to model human movement, which captures differences in human navigational behaviour depending on their position, heading, and movement speed. We assume a noisily rational model of human behaviour that incorporates a) human navigational intent that may change over time, b) how optimal a person's actions are given the navigational intent, and c) how far ahead in time a person considers when choosing navigational actions. These parameters are recursively and continuously updated in a Bayesian fashion. To make the Bayesian update and inference tractable, we exploit properties of the time-to-reach value function from optimal control and the extended Dubins car dynamics to construct a utility function on which the human policy is based, and employ particle representations of probability distributions where necessary. We demonstrate the effectiveness of our method by comparing our results with a recent approach using synthetic data and validate it on real world data.
Knowledge-Powered Explainable Artificial Intelligence for Network Automation toward 6G

Communication networks are becoming increasingly complex toward 6G. Manual management is no longer an option for network operators. Network automation has been widely discussed in the networking community, and it is a sensible means to manage the complex communication network. Deep learning models developed to enable network automation for given operation practices have the limitations of lack of explainability and inapplicability across different networks and/or network settings. To tackle the above issues, in this article we propose a new knowledge-powered framework that provides a human-understandable explainable artificial intelligence (XAI) agent for network automation. A case study of path selection is developed to demonstrate the feasibility of the proposed framework. Research on network automation is still in its infancy. Therefore, at the end of this article, we provide a list of challenges and open issues that can guide further research in this important area.
Abnormal Behavior Learning Based on Edge Computing toward a Crowd Monitoring System

Abnormal behavior poses a great threat to social security and stability. The resulting violence or crime leads to terrible consequences. How to utilize reasonable means to predict the dangerous intentions of massive crowds and prevent the potential hazard to the public is significant for social security. A crowd monitoring and management system is an effective way to detect abnormal behavior. In this article, we release unmanned aerial vehicles as well as fixed ground devices to achieve multi-level and multi-modal behavioral sensing on a massive crowd, deploy a hybrid model in edge cloud to extract global features from behavioral data of a massive crowd, and then utilize these global features to construct decent classification algorithms for action recognition and behavioral semantic cognition. With the cooperation of behavioral data and cognitive algorithms, we can understand the instantaneous emotions of the crowd. On the basis of behavioral data and the emotional state of the crowd, the correlation between daily behavior and any dangerous intention of a massive crowd has been revealed by utilizing behavioral big data analysis, which is a key foundation for predicting people's dangerous intentions. Finally, we conduct a case study of abnormal behavior detection based on pix2pix and continuous video frames. The experimental results show that the performance of our method is better than other algorithms in both public datasets and the customized Hajj dataset. The proposed novel pattern for the effective learning of a massive crowd is validated to effectively eliminate some of the possible dangers caused by abnormal behavior.
Standardization of Smart Contracts for Energy Markets and Operation

This work presents a formal review of smart contracts, including definitions, technical requirements, and potential power and energy-related use cases. This includes in-depth discussions covering cybersecurity, legality and interoperability goals that must be taken into consideration by potential end-users. The paper presents a first attempt towards the standardization of smart contracts (SCs) within the field of power and energy as a work in progress activity under the IEEE Standards Association (IEEE SA) P2418.5 Working Group. This work also proposes a holistic, language-agnostic reference model that is intended to accelerate the adoption of Distributed Ledger Technology (DLT) by industry stakeholders by providing standardized processes. Finally, the paper discusses key takeaways that must continue to be developed to increase SC usage within the energy industry.
A Gaussian Mixture Model Based Fast Motion Planning Method Through Online Environmental Feature Learning

Rapidly exploring random tree and its applications for mobile robot navigation have attracted extensive interest in the human–robot interaction environment. However, most current sampling-based path planning algorithms are easy to fall into the trap space in complex and dynamic environments. In this article, a fast path planning algorithm based on Gaussian mixture model (GMM) is proposed to allow robotic systems to efficiently extract feature nodes and generate collision-free paths considering the pedestrian density and the environmental structure. First, the preliminary feature nodes are obtained by training GMM, in which the covariance matrix of each feature node contains the environmental structure information. Second, an online feature learning method is presented to extract features during the movement of the robot by the dual-channel scale filter and the secondary distance fusion. Once the obstacle-free grid points can be connected to the feature nodes without any collision, a heuristic path is generated directly to guide the robot to avoid the crowds according to the proposed pedestrian matrix. For the same environment, the learned information can be reused as prior knowledge in the next path planning to further improve the efficiency of path replanning. A series of experimental studies demonstrate that our proposed method can remarkably reduce the time taken to reach a goal and enhance the success rate of navigation in trapped and narrow environments.
Privacy-Preserving Data Integrity Verification for Secure Mobile Edge Storage

Mobile edge computing (MEC) is proposed as an extension of cloud computing in the scenarios where the end devices desire better services in terms of response time. Because the edges are usually owned by individuals or small organizations with limited operation capabilities, the data on the edges are easily corrupted (due to external attacks or internal hardware failures). Therefore, it is essential to verify data integrity in the MEC. We propose two Integrity Checking protocols for the mobile Edge storage, called ICE-basic and ICE-batch. Our protocols allow a third-party verifier to check the data integrity on the edges without violating users data privacy and query pattern privacy. We rigorously prove the security and privacy guarantees of the protocols. In addition, we have investigated how to let the end devices cache some verification tags such that the communication cost between end devices and the cloud can be further reduced when a user connects to multiple edges in sequence. We have implemented a proof-of-concept system that runs ICE, and extensive experiments are conducted to evaluate the performance of the proposed protocols. The theoretical analysis and experimental results demonstrate the proposed protocols are efficient both in computation and communication.
Semisupervised Heterogeneous Domain Adaptation via Dynamic Joint Correlation Alignment Network for Ship Classification in SAR Imagery

Improving ship classification performance in synthetic aperture radar (SAR) imagery by transferring knowledge from the related domain is a newly emerging research topic. Existing methods follow supervised or unsupervised homogeneous transfer learning techniques with certain restrictions on the use of features (homogeneous rather than heterogeneous) and data (ignoring excavate the potential of unlabeled target domain data), which may hinder further performance improvements. To address these problems, this letter proposes a dynamic joint correlation alignment (DJ-CORAL) network to conduct semi-supervised heterogeneous domain adaptation (HDA). Specifically, DJ-CORAL firstly transforms the heterogeneous features from the source and target domains into a common subspace to eliminate the heterogeneity, then simultaneously performs classifier adaptation and joint marginal and conditional distribution alignment to facilitate the domain shift minimization. Comprehensive experiments validate the superiority of the proposed DJ-CORAL network against state-of-the-art HDA methods. The codes are available at https://github.com/BUCT-RS-ML/DJ-CORAL.
TSF: Two-Stage Sequential Fusion for 3D Object Detection

There have been significant advances in 3D object detection using LiDAR and camera fusion for autonomous driving.However, it is surprisingly difficult to effectively design fusion location and fusion strategies for point cloud-based 3D object detection networks.In this paper, we propose a novel two-stage sequential fusion (TSF) method.In the first stage of fusion, TSF generates the enhanced point cloud by combining the raw point cloud and semantic information of image instance segmentation.In the second stage, the proposals generated by LiDAR baseline is used to complete the No-Maximum Suppression (NMS) together with the 2D object detection results.Numerous experiments on the KITTI validation set show that our method outperforms stateof-the-art multimodal fusion-based methods on the three classes in 3D performance (Easy, Moderate, Hard): cars (89.94%, 82.76%, 76.04), pedestrians (70.74%, 63.47%, 56.56%), and cyclists (84.72%, 64.22%, 56.78%).In ablation, we analyze the augmented effect of fusion module on the LiDAR baseline detection capability, and study the best trade-off between running time and accuracy. Index Terms-3D object
Improving Multiscale Object Detection With Off-Centered Semantics Refinement

Feature Pyramid (FP) is typically a fundamental component for detecting multi-scale objects. However, as the network deepens, FP faces two problems: (1) Information loss caused by channel reduction. (2) The insufficient effective receptive field due to convolution with the sliding window mode. We found that the above problems can be alleviated by increasing the semantics extraction weights of the off-centered feature map. In this paper, a new feature pyramid architecture named Off-Centered Semantics Refinement Feature Pyramid Network (OSR-FPN) is proposed. Specifically, OSR-FPN contains two components exploiting the Off-Centered Semantics Refinement (OSR) mechanism: Features Supplement Module (FSM) and Receptive Field Enlargement Module (RFEM). FSM and RFEM are respectively designed to complement the lost context at the highest pyramid level and enrich the semantics by expanding the receptive field. In addition, we propose the Sigmoid-interpolation Padding method to enhance our OSR. Experiments on MS COCO dataset and UAVDT object detection benchmarks demonstrate the effectiveness of our method. As a result, OSR-FPN achieves a better accuracy of complex object detection.
Accelerating Edit-Distance Sequence Alignment on GPU Using the Wavefront Algorithm

Sequence alignment remains a fundamental problem with practical applications ranging from pattern recognition to computational biology. Traditional algorithms based on dynamic programming are hard to parallelize, require significant amounts of memory, and fail to scale for large inputs. This work presents eWFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute the exact edit-distance sequence alignment based on the wavefront alignment algorithm (WFA). This approach exploits the similarities between the input sequences to accelerate the alignment process while requiring less memory than other algorithms. Our implementation takes full advantage of the massive parallel capabilities of modern GPUs to accelerate the alignment process. In addition, we propose a succinct representation of the alignment data that successfully reduces the overall amount of memory required, allowing the exploitation of the fast shared memory of a GPU. Our results show that our GPU implementation outperforms by 3-9&#x00D7; the baseline edit-distance WFA implementation running on a 20 core machine. As a result, eWFA-GPU is up to 265 times faster than state-of-the-art CPU implementation, and up to 56 times faster than state-of-the-art GPU implementations.
Materialities of digital disease control in Taiwan during COVID-19

During the course of the COVID-19 pandemic, a wide range of digital technologies and data analytics have been incorporated into pandemic response models globally, in the hope of better detecting, tracking, monitoring and containing outbreaks. This increased digital involvement in disease control has offered the prospect of heightened effectiveness in all of the above, but not without raising other concerns. This paper contributes to ongoing discussions of the digital transformation in disease control by proposing a materialist analysis of how such control has become operative and what its effects may be, both now and in the future. Using Taiwan's digital pandemic response as a case study, the paper explores specific ways in which material processes and arrangements have shaped digital measures, as well as the actions that rendered such measures operable, with their ensuing consequences. This analysis illustrates the importance of historical, material and technological specificities and contingencies to our understanding of how digital disease control takes a particular shape. It also demonstrates how shifting regimes of practice continually reconfigure the ways in which digital disease control functions. The paper argues that paying greater attention to the materialities of digital disease control can provide a more nuanced understanding of the complex ways in which society may be protected or harmed by its use, possibly simultaneously. It is hoped that such increased attentiveness may inform more considered and careful preparation for subsequent pandemics.
The Interactive Design and User Experience of Virtual Museums: Case Study of the Virtual Palace Museum

One of the fundamental functions of a museum is to collect and preserve ancient cultural relics. Nowadays, some museums also undertake the roles as learning center, communication and social place, leisure center, and cultural communication center, by offering comprehensive experience of multisensory feelings, intelligence, aesthetics, social communication. As active participators and operators in virtual environment, the visitors show stronger engagement and better use experience when using virtual reality applications. A good interactive design in the virtual museum system will enhance the quality of remediation, which will encourage users to be active and immersed in the virtual tour. This study aims to analyze the interaction design of the virtual Palace Museum and examine virtual experience designed for visitors. After analyzed the virtual applications of the Palace Museum, we found that virtual museums can deliver knowledge in an attractive and accessible way for better cultural experience. Virtual reality is not to replace the unique experience of appreciating real objects, but it can greatly improve our understanding of artifacts and allow us to better understand their stories. The virtual museum enables immersive sense, good storytelling, interactive learning, and sharing between peers, which turns passive visitors to active explorers. In the digital age and the epidemic, we must re-examine the positioning and mission of museums from a longer-term perspective, dig and interpret the diverse values of cultural heritage, and give new vitality to cultural heritage.
Design and Evaluation of Countermeasures Against Fault Injection Attacks and Power Side-Channel Leakage Exploration for AES Block Cipher

Differential Fault Analysis (DFA) and Power Analysis (PA) attacks, have become the main methods for exploiting the vulnerabilities of physical implementations of block ciphers, currently used in a multitude of applications, such as the Advanced Encryption Standard (AES).In order to minimize these types of vulnerabilities, several mechanisms have been proposed to detect fault attacks.However, these mechanisms can have a significant cost, not fully covering the implementations against fault attacks or not taking into account the leakage of the information exploitable by the power analysis attacks.In this paper, four different approaches are proposed with the aim of protecting the AES block cipher against DFA.The proposed solutions are based on Hamming code and parity bits as signature generators for the internal state of the AES cipher.These allow to detect DFA exploitable faults, from bit to byte level.The proposed solutions have been applied to a T-box based AES block cipher implemented on Field Programmable Gate Array (FPGA).Experimental results suggest a fault coverage of 98.5% and 99.99% with an area penalty of 9% and 36% respectively, for the parity bit signature generators and a fault coverage of 100% with an area penalty of 18% and 42% respectively when Hamming code signature generator is used.In addition, none of the proposed countermeasures impose a frequency degradation, in respect to the unprotected cipher.The proposed work goes further in the evaluation of the proposed DFA countermeasures by evaluating the impact of these structures in terms of power side-channel.The obtained results suggest that no extra information leakage is produced that can be exploited by PA.Overall, the proposed DFA countermeasures provide a high fault coverage protection with a low cost in terms of area and power consumption and no PA security degradation.