Reviewerly/exHarmony huggingface dataset:
PCG-TAL: Progressive Cross-Granularity Cooperation for Temporal Action Localization

There are two major lines of works, i.e., anchor-based and frame-based approaches, in the field of temporal action localization. But each line of works is inherently limited to a certain detection granularity and cannot simultaneously achieve high recall rates with accurate action boundaries. In this work, we propose a progressive cross-granularity cooperation (PCG-TAL) framework to effectively take advantage of complementarity between the anchor-based and frame-based paradigms, as well as between two-view clues (i.e., appearance and motion). Specifically, our new Anchor-Frame Cooperation (AFC) module can effectively integrate both two-granularity and two-stream knowledge at the feature and proposal levels, as well as within each AFC module and across adjacent AFC modules. Specifically, the RGB-stream AFC module and the flow-stream AFC module are stacked sequentially to form a progressive localization framework. The whole framework can be learned in an end-to-end fashion, whilst the temporal action localization performance can be gradually boosted in a progressive manner. Our newly proposed framework outperforms the state-of-the-art methods on three benchmark datasets the THUMOS14, ActivityNet v1.3 and UCF-101-24, which clearly demonstrates the effectiveness of our framework.
Review on Chaotic Lasers and Measurement Applications

Chaotic laser is considered as a novel signal source and extensively applied in the areas of secure communication, random number generation, optical sensing. This review article provides an overview of our recent progresses in high-performance chaos generation and its unique measurement applications. Several creative methods underpinning chaos optimization, including bandwidth enhancement, time delay signature suppression, and photonic integration, are discussed and experimentally verified respectively. Then the long-range high-resolution measurement application of chaotic Brillion optical correlation-domain analysis and chaos optical time-domain reflectometer are summarized. Avenues for future research and development of chaotic laser and chaos measurement technology are also prospected.
Cross-domain recommendation with user personality

Recommender systems have long played indispensable roles in e-commerce by helping users find their preferred products efficiently and accurately. In recent years, some pioneering works have shown that users' intrinsic characteristics, such as personality traits, can help improve the performance of recommender systems. However, they generally do not consider how to obtain users' personality traits effectively without burdensome surveys and, therefore, cannot be used in new application domains without adequate personality information. Therefore, we propose a novel framework for building cross-domain personality-based recommender systems, especially for personality-scarce target domains. Specifically, we define the cross-domain personality trait classification problem and solve it in a semisupervised manner by leveraging the predictive text embedding method as the method for transfer learning from the source to the target domain. We then design a personality-boosted probabilistic matrix factorization method for personality-based recommendations. Extensive experiments conducted on five real-world datasets demonstrate that users' personality traits can be recognized more precisely with cross-domain transfer learning, and recommendation performance is improved accordingly.
Uncertainty-Aware Semantic Guidance and Estimation for Image Inpainting

Completing a corrupted image by filling in correct structures and reasonable textures for a complex scene remains an elusive challenge. In case that a missing hole involves diverse semantic information, conventional two-stage approaches based on structural information often lead to unreliable structural prediction and ambiguous visual texture generation. To address the problem, we propose a SEmantic GUidance and Estimation Network (SeGuE-Net) that iteratively evaluates the uncertainty of inpainted visual contents based on pixel-wise semantic inference and optimize structural priors and inpainted contents alternatively. Specifically, SeGuE-Net utilizes semantic segmentation maps as guidance in each iteration of image inpainting, under which location-dependent inferences are re-estimated, and, accordingly, poorly-inferred regions are refined in subsequent iterations. Extensive experiments on real-world images demonstrate the superiority of our proposed method over state-of-the-art approaches in terms of clear boundaries and photo-realistic textures.
Dynamic batch size tuning based on stopping criterion for neural network training

In the neural network training, the selection of the minibatch size affects not only the computational cost but also the training performance, as it underlies the loss function. Generally, an approach based on increasing the batch size according to the linear and step functions during the training process is known to be effective in improving the generalization performance of a network. However, it requires specifying the way of increasing the batch size beforehand. In this study, we propose a more flexible method that implies temporarily varying a small batch size to destabilize the loss function when a change in the training loss satisfies the predefined stopping criterion. Repeating the destabilization step allows a parameter to avoid being trapped at the local minima and to converge at a robust minimum, thereby improving generalization performance. We experimentally demonstrate the superiority of the proposed method considering several benchmark datasets and neural network models.
MoGaze: A Dataset of Full-Body Motions that Includes Workspace Geometry and Eye-Gaze

As robots become more present in open human environments, it will become crucial for robotic systems to understand and predict human motion. Such capabilities depend heavily on the quality and availability of motion capture data. However, existing datasets of full-body motion rarely include 1) long sequences of manipulation tasks, 2) the 3D model of the workspace geometry, and 3) eye-gaze, which are all important when a robot needs to predict the movements of humans in close proximity. Hence, in this letter, we present a novel dataset of full-body motion for everyday manipulation tasks, which includes the above. The motion data was captured using a traditional motion capture system based on reflective markers. We additionally captured eye-gaze using a wearable pupil-tracking device. As we show in experiments, the dataset can be used for the design and evaluation of full-body motion prediction algorithms. Furthermore, our experiments show eye-gaze as a powerful predictor of human intent. The dataset includes 180 min of motion capture data with 1627 pick and place actions being performed. It is available at https://humans-to-robots-motion.github.io/mogaze/MoGaze, Dataset and is planned to be extended to collaborative tasks with two humans in the near future.
Predicting Level of Visual Focus of Human’s Attention Using Machine Learning Approaches

Attention is a human’s concentration of awareness on a specific moment or information. The level of attention describes how deep the focus is on that specific moment or information. The level of attention generally depends on the environment and the mental condition of a human. This paper has offered various machine learning approaches for predicting the level of visual focus of human’s attention. Using data collected from survey reports (environmental data) and eyeball data (eyeball movement, reading time, and head turn) while reading an article, a dataset with each participant’s attention level was formed. Eight different classifiers were trained, namely: Logistic Regression, Support Vector Machine (SVM), Decision Tree, K-Nearest Neighbor (KNN), AdaBoost, Multilayer Perceptron (MLP), Extra Tree Classifier, and Voting Classifier for classifying the participant’s attention level into three classes: High, Average, and Low. The Logistic Regression achieved the highest accuracy of 96% beating others in predicting these classes. The aggregated weighted Voting Classifier attained an accuracy of 95%.
URNet: A U-Net based residual network for image dehazing

Low visibility in hazy weather causes the loss of image details in digital images captured by some imaging devices such as monitors. This paper proposes an end-to-end U-Net based residual network (URNet) to improve the visibility of hazy images. The encoder module of URNet uses hybrid convolution combining standard convolution with dilated convolution to expand the receptive field for extracting image features with more details. The URNet embeds several building blocks of ResNet into the junction between the encoder module and the decoder module. This prevents network performance degradation due to the vanishing gradient. After considering large absolute difference on image saturation and value components between hazy images and haze-free images in the HSV color space, the URNet defines a new loss function to better guide the network training. Experimental results on synthetic hazy images and real hazy images show that the URNet significantly improves the image dehazing effect compared to the state-of-the-art methods.
DeepQuake — An application of CNN for seismo-acoustic event classification in The Netherlands

Recent developments in instrumentation and methodology are major driving forces in the advances of solid Earth sciences. The deployment of large and dense sensor networks enables data centres to acquire data of increased volume and quality. The analysis of such data provides scientists with a better understanding about natural phenomena in the subsurface. At the same time new challenges arise to exploit the growing information potential. Innovative methods based on Artificial Intelligence offer concrete opportunities to tackle those challenges. In this paper we investigate Convolutional Neural Networks (CNN) for seismo-acoustic event classification in The Netherlands. A supervised deep learning approach is applied to continuous seismic waveforms in order to automatically detect seismo-acoustic events and discriminate their sources. Our region of interest is characterised by high seismic noise conditions and intense anthropogenic activity which deeply influence the recorded signals. Site conditions and our aim for a tool that could support operational earthquake analysis motivated us to devise a novel specialised CNN architecture. Building on existing approaches, we designed, trained and evaluated two CNN models: arch-time and arch-spect. Our results suggest that as CNN inputs, spectrograms are more suitable than continuous waveforms. Therefore, we propose a model (arch-spect) that achieves a performance among the highest in modern source detection systems. Furthermore, we focus on explainability aspects and propose a method to gain insights in the behaviour of a ‘black-box’ deep learning classification system. By generating visualisations of activations we aim to empower domain experts with a useable cross-checking tool and pave the way for a broader operational uptake of Artificial Intelligence based methods.
RRT*N: an efficient approach to path planning in 3D for Static and Dynamic Environments

In this paper, a modified algorithm of the Rapidly exploring Random Tree star (RRT*) is proposed. This method is called RRT*N. The main advantage of this method is its speed and robustness in finding the path to the target. An extension of this method to the 3D case is also presented and its capability of handling static obstacles and dynamic unknown moving obstacles in 2D and 3D environments, is shown. This improved method uses a probability distribution to generate new nodes. The nodes closest to the target have higher probability, which generates a tree centered on the line joining the robot to the target. It is shown that this method can be three times faster in finding the path to the target than the regular RRT* in the same environment. Simulation and experimental results are presented to show the robustness of the proposed RRT*N method.
An IoT-Applicable Access Control Model Under Double-Layer Blockchain

Access control in Internet of Things (IoT) systems has long been facing with single-point-of-failure, collusion and other security issues. Deploying most of the access control procedures on blockchain makes it more trustworthy. However, the miscellaneous transaction data on the public blockchain incurs an extra burden on computing and communication overhead, which hinders its application for IoT devices. Additionally, the demands of rapid response in industrial IoT scenarios (e.g., edge computing scenarios) is hardly realized under a public blockchain. In this brief, we propose an IoT-applicable access control model under an original double-layer blockchain architecture, in which the mortgage/registration and control processes can be disaggregated. Depending on this architecture, the communication overhead for IoT devices is reduced and a rapid response is achieved. With linear secret sharing scheme, attribute-based access policies for resource objects are established to ensure fine-grained access control. We further implement a smart contract prototype on Ethereum and Fisco Bcos. Extensive experiments and numerical analysis demonstrate the feasibility of our proposed model.
How ageing shapes body and space representations: A comparison study between healthy young and older adults

To efficiently interact with the external world, the brain needs to represent the size of the involved body parts - body representations (BR) - and the space around the body in which the interactions with the environment take place - peripersonal space representation (PPS). BR and PPS are both highly flexible, being updated by the continuous flow of sensorimotor signals between the brain and the body, as observed for example after tool-use or immobilization. The progressive decline of sensorimotor abilities typically described in ageing could thus influence BR and PPS representations in the older adults. To explore this hypothesis, we compared BR and PPS in healthy young and older participants. By focusing on the upper limb, we adapted tasks previously used to evaluate BR and PPS plasticity, i.e., the body-landmarks localization task and audio-tactile interaction task, together with a new task targeting explicit BR (avatar adjustment task, AAT). Results show significantly higher distortions in the older rather than young participants in the perceived metric characteristic of the upper limbs. We found significant modifications in the implicit BR of the global shape (length and width) of both upper limbs, together with an underestimation in the arm length. Similar effects were also observed in the AAT task. Finally, both young and older adults showed equivalent multisensory facilitation in the space close to the hand, suggesting an intact PPS representation. Together, these findings demonstrated significant alterations of implicit and explicit BR in the older participants, probably associated with a less efficient contribution of bodily information typically subjected to age-related decline, whereas the comparable PPS representation in both groups could be supported by preserved multisensory abilities in older participants. These results provide novel empirical insight on how multiple representations of the body in space, subserving actions and perception, are shaped by the normal course of life.
A dynamic load scheduling in IaaS cloud using binary JAYA algorithm

Datacenters receive dynamic workloads with disparate specifications to be scheduled on virtual machines (VMs). These unpredictable, alarmingly growing workloads with varying resource specifications may bring down the servers of datacenters into an imbalanced state. Thus, resulting in low resource utilization and high energy consumption among the servers. To cater to the need of fluctuating on-demand resource provisioning, it is essential to scale up the ability and capacityof existing infrastructure through virtualization. Moreover, due to the involvement of conflicting scheduling constraints, load scheduling in cloud computing fall under NP-hard problem. An effective scheduling mechanism in amalgamation with a load balancing strategy based on binary JAYA is implemented to alleviate the challenges as mentioned above. This technique not only improves resource utilization but also brings down the degree of energy consumption and makespan while keeping the whole system balanced. At first, it focuses on evoking a load balancing procedure to uniformly disperse the loads among VMs based on the compatibility between the tasks and VMs and secondly, JAYA algorithm is executed to find the best possible mapping of tasks onto VMs. In order to appraise the efficacy of the proposed algorithm, the scheduling of independent and non-preemptive tasks is simulated in CloudSim using a benchmark parallel workload by NASA-iPSC. Experiments are conducted in both homogenous and heterogeneous environments. The proposed algorithm is statistically validated using the Friedman test, followed by a Holm's test. The proposed approach is compared over other algorithms such as Round Robin (RR), binary Particle Swarm Optimization (PSO) and Genetic Algorithm (GA). The simulation results show a notable amelioration over other mentioned algorithms by an increase of resource utilization with 18.47% (GA), 12.65% (BPSO) and 4.18% (GA), 2.51% (BPSO) and a reduction of makespan by 6.47% (GA), 4.35% (BPSO) and 4.17% (GA), 2.20% (BPSO) with an increasing number of tasks and VMs in two different test cases respectively.
RusTEE: Developing Memory-Safe ARM TrustZone Applications

In the past decade, Trusted Execution Environment (TEE) provided by ARM TrustZone is becoming one of the primary techniques for enhancing the security of mobile devices. The isolation enforced by TrustZone can protect the trusted applications running in the TEE against malicious software in the untrusted rich execution environment (REE). However, TrustZone cannot completely prevent vulnerabilities in trusted applications residing in the TEE, which can then be used to attack other trusted applications or even the trusted OS. Previously, a number of memory corruption vulnerabilities have been reported on different TAs, which are written in memory-unsafe languages like C.
IoT Forensics: An Overview of the Current Issues and Challenges

The pursuit of cybercrime in an IoT environment often requires complex investigations where the traditional digital forensics methodology may struggle to support the forensics investigators. This is due to the nature of the technologies such as RFID, sensors and cloud computing, used in IoT environments together with the huge volume and heterogeneous information and borderless cyber infrastructure, rising new challenges in modern digital forensics. In the last few years, many researches have been conducted discussing the challenges facing digital forensic investigators and the impact of these challenges bring upon the field. Some of these challenges include the ambiguity of data location, data acquisition, diversity of devices, various data types, volatility of data and the lack of adequate forensics tools. Moreover, while there are many technical challenges in IoT forensics, there are also non-technical challenges such as determining what are IoT devices, how to forensically acquire data and secure the chain of custody among other unexplored areas, including resources required for training or the type of applied forensics tools. A profound understanding of the challenges found in the literature will help the researchers in identifying future research directions and provide some guidelines to support forensics investigators. This study presents a succinct overview of IoT forensics challenges focusing on a typical smart home investigation and a comparison of the existing frameworks to conduct forensics investigations in the IoT environment.
POISON: Human Pose Estimation in Insufficient Lighting Conditions Using Sensor Fusion

In this article, we present POISON, an end-to-end deep neural network (DNN) method to estimate the human pose in real-time and under the variability of lighting conditions. A lightweight version of the open-source library, OpenPose, is used to extract human keypoints from RGB and infrared (IR) images, and the contribution of each image is combined by a fusion step. We propose a method to fuse these two types of information, using a fusion strategy followed by a refinement stage DNN, which aims to identify complex relationships between extracted keypoints. Finally, the combined information is used to infer the human pose. The experimental results validate that POISON improves the overall performance of conventional single-camera methods by a factor 1.79×. On a custom data set of RGB/IR pair images captured in challenging low-light conditions, POISON correctly detects up to 86% of human keypoints, outperforming conventional single-camera methods, which can detect up to 48% of keypoints. We also provide a comparison of POISON against the current state-of-the-art image fusion methods based on DNN techniques. We perform extensive quantitative evaluation and show that POISON outperforms existing approaches for the task of estimating the human pose in challenging lighting environments. POISON runs at 17 frames/s on Intel Core i7 CPU with Nvidia GeForce GTX-1080 GPU dedicated to DNN operations, making the solution suitable for real-time performance.
FPGA-based low-batch training accelerator for modern CNNs featuring high bandwidth memory

Training convolutional neural networks (CNNs) requires intensive computations as well as a large amount of storage and memory access. While low bandwidth off-chip memories in prior FPGA works have hindered the system-level performance, modern FPGAs offer high bandwidth memory (HBM2) that unlocks opportunities to improve the throughput/energy of FPGA-based CNN training. This paper presents a FPGA accelerator for CNN training which (1) uses HBM2 for efficient off-chip communication, and (2) supports various training operations (e.g. residual connections, stride-2 convolutions) for modern CNNs. We analyze the impact of HBM2 on CNN training workloads, provide a comprehensive comparison with DDR3, and present the strategies to efficiently use HBM2 features for enhanced CNN training performance. For training ResNet-20/VGG-like CNNs for CIFAR-10 dataset with low batch size of 2, the proposed CNN training accelerator on Intel Stratix-10 MX FPGA demonstrates 1.4/1.7X energy-efficiency improvement compared to Stratix-10 GX FPGA with DDR3 memory, and 4.5/9.7 X energy-efficiency improvement compared to Tesla V100 GPU.
Design Patterns for Blockchain-based Self-Sovereign Identity

Self-sovereign identity is a new identity management paradigm that allows entities to really have the ownership of their identity data and control their use without involving any intermediary. Blockchain is an enabling technology for building self-sovereign identity systems by providing a neutral and trustable storage and computing infrastructure, and can be viewed as a component of the systems. Both blockchain and self-sovereign identity are emerging technologies which could present a steep learning curve for architects. We collect and propose 12 design patterns for blockchain-based self-sovereign identity systems to help the architects understand and easily apply the concepts in system design. Based on the lifecycles of three main objects involved in self-sovereign identity, we categorise the patterns into three groups: key management patterns, decentralised identifier management patterns, and credential design patterns. The proposed patterns provide a systematic and holistic guide for architects to design the architecture of blockchain-based self-sovereign identity systems.