Reviewerly/exHarmony huggingface dataset:
Integrating Social Assistive Robots, IoT, Virtual Communities and Smart Objects to Assist at-Home Independently Living Elders: the MoveCare Project

The integration of Ambient Assisted Living (AAL) frameworks with Socially Assistive Robots (SARs) has proven useful for monitoring and assisting older adults in their own home. However, the difficulties associated with long-term deployments in real-world complex environments are still highly under-explored. In this work, we first present the MoveCare system, an unobtrusive platform that, through the integration of a SAR into an AAL framework, aimed to monitor, assist and provide social, cognitive, and physical stimulation in the own houses of elders living alone and at risk of falling into frailty. We then focus on the evaluation and analysis of a long-term pilot campaign of more than 300 weeks of usages. We evaluated the system's acceptability and feasibility through various questionnaires and empirically assessed the impact of the presence of an assistive robot by deploying the system with and without it. Our results provide strong empirical evidence that Socially Assistive Robots integrated with monitoring and stimulation platforms can be successfully used for long-term support to older adults. We describe how the robot's presence significantly incentivised the use of the system, but slightly lowered the system's overall acceptability. Finally, we emphasise that real-world long-term deployment of SARs introduces a significant technical, organisational, and logistical overhead that should not be neglected nor underestimated in the pursuit of long-term robust systems. We hope that the findings and lessons learned from our work can bring value towards future long-term real-world and widespread use of SARs.
A novel biologically-inspired method for underwater image enhancement

Underwater images are usually characterized by color distortion, blurry, and severe noise, because light is severely scattered and absorbed when traveling in the water. In this paper, we propose a novel method motivated by the astonishing capability of the biological vision to address the low visibility of the real-world underwater images. Firstly, we simply imitate the color constancy mechanism in photoreceptors and horizontal cells (HCs) to correct the color distortion. In particular, HCs modulation provides a global color correction with gain control, in which light wavelength-dependent absorption is taken into account. Then, to solve the problems of blurry and noise, we introduce a straightforward and effective two-pathway dehazing method. The core idea is to decompose the color corrected image into structure-pathway and texture-pathway, corresponding to the Magnocellular (M-) and Parvocellular (P-) pathway in the early visual system. In the structure-pathway, we design an innovative biological normalization model to adjust the dynamic range of luminance by integrating the bright and dark regions. By using this approach, the proposed method leads to significant improvement in the contrast degradation of underwater images. Additionally, the detail preservation and noise suppression are implemented on the textural information. Finally, we merge the outputs of structure and texture pathways to reconstruct the enhanced underwater image. Both qualitative and quantitative evaluations show that the proposed biologically-inspired method achieves better visual quality, when compared with several related methods.
Pose and Joint-Aware Action Recognition

Recent progress on action recognition has mainly focused on RGB and optical flow features. In this paper, we approach the problem of joint-based action recognition. Unlike other modalities, constellation of joints and their motion generate models with succinct human motion information for activity recognition. We present a new model for joint-based action recognition, which first extracts motion features from each joint separately through a shared motion encoder before performing collective reasoning. Our joint selector module re-weights the joint information to select the most discriminative joints for the task. We also propose a novel joint-contrastive loss that pulls together groups of joint features which convey the same action. We strengthen the joint-based representations by using a geometry-aware data augmentation technique which jitters pose heatmaps while retaining the dynamics of the action. We show large improvements over the current state-of-the-art joint-based approaches on JHMDB, HMDB, Charades, AVA action recognition datasets. A late fusion with RGB and Flow-based approaches yields additional improvements. Our model also outperforms the existing baseline on Mimetics, a dataset with out-of-context actions.
Metacognitive improvement: Disentangling adaptive training from experimental confounds.

Metacognition is defined as the capacity to monitor and control one's own cognitive processes. Recently, Carpenter and colleagues (2019) reported that metacognitive performance can be improved through adaptive training: healthy participants performed a perceptual discrimination task, and subsequently indicated confidence in their response. Metacognitive performance, defined as how much information these confidence judgments contain about the accuracy of perceptual decisions, was found to increase in a group of participants receiving monetary reward based on their confidence judgments over hundreds of trials and multiple sessions. By contrast, in a control group where only perceptual performance was incentivized, metacognitive performance remained constant across experimental sessions. We identified two possible confounds that may have led to an artificial increase in metacognitive performance, namely the absence of reward in the initial session and an inconsistency between the reward scheme and the instructions about the confidence scale. We thus conducted a preregistered conceptual replication where all sessions were rewarded and where instructions were consistent with the reward scheme. Critically, once these two confounds were corrected we found moderate evidence for an absence of metacognitive training. Our data thus suggest that previous claims about metacognitive training are premature, and calls for more research on how to train individuals to monitor their own performance. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Perceptual Consistency in Video Segmentation

In this paper, we present a novel perceptual consistency perspective on video semantic segmentation, which can capture both temporal consistency and pixel-wise correctness. Given two nearby video frames, perceptual consistency measures how much the segmentation decisions agree with the pixel correspondences obtained via matching general perceptual features. More specifically, for each pixel in one frame, we find the most perceptually correlated pixel in the other frame. Our intuition is that such a pair of pixels are highly likely to belong to the same class. Next, we assess how much the segmentation agrees with such perceptual correspondences, based on which we derive the perceptual consistency of the segmentation maps across these two frames. Utilizing perceptual consistency, we can evaluate the temporal consistency of video segmentation by measuring the perceptual consistency over consecutive pairs of segmentation maps in a video. Furthermore, given a sparsely labeled test video, perceptual consistency can be utilized to aid with predicting the pixel-wise correctness of the segmentation on an unlabeled frame. More specifically, by measuring the perceptual consistency between the predicted segmentation and the available ground truth on a nearby frame and combining it with the segmentation confidence, we can accurately assess the classification correctness on each pixel. Our experiments show that the proposed perceptual consistency can more accurately evaluate the temporal consistency of video segmentation as compared to flow-based measures. Furthermore, it can help more confidently predict segmentation accuracy on unlabeled test frames, as compared to using classification confidence alone. Finally, our proposed measure can be used as a regularizer during the training of segmentation models, which leads to more temporally consistent video segmentation while maintaining accuracy.
Towards Accurate Oriented Object Detection in Aerial Images with Adaptive Multi-level Feature Fusion

Detecting objects in aerial images is a long-standing and challenging problem since the objects in aerial images vary dramatically in size and orientation. Most existing neural network based methods are not robust enough to provide accurate oriented object detection results in aerial images since they do not consider the correlations between different levels and scales of features. In this paper, we propose a novel two-stage network-based detector with a daptive f eature f usion towards highly accurate oriented object det ection in aerial images, named AFF-Det . First, a multi-scale feature fusion module (MSFF) is built on the top layer of the extracted feature pyramids to mitigate the semantic information loss in the small-scale features. We also propose a cascaded oriented bounding box regression method to transform the horizontal proposals into oriented ones. Then the transformed proposals are assigned to all feature pyramid network (FPN) levels and aggregated by the weighted RoI feature aggregation (WRFA) module. The above modules can adaptively enhance the feature representations in different stages of the network based on the attention mechanism. Finally, a rotated decoupled-RCNN head is introduced to obtain the classification and localization results. Extensive experiments are conducted on the DOTA and HRSC2016 datasets to demonstrate the advantages of our proposed AFF-Det. The best detection results can achieve 80.73% mAP and 90.48% mAP, respectively, on these two datasets, outperforming recent state-of-the-art methods.
GC–HGNN: A global-context supported hypergraph neural network for enhancing session-based recommendation

Session-based recommendation (SBR) aims to predict the next most likely item to be interacted based on the current session. Given a short individual session, it is difficult to make accurate recommendations by using only the information of the sequence itself. To improve the recommendation effect, it has become a trend to comprehensively consider the interactive information of other session sequences. In this paper, GC–HGNN, a hypergraph neural network, is proposed to enhance the SBR effects. The model first learns the preferences of current users by transiting information among items in all session sequences. On this basis, the GC–HGNN model fully considers the global context information and local context information of items, and constructs the global session hypergraph and local session graph. The global context information is obtained by the hypergraph convolutional neural network, and the graph attention network calculates the local context information by learning the pairwise relationship between items. The sum-pooling method is utilized to fuse the two kinds of information and the fused information is applied to accomplish the prediction. Extensive experiments were conducted on three public datasets, Diginetica, Tmall and Nowplaying, and the experimental results demonstrate that our proposed GC-HGNN model outperforms the baseline models in SBR.
Occlusion Resistant Network for 3D Face Reconstruction

3D face reconstruction from a monocular face image is a mathematically ill-posed problem. Recently, we observed a surge of interest in deep learning-based approaches to address the issue. These methods possess extreme sensitivity towards occlusions. Thus, in this paper, we present a novel context-learning-based distillation approach to tackle the occlusions in the face images. Our training pipeline focuses on distilling the knowledge from a pre-trained occlusion-sensitive deep network. The proposed model learns the context of the target occluded face image. Hence our approach uses a weak model (unsuitable for occluded face images) to train a highly robust network towards partially and fully-occluded face images. We obtain a landmark accuracy of 0.77 against 5.84 of recent state-of-the-art-method for real-life challenging facial occlusions. Also, we propose a novel end-to-end training pipeline to reconstruct 3D faces from multiple variations of the target image per identity to emphasize the significance of visible facial features during learning. For this purpose, we leverage a novel composite multi-occlusion loss function. Our multi-occlusion per identity model shows a dip in the landmark error by a large margin of 6.67 in comparison to a recent state-of-the-art method. We deploy the occluded variations of the CelebA validation dataset and AFLW2000-3D face dataset: naturally-occluded and artificially occluded, for the comparisons. We comprehensively compare our results with the other approaches concerning the accuracy of the reconstructed 3D face mesh for occluded face images.
MTGLS: Multi-Task Gaze Estimation with Limited Supervision

Robust gaze estimation is a challenging task, even for deep CNNs, due to the non-availability of large-scale labeled data. Moreover, gaze annotation is a time-consuming process and requires specialized hardware setups. We propose MTGLS: a Multi-Task Gaze estimation framework with Limited Supervision, which leverages abundantly available non-annotated facial image data. MTGLS distills knowledge from off-the-shelf facial image analysis models, and learns strong feature representations of human eyes, guided by three complementary auxiliary signals: (a) the line of sight of the pupil (i.e. pseudo-gaze) defined by the localized facial landmarks, (b) the head-pose given by Euler angles, and (c) the orientation of the eye patch (left/right eye). To overcome inherent noise in the supervisory signals, MT-GLS further incorporates a noise distribution modelling approach. Our experimental results show that MTGLS learns highly generalized representations which consistently perform well on a range of datasets. Our proposed framework outperforms the unsupervised state-of-the-art on CAVE (by ∼ 6.43%) and even supervised state-of-the-art methods on Gaze360 (by ∼ 6.59%) datasets.
Automatic modulation classification: Cauchy-Score-function-based cyclic correlation spectrum and FC-MLP under mixed noise and fading channels

Automatic modulation classification (AMC), also termed blind signal modulation recognition, plays a critical role in various civilian and military applications. Although existing approaches have made substantial contributions in this area, most fail to simultaneously consider mixed noise, different channel modes and low-power scenarios. In this paper, a novel cyclic correlation spectrum based on the Cauchy Score function is first proposed as the robust pattern for the outliers in the Gaussian and non-Gaussian mixed noise. Besides, its properties are studied and proved for further potential applications in wireless signal processing. Furthermore, a modified lightweight neural network, termed feature-coupling multi-layer perceptron (FC-MLP), is designed to avoid the potential risk of overfitting and meet the needs when applied in low-power chips. In addition, a novel AMC method is proposed under a dual-layer decision tree framework, and different patterns are adopted in different layers to make full use of the robustness of the proposed pattern and the information embedded in the time sequence. Meanwhile, different classifiers are also chosen according to the characteristics of the patterns. In the simulations, state-of-the-art machine learning techniques, neural networks and patterns are employed as comparison candidates to verify the superiority of the proposed pattern, the lightweight classifier and the novel AMC approach in the scenario of Rayleigh channel or Rician channel with additive mixed noise.
A New V-Net Convolutional Neural Network Based on Four-Dimensional Hyperchaotic System for Medical Image Encryption

In the transmission of medical images, if the image is not processed, it is very likely to leak data and personal privacy, resulting in unpredictable consequences. Traditional encryption algorithms have limited ability to deal with complex data. The chaotic system is characterized by randomness and ergodicity, which has advantages over traditional encryption algorithms in image encryption processing. A novel V-net convolutional neural network (CNN) based on four-dimensional hyperchaotic system for medical image encryption is presented in this study. Firstly, the plaintext medical images are processed into 4D hyperchaotic sequence images, including image segmentation, chaotic system processing, and pseudorandom sequence generation. Then, V-net CNN is used to train chaotic sequences to eliminate the periodicity of chaotic sequences. Finally, the chaotic sequence image is diffused to change the raw image pixel to realize the encryption processing. Simulation test analysis demonstrates that the proposed algorithm has better effect, robustness, and plaintext sensitivity.
Visual explanation of black-box model: Similarity Difference and Uniqueness (SIDU) method

Explainable Artificial Intelligence (XAI) has in recent years become a well-suited framework to generate human understandable explanations of ‘black- box’ models. In this paper, a novel XAI visual explanation algorithm known as the Similarity Difference and Uniqueness (SIDU) method that can effectively localize entire object regions responsible for prediction is presented in full detail. The SIDU algorithm robustness and effectiveness is analyzed through various computational and human subject experiments. In particular, the SIDU algorithm is assessed using three different types of evaluations (Application, Human and Functionally-Grounded) to demonstrate its superior performance. The robustness of SIDU is further studied in the presence of adversarial attack on ’black-box’ models to better understand its performance. Our code is available at: https://github.com/satyamahesh84/SIDU_XAI_CODE.
Hierarchical blockchain structure for node authentication in IoT networks

Internet of Things (IoT) networks are large peer-to-peer networks of small devices that require a competent security system that is scalable and adaptable to the limited resources of the IoT devices. Node authentication is a crucial part of IoT security. The current authentication solutions require a centralized trusted party for authentication, which presents a single point of failure. Blockchain as a peer-to-peer network with decentralized authentication can provide a decentralized solution for node authentication. In existing literature, most blockchain applications in IoT are connected to existing blockchain networks by more computationally capable devices, thereby limiting their adaptability for IoT networks and presenting single point of failure problem. Considering the issues, this paper proposes a blockchain-based decentralized structure for authentication by arranging the IoT devices into clusters based on their computational capability, energy reserve and their location. The devices in each cluster are authenticated by a hierarchical structure of interconnected blockchains. To reduce the processing load we introduced a consensus protocol based on verifying identity-based encryption key signature of the device and its related cluster. The proposed structure simulation has shown a reduction of the processor and memory load of IoT devices. Further testing using Docker container network and Raspberry Pi devices network has shown that the proposed blockchain structure and consensus algorithm have reduced computational load. The analysis of the structure security and performance shows it offers comprehensive security protection while being lightweight and scalable.
PoVRPoint: Authoring Presentations in Mobile Virtual Reality

Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing traditional input devices with a large three-dimensional output space and spatial input. Previous research on supporting VR knowledge work explored domains such as text entry using physical keyboards and spreadsheet interaction using combined pen and touch input. Inspired by such work, this paper probes the VR design space for authoring presentations in mobile settings. We propose PoVRPoint-a set of tools coupling pen- and touch-based editing of presentations on mobile devices, such as tablets, with the interaction capabilities afforded by VR. We study the utility of extended display space to, for example, assist users in identifying target slides, supporting spatial manipulation of objects on a slide, creating animations, and facilitating arrangements of multiple, possibly occluded shapes or objects. Among other things, our results indicate that 1) the wide field of view afforded by VR results in significantly faster target slide identification times compared to a tablet-only interface for visually salient targets; and 2) the three-dimensional view in VR enables significantly faster object reordering in the presence of occlusion compared to two baseline interfaces. A user study further confirmed that the interaction techniques were found to be usable and enjoyable.
Improving Depth Estimation by Embedding Semantic Segmentation: A Hybrid CNN Model

Single image depth estimation works fail to separate foreground elements because they can easily be confounded with the background. To alleviate this problem, we propose the use of a semantic segmentation procedure that adds information to a depth estimator, in this case, a 3D Convolutional Neural Network (CNN)-segmentation is coded as one-hot planes representing categories of objects. We explore 2D and 3D models. Particularly, we propose a hybrid 2D-3D CNN architecture capable of obtaining semantic segmentation and depth estimation at the same time. We tested our procedure on the SYNTHIA-AL dataset and obtained σ3=0.95, which is an improvement of 0.14 points (compared with the state of the art of σ3=0.81) by using manual segmentation, and σ3=0.89 using automatic semantic segmentation, proving that depth estimation is improved when the shape and position of objects in a scene are known.
Multiple-Layer Security Threats on the Ethereum Blockchain and Their Countermeasures

Blockchain technology has been widely used in digital currency, Internet of Things, and other important fields because of its decentralization, nontampering, and anonymity. The vigorous development of blockchain cannot be separated from the security guarantee. However, there are various security threats within the blockchain that have shown in the past to cause huge financial losses. This paper aims at studying the multi-level security threats existing in the Ethereum blockchain, and exploring the security protection schemes under multiple attack scenarios. There are ten attack scenarios studied in this paper, which are replay attack, short url attack, false top-up attack, transaction order dependence attack, integer overflow attack, re-entrancy attack, honeypot attack, airdrop hunting attack, writing of arbitrary storage address attack, and gas exhaustion denial of service attack. This paper also proposes protection schemes. Finally, these schemes are evaluated by experiments. Experimental results show that our approach is efficient and does not bring too much extra cost and that the time cost has doubled at most.