Reviewerly/exHarmony huggingface dataset:
A Survey and Comparison of Activities of Daily Living Datasets in Real-life and Virtual Spaces

Academics, researchers, and industrial experts have made significant efforts in human activity recognition by considering different perspectives, such as benchmark datasets, utilization of smart sensors, and development of recognition algorithms. In addition, the emerging trend of avatar technologies has drawn the attention of researchers, which assemble cognitive abilities, including activity recognition, to realize social activities and overcome boundaries of human energy, time, and environment. These could widely assist older adults, independent living care, and medical care, by the data obtained from human activities. Since human behavior in real-life is extremely complex, assembling meaningful recognition is paramount. The performance of existing recognition models depends heavily on the datasets. However, it has constraints to acquiring datasets with rich information due to the limited budgets, the specific areas that can be used for human activity, the limited number of actors, and other ethical reasons. This paper considers the critical criteria to fill the requirements for human daily activity recognition and presents a survey of the existing datasets of activities of daily living in both real-life settings and virtual spaces. It also presents new challenges and potential advances in behavior recognition technology.
ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition

Sign language recognition has attracted the interest of researchers in recent years. While numerous approaches have been proposed for European and Asian sign languages recognition, very limited attempts have been made to develop similar systems for the Arabic sign language (ArSL). This can be attributed partly to the lack of a dataset at the sentence level. In this paper, we aim to make a significant contribution by proposing ArabSign, a continuous ArSL dataset. The proposed dataset consists of 9,335 samples performed by 6 signers. The total time of the recorded sentences is around 10 hours and the average sentence's length is 3.1 signs. ArabSign dataset was recorded using a Kinect V2 camera that provides three types of information (color, depth, and skeleton joint points) recorded simultaneously for each sentence. In addition, we provide the annotation of the dataset according to ArSL and Arabic language structures that can help in studying the linguistic characteristics of ArSL. To benchmark this dataset, we propose an encoder-decoder model for Continuous ArSL recognition. The model has been evaluated on the proposed dataset, and the obtained results show that the encoder-decoder model outperformed the attention mechanism with an average word error rate (WER) of 0.50 compared with 0.62 with the attention mechanism. The data and code are available at https://github.com/Hamzah-Luqman/rabSign
Blockchain Forensics and Crypto-Related Cybercrimes

This handbook provides an overview of various criminal activities related to cryptocurrencies, including investment scams, Ponzi schemes, rug pulls, ransomware attacks, money laundering, and darknet markets. We discuss the need for user protection and education, centralized entities and regulators' efforts to prevent scams, and the use of blockchain forensic techniques to investigate and curb cybercrimes such as ransomware attacks and money laundering. We also go beyond blockchain and access how other statistical analyses of crypto-related data emerge as a powerful tool to examine wash trading and tax evasion. Despite some success by authorities in shutting down cybercriminal activities, using cryptocurrencies and other technologies makes it challenging to identify and prosecute those involved in illicit activities, and increased collaboration between law enforcement agencies and the technology industry is needed. This handbook is an ongoing effort by the authors to provide an overall anatomy of cybercrime and potential solutions in the crypto space.
Latent Generative Replay for Resource-Efficient Continual Learning of Facial Expressions

Real-world Facial Expression Recognition (FER) systems require models to constantly learn and adapt with novel data. Traditional Machine Learning (ML) approaches struggle to adapt to such dynamics as models need to be re-trained from scratch with a combination of both old and new data. Replay-based Continual Learning (CL) provides a solution to this problem, either by storing previously seen data samples in memory, sampling and interleaving them with novel data (rehearsal) or by using a generative model to simulate pseudo-samples to replay past knowledge (pseudo-rehearsal). Yet, the high memory footprint of rehearsal and the high computational cost of pseudo-rehearsal limit the real-world application of such methods, especially on resource-constrained devices. To address this, we propose Latent Generative Replay (LGR) for pseudo-rehearsal of low-dimensional latent features to mitigate forgetting in a resource-efficient manner. We adapt popular CL strategies to use LGR instead of generating pseudo-samples, resulting in performance upgrades when evaluated on the CK+, RAF-DB and AffectNet FER benchmarks where LGR significantly reduces the memory and resource consumption of replay-based CL without compromising model performance.
StyleMask: Disentangling the Style Space of StyleGAN2 for Neural Face Reenactment

In this paper we address the problem of neural face reenactment, where, given a pair of a source and a target facial image, we need to transfer the target's pose (defined as the head pose and its facial expressions) to the source image, by preserving at the same time the source's identity characteristics (e.g., facial shape, hair style, etc), even in the challenging case where the source and the target faces belong to different identities. In doing so, we address some of the limitations of the state-of-the-art works, namely, a) that they depend on paired training data (i.e., source and target faces have the same identity), b) that they rely on labeled data during inference, and c) that they do not preserve identity in large head pose changes. More specifically, we propose a framework that, using unpaired randomly generated facial images, learns to disentangle the identity characteristics of the face from its pose by incorporating the recently introduced style space S [1] of StyleGAN2 [2], a latent representation space that exhibits remarkable disentanglement properties. By capitalizing on this, we learn to successfully mix a pair of source and target style codes using supervision from a 3D model. The resulting latent code, that is subsequently used for reenactment, consists of latent units corresponding to the facial pose of the target only and of units corresponding to the identity of the source only, leading to notable improvement in the reenactment performance compared to recent state-of-the-art methods. In comparison to state of the art, we quantitatively and qualitatively show that the proposed method produces higher quality results even on extreme pose variations. Finally, we report results on real images by first embedding them on the latent space of the pretrained generator. We make the code and the pretrained models publicly available at: https://github.com/StelaBou/StyleMask.
S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image

We present a novel face reconstruction method capable of reconstructing detailed face geometry, spatially varying face reflectance from a single monocular image. We build our work upon the recent advances of DNN-based auto-encoders with differentiable ray tracing image formation, trained in self-supervised manner. While providing the advantage of learning-based approaches and real-time reconstruction, the latter methods lacked fidelity. In this work, we achieve, for the first time, high fidelity face reconstruction using self-supervised learning only. Our novel coarse-to-fine deep architecture allows us to solve the challenging problem of decoupling face reflectance from geometry using a single image, at high computational speed. Compared to state-of-the-art methods, our method achieves more visually appealing reconstruction.
Smartverse: Blockchain based Crowdsourced V2X Message Verification and Dissemination System

Intelligent transportation systems with Vehicle to Everything (V2X) connectivity can help smart cities solve problems like pedestrian jaywalking, vehicle collision, and traffic congestion. V2X offers seamless connectivity between road in-frastructures and driver assistance systems with crowd-sourced data to guide design and feature sets. V2X applications raise security and privacy risks despite their benefits. Risks include data loss, device failure, and environmental damage. The current PKI-based V2X ecosystem requires many pseudonym keys to work successfully. Public Key Infrastructure (PKI) often encoun-ters bottlenecks due to the enormous volume of authorisation, registration, and confirmation requests. When we transit to a means of fully autonomous transport, an enormous amount of data related to vehicle movement will flood the V2X network. A considerable infrastructure setup is needed to validate these mes-sages. Blockchain technology offers distributed ledger technology and a decentralised data processing mechanism. So blockchain can solve the issue of validating the messages in the V2X network. This paper proposes Smartverse, a blockchain technology-based authentication method to validate messages in the V2X ecosystem. We used the Ethereum blockchain platform, Interplanatary File System (IPFS) and developed algorithms to validate the crowd-sourced data in the V2X network. When a user submits data, say vehicle collision, other users in the V2X network validate the data by a voting mechanism. Once the messages are marked valid, other V2X users will receive this message. This paper explains the blockchain-based V2X network architecture and message validation algorithm.
RGB-Guided Depth Map Recovery by Two-Stage Coarse-to-Fine Dense CRF Models

Depth maps generally suffer from large erroneous areas even in public RGB-Depth datasets. Existing learning-based depth recovery methods are limited by insufficient high-quality datasets and optimization-based methods generally depend on local contexts not to effectively correct large erroneous areas. This paper develops an RGB-guided depth map recovery method based on the fully connected conditional random field (dense CRF) model to jointly utilize local and global contexts of depth maps and RGB images. A high-quality depth map is inferred by maximizing its probability conditioned upon a low-quality depth map and a reference RGB image based on the dense CRF model. The optimization function is composed of redesigned unary and pairwise components, which constraint local structure and global structure of depth map, respectively, with the guidance of RGB image. In addition, the texture-copy artifacts problem is handled by two-stage dense CRF models in a coarse-to-fine way. A coarse depth map is first recovered by embedding RGB image in a dense CRF model in unit of 3×3 blocks. It is refined afterward by embedding RGB image in another model in unit of individual pixels and restricting the model mainly work in discontinued regions. Extensive experiments on six datasets verify that the proposed method considerably outperforms a dozen of baseline methods in correcting erroneous areas and diminishing texture-copy artifacts of depth maps.
Quality-Constrained Encoding Optimization for Omnidirectional Video Streaming

Omnidirectional video streaming is usually implemented based on the representations of tiles, where the tiles are obtained by splitting the video frame into several rectangular areas and each tile is converted into multiple representations with different resolutions and encoded at different bitrates. One key issue in omnidirectional video streaming is how to choose the optimal representations for each tile at the server to save the overall transmission bitrate to all users while offering them satisfactory quality. This is different from the adaptive bitrate-based method that optimizes the downloading procedure of individual users, where the given video representations are stored on the server. In this work, we focus on optimization for the encoding of omnidirectional video streaming by using the optimal combination of tile representations. To achieve our goal, we formulate the selection of the representations into an optimization problem in which the transmission bitrate of all the representations is minimized with a quality constraint. By using this constraint, we can improve the average quality of omnidirectional videos for users. More specifically, we first construct the tile-level rate-distortion (R-D) model and determine the available tile bandwidth based on the previous viewers’ statistics. Then, we formulate the representation selection problem based on the obtained R-D model and tile bandwidth. Finally, we solve this problem to obtain the optimal combination of tile representations so that we can transmit the omnidirectional video to users with satisfactory quality but low bitrate. The experimental results demonstrate the effectiveness of our proposed approach when it is applied to omnidirectional video streaming.
Multi-Channel Augmented Graph Embedding Convolutional Network for Multi-View Clustering

With the explosive development of multi-view data from diverse sources, multi-view clustering (MVC) has drawn widespread attention. Existing MVC methods still have several limitations. First, it is difficult to sufficiently consider the local invariance within data views. Second, view fusion usually utilizes weighted averages, thus how to fuse views is warranting further exploration. Towards these two issues, this paper proposes a multi-channel augmented graph embedding convolutional network (MAGEC-Net) for multi-view clustering and its extended end-to-end model (EMAGEC-Net). The proposed frameworks are dedicated to exploring the consistency and complementarity of multi-view data. Specifically, on one hand, the augmented graphs are derived from generative adversarial networks, which explore the information and features of a single view more comprehensively. On the other hand, each augmented view is considered as a channel and fused by a deep fusion network, thus this method effectively improves the complementary information across views. Finally, feature extraction is performed on the fused consistent graphs to enable better clustering. Extensive experiments on six real challenging datasets demonstrate the effectiveness of the proposed method and its superiority over eight compared state-of-the-art methods.
DNN Partitioning for Inference Throughput Acceleration at the Edge

Deep neural network (DNN) inference on streaming data requires computing resources to satisfy inference throughput requirements. However, latency and privacy sensitive deep learning applications cannot afford to offload computation to remote clouds because of the implied transmission cost and lack of trust in third-party cloud providers. Among solutions to increase performance while keeping computation on a constrained environment, hardware acceleration can be onerous, and model optimization requires extensive design efforts while hindering accuracy. DNN partitioning is a third complementary approach, and consists of distributing the inference workload over several available edge devices, taking into account the edge network properties and the DNN structure, with the objective of maximizing the inference throughput (number of inferences per second). This paper introduces a method to predict inference and transmission latencies for multi-threaded distributed DNN deployments, and defines an optimization process to maximize the inference throughput. A branch and bound solver is then presented and analyzed to quantify the achieved performance and complexity. This analysis has led to the definition of the acceleration region, which describes deterministic conditions on the DNN and network properties under which DNN partitioning is beneficial. Finally, experimental results confirm the simulations and show inference throughput improvements in sample edge deployments.
DEAR-GAN: Degradation-Aware Face Restoration With GAN Prior

With the development of generative adversarial networks (GANs), recent face restoration (FR) methods often utilize pre-trained GAN models (i.e.,, StyleGAN2) as prior to generate rich details. However, these methods usually struggle to balance realness and fidelity when facing various degradation levels. In this paper, we propose a novel DEgradation-Aware Restoration network with GAN prior, dubbed DEAR-GAN, for FR tasks by explicitly learning the degradation representations (DR) to adapt to various degradation. Specifically, an unsupervised degradation representation learning (UDRL) strategy is first developed to extract DR of the input degraded images. Then, a degradation-aware feature interpolation (DAFI) module is proposed to dynamically fuse the two types of informative features (i.e.,, features from degraded images and features from GAN prior network) with flexible adaption to various degradation based on DR. Extensive experiments show that our proposed DEAR-GAN outperforms the state-of-the-art methods for face restoration under multiple degradation and face super-resolution, and demonstrate the effectiveness of feature interpolation, which can be extended to face inpainting to achieve excellent results.
CMTSNN: A Deep Learning Model for Multiclassification of Abnormal and Encrypted Traffic of Internet of Things

With the increasing types and number of Internet of Things (IoT) devices and malicious programs and the popularization of encryption technology in the communication process between the Internet and the IoT, a large amount of encrypted abnormal traffic among devices endangers IoT cybersecurity. How to identify abnormal encrypted traffic of the IoT has become the premise of cybersecurity. Presently, most of the detection methods for traffic in the IoT have problems, such as simple data set processing, imperfect feature extraction, data imbalance, and low multiclassification accuracy. In this article, we propose a multiclassification deep learning model named the cost matrix time–space neural network (CMTSNN) for abnormal and encrypted IoT traffic. The CMTSNN is divided into three parts. The first part is the preprocessing stage of the data set, which needs to retain the timing relation between two data packets in the stream and create a cost penalty matrix according to the sample distribution. Aimed at the robustness of feature extraction in network flow, the second part extracts time series features and then space features to ensure the robustness of feature extraction. The third part is aimed at the problem of data imbalance. The cost penalty matrix is applied to the cost penalty layer in the training process, and then the improved cross-entropy loss function is used to calculate the loss to improve the classification accuracy of minority categories and increase the overall multiclassification performance of the model. Experiments were carried out with the ToN-IoT, BoT-IoT, and ISCX VPN-NonVPN data sets. Compared with current methods, the proposed method shows better performances, including accuracy, precision, recall, F1 Score, and false alarm rate.
High-Intensified Resemblance and Statistic-Restructured Alignment in Few-Shot Domain Adaptation for Industrial-Specialized Employment

The industrial status quo has been changed hastily. Facilities require new technological factors and are less data-driven to make precise inferences at most. Thus, the use of virtual knowledge can be adapted to practical applications to limit human intervention in explanation, namely, comprehensive digital transformation. Deep transfer learning has the role of enabling transferable knowledge and awareness practice of annotating endpoints. However, practical transfer tasks are burdened while considering scarce or non-annotated instances accumulated from external sampling, and these may inevitably degenerate knowledge during training and further degrade performance. In this study, a novel framework composed of the spectral correlation alignment is proposed to enable spectral patterned structure for statistical criterion to diminish the domain disparity under a few-shot alignment and multiple attention orchestration to accelerate a few-shot adaptation. This technique also increases high-concentrated recognition in cross-domain similarity verification. The demonstrations are conducted under public visual adaptation bench-marks and realistic deployment. The empirical experiment illustrates that our approach is better in efficacy and adaptability under data-limited conditions. Besides, in similar realistic applications, it is evident that the proposed scheme is deployable and can be re-practicable to less time as well as data consumption, yet is better in performance.
Ray-Space Motion Compensation for Lenslet Plenoptic Video Coding

Plenoptic images and videos bearing rich information demand a tremendous amount of data storage and high transmission cost. While there has been much study on plenoptic image coding, investigations into plenoptic video coding have been very limited. We investigate the motion compensation (or so-called temporal prediction) for plenoptic video coding from a slightly different perspective by looking at the problem in the ray-space domain instead of in the conventional pixel domain. Here, we develop a novel motion compensation scheme for lenslet video under two sub-cases of ray-space motion, that is, integer ray-space motion and fractional ray-space motion. The proposed new scheme of light field motion-compensated prediction is designed such that it can be easily integrated into well-known video coding techniques such as HEVC. Experimental results compared to relevant existing methods have shown remarkable compression efficiency with an average gain of 20.03% and 21.76% respectively under "Low delayed B " and "Random Access" configurations of HEVC.
3-D Computed Laminography Based on Prior Images and Total Variation

The 3-D sparse-view computed laminography (CL) is an important research issue due to its considerable potential in lowering costs. To solve the artifacts and noise caused by sparse-view scanning, the total variation (TV) method is widely used. However, TV only utilizes the prior information of the reconstructed image itself. Therefore, the prior-image-based reconstruction method (PIBR) is proposed to reduce the artifacts by using a prior image. In this study, we proposed a 3-D CL method based on the final prior image and TV for sparse-view scanning, which is divided into two subproblems corresponding to two solution steps: the novel PIBR step and the denoising step. In the first step, we devise the final prior image and its corresponding prior mask to ensure the effectiveness of the PIBR. The final prior image is obtained based on the prior images, which are reconstructed from two different oblique angle projections. The prior mask is a binary mask, which is used to record the position of the prior image information. Then, the denoising step is based on the TV minimization method using the split-Bregman (SB) frame. Soft shrinkage operations and fast Fourier transform (FFT) are used to efficiently implement the SB frame. Compared with several iterative reconstruction methods, the experimental results demonstrate the effectiveness of the proposed method in terms of preserving edges, suppressing interslices aliasing and denoising.