Reviewerly/exHarmony huggingface dataset:
Color Invariant Skin Segmentation

This paper addresses the problem of automatically detecting human skin in images without reliance on color information. A primary motivation of the work has been to achieve results that are consistent across the full range of skin tones, even while using a training dataset that is significantly biased toward lighter skin tones. Previous skin-detection methods have used color cues almost exclusively, and we present a new approach that performs well in the absence of such information. A key aspect of the work is dataset repair through augmentation that is applied strategically during training, with the goal of color invariant feature learning to enhance generalization. We have demonstrated the concept using two architectures, and experimental results show improvements in both precision and recall for most Fitzpatrick skin tones in the benchmark ECU dataset. We further tested the system with the RFW dataset to show that the proposed method performs much more consistently across different ethnicities, thereby reducing the chance of bias based on skin color. To demonstrate the effectiveness of our work, extensive experiments were performed on grayscale images as well as images obtained under unconstrained illumination and with artificial filters. Source code: https://github.com/HanXuMartin/Color-Invariant-Skin-Segmentation
Deep-FlexISP: A Three-Stage Framework for Night Photography Rendering

Night photography rendering is challenging due to images’ high noise level, less vivid color, and low dynamic range. In this work, we propose a three-stage cascade framework named Deep-FlexISP, which decomposes the ISP into three weakly correlated sub-tasks: raw image de-noising, white balance, and Bayer to sRGB mapping, for the following considerations. First, task decomposition can enhance the learning ability of the framework and make it easier to converge. Second, weak correlation sub-tasks do not influence each other too much, so the framework has a high degree of freedom. Finally, noise, color, and brightness are essential for night photographs. Our framework can flexibly adjust different styles according to personal preferences with the vital learning ability and the degree of freedom. Compared with the other Deep-ISP methods, our proposed Deep-FlexISP shows state-of-the-art performance and achieves first place in people’s choice and photographer’s choice in NTIRE 2022 Night Photography Render Challenge.
A Lightweight Network for High Dynamic Range Imaging

Multi-frame high dynamic range (HDR) reconstruction methods try to expand the range of illuminance with differently exposed images. They suffer from ghost artifacts when camera jittering or object moving. Several methods can generate high-quality HDR images with high computational complexity, but the inference process is too slow. However, the network with small parameters will produce unsatisfactory results. To balance the quality and computational complexity, we propose a lightweight network for HDR imaging that has small parameters and fast speed. Specifically, following AHDRNet, we employ a spatial attention module to detect the misaligned regions to avoid ghost artifacts. Considering the missing details in over-/under-exposure regions, we propose a dual attention module for selectively retaining information to force the fusion network to learn more details for degenerated regions. Furthermore, we employ an encoder-decoder structure with a lightweight block to achieve the fusion process. As a result, the high-quality content and features can be reconstructed after the attention module. Finally, we fuse high-resolution features and the encoder-decoder features into the HDR imaging results. Experimental results demonstrate that the proposed method performs favorably against the state-of-the-art methods, achieving a PSNR of 39.05 and a PSNR-µ of 37.27 with 156.12 GMAcs in NTIRE 2022 HDR Challenge (Track 2 Fidelity).
Online Unsupervised Domain Adaptation for Person Re-identification

Unsupervised domain adaptation for person re-identification (Person Re-ID) is the task of transferring the learned knowledge on the labeled source domain to the unlabeled target domain. Most of the recent papers that address this problem adopt an offline training setting. More precisely, the training of the Re-ID model is done assuming that we have access to the complete training target domain data set. In this paper, we argue that the target domain generally consists of a stream of data in a practical real-world application, where data is continuously increasing from the different network’s cameras. The Re-ID solutions are also constrained by confidentiality regulations stating that the collected data can be stored for only a limited period, hence the model can no longer get access to previously seen target images. Therefore, we present a new yet practical online setting for Unsupervised Domain Adaptation for person Re-ID with two main constraints: Online Adaptation and Privacy Protection. We then adapt and evaluate the state-of-the-art UDA algorithms on this new online setting using the well-known Market-1501, Duke, and MSMT17 benchmarks.
Understanding the Role of Weather Data for Earth Surface Forecasting using a ConvLSTM-based Model

Climate change is perhaps the biggest single threat to humankind and the environment, as it severely impacts our terrestrial surface, home to most of the living species.Inspired by video prediction and exploiting the availability of Copernicus Sentinel-2 images, recent studies have attempted to forecast the land surface evolution as a function of past land surface evolution, elevation, and weather.Further extending this paradigm, we propose a model based on convolutional long short-term memory (ConvLSTM) that is computationally efficient (lightweight), however obtains superior results to the previous baselines.By introducing a ConvLSTM-based architecture to this problem, we can not only ingest the heterogeneous data sources (Sentinel-2 time-series, weather data, and a Digital Elevation Model (DEM)) but also explicitly condition the future predictions on the weather.Our experiments confirm the importance of weather parameters in understanding the land cover dynamics and show that weather maps are significantly more important than the DEM in this task.Furthermore, we perform generative simulations to investigate how varying a single weather parameter can alter the evolution of the land surface.All studies are performed using the EarthNet2021 dataset.The code, additional materials and results can be found at https://github.com/dcodrut/weather2land.
Multi-Camera Vehicle Tracking Based on Occlusion-aware and Inter-vehicle Information

With the demands of analyzing and predicting traffic flow for applications in smart cities, Multi-Target Multi-Camera vehicle Tracking(MTMCT) at the city scale has become a fundamental problem. The MTMCT is challenging due to the view variations, frequent occlusions, and similar vehicle models in the same camera. This work proposes an MTMCT framework based on occlusion-aware and inter-vehicle information that can effectively match vehicle track-lets. The occlusion-aware module segments the tracklets of an occluded and occluding vehicle pair. It recalculates the similarity of the complete tracklets, which can handle the occlusions and suppress false detections. This work proposes an inter-vehicle information module to improve the matching accuracy. The module can enhance the ability to distinguish similar vehicles under the same camera at different times. The proposed whole framework consists of four modules: (1) vehicle detection and feature extraction by re-identification models, (2) single-camera tracking (SCT) to produce initial tracklets with an occlusion-aware module, (3) tracklets similarity by inter-vehicle association, (4) clustering in adjacent cameras for multi-camera track-lets matching. The proposed method obtains IDF <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> score of 0.8285 on the Track-1 multi-camera vehicle tracking task of the 2022 AI City Challenge.
The sentiment of a virtual rock concert

Abstract We created a virtual reality version of a 1983 performance by Dire Straits, this being a highly complex scenario consisting of both the virtual band performance and the appearance and behaviour of the virtual audience surrounding the participants. Our goal was to understand the responses of participants, and to learn how this type of scenario might be improved for later reconstructions of other concerts. To understand the responses of participants we carried out two studies which used sentiment analysis of texts written by the participants. Study 1 ( n = 25) (Beacco et al. in IEEE Virtual Reality: 538–545, 2021) had the unexpected finding that negative sentiment was caused by the virtual audience, where e.g. some participants were fearful of being harassed by audience members. In Study 2 ( n = 26) notwithstanding some changes, the audience again led to negative sentiment—e.g. a feeling of being stared at. For Study 2 we compared sentiment with questionnaire scores, finding that the illusion of being at the concert was associated with positive sentiment for males but negative for females. Overall, we found sentiment was dominated by responses to the audience rather than the band. Participants had been placed in an unusual situation, being alone at a concert, surrounded by strangers, who seemed to pose a social threat for some of them. We relate our findings to the concept of Plausibility, the illusion that events and situations in the VR are really happening. The results indicate high Plausibility, since the negative sentiment, for example in response to being started at, only makes sense if the events are experienced as actually happening. We conclude with the need for co-design of VR scenarios, and the use of sentiment analysis in this process, rather than sole reliance on concepts proposed by researchers, typically expressed through questionnaires, which may not reflect the experiences of participants.
SkeletonPose: Exploiting human skeleton constraint for 3D human pose estimation

We present SkeletonPose, a novel method to improve the accuracy of 3D human pose estimation via human skeleton constraints. In contrast to other state-of-the-art 3D pose estimation approaches, which use deep convolutional networks to regress the human pose, the proposed method employs a combination of data-driven and calculation methods. Our approach explicitly exploits the skeleton length prior during testing which can decrease the predicted human skeleton length error. As a result, the predicted 3D human pose error deceases accordingly. First, the proposed approach uses deep convolutional networks to regress the z -coordinate of the root joint. Second, the predicted outputs of the networks are used to calculate the 3D human pose according to the skeleton length invariance constraint. This combined method increases the accuracy of pose estimation because the skeleton length prior restricts the human pose space. Moreover, to eliminate the ambiguity of the z -coordinate difference between two connected joints, we propose a step-wise refinement to reduce the adverse effect caused by inaccurate predictions. Thorough evaluations were performed on three public databases; the Human3.6M dataset, the Human Eva-I database and the MPI-INF-3DHP dataset. Comparisons illustrate that SkeletonPose achieved better performance with respect to other state-of-the-art pose estimation approaches. The code and data are available at https://github.com/XTU-PR-LAB/skeleton-pose .
Neural Fairness Blockchain Protocol Using an Elliptic Curves Lottery

To protect participants’ confidentiality, blockchains can be outfitted with anonymization methods. Observations of the underlying network traffic can identify the author of a transaction request, although these mechanisms often only consider the abstraction layer of blockchains. Previous systems either give topological confidentiality that may be compromised by an attacker in control of a large number of nodes, or provide strong cryptographic confidentiality but are so inefficient as to be practically unusable. In addition, there is no flexible mechanism to swap confidentiality for efficiency in order to accommodate practical demands. We propose a novel approach, the neural fairness protocol, which is a blockchain-based distributed ledger secured using neural networks and machine learning algorithms, enabling permissionless participation in the process of transition validation while concurrently providing strong assurance about the correct functioning of the entire network. Using cryptography and a custom implementation of elliptic curves, the protocol is designed to ensure the confidentiality of each transaction phase and peer-to-peer data exchange.
Artificial Intelligence and Cyber Defense System for Banking Industry: A Qualitative Study of AI Applications and Challenges

Cyberattacks are becoming more and more intense in the banking industry (Ryzhkova et al. 2020).Banking industry is trying to adopt artificial intelligence to create cyber defence system so that the unauthorized access and cyber attacks would be minimized.Banks in Qatar acknowledge the threat of cybercrimes and the role of cybersecurity in sustainable growth.Meanwhile, the banking industry is experiencing a major technological disruption.It becomes important to understand the implications of such technologies as artificial intelligence (AI) on the cybersecurity of banks.The present paper aims to explore the impacts of AI on the cybersecurity of banks in Qatar.A thematic analysis of interviews of 9 experts in the banking industry of Qatar was conducted.A qualitative thematic analysis was conducted using NVIVO 12 tool.Four key themes were identified: (1) AI is a major tool for enhancing the cybersecurity of banks in Qatar; (2) banks face challenges in using AI for improving cybersecurity; (3) AI can be used destructively and thus poses a threat to cybersecurity in banks of Qatar; (4) employed AI-based tools have vulnerabilities that can be exploited.Overall, Qatar banks can be expected to face new challenges in the future, due to changes in regulatory frameworks and increasing availability of AI-powered malware.
An Evaluation of Hand-Based Algorithms for Sign Language Recognition

Sign language recognition is an evolving research field in computer vision, assisting communication between hearing disabled people. Hand gestures contain the majority of the information when signing. Focusing on feature extraction methods to obtain the information stored in hand data in sign language recognition may improve classification accuracy. Pose estimation is a popular method for extracting body and hand landmarks. We implement and compare different feature extraction and segmentation algorithms, focusing on the hands only on the LSA64 dataset. To extract hand landmark coordinates, MediaPipe Holistic is implemented on the sign images. Classification is performed using poplar CNN architectures, namely ResNet and a Pruned VGG network. A separate 1D-CNN is utilised to classify hand landmark coordinates extracted using MediaPipe. The best performance was achieved on the unprocessed raw images using a Pruned VGG network with an accuracy of 95.50%. However, the more computationally efficient model using the hand landmark data and 1D-CNN for classification achieved an accuracy of 94.91%.
Evidence-driven model for implementing Blockchain in food supply chains

Blockchain technology has been identified as a possible solution to address critical challenges faced by the food sector.Building on the potential of Blockchain within Food Supply Chains (FSC), this study aims to develop an evidence-based implementation model for Blockchain in the food industry.Innovation Adoption and other prominent theories are integrated to first develop a conceptual framework, which is later validated following an analysis of the qualitative data.Fifteen semistructured expert interviews are used to develop an evidence-driven, applied model for implementing Blockchain; providing detailed insights into typical stages, associated activities, and contextual determinants needed for successful integration.An empirically validated implementation model advances the extant academic literature and further provides a detailed roadmap for food practitioners, while initiating Blockchain projects with their firms and/or supply chains.
Overview of JOKER@CLEF 2022: Automatic Wordplay and Humour Translation Workshop

While humour and wordplay are among the most intensively studied problems in the field of translation studies, they have been almost completely ignored in machine translation. This is partly because most AI-based translation tools require a quality and quantity of training data (e.g., parallel corpora) that has historically been lacking for humour and wordplay. The goal of the JOKER@CLEF 2022 workshop was to bring together translators and computer scientists to work on an evaluation framework for wordplay, including data and metric development, and to foster work on automatic methods for wordplay translation. To this end, we defined three pilot tasks: (1) classify and explain instances of wordplay, (2) translate single terms containing wordplay, and (3) translate entire phrases containing wordplay (punning jokes). This paper describes and discusses each of these pilot tasks, as well as the participating systems and their results.
Review of optical measurement techniques for measuring three-dimensional topography of inner-wall-shaped parts

Inner-wall-shaped parts refer to special parts with semi-closed structures, such as bearings and pipes, that are widely used in various fields. Three-dimensional (3D) topography reflects the geometric characteristics and surface quality of a part, so it is essential to measure it. Optical measurement techniques are efficient, non-destructive, and have become the main methods for measuring such parts. This paper evaluates the optical measurement techniques that meet the 3D topography measurement requirements of inner-wall-shaped parts in different frequency bands. The key points, advantages, and disadvantages of the different measurement techniques and their improvement directions in terms of systems and algorithms are analyzed systematically. The relevant processing algorithms adapted to topography features are also introduced. Finally, this review tries to determine the future development trends of 3D topography measurement technology for inner-wall-shaped parts according to the current research and application status.
Cε-LDE: A lightweight variant of differential evolution algorithm with combined ε constrained method and Lévy flight for constrained optimization problems

Differential evolution (DE) algorithm is popular to tackle real-world optimization problems. The standard DE, however, is not able to solve constrained optimization problems (COPs) for their complicated linear or nonlinear constraints. The transformation of constraints, which is realized via constraint handling technique, has a great impact on dealing with constrained algorithms. In this paper, a lightweight and efficient variant of DE named Cε-LDE is proposed to solve constrained single-objective optimization problems. This study firstly introduces a combined constraint handling method, which is designed based on the difference and relationship between two different ε constrained methods and balances infeasible solution and feasible solution with rules from the perspective of probability allocation. In addition, an extra control parameter is redefined in terms of different initial ε level in consideration of the complexity and diversity of COPs. On the other hand, DE evolves by opposition-based learning initialization and modifications on mutation operator and crossover rate selection, respectively. Mutation strategies frequently named rand/1 and best/1 are adopted and Lévy flight is added to the mutation strategy as a multiplier with scale factor F for a long jump. The crossover rate CR tends to be a larger or smaller stochastic value according to the pros and cons of the selected vector. A set of 28 problems under different dimension settings (D = 10, 30, 50 and 100) used for the CEC 2017 Competition is employed to evaluate the performance of the proposed Cε-LDE. The computational results demonstrate the effectiveness and superiority of the proposed combined ε constrained method. A comparative study of Cε-LDE with state-of-art algorithms is conducted on the obtained experimental results under the rules of CEC 2017 Competition. The ranking outcomes reveal that Cε-LDE is equipped with high competitiveness. Furthermore, the applications in several standard real-life engineering problems verify the effectiveness and practicability of the proposed scheme.
CMAFGAN: A Cross-Modal Attention Fusion based Generative Adversarial Network for attribute word-to-face synthesis

Face synthesis based on attribute words is a novel and challenging topic in computer vision, which has various application potentials in public security and multimedia. Existing attribute vector-to-face (V2F) synthesis methods mainly generate faces based on attribute label vectors that lack rich semantic feature information, which leads to low-quality generated face images. To address this challenge, we advocate attribute word-to-face (W2F) synthesis, using attribute-word sequences that contain rich semantic information as input. A novel Cross-Modal Attention Fusion based Generative Adversarial Network (CMAFGAN) is proposed to generate faces from facial attribute words. CMAFGAN is highlighted by two blocks, cross-modal attention fusion (CMAF) and word feature transformation (WFT), which are proposed to explore the correlation between image features and the corresponding attribute word features. Experimental results on the CelebA and LFW datasets demonstrate that our CMAFGAN achieves state-of-the-art performance, effectively improving the quality of the synthesised faces. In particular, the consistency between the predicted images and input attribute words (R-precision) on the CelebA and LFW datasets achieved 61.24% and 64.46% respectively, which is significantly better than previous methods. In addition, CMAFGAN achieves comparable or better performance than the current best methods of text-to-image synthesis (R-precision 83.41% on caltech-ucsd birds-200-2011, CUB).