Reviewerly/exHarmony huggingface dataset:
The Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation

Recently, the leading performance of human pose estimation is dominated by top-down methods. Being a fundamental component in training and inference, data processing has not been systematically considered in pose estimation community, to the best of our knowledge. In this paper, we focus on this problem and find that the devil of top-down pose estimator is in the biased data processing. Specifically, by investigating the standard data processing in state-of-the-art approaches mainly including data transformation and encoding-decoding, we find that the results obtained by common flipping strategy are unaligned with the original ones in inference. Moreover, there is statistical error in standard encoding-decoding during both training and inference. Two problems couple together and significantly degrade the pose estimation performance. Based on quantitative analyses, we then formulate a principled way to tackle this dilemma. Data is processed in continuous space based on unit length (the intervals between pixels) instead of in discrete space with pixel, and a combined classification and regression approach is adopted to perform encoding-decoding. The Unbiased Data Processing (UDP) for human pose estimation can be achieved by combining the two together. UDP not only boosts the performance of existing methods by a large margin but also plays a important role in result reproducing and future exploration. As a model-agnostic approach, UDP promotes SimpleBaseline-ResNet50-256x192 by 1.5 AP (70.2 to 71.7) and HRNet-W32-256x192 by 1.7 AP (73.5 to 75.2) on COCO test-dev set. The HRNet-W48-384x288 equipped with UDP achieves 76.5 AP and sets a new state-of-the-art for human pose estimation. The source code is publicly available for further research.
Eff-UNet: A Novel Architecture for Semantic Segmentation in Unstructured Environment

Since the last few decades, the number of road causalities has seen continuous growth across the globe. Nowa-days intelligent transportation systems are being developed to enable safe and relaxed driving and scene understanding of the surrounding environment is an integral part of it. While several approaches are being developed for semantic scene segmentation based on deep learning and Convolutional Neural Network (CNN), these approaches assume well structured road infrastructure and driving environment. We focus our work on recent India Driving Lite Dataset (IDD), which contains data from unstructured driving environment and was hosted as an online challenge in NCVPRIPG 2019. We propose a novel architecture named as Eff-UNet which combines the effectiveness of compound scaled EfficientNet as the encoder for feature extraction with UNet decoder for reconstructing the fine-grained segmentation map. High level feature information as well as low level spatial information useful for precise segmentation are combined. The proposed architecture achieved 0.7376 and 0.6276 mean Intersection over Union (mIoU) on validation and test dataset respectively and won first prize in IDD lite segmentation challenge outperforming other approaches in the literature.
Distilling Image Dehazing With Heterogeneous Task Imitation

State-of-the-art deep dehazing models are often difficult in training. Knowledge distillation paves a way to train a student network assisted by a teacher network. However, most knowledge distill methods are used for image classification and segmentation as well as object detection, and few investigate distilling image restoration and use different task for knowledge transfer. In this paper, we propose a knowledge-distill dehazing network which distills image dehazing with the heterogeneous task imitation. In our network, the teacher is an off-the-shelf auto-encoder network and is used for image reconstruction. The dehazing network is trained assisted by the teacher network with the process-oriented learning mechanism. The student network imitates the task of image reconstruction in the teacher network. Moreover, we design a spatial-weighted channel-attention residual block for the student image dehazing network to adaptively learn the content-aware channel level attention and pay more attention to the features for dense hazy regions reconstruction. To evaluate the effectiveness of the proposed method, we compare our method with several state-of-the-art methods on two synthetic and real-world datasets, as well as real hazy images.
MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation

With the success of deep learning in classifying short trimmed videos, more attention has been focused on temporally segmenting and classifying activities in long untrimmed videos. State-of-the-art approaches for action segmentation utilize several layers of temporal convolution and temporal pooling. Despite the capabilities of these approaches in capturing temporal dependencies, their predictions suffer from over-segmentation errors. In this paper, we propose a multi-stage architecture for the temporal action segmentation task that overcomes the limitations of the previous approaches. The first stage generates an initial prediction that is refined by the next ones. In each stage we stack several layers of dilated temporal convolutions covering a large receptive field with few parameters. While this architecture already performs well, lower layers still suffer from a small receptive field. To address this limitation, we propose a dual dilated layer that combines both large and small receptive fields. We further decouple the design of the first stage from the refining stages to address the different requirements of these stages. Extensive evaluation shows the effectiveness of the proposed model in capturing long-range dependencies and recognizing action segments. Our models achieve state-of-the-art results on three datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.
The Hourglass Model Revisited

Recent developments in the field of AI have fostered multidisciplinary research in various disciplines, including computer science, linguistics, and psychology. Intelligence, in fact, is much more than just IQ: it comprises many other kinds of intelligence, including physical intelligence, cultural intelligence, linguistic intelligence, and emotional intelligence (EQ). While traditional classification tasks and standard phenomena in computer science are easy to define, however, emotions are still a rather mysterious subject of study. That is why so many different emotion classifications have been proposed in the literature and there is still no common agreement on a universal emotion categorization model. In this article, we revisit the Hourglass of Emotions, an emotion categorization model optimized for polarity detection, based on some recent empirical evidence in the context of sentiment analysis. This new model does not claim to offer the ultimate emotion categorization but it proves the most effective for the task of sentiment analysis.
Quantum equation of motion for computing molecular excitation energies on a noisy quantum processor

The computation of molecular excitation energies is essential for predicting photo-induced reactions of chemical and technological interest. While the classical computing resources needed for this task scale poorly, quantum algorithms emerge as promising alternatives. In particular, the extension of the variational quantum eigensolver algorithm to the computation of the excitation energies is an attractive option. However, there is currently a lack of such algorithms for correlated molecular systems that is amenable to near-term, noisy hardware. In this work, we propose an extension of the well-established classical equation of motion approach to a quantum algorithm for the calculation of molecular excitation energies on noisy quantum computers. In particular, we demonstrate the efficiency of this approach in the calculation of the excitation energies of the LiH molecule on an IBM Quantum computer.3 MoreReceived 22 January 2020Accepted 23 September 2020DOI:https://doi.org/10.1103/PhysRevResearch.2.043140Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum algorithmsQuantum simulationQuantum Information
Adversarial T-Shirt! Evading Person Detectors in a Physical World

It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person’s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in the digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously.
A Novel Patient-Centric Architectural Framework for Blockchain-Enabled Healthcare Applications

With the proliferation of information and communication technology in every walks of the society, including healthcare services, digitization, and increased sophistication have been gaining pace, digital healthcare alternatives such as electronic healthcare record (EHR) have gained prominence with increased patients' data volume. However, traditional EHR-based systems are plagued by data loss risks, security and immutability consensus over health records, gapped communication among constituted hospitals, and inefficient clinical data retrieval systems, among others. Blockchain has been developed as a decentralized technology that holds the promise to address the aforesaid facilities in EHR-based systems. This article presents a patient-centric design of a decentralized healthcare management system with blockchain-based EHR using javascript-based smart contracts. A working prototype based on hyperledger fabric and composer technology has also been implemented which guarantees the security of the proposed model. Experiments with the hyperledger caliper benchmarking tool provide performance such as latency, throughput, resource utilization, and so on under varied scenarios and control parameters. The results affirm the efficacy of the proposed approach.
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation

Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion to obtain better feature representations to achieve more accurate segmentation. This, however, may not lead to satisfactory results as actual depth data are generally noisy, which might worsen the accuracy as the networks go deeper. In this paper, we propose a unified and efficient Cross-modality Guided Encoder to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively. The key of the proposed architecture is a novel Separation-and-Aggregation Gating operation that jointly filters and recalibrates both representations before cross-modality aggregation. Meanwhile, a Bi-direction Multi-step Propagation strategy is introduced, on the one hand, to help to propagate and fuse information between the two modalities, and on the other hand, to preserve their specificity along the long-term propagation process. Besides, our proposed encoder can be easily injected into the previous encoder-decoder structures to boost their performance on RGB-D semantic segmentation. Our model outperforms state-of-the-arts consistently on both in-door and out-door challenging datasets (Code of this work is available at https://charlescxk.github.io/ ).
How Do AI-driven Chatbots Impact User Experience? Examining Gratifications, Perceived Privacy Risk, Satisfaction, Loyalty, and Continued Use

(2020). How Do AI-driven Chatbots Impact User Experience? Examining Gratifications, Perceived Privacy Risk, Satisfaction, Loyalty, and Continued Use. Journal of Broadcasting & Electronic Media: Vol. 64, No. 4, pp. 592-614.
Lightweight Encryption Technique to Enhance Medical Image Security on Internet of Medical Things Applications

The importance of image security in the field of medical imaging is challenging.Several research works have been conducted to secure medical healthcare images.Encryption, not risking loss of data, is the right solution for image confidentiality.Due to data size limitations, redundancy, and capacity, traditional encryption techniques cannot be applied directly to e-health data, especially when patient data are transferred over the open channels.Therefore, patients may lose the privacy of data contents since images are different from the text because of their two particular factors of loss of data and confidentiality.Researchers have identified such security threats and have proposed several image encryption techniques to mitigate the security problem.However, the study has found that the existing proposed techniques still face application-specific several security problems.Therefore, this paper presents an efficient, lightweight encryption algorithm to develop a secure image encryption technique for the healthcare industry.The proposed lightweight encryption technique employs two permutation techniques to secure medical images.The proposed technique is analyzed, evaluated, and then compared to conventionally encrypted ones in security and execution time.Numerous test images have been used to determine the performance of the proposed algorithm.Several experiments show that the proposed algorithm for image cryptosystems provides better efficiency than conventional techniques.
Contrastive Learning for Sequential Recommendation

Sequential recommendation methods play a crucial role in modern recommender systems because of their ability to capture a user's dynamic interest from her/his historical interactions. Despite their success, we argue that these approaches usually rely on the sequential prediction task to optimize the huge amounts of parameters. They usually suffer from the data sparsity problem, which makes it difficult for them to learn high-quality user representations. To tackle that, inspired by recent advances of contrastive learning techniques in the computer version, we propose a novel multi-task model called \textbf{C}ontrastive \textbf{L}earning for \textbf{S}equential \textbf{Rec}ommendation~(\textbf{CL4SRec}). CL4SRec not only takes advantage of the traditional next item prediction task but also utilizes the contrastive learning framework to derive self-supervision signals from the original user behavior sequences. Therefore, it can extract more meaningful user patterns and further encode the user representation effectively. In addition, we propose three data augmentation approaches to construct self-supervision signals. Extensive experiments on four public datasets demonstrate that CL4SRec achieves state-of-the-art performance over existing baselines by inferring better user representations.
Joint embedding VQA model based on dynamic word vector

The existing joint embedding Visual Question Answering models use different combinations of image characterization, text characterization and feature fusion method, but all the existing models use static word vectors for text characterization. However, in the real language environment, the same word may represent different meanings in different contexts, and may also be used as different grammatical components. These differences cannot be effectively expressed by static word vectors, so there may be semantic and grammatical deviations. In order to solve this problem, our article constructs a joint embedding model based on dynamic word vector-none KB-Specific network (N-KBSN) model which is different from commonly used Visual Question Answering models based on static word vectors. The N-KBSN model consists of three main parts: question text and image feature extraction module, self attention and guided attention module, feature fusion and classifier module. Among them, the key parts of N-KBSN model are: image characterization based on Faster R-CNN, text characterization based on ELMo and feature enhancement based on multi-head attention mechanism. The experimental results show that the N-KBSN constructed in our experiment is better than the other 2017-winner (glove) model and 2019-winner (glove) model. The introduction of dynamic word vector improves the accuracy of the overall results.
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning

Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level. We first utilize a pre-trained detector to detect objects. Then, we train a 3D convolutional neural network to produce discriminative anomaly-specific information by jointly learning multiple proxy tasks: three self-supervised and one based on knowledge distillation. The self-supervised tasks are: (i) discrimination of forward/backward moving objects (arrow of time), (ii) discrimination of objects in consecutive/intermittent frames (motion irregularity) and (iii) reconstruction of object-specific appearance information. The knowledge distillation task takes into account both classification and detection information, generating large prediction discrepancies between teacher and student models when anomalies occur. To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture. Our lightweight architecture outperforms the state-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSD Ped2. Additionally, we perform an ablation study demonstrating the importance of integrating self-supervised learning and normality-specific distillation in a multi-task learning setting.
Cross-Modal Contrastive Learning for Text-to-Image Generation

The output of text-to-image synthesis systems should be coherent, clear, photo-realistic scenes with high semantic fidelity to their conditioned text descriptions. Our Cross-Modal Contrastive Generative Adversarial Network (XMC-GAN) addresses this challenge by maximizing the mutual information between image and text. It does this via multiple contrastive losses which capture inter-modality and intra-modality correspondences. XMC-GAN uses an attentional self-modulation generator, which enforces strong text-image correspondence, and a contrastive discriminator, which acts as a critic as well as a feature encoder for contrastive learning. The quality of XMC-GAN’s output is a major step up from previous models, as we show on three challenging datasets. On MS-COCO, not only does XMC-GAN improve state-of-the-art FID from 24.70 to 9.33, but– more importantly–people prefer XMC-GAN by 77.3% for image quality and 74.1% for image-text alignment, compared to three other recent models. XMC-GAN also generalizes to the challenging Localized Narratives dataset (which has longer, more detailed descriptions), improving state-of-the-art FID from 48.70 to 14.12. Lastly, we train and evaluate XMC-GAN on the challenging Open Images data, establishing a strong benchmark FID score of 26.91.
Review of Wearable Devices and Data Collection Considerations for Connected Health

Wearable sensor technology has gradually extended its usability into a wide range of well-known applications. Wearable sensors can typically assess and quantify the wearer’s physiology and are commonly employed for human activity detection and quantified self-assessment. Wearable sensors are increasingly utilised to monitor patient health, rapidly assist with disease diagnosis, and help predict and often improve patient outcomes. Clinicians use various self-report questionnaires and well-known tests to report patient symptoms and assess their functional ability. These assessments are time consuming and costly and depend on subjective patient recall. Moreover, measurements may not accurately demonstrate the patient’s functional ability whilst at home. Wearable sensors can be used to detect and quantify specific movements in different applications. The volume of data collected by wearable sensors during long-term assessment of ambulatory movement can become immense in tuple size. This paper discusses current techniques used to track and record various human body movements, as well as techniques used to measure activity and sleep from long-term data collected by wearable technology devices.
EDMF: Efficient Deep Matrix Factorization With Review Feature Learning for Industrial Recommender System

Recommendation accuracy is a fundamental problem in the quality of the recommendation system. In this article, we propose an efficient deep matrix factorization (EDMF) with review feature learning for the industrial recommender system. Two characteristics in user’s review are revealed. First, interactivity between the user and the item, which can also be considered as the former’s scoring behavior on the latter, is exploited in a review. Second, the review is only a partial description of the user’s preferences for the item, which is revealed as the sparsity property. Specifically, in the first characteristic, EDMF extracts the interactive features of onefold review by convolutional neural networks with word-attention mechanism. Subsequently, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${L}_{0}$</tex-math></inline-formula> norm is leveraged to constrain the review considering that the review information is a sparse feature, which is the second characteristic. Furthermore, the loss function is constructed by maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> estimation theory, where the interactivity and sparsity property are converted as two prior probability functions. Finally, the alternative minimization algorithm is introduced to optimize the loss functions. Experimental results on several datasets demonstrate that the proposed methods, which show good industrial conversion application prospects, outperform the state-of-the-art methods in terms of effectiveness and efficiency.
MetaSocieties in Metaverse: MetaEconomics and MetaManagement for MetaEnterprises and MetaCities

Welcome to the first issue of the IEEE Transactions on Computational Social Systems (TCSS) of 2022. We would like to take this opportunity to express our sincere thanks to our associate editors, reviewers, authors, and readers for your great support and effort devoted to IEEE TCSS. Happy New Year to you all, and cheers to health, happiness, and high-producing in 2022!