Reviewerly/exHarmony huggingface dataset:
Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution

3D human pose estimation (HPE) is crucial in many fields, such as human behavior analysis, augmented reality/virtual reality (AR/VR) applications, and self-driving industry. Videos that contain multiple potentially occluded people captured from freely moving monocular cameras are very common in realworld scenarios, while 3D HPE for such scenarios is quite challenging, partially because there is a lack of such data with accurate 3D ground truth labels in existing datasets. In this paper, we propose a temporal regression network with a gated convolution module to transform 2D joints to 3D and recover the missing occluded joints in the meantime. A simple yet effective localization approach is further conducted to transform the normalized pose to the global trajectory. To verify the effectiveness of our approach, we also collect a new moving camera multi-human (MMHuman) dataset that includes multiple people with heavy occlusion captured by moving cameras. The 3D ground truth joints are provided by accurate motion capture (MoCap) system. From the experiments on static-camera based Human3.6M data and our own collected moving-camera based data, we show that our proposed method outperforms most state-of-the-art 2D-to-3D pose estimation methods, especially for the scenarios with heavy occlusions.
Speech Rate Adjustments in Conversations With an Amazon Alexa Socialbot

This paper investigates users’ speech rate adjustments during conversations with an Amazon Alexa socialbot in response to situational (in-lab vs. at-home) and communicative (ASR comprehension errors) factors. We collected user interaction studies and measured speech rate at each turn in the conversation and in baseline productions (collected prior to the interaction). Overall, we find that users slow their speech rate when talking to the bot, relative to their pre-interaction productions, consistent with hyperarticulation. Speakers use an even slower speech rate in the in-lab setting (relative to at-home). We also see evidence for turn-level entrainment: the user follows the directionality of Alexa’s changes in rate in the immediately preceding turn. Yet, we do not see differences in hyperarticulation or entrainment in response to ASR errors, or on the basis of user ratings of the interaction. Overall, this work has implications for human-computer interaction and theories of linguistic adaptation and entrainment.
Medication Name Comprehension of Intelligent Virtual Assistants: A Comparison of Amazon Alexa, Google Assistant, and Apple Siri Between 2019 and 2021

The current study was a replication and comparison of our previous research which examined the comprehension accuracy of popular intelligent virtual assistants, including Amazon Alexa, Google Assistant, and Apple Siri for recognizing the generic and brand names of the top 50 most dispensed medications in the United States. Using the exact same voice recordings from 2019, audio clips of 46 participants were played back to each device in 2021. Google Assistant achieved the highest comprehension accuracy for both brand medication names (86.0%) and generic medication names (84.3%), followed by Apple Siri (brand names = 78.4%, generic names = 75.0%), and the lowest accuracy by Amazon Alexa (brand names 64.2%, generic names = 66.7%). These findings represent the same trend of results as our previous research, but reveal significant increases of ~10–24% in performance for Amazon Alexa and Apple Siri over the past 2 years. This indicates that the artificial intelligence software algorithms have improved to better recognize the speech characteristics of complex medication names, which has important implications for telemedicine and digital healthcare services.
Application of Blockchain in Education: GDPR-Compliant and Scalable Certification and Verification of Academic Information

Blockchain technologies are awakening in recent years the interest of different actors in various sectors and, among them, the education field, which is studying the application of these technologies to improve information traceability, accountability, and integrity, while guaranteeing its privacy, transparency, robustness, trustworthiness, and authenticity. Different interesting proposals and projects were launched and are currently being developed. Nevertheless, there are still issues not adequately addressed, such as scalability, privacy, and compliance with international regulations such as the General Data Protection Regulation in Europe. This paper analyzes the application of blockchain technologies and related challenges to issue and verify educational data and proposes an innovative solution to tackle them. The proposed model supports the issuance, storage, and verification of different types of academic information, both formal and informal, and complies with applicable regulations, protecting the privacy of users’ personal data. This proposal also addresses the scalability challenges and paves the way for a global academic certification system.
Secure and efficient computing resource management in blockchain-based vehicular fog computing

In vehicular fog computing (VFC), the resource transactions in the Internet of Vehicles (IoV) have become a novel resource management scheme that can improve system resource utilization and the quality of vehicle services. In this paper, in order to improve the security and fairness of resource transactions, we design a blockchain-based resource management scheme for VFC. First, we propose the concept of resource coin (RC) and develop a blockchain-based secure computing reource trading mechanism in terms of RC. As a node of the blockchain network, the roadside unit (RSU) participates in verifying the legitimacy of transactions and the creation of new blocks. Next, we propose a resource management scheme based on contract theory, encouraging parked vehicles to contribute computing resource so that RSU could complete proof of work (PoW) quickly, improve the success probability of block creation and get RC rewards. We use the gradient descent method to solve the computing resource utilization that can maximize the RC revenue of RSUs and vehicles during the block creation. Finally, the performance of this model is validated in simulation result and analysis.
Restoration of non-structural damaged murals in Shenzhen Bao’an based on a generator–discriminator network

Abstract Shenzhen is a modern metropolis, but it hides a variety of valuable cultural heritage, such as ancient murals. How to effectively preserve and repair the murals is a worthy of discussion question. Here, we propose a generation-discriminator network model based on artificial intelligence algorithms to perform digital image restoration of ancient damaged murals. In adversarial learning, this study optimizes the discriminative network model. First, the real mural images and damaged images are spliced together as input to the discriminator network. The network uses a 5-layer encoder unit to down-sample the 1024 × 1024 × 3 image to 32 × 32 × 256. Then, we connect a layer of ZeroPadding2D to expand the image to 34 × 34 × 256, and pass the Conv2D layer, down-sample to 31 × 31 × 256, perform batch normalization, and repeat the above steps to get a 30 × 30 × 1 matrix. Finally, this part of the loss is emphasized in the loss function as needed to improve the texture detail information of the image generated by the Generator. The experimental results show that compared with the traditional algorithm, the PSNR value of the algorithm proposed in this paper can be increased by 5.86 db at most. The SSIM value increased by 0.13. Judging from subjective vision. The proposed algorithm can effectively repair damaged murals with dot-like damage and complex texture structures. The algorithm we proposed may be helpful for the digital restoration of ancient murals, and may also provide reference for mural restoration workers.
Exploring Sparse Expert Models and Beyond

Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling. Still it is a mystery how MoE layers bring quality gains by leveraging the parameters with sparse activation. In this work, we investigate several key factors in sparse expert models. We observe that load imbalance may not be a significant problem affecting model quality, contrary to the perspectives of recent studies, while the number of sparsely activated experts $k$ and expert capacity $C$ in top-$k$ routing can significantly make a difference in this context. Furthermore, we take a step forward to propose a simple method called expert prototyping that splits experts into different prototypes and applies $k$ top-$1$ routing. This strategy improves the model quality but maintains constant computational costs, and our further exploration on extremely large-scale models reflects that it is more effective in training larger models. We push the model scale to over $1$ trillion parameters and implement it on solely $480$ NVIDIA V100-32GB GPUs, in comparison with the recent SOTAs on $2048$ TPU cores. The proposed giant model achieves substantial speedup in convergence over the same-size baseline.
Extending Multi-Document Summarization Evaluation to the Interactive Setting

Allowing users to interact with multidocument summarizers is a promising direction towards improving and customizing summary results.Different ideas for interactive summarization have been proposed in previous work but these solutions are highly divergent and incomparable.In this paper, we develop an end-to-end evaluation framework for interactive summarization, focusing on expansion-based interaction, which considers the accumulating information along a user session.Our framework includes a procedure of collecting real user sessions, as well as evaluation measures relying on summarization standards, but adapted to reflect interaction.All of our solutions and resources are available publicly as a benchmark, allowing comparison of future developments in interactive summarization, and spurring progress in its methodological evaluation.We demonstrate the use of our framework by evaluating and comparing baseline implementations that we developed for this purpose, which will serve as part of our benchmark.Our extensive experimentation and analysis motivate the proposed evaluation framework design and support its viability.
Causal Effects of Linguistic Properties

We consider the problem of using observational data to estimate the causal effects of linguistic properties.For example, does writing a complaint politely lead to a faster response time?How much will a positive product review increase sales?This paper addresses two technical challenges related to the problem before developing a practical method.First, we formalize the causal quantity of interest as the effect of a writer's intent, and establish the assumptions necessary to identify this from observational data.Second, in practice, we only have access to noisy proxies for the linguistic properties of interest-e.g., predictions from classifiers and lexicons.We propose an estimator for this setting and prove that its bias is bounded when we perform an adjustment for the text.Based on these results, we introduce TEXTCAUSE, an algorithm for estimating causal effects of linguistic properties.The method leverages (1) distant supervision to improve the quality of noisy proxies, and (2) a pre-trained language model (BERT) to adjust for the text.We show that the proposed method outperforms related approaches when estimating the effect of Amazon review sentiment on semi-simulated sales figures.Finally, we present an applied case study investigating the effects of complaint politeness on bureaucratic response times.
Medical Information Mart for Intensive Care: A Foundation for the Fusion of Artificial Intelligence and Real-World Data

OPINION article Front. Artif. Intell., 31 May 2021Sec. Medicine and Public Health https://doi.org/10.3389/frai.2021.691626
On Physical-Layer Authentication via Online Transfer Learning

This article introduces a novel physical-layer (PHY-layer) authentication scheme, called transfer learning-based PHY-layer authentication (TL-PHA), aiming to achieve fast online user authentication that is highly desired for latency-sensitive applications such as edge computing. The proposed TL-PHA scheme is characterized by incorporating with a novel convolutional neural network architecture, namely, the triple-pool network (TP-Net), for achieving lightweight and online classification, as well as effective data augmentation methods for generation of data set samples for the network model training. To assess the performance of the proposed scheme, we conducted two sets of experiments, including the one using computer-simulated channel data and the other utilizing real experiment data generated by our wireless testbed. The results demonstrate the superiority of the proposed scheme in terms of authentication accuracy, detection rate, and training complexity compared to all the considered counterparts.
CMCL 2021 Shared Task on Eye-Tracking Prediction

Eye-tracking data from reading represent an important resource for both linguistics and natural language processing.The ability to accurately model gaze features is crucial to advance our understanding of language processing.This paper describes the Shared Task on Eye-Tracking Data Prediction, jointly organized with the eleventh edition of the Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2021).The goal of the task is to predict 5 different token-level eyetracking metrics from the Zurich Cognitive Language Processing Corpus (ZuCo).Eyetracking data were recorded during natural reading of English sentences.In total, we received submissions from 13 registered teams, whose systems include boosting algorithms with handcrafted features, neural models leveraging transformer language models, or hybrid approaches.The winning system used a range of linguistic and psychometric features in a gradient boosting framework.
WRIME: A New Dataset for Emotional Intensity Estimation with Subjective and Objective Annotations

We annotate 17,000 SNS posts with both the writer's subjective emotional intensity and the reader's objective one to construct a Japanese emotion analysis dataset.In this study, we explore the difference between the emotional intensity of the writer and that of the readers with this dataset.We found that the reader cannot fully detect the emotions of the writer, especially anger and trust.In addition, experimental results in estimating the emotional intensity show that it is more difficult to estimate the writer's subjective labels than the readers'.The large gap between the subjective and objective emotions implies the complexity of the mapping from a post to the subjective emotional intensities, which also leads to a lower performance with machine learning models.
Multilingual BERT Post-Pretraining Alignment

We propose a simple method to align multilingual contextual embeddings as a postpretraining step for improved cross-lingual transferability of the pretrained language models.Using parallel data, our method aligns embeddings on the word level through the recently proposed Translation Language Modeling objective as well as on the sentence level via contrastive learning and random input shuffling.We also perform sentence-level code-switching with English when finetuning on downstream tasks.On XNLI, our best model (initialized from mBERT) improves over mBERT by 4.7% in the zero-shot setting and achieves comparable result to XLM for translate-train while using less than 18% of the same parallel data and 31% fewer model parameters.On MLQA, our model outperforms XLM-R Base , which has 57% more parameters than ours.
Enhancing users involvement in architectural design using mobile augmented reality

Purpose Mobile augmented reality (MAR) is one of the advanced three-dimensional (3D) representation tools that has been recently utilized in the construction industry. This paper aims to assess a user’s involvement levels through MAR application that has been experimented against traditional involvement techniques through an existing facility, plan re-designing scenario. Design/methodology/approach Through reviewing related literature studies in the MAR field, an application has been developed that can superimpose real design alternatives on paper-based markers, allowing for flexible wall positioning, interior and exterior wall material application. As such, an enhanced user involvement experience is created. To measure user involvement levels, the application is experimented with 33 participants having the British University in Egypt’s library building as a case study, followed by survey questionnaires to gather and evaluate user responses. Findings The results of the analyzed data using SPSS indicated that MAR showed a positive impact on enhancing user involvement and better understanding of design projects. It also allowed users to produce different design alternatives in comparison to the traditional involvement approaches where users showed low design interaction and understanding. Originality/value The interactive features of the proposed application facilitate implementing ideas in design of construction projects that require user involvement.
Mirror-assisted multiview DIC for 360° panoramic large deformation measurement

In this paper, we propose a new technique to achieve 360° panoramic large deformation measurement with the help of trinocular digital image correlation (3DT-DIC) system assisted by two mirrors. Two reflecting mirrors can image the rear of the specimen with virtual reality (secondary reflection). The trinocular vision system simultaneously captures the deformation images of the front and rear of the specimen and completes 360° profile reconstruction and large deformation measurement. The 3DT-DIC system can effectively solve the problems of field-of-view occlusion and receding correlation during the large deformation and distortion of the specimen. A printed speckle pattern for precise positioning of the two reflecting mirrors’ poses and virtual reality transformation during the mirror imaging are introduced. Static and dynamic experiments are conducted separately. Experimental results show that the proposed method has better 360° panoramic reconstruction and stress–strain measurement capability. Compared with the binocular DIC system, the proposed method can calculate more effective points and has better 3D reconstruction capability under distorted large deformation. The proposed method can effectively solve the problems of field-of-view occlusion and receding correlation under large deformation. It provides a new idea for realizing 360° panoramic large deformation measurement.
Project Gradient Descent Adversarial Attack against Multisource Remote Sensing Image Scene Classification

Deep learning technology (a deeper and optimized network structure) and remote sensing imaging (i.e., the more multisource and the more multicategory remote sensing data) have developed rapidly. Although the deep convolutional neural network (CNN) has achieved state-of-the-art performance on remote sensing image (RSI) scene classification, the existence of adversarial attacks poses a potential security threat to the RSI scene classification task based on CNN. The corresponding adversarial samples can be generated by adding a small perturbation to the original images. Feeding the CNN-based classifier with the adversarial samples leads to the classifier misclassify with high confidence. To achieve a higher attack success rate against scene classification based on CNN, we introduce the projected gradient descent method to generate adversarial remote sensing images. Then, we select several mainstream CNN-based classifiers as the attacked models to demonstrate the effectiveness of our method. The experimental results show that our proposed method can dramatically reduce the classification accuracy under untargeted and targeted attacks. Furthermore, we also evaluate the quality of the generated adversarial images by visual and quantitative comparisons. The results show that our method can generate the imperceptible adversarial samples and has a stronger attack ability for the RSI scene classification.
A Multi-Model Approach for User Portrait

Age, gender, educational background, and so on are the most basic attributes for identifying and portraying users. It is also possible to conduct in-depth mining analysis and high-level predictions based on such attributes to learn users’ preferences and personalities so as to enhance users’ online experience and to realize personalized services in real applications. In this paper, we propose using classification algorithms in machine learning to predict users’ demographic attributes, such as gender, age, and educational background, based on one month of data collected with the Sogou search engine with the goal of making user portraits. A multi-model approach using the fusion algorithms is adopted and hereby described in the paper. The proposed model is a two-stage structure using one month of data with demographic labels as the training data. The first stage of the structure is based on traditional machine learning models and neural network models, whereas the second one is a combination of the models from the first stage. Experimental results show that our proposed multi-model method can achieve more accurate results than the single-model methods in predicting user attributes. The proposed approach also has stronger generalization ability in predicting users’ demographic attributes, making it more adequate to profile users.
Having Difficulties Reading the Facial Expression of Older Individuals? Blame It on the Facial Muscles, Not the Wrinkles

Previous studies have found it is more difficult identifying an emotional expression displayed by an older than a younger face. It is unknown whether this is caused by age-related changes such as wrinkles and folds interfering with perception, or by the aging of facial muscles, potentially reducing the ability of older individuals to display an interpretable expression. To discriminate between these two possibilities, participants attempted to identify facial expressions under different conditions. To control for the variables (wrinkles/folds vs facial muscles), we used Generative Adversarial Networks to make faces look older or younger. Based upon behavior data collected from 28 individuals, our model predicts that the odds of correctly identifying the expressed emotion of a face reduced 16.2% when younger faces (condition 1) are artificially aged (condition 3). Replacing the younger faces with natural old-looking faces (Condition 2), however, results in an even stronger effect (odds of correct identification decreased by 50.9%). Counterintuitively, making old faces (Condition 2) look young (Condition 4) results in the largest negative effect (odds of correct identification decreased by 74.8% compared with natural young faces). Taken together, these results suggest that both age-related decline in the facial muscles’ ability to express facial emotions and age-related physical changes in the face, explain why it is difficult to recognize facial expressions from older faces; the effect of the former, however, is much stronger than that of the latter. Facial muscle exercises, therefore, might improve the capacity to convey facial emotional expressions in the elderly.