Reviewerly/exHarmony huggingface dataset:
Speech driven facial animation generation based on GAN

A facial animation generation model can generate talking face videos from speech audio clips and face images. The frame sequence in the generated animation should be well synchronized with the source audio clips. A Facial Animation generation model based on an Adversarial Network (FAAN) model is proposed in this paper to generate video-speech synchronized facial animation from real human speech and a face image. The model maps features of the face image and the natural speech to a public space during the encoding process and then generates a frame sequence of a talking face according to the temporal coherence features contained in the speech fragments. To improve the synchronization accuracy between the generated video frame and source audio sequences, the model design should be a conditional least squares GAN, in which the temporal sequence of features extracted from the speech audio clips is inputted as a condition of the sequence discriminator. Furthermore, the temporal sequence of features extracted from the speech audio clips is added to the public space after the audio features are coupled with the facial features. Another conditional GAN, in which the input video frame is used as the condition of the frame discriminator, is designed in this model to improve the authenticity of the generated video frames. The model is ablated using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Frechette Inception Distance (FID). The experimental results show that the PSNR and SSIM scores of the FAAN model are better than those of the latest model on the GRID dataset; on the LRW dataset, the FANN model has the highest PSNR score. Finally, the validity of the proposed model is demonstrated by generating an optical flow diagram of the video frames, which shows that the model can elaborately drive the animation of the mouth in the generated video frames. Source code and videos are available at https://github.com/zjy-2020/Speech2video-FAAN/
A zero-watermark algorithm for multiple images based on visual cryptography and image fusion

At present, it is difficult for the multiple images zero-watermark algorithm to protect all the images in the image set, and repeated operations will reduce the efficiency of the algorithm. To solve these issues, the proposed algorithm can design a reasonable copyright protection scheme according to the number of images in the image set to realize the protection of all images, and reduce the cost of time and storage. The gray-weighted average image fusion method is used to fuse multiple normalized standard images into one image. The LWT(Lifting the Wavelet Transform)-QR decomposition is applied to the effective area of the fusion image to obtain the robust feature image. Non-extended visual cryptography is used to enhance the security of the algorithm. A zero-watermark image is obtained by using the XOR manipulation for the feature image and the public shared image. Experimental results demonstrate that the proposed algorithm has good performance.
A Mask-Guided Transformer Network with Topic Token for Remote Sensing Image Captioning

Remote sensing image captioning aims to describe the content of images using natural language. In contrast with natural images, the scale, distribution, and number of objects generally vary in remote sensing images, making it hard to capture global semantic information and the relationships between objects at different scales. In this paper, in order to improve the accuracy and diversity of captioning, a mask-guided Transformer network with a topic token is proposed. Multi-head attention is introduced to extract features and capture the relationships between objects. On this basis, a topic token is added into the encoder, which represents the scene topic and serves as a prior in the decoder to help us focus better on global semantic information. Moreover, a new Mask-Cross-Entropy strategy is designed in order to improve the diversity of the generated captions, which randomly replaces some input words with a special word (named [Mask]) in the training stage, with the aim of enhancing the model’s learning ability and forcing exploration of uncommon word relations. Experiments on three data sets show that the proposed method can generate captions with high accuracy and diversity, and the experimental results illustrate that the proposed method can outperform state-of-the-art models. Furthermore, the CIDEr score on the RSICD data set increased from 275.49 to 298.39.
The effect of virtual reality on the marketing of residential property

Property technology has ushered in new possibilities for the real estate industry, including the use of virtual reality (VR) technologies by real estate agents when marketing properties. Previous studies have identified factors in determining purchase intents within a virtual setting, but little is known about how such technologies affect homebuyers' purchase decisions. In this study, a family-home purchase decision-making model is used to conceptualise how VR can affect buyers' involvement in the property purchasing process. Using transaction data from Wuhan City, China, the study found that a 1% increase in the number of followers visiting an online property portal resulted in a 21% increase in physical home visits. The study also indicates that VR tours shorten the marketing time of property by 6.4% and narrow the bid-ask spread by 2%, ceteris paribus. These findings suggest that emerging property technologies such as VR can enhance the purchase decision-making process and reshape the role of the real estate agent.
The role of institutional factors and cognitive absorption on students’ satisfaction and performance in online learning during COVID 19

With the rise of the Covid-19 pandemic, there has been a severe negative impact on all aspects of life, whether it be a job, business, health, education, etc. As a result, institutions, schools, colleges and universities are being shut down globally to control the spread of Covid-19. Due to this reason, the mode of education has a dramatic shift from on-campus to online learning with virtual teaching using digital technologies. This sudden shift has elevated the stress level among the students because they were not mentally prepared for it, and hence their academic performance has been adversely affected. So, there needs to figure out the underlying process to make online learning more productive. Thus, to obtain this objective, the present study has integrated the modified Technology Acceptance Model (TAM), Task Technology Fit Model (TTF), DeLone and McLean Model of Information Systems Success (DMISM) and Unified Theory of Acceptance and Use of Technology (UTAUT) model. A sample of 404 students was obtained, where 202 students were from the top ten public sector universities, and 202 were from the top ten private sector universities of Punjab. Structural Equation Modelling (SEM) was used to analyze the hypothesized framework using AMOS. The results reveal that institutional factors positively impact students' performance mediated by user satisfaction and task technology fit. Similarly, institutional factors affect performance through mediation by user satisfaction and actual usage in sequence. Cognitive absorption was used as a moderator between institutional factors and user satisfaction. In the end, theoretical and practical inferences have also been discussed.
FDG-PET to T1 Weighted MRI Translation with 3D Elicit Generative Adversarial Network (E-GAN)

Objective: With the strengths of deep learning, computer-aided diagnosis (CAD) is a hot topic for researchers in medical image analysis. One of the main requirements for training a deep learning model is providing enough data for the network. However, in medical images, due to the difficulties of data collection and data privacy, finding an appropriate dataset (balanced, enough samples, etc.) is quite a challenge. Although image synthesis could be beneficial to overcome this issue, synthesizing 3D images is a hard task. The main objective of this paper is to generate 3D T1 weighted MRI corresponding to FDG-PET. In this study, we propose a separable convolution-based Elicit generative adversarial network (E-GAN). The proposed architecture can reconstruct 3D T1 weighted MRI from 2D high-level features and geometrical information retrieved from a Sobel filter. Experimental results on the ADNI datasets for healthy subjects show that the proposed model improves the quality of images compared with the state of the art. In addition, the evaluation of E-GAN and the state of art methods gives a better result on the structural information (13.73% improvement for PSNR and 22.95% for SSIM compared to Pix2Pix GAN) and textural information (6.9% improvements for homogeneity error in Haralick features compared to Pix2Pix GAN).
How Human-like Behavior of Service Robot Affects Social Distance: A Mediation Model and Cross-Cultural Comparison

Previous studies on the human likeness of service robots have focused mainly on their human-like appearance and used psychological constructs to measure the outcomes of human likeness. Unlike previous studies, this study focused on the human-like behavior of the service robot and used a sociological construct, social distance, to measure the outcome of human likeness. We constructed a conceptual model, with perceived competence and warmth as mediators, based on social-identity theory. The hypotheses were tested through online experiments with 219 participants from China and 180 participants from the US. Similar results emerged for Chinese and American participants in that the high (vs. low) human-like behavior of the service robot caused the participants to have stronger perceptions of competence and warmth, both of which contributed to a smaller social distance between humans and service robots. Perceptions of competence and warmth completely mediated the positive effect of the human-like behavior of the service robot on social distance. Furthermore, Chinese participants showed higher anthropomorphism (perceived human-like behavior) and a stronger perception of warmth and smaller social distance. The perception of competence did not differ across cultures. This study provides suggestions for the human-likeness design of service robots to promote natural interaction between humans and service robots and increase human acceptance of service robots.
The User Experience of Journeys in the Realm of Augmented Reality Television

Augmented Reality Television (ARTV) can take many forms, from AR content displayed outside the TV frame to video-projected TV screens to social TV watching in VR to immersive holograms in the living room. While the user experience (UX) of individual forms of ARTV has been documented before, “journeys” as transitions between such forms have not. In this work, we examine the UX of watching TV when switching between various levels of augmentation. Our findings from an experiment with fourteen participants reveal an UX characterized by high perceived usability, captivation, and involvement with a low to medium workload and a moderate feeling of dissociation from the physical world. We interpret our results in the context of Garrett’s established five-plane model of UX—strategy, scope, structure, skeleton, and surface—and propose a sixth plane, “switch,” which separates conceptually the design of user journeys in ARTV from the specifics of the other UX planes.
Perspectives of public health professionals on border control practices for COVID-19 management in Europe

During the COVID-19 pandemic, internal European borders were temporarily re-established to mitigate the outbreak. Much research on pandemic border control measures has focused on quantifying their effectiveness for infectious disease control as well as on their social consequences for cross-border life in the European Union. However, little attention has been paid to the impacts for the practice and organisation of cross-border public health. To address this gap, the present study analysed the experiences and perspectives of public health professionals working in European border regions regarding border control measures in the pandemic.Qualitative interview-based study.In total, 27 semistructured interviews with public health professionals were conducted in the border regions between Germany, the Netherlands and Belgium. Participants were asked about their perspectives on border control and the spread of COVID-19 in the region. Interviews were performed between December 2020 and April 2021 and carried out in German, English, Dutch and French.Before the COVID-19 pandemic, borders had become largely invisible with extensive cross-border social life and mobility. Participants were sceptical about the role of cross-border mobility as a pandemic driver and consequently the effectiveness of enforcing border control for reducing the spread of COVID-19 in their border regions. At the same time, participants raised concerns about the negative consequences for the social fabric and provision of cross-border public health.Public health professionals highlighted the uncertain role of border control measures for regional infectious disease control in border regions. Rather than border control, sustainable cross-border communication and collaboration is crucial to ensure effective pandemic management in border regions.
Towards Adversarial Attack on Vision-Language Pre-training Models

While vision-language pre-training model (VLP) has shown revolutionary improvements on various vision-language (V+L) tasks, the studies regarding its adversarial robustness remain largely unexplored. This paper studied the adversarial attack on popular VLP models and V+L tasks. First, we analyzed the performance of adversarial attacks under different settings. By examining the influence of different perturbed objects and attack targets, we concluded some key observations as guidance on both designing strong multimodal adversarial attack and constructing robust VLP models. Second, we proposed a novel multimodal attack method on the VLP models called Collaborative Multimodal Adversarial Attack (Co-Attack), which collectively carries out the attacks on the image modality and the text modality. Experimental results demonstrated that the proposed method achieves improved attack performances on different V+L downstream tasks and VLP models. The analysis observations and novel attack method hopefully provide new understanding into the adversarial robustness of VLP models, so as to contribute their safe and reliable deployment in more real-world scenarios.
Practices of Linked Open Data in Archaeology and Their Realisation in Wikidata

In this paper, we introduce Linked Open Data (LOD) in the archaeological domain as a means to connect dispersed data sources and enable cross-querying. The technology behind the design principles and how LOD can be created and published is described to enable less-familiar researchers to understand the presented benefits and drawbacks of LOD. Wikidata is introduced as an open knowledge hub for the creation and dissemination of LOD. Different actors within archaeology have implemented LOD, and we present which challenges have been and are being addressed. A selection of projects showcases how Wikidata is being used by archaeologists to enrich and open their databases to the general public. With this paper, we aim to encourage the creation and re-use of LOD in archaeology, as we believe it offers an improvement on current data publishing practices.
Closing the Locality and Detection Loopholes in Multiparticle Entanglement Self-Testing

First proposed by Mayers and Yao, self-testing provides a certification method to infer the underlying physics of quantum experiments in a black-box scenario. Numerous demonstrations have been reported to self-test various types of entangled states. However, all the multiparticle self-testing experiments reported so far suffer from both detection and locality loopholes. Here, we report the first experimental realization of multiparticle entanglement self-testing closing the locality loophole in a photonic system, and the detection loophole in a superconducting system, respectively. We certify three-party and four-party GHZ states with at least 0.84(1) and 0.86(3) fidelities in a device-independent way. These results can be viewed as a meaningful advance in multiparticle loophole-free self-testing, and also significant progress on the foundations of quantum entanglement certification.
Trusting deep learning natural-language models via local and global explanations

Abstract Despite the high accuracy offered by state-of-the-art deep natural-language models (e.g., LSTM, BERT), their application in real-life settings is still widely limited, as they behave like a black-box to the end-user. Hence, explainability is rapidly becoming a fundamental requirement of future-generation data-driven systems based on deep-learning approaches. Several attempts to fulfill the existing gap between accuracy and interpretability have been made. However, robust and specialized eXplainable Artificial Intelligence solutions, tailored to deep natural-language models, are still missing. We propose a new framework, named T-EBAnO , which provides innovative prediction-local and class-based model-global explanation strategies tailored to deep learning natural-language models. Given a deep NLP model and the textual input data, T-EBAnO provides an objective, human-readable, domain-specific assessment of the reasons behind the automatic decision-making process. Specifically, the framework extracts sets of interpretable features mining the inner knowledge of the model. Then, it quantifies the influence of each feature during the prediction process by exploiting the normalized Perturbation Influence Relation index at the local level and the novel Global Absolute Influence and Global Relative Influence indexes at the global level. The effectiveness and the quality of the local and global explanations obtained with T-EBAnO are proved on an extensive set of experiments addressing different tasks, such as a sentiment-analysis task performed by a fine-tuned BERT model and a toxic-comment classification task performed by an LSTM model. The quality of the explanations proposed by T-EBAnO , and, specifically, the correlation between the influence index and human judgment, has been evaluated by humans in a survey with more than 4000 judgments. To prove the generality of T-EBAnO and its model/task-independent methodology, experiments with other models (ALBERT, ULMFit) on popular public datasets (Ag News and Cola) are also discussed in detail.
Predicting health indicators for open source projects (using hyperparameter optimization)

Software developed on public platform is a source of data that can be used to make predictions about those projects. While the individual developing activity may be random and hard to predict, the developing behavior on project level can be predicted with good accuracy when large groups of developers work together on software projects. To demonstrate this, we use 64,181 months of data from 1,159 GitHub projects to make various predictions about the recent status of those projects (as of April 2020). We find that traditional estimation algorithms make many mistakes. Algorithms like k-nearest neighbors (KNN), support vector regression (SVR), random forest (RFT), linear regression (LNR), and regression trees (CART) have high error rates. But that error rate can be greatly reduced using hyperparameter optimization. To the best of our knowledge, this is the largest study yet conducted, using recent data for predicting multiple health indicators of open-source projects. To facilitate open science (and replications and extensions of this work), all our materials are available online at https://github.com/arennax/Health_Indicator_Prediction .
Markov‐GAN: Markov image enhancement method for malicious encrypted traffic classification

The rapidly growing encrypted traffic hides a large number of malicious behaviours. The difficulty of collecting and labelling encrypted traffic makes the class distribution of dataset seriously imbalanced, which leads to the poor generalisation ability of the classification model. To solve this problem, a new representation learning method in encrypted traffic and its diversity enhancement model are proposed, which uses the diversity of images to represent the diversity of traffic samples. First, the encrypted traffic is transformed into Markov images. Then, a diversity maximisation Markov-GAN based on the Simpson index is designed to generate new Markov images. Finally, the balanced Markov image set is sent to the CNN for classification. Experimental results show that the proposed method can predict the whole dataset space with only a few original samples. And the classification accuracies under different imbalance degrees are significantly improved, all of which are over 90%. The enhanced Markov image set can effectively alleviate performance generalisation deviation caused by different network depths. Even an ordinary CNN has almost the same classification effect as VGG13 and VGG16. Compared with other data enhancement methods, the Markov-GAN only needs to balance the transform domain dataset, which is lightweight, easy to train and has stronger amplification ability.
Nonparametric regression for interval-valued data based on local linear smoothing approach

In this paper, we propose an interval local linear method (ILLM) to fit the regression model with interval-valued explanatory and response variables. The proposed method has no restriction on the form of the regression function. Moreover, it reduces the boundary effect of interval kernel method. Some experimental studies including two simulations and four real datasets are examined to evaluate the proposed method. Experimental results show that our method has higher predictive accuracy than some existed methods, including the center and range method, the interval least absolute method, the interval kernel method, multi-output support vector regression and the interval multilayer perceptron.
Toward Physics-Guided Safe Deep Reinforcement Learning for Green Data Center Cooling Control

Deep reinforcement learning (DRL) has shown good performance in tackling Markov decision process (MDP) problems. As DRL opti-mizes a long-term reward, it is a promising approach to improving the energy efficiency of data center cooling. However, enforcement of thermal safety constraint during DRL's state exploration is a main challenge. The widely adopted reward shaping approach adds negative reward when the exploratory action results in unsafety. Thus, it needs to experience sufficient unsafe states before it learns how to prevent unsafety. In this paper, we propose a safety-aware DRL framework for single-hall data center cooling control. It applies offline imitation learning and online post-hoc rectification to holis-tically prevent thermal unsafety during online DRL. In particular, the post-hoc rectification searches for the minimum modification to the DRL-recommended action such that the rectified action will not result in unsafety. The rectification is designed based on a thermal state transition model that is fitted using historical safe operation traces and able to extrapolate the transitions to unsafe states ex-plored by DRL. Extensive evaluation for chilled water and direct expansion cooled data centers in two climate conditions shows that our approach saves 22.7% to 26.6% total data center power compared with conventional control, reduces safety violations by 94.5% to 99% compared with reward shaping.