Reviewerly/exHarmony huggingface dataset:
HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces

In this paper, we present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity, driven by a target facial pose. Existing state-of-the-art face reenactment methods train controllable generative models that learn to synthesize realistic facial images, yet producing reenacted faces that are prone to significant visual artifacts, especially under the challenging condition of extreme head pose changes, or requiring expensive few-shot fine-tuning to better preserve the source identity characteristics. We propose to address these limitations by leveraging the photorealistic generation ability and the disentangled properties of a pretrained StyleGAN2 generator, by first inverting the real images into its latent space and then using a hypernetwork to perform: (i) refinement of the source identity characteristics and (ii) facial pose re-targeting, eliminating this way the dependence on external editing methods that typically produce artifacts. Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring any subject-specific fine-tuning. We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2, demonstrating the superiority of our approach in producing artifact-free images, exhibiting remarkable robustness even under extreme head pose changes. We make the code and the pretrained models publicly available at: https://github.com/StelaBou/HyperReenact.
Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Denoising diffusion models (DDMs) have attracted attention for their exceptional generation quality and diversity. This success is largely attributed to the use of class- or text-conditional diffusion guidance methods, such as classifier and classifier-free guidance. In this paper, we present a more comprehensive perspective that goes beyond the traditional guidance methods. From this generalized perspective, we introduce novel condition- and training-free strategies to enhance the quality of generated images. As a simple solution, blur guidance improves the suitability of intermediate samples for their fine-scale information and structures, enabling diffusion models to generate higher quality samples with a moderate guidance scale. Improving upon this, Self-Attention Guidance (SAG) uses the intermediate self-attention maps of diffusion models to enhance their stability and efficacy. Specifically, SAG adversarially blurs only the regions that diffusion models attend to at each iteration and guides them accordingly. Our experimental re sults show that our SAG improves the performance of various diffusion models, including ADM, IDDPM, Stable Diffusion, and DiT. Moreover, combining SAG with conventional guidance methods leads to further improvement.
3D Segmentation of Humans in Point Clouds with Synthetic Data

Segmenting humans in 3D indoor scenes has become increasingly important with the rise of human-centered robotics and AR/VR applications. To this end, we propose the task of joint 3D human semantic segmentation, instance segmentation and multi-human body-part segmentation. Few works have attempted to directly segment humans in cluttered 3D scenes, which is largely due to the lack of annotated training data of humans interacting with 3D scenes. We address this challenge and propose a framework for generating training data of synthetic humans interacting with real 3D scenes. Furthermore, we propose a novel transformer-based model, Human3D, which is the first end-to-end model for segmenting multiple human instances and their body-parts in a unified manner. The key advantage of our synthetic data generation framework is its ability to generate diverse and realistic human-scene interactions, with highly accurate ground truth. Our experiments show that pre-training on synthetic data improves performance on a wide variety of 3D human segmentation tasks. Finally, we demonstrate that Human3D outperforms even task-specific state-of-the-art 3D segmentation methods.
What can Discriminator do? Towards Box-free Ownership Verification of Generative Adversarial Networks

In recent decades, Generative Adversarial Network (GAN) and its variants have achieved unprecedented success in image synthesis. However, well-trained GANs are under the threat of illegal steal or leakage. The prior studies on remote ownership verification assume a black-box setting where the defender can query the suspicious model with specific inputs, which we identify is not enough for generation tasks. To this end, in this paper, we propose a novel IP protection scheme for GANs where ownership verification can be done by checking outputs only, without choosing the inputs (i.e., box-free setting). Specifically, we make use of the unexploited potential of the discriminator to learn a hypersphere that captures the unique distribution learned by the paired generator. Extensive evaluations on two popular GAN tasks and more than 10 GAN architectures demonstrate our proposed scheme to effectively verify the ownership. Our proposed scheme shown to be immune to popular input-based removal attacks and robust against other existing attacks. The source code and models are available at https://github.com/AbstractTeen/gan_ownership_verification.
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions

Audio-driven portrait animation aims to synthesize portrait videos that are conditioned by given audio. Animating high-fidelity and multimodal video portraits has a variety of applications. Previous methods have attempted to capture different motion modes and generate high-fidelity portrait videos by training different models or sampling signals from given videos. However, lacking correlation learning between lip-sync and other movements (e.g., head pose/eye blinking) usually leads to unnatural results. In this paper, we propose a unified system for multi-person, diverse, and high-fidelity talking portrait generation. Our method contains three stages, i.e., 1) Mapping-Once network with Dual Attentions (MODA) generates talking representation from given audio. In MODA, we design a dual-attention module to encode accurate mouth movements and diverse modalities. 2) Facial composer network generates dense and detailed face landmarks, and 3) temporal-guided renderer syntheses stable videos. Extensive evaluations demonstrate that the proposed system produces more natural and realistic video portraits compared to previous methods.
RecursiveDet: End-to-End Region-based Recursive Object Detection

End-to-end region-based object detectors like Sparse R-CNN usually have multiple cascade bounding box decoding stages, which refine the current predictions according to their previous results. Model parameters within each stage are independent, evolving a huge cost. In this paper, we find the general setting of decoding stages is actually redundant. By simply sharing parameters and making a recursive decoder, the detector already obtains a significant improvement. The recursive decoder can be further enhanced by positional encoding (PE) of the proposal box, which makes it aware of the exact locations and sizes of input bounding boxes, thus becoming adaptive to proposals from different stages during the recursion. Moreover, we also design centerness-based PE to distinguish the RoI feature element and dynamic convolution kernels at different positions within the bounding box. To validate the effectiveness of the proposed method, we conduct intensive ablations and build the full model on three recent mainstream region-based detectors. The RecusiveDet is able to achieve obvious performance boosts with even fewer model parameters and slightly increased computation cost. Codes are available at https://github.com/bravezzzzzz/RecursiveDet.
MUVA: A New Large-Scale Benchmark for Multi-view Amodal Instance Segmentation in the Shopping Scenario

Amodal Instance Segmentation (AIS) endeavors to accurately deduce complete object shapes that are partially or fully occluded. However, the inherent ill-posed nature of single-view datasets poses challenges in determining occluded shapes. A multi-view framework may help alleviate this problem, as humans often adjust their perspective when encountering occluded objects. At present, this approach has not yet been explored by existing methods and datasets. To bridge this gap, we propose a new task called Multi-view Amodal Instance Segmentation (MAIS) and introduce the MUVA dataset, the first MUlti-View AIS dataset that takes the shopping scenario as instantiation. MUVA provides comprehensive annotations, including multi-view amodal/visible segmentation masks, 3D models, and depth maps, making it the largest image-level AIS dataset in terms of both the number of images and instances. Additionally, we propose a new method for aggregating representative features across different instances and views, which demonstrates promising results in accurately predicting occluded objects from one viewpoint by leveraging information from other viewpoints. Besides, we also demonstrate that MUVA can benefit the AIS task in real-world scenarios. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Polar Constrained Image Stitching Algorithm for Unmanned Aerial Vehicle Based on Crossing Area

Stitching the aerial images from an Unmanned Aerial Vehicle (UAV) is usually needed to obtain a more comprehensive image information, but the image stitching technology is limited by the requirements of stitching quality and stitching speed.This study proposed a method to achieve good quality, fast stitching speed, and good robustness in image stitching.A large number of feature points in the intersection region were extracted and incorrect feature matching point pairs were accurately filtered out.The area with large cross area of multiple images was extracted as the target area of feature points to be extracted.The Scale-Invariant Feature Transform (SIFT) algorithm was employed to extract the feature points in the image which was not invariant with illumination, environment, and scale.Given that Brute Force (BF) matching soundly produces large number of wrong feature point pairs, the polar constraint was added to improve the matching accuracy, and the feature point pairs were further filtered through the Progressive Sample Consensus (PROSAC) algorithm.Finally, the homograph transformation matrix was used for image registration, and the fusion algorithm was applied to naturally fuse the two images together.Results demonstrate that this algorithm largely reduces the computational complexity, improves the speed by 0.124% compared with other algorithms, and improves the performance of Root Mean Square Error (RMSE) and Mean Square Error (MAE) by 0.224% and 0.173%, respectively.The stitched image quality is obviously better than the general algorithm process.The proposed method provides a certain reference for UAV aerial image Mosaic methods.
CaT: Balanced Continual Graph Learning with Graph Condensation

Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Since the model easily forgets previously learned knowledge when training with new-coming data, the catastrophic forgetting problem has been the major focus in CGL. Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution of historical data. After updating the model, a new replayed graph sampled from the incoming graph will be added to the existing memory bank. Despite these methods are intuitive and effective for the CGL, two issues are identified in this paper. Firstly, most samplingbased methods struggle to fully capture the historical distribution when the storage budget is tight. Secondly, a significant data imbalance exists in terms of the scales of the complex newcoming graph data and the lightweight memory bank, resulting in unbalanced training. To solve these issues, a Condense and Train (CaT) framework is proposed in this paper. Prior to each model update, the new-coming graph is condensed to a small yet informative synthesised replayed graph, which is then stored in a Condensed Graph Memory with historical replay graphs. In the continual learning phase, a Training in Memory scheme is used to update the model directly with the Condensed Graph Memory rather than the whole new-coming graph, which alleviates the data imbalance problem. Extensive experiments conducted on four benchmark datasets successfully demonstrate superior performances of the proposed CaT framework in terms of effectiveness and efficiency. The code has been released on https://github.com/superallen13/CaT-CGL.
Method and means of testing specialized components of a mobile robotics platform at operating clock frequencies

Processes of development of test scenarios, selection of technological means of testing hardware and software of specialized components of mobile robotic platform are analyzed. The methods and means of testing specialized components of the mobile robotic platform when operating at working clock frequencies are studied. It is improved the method of testing the hardware and software of the specialized components of the mobile robotics platform. This improvement is made due to the development of specialized scenarios and adaptation of the technological environment to the requirements of a specific application, ensures an increase in the quality of testing in real time. It is shown that the main stages of testing specialized hardware at working clock frequencies are: the development of a test plan, setting of a working clock frequency, creation of a test environment, execution of tests, comparison of test results with reference results, analysis of comparison results. For testing, two environments and two test scenarios are developed: testing of means of encryption and masking of control commands of the mobile robotics platform; testing means of unmasking and deciphering control commands of a mobile robotics platform. It is developed a neurofuzzy control system for the autonomous control of the movement of a wheeled mobile robotic platform, the main components of which are intelligent remote navigation sensors, a rule base, fuzzification, decision-making and defuzzification blocks. The structure of means and scenarios for testing blocks of neuro-like encryption/decryption and masking/unmasking of mobile robotic platform control commands are developed, which provide joint testing of both software and hardware at working clock frequencies. It is developed the structure of tools and scenarios for testing the fuzzy mobile robotic platform control system, which are focused on sequential testing of fuzzification, decision-making, and defuzzification blocks and provide joint testing of both software and hardware tools at working clock frequencies in real time. Using the improved method, testing of the mobile robotic platform control system is performed, which confirmed the feasibility of the chosen approach. The implementation of the latest methods and means of testing specialized components of complex systems saves time and financial costs and contributes to the long-term operation of systems as a whole as a result of identifying problems in the process of their creation.
The Effects of Entrepreneurial Leadership, and Transformational Leadership on Innovation Behavior

This study examines the impact of entrepreneurial and transformational leadership on employee innovative behaviour, mediated by an innovation climate with intellectual agility and moderated by intrinsic motivation. Data from 341 full-time IT sector employees in Pakistan were collected using a self-administered questionnaire. Structural equation modelling was employed for analysis. Results indicate that both leadership styles enhance employees' intellectual agility and innovative behaviour. The study underscores the significant role of innovation climate in fostering innovative behaviour. Furthermore, intrinsic motivation moderates the relationship between transformational leadership and innovative behaviour. These findings empower leaders to foster innovation, cultivate conducive environments, and encourage open idea-sharing. The study sheds light on implications, limitations, and avenues for future research.
Webcam Eye Tracking for Desktop and Mobile Devices: A Systematic Review

Building the Internet of Behaviors (IOB) obviously requires capturing human behavior. Sensor input from eye tracking has been widely used for profiling in market research, adaptive user interfaces, and other smart systems, but requires dedicated hardware. The wide spread of webcams in consumer devices like phones, tablets, notebooks, and smart TVs has fostered eye tracking with commodity cameras. In this paper, we present a systematic review across the IEEE and ACM databases -- complemented by snowballing and input from eye tracking experts at CHI 2021 -- to list and characterize publicly available webcam eye trackers that estimate the point-of-regard on devices with no additional hardware. Information from articles was supplemented by searching author websites and code repositories, and contacting authors. 16 eye trackers were found that can be used. The restrictions regarding license terms and technical performance are presented, enabling developers to choose an appropriate software for their IoB application.
Not Everything is a Metaverse?! A Practitioners Perspective on Characterizing Metaverse Platforms

Organizations claim to host what is called a metaverse– an extended version our real world. First rudimental realizations of such metaverses can be found throughout the internet, e.g., Epic Game’s Fortnite. At the same time, research and practice struggle to specify what a metaverse truly is and how we can characterize it. With our work, we analyze the proximity of the realization of a holistic metaverse platform and present the results of a qualitative interview study (n=30). The goal of our work is to use the expertise of practitioners to discuss different examples that claim to represent a metaverse, e.g., Second Life and Decentraland. To achieve this goal, we develop a typology we call the Metagon and use it to evaluate existing metaverse platforms. We contribute to theory by clarifying the meaning of metaverse platforms. Practitioners are guided by a demonstration of metaverse characteristics.
Trust and Vulnerability in the Cybersecurity Context

Cybersecurity attacks offer a pertinent context within which to examine the currently under-explored dynamics of trust and distrust and their consequences for organisations and their employees. Such attacks wreak havoc in organisations, costly both in financial and reputational terms. We outline the dynamic, multi-level processes following initial attack making salient vulnerability to an unseen exploiter, and then making germane trust and trustworthiness in a number of relationships in an employing organisation. Drawing from events theory, we devise a multi-level conceptual trust model of the interrelations between the emotional, cognitive and social processes an attack can produce, to distinguish two paths revealing their markedly different durations, magnitudes, and level of relational and threat consequences. We explore these as dynamic experiences of trust to elucidate vulnerability and its experience and management for the targeted individual, with key organisational actors. This leads to the formation of an anchoring event that creates enduring changes to multiple relationships within the organisation with consequences for individual, team and organisational resilience and risk.
Can Chatbots Be Persuasive? How to Boost the Effectiveness of Chatbot Recommendations for Increasing Purchase Intention

Firms increasingly invest in chatbots that provide purchase recommendations. However, customers often reject recommendations by chatbots because they find neither the contents of the recommendation (message-level persuasiveness) nor the chatbot itself (source-level persuasiveness) persuasive. To overcome these barriers and increase purchase intention, this study examines how the content of recommendation messages should be designed and which communication style the chatbot should use to provide recommendation messages. Results of a 2 (two-sided vs. one-sided recommendation message) ✕ 3 (warm vs. competent vs. neutral communication style) between-subject online experiment show that a two-sided recommendation message increases purchase intention, but only for chatbots using a warm or competent communication style. Whereas a warm chatbot leads to higher purchase intentions of a recommendation through promoting its source persuasiveness, a competent chatbot increases recommendation effectiveness by promoting message persuasiveness. Therefore, firms should refine a chatbot’s communication style for providing recommendations that persuade customers to purchase.
Is There More Than Pokémon Go? – Exploring the State of Research on Causal Modeling in the Field of Augmented Reality

The paper explores how scholars apply causal modeling to gain an understanding of augmented reality as innovative technology and its potential for application. To do so, we conducted a structured literature review and applied a graph database-driven approach to analyze how scholars research augmented reality. Such an approach enables in-depth analysis of the body of knowledge that is not accessible in traditional ways of exploring literature. The results help to understand where we as a community stand and how directions for future research can help reshape the understanding of augmented reality and its application.
Federated Learning as a Solution for Problems Related to Intergovernmental Data Sharing

To address global problems, intergovernmental collaboration is needed. Modern solutions to these problems often include data-driven methods like artificial intelligence (AI), which require large amounts of data to perform well. However, data sharing between governments is limited. A possible solution is federated learning (FL), a decentralised AI method created to utilise personal information on edge devices. Instead of sharing data, governments can build their own models and just share the model parameters with a centralised server aggregating all parameters, resulting in a superior overall model. By conducting a structured literature review, we show how major intergovernmental data sharing challenges like disincentives, legal and ethical issues as well as technical constraints can be solved through FL. Enhanced AI while maintaining privacy through FL thus allows governments to collaboratively address global problems, which will positively impact governments and citizens.