Reviewerly/exHarmony huggingface dataset:
ActiveFace: A Synthetic Active Perception Dataset for Face Recognition

Active vision aims to enhance the efficiency of computer vision methods by enabling the capturing sensor, usually placed on a robot or, more generally, an autonomous system, to dynamically adjust its viewpoint, position or parameters in real-time. This capability allows for more precise decision-making by the model. However, training and evaluating an active vision model often necessitates a substantial number of annotated images, which must be captured under various sensor and environmental settings. These diverse images enable the model to learn the underlying dynamics of the active perception process. Unfortunately, collecting and annotating such datasets is a challenging and expensive task. It involves not only providing hand-crafted ground truth annotations but also ensuring that actions, such as moving around / towards / away from a person, are properly “imitated” to enable active vision approaches to model the corresponding active perception dynamics. To address these limitations, in this paper we propose a synthetic facial image generation pipeline specifically designed to support active face recognition, developed using a highly realistic simulation framework based on Unity. The developed pipeline allows for the generation of facial images for a set of persons at various view angles, distances, illumination conditions, and backgrounds. We demonstrate the effectiveness of our approach by training and evaluating a recently proposed embedding-based active face recognizer, as well as extending it to perform 2 axis control, leveraging the additional information provided by the generated dataset. To facilitate replication and encourage the use of the generated dataset for training and evaluating other active vision approaches, we also provide the associated assets and the developed dataset generation pipeline.
Multi-Scale Dilated Convolution Transformer for Single Image Deraining

Recently, Transformer-based methods have achieved significant improvements over convolutional neural networks (CNNs) in single image deraining, due to the powerful ability of modeling non-local information. In fact, rich local-global information representations are equally important for better satisfying rain removal. In this paper, we propose an effective image deraining method by integrating a CNN model into the Transformer backbone to accelerate network convergence, called Multi-scale Dilated-convolution Transformer (MDT), which fully leverages the learning capabilities of Transformers on non-local features, seamlessly integrating local detail extraction and global structural representation. The fundamental building unit of our framework is the Multi-scale Dilated-convolution Transformer Block (MDTB) with different dilation rates, which consists of the Dilconv Self-Attention (DSA) and the Dilconv Feed-Forward Network (DFN). Specifically, the former processes the contextual information via dilated convolutions and enables the model to emphasize spatially-varying rain distribution features, while the latter integrates the dual-branch information to facilitate the local feature learning for better feature aggregation. Extensive evaluations demonstrate that our model reaches superior performance, significantly improving the image deraining quality.
Predicting the Invariance Behind Residuals: A Novel GAN Inversion Method for Image Editing and Detail Retaining

Generative adversarial network (GAN) inversion has been serving as the vehicle to enable the restoration of real-world images by GANs, rather than the realistic generation from random noise. Existing GAN inversion methods, however, suffer from the fidelity-editability trade-off, which mainly invert the semantics within images and fail to reconstruct the details. To address this issue, we propose a novel adaptive detail compensation method for GAN inversion (ADC-GInv), which automatically locates and restores the non-semantic details, whilst maintaining the editability on the semantics of real-world images. More specifically, we first develop an adversarial reciprocal learning GAN (ARL-GAN) so as to seamlessly optimise the reconstruction during the training of GANs, followed by a sophisticated fine-tuning technique for ARL-GAN inversion. This ensures superior restoration and editability on the semantic cues of images. Then, regarding the non-semantic details, our ADC-GInv method adaptively locates the details by predicting the invariance given edited and non-edited residuals of restoration, which are then compensated at the pixel-level for high fidelity. As a consequence, the experimental results have verified the superior performance of our ADC-GInv, on both fidelity and editability during inversion.
Multi-Label Deepfake Classification

In this paper, we investigate the suitability of current multi-label classification approaches for deepfake detection.With the recent advances in generative modeling, new deepfake detection methods have been proposed.Nevertheless, they mostly formulate this topic as a binary classification problem, resulting in poor explainability capabilities.Indeed, a forged image might be induced by multi-step manipulations with different properties.For a better interpretability of the results, recognizing the nature of these stacked manipulations is highly relevant.For that reason, we propose to model deepfake detection as a multi-label classification task, where each label corresponds to a specific kind of manipulation.In this context, state-of-the-art multi-label image classification methods are considered.Extensive experiments are performed to assess the practical use case of deepfake detection.
Toward Digits Recognition Using Continual Learning

Over the past few decades, handwriting recognition has made tremendous progress based on deep learning architectures. Most commonly, static learning has achieved impressive results on various datasets. However, this approach relies predominantly on fixed datasets and stationary environments, making it challenging to handle continuous streams of data without forgetting previously learned knowledge. Conversely, Continual learning offers a more efficient and flexible framework for learning from a stream of data over time. However, it can be challenging to compare the performance of these approaches due to differences in evaluation protocols and a lack of a common framework. Moreover, prior research has primarily focused on updating models to perform well on specific task or domain scenarios. Yet, the Class-Incremental Learning scenario, which involves learning new classes incrementally over time, has not been as well addressed as it is considered a more challenging task. To address this gap, this paper provides a comprehensive empirical comparison of some of the currently used continual learning strategies, specifically focusing on the case where the model must learn to recognize continuously new classes while the task-ID is unknown. We conducted a deep overview of recent research from different perspectives and show that well-known approaches such as EWC, LWF, and SI are not ideal for the class-IL scenarios. We discuss the results in more detail and highlight some of the limitations that we believe require further investigation.
Learning Stigmergic Communication for Self-organising Coordination

Self-organisation in multi-agent systems (MAS) requires agents to coordinate their actions according to the specific goal to be pursued by the system as a whole. When agents can fully observe their peers, coordination can be implicit, that is, relying solely on appropriate reactions to others’ actions and their effects. When observability is limited, instead, explicit communication is needed. In this case, it is often taken as a built-in capability of agents, and the communication policy is defined at design time. But learning to communicate may enable the MAS to better adapt to novel deployments, run-time changes, and/or changing goals. In this paper, we study learning stigmergic communication in a MAS whose self-organising goal changes between aggregating agents in clusters and its opposite— scattering them around. Through experiments in NetLogo we show four results: (i) learning to communicate is possible, and can also be more effective than built-in communication; (ii) agents learn to self-organise so that the "correct" (i.e. leading to goal achievement) global sequencing of actions arises (i.e. few agents deposit pheromones while most follow pheromone trails, in the case of aggregation goal); and (iii) learning communication policies instead of having them built-in enables self-(re)organisation towards a new goal without heavy intervention in the learning process itself (i.e. by simply changing the reward scheme).
CLAPS: Curriculum Learning-Based Adaptive Bitrate and Preloading for Short Video Streaming

To provide high user QoE while maintaining low bandwidth waste, it is important to design adaptive bitrate and preloading algorithms for short video streaming. Current solutions either have relatively low performance, as observed in heuristic algorithms, or suffer the problem of poor generalization, as seen in deep reinforcement learning (DRL)-based algorithms. To address this issue, we propose CLAPS, a curriculum learning-based DRL model that enhances the generalization of the DRL model across a wide range of data, while ensuring high performance. CLAPS introduces a comprehensive metric that measures the curriculum difficulty by combing the performance gap between an existing heuristic algorithm and the DRL model with the prediction error of network bandwidth. Moreover, we design a training scheduler to control sampling proportion based on Markov transition probabilities to address the model forgetting problem. Extensive evaluations using real video datasets and network traces including 5G, 4G, and Wi-Fi demonstrate that CLAPS outperforms all the baseline algorithms. Specifically, CLAPS improves the overall performance by 10.04%-13.84% and the generalization by 22.52% -35.77% compared to the best-performing DRL baseline.
Chinese pre-service chemistry teachers’ perception of augmented reality-assisted secondary chemistry learning

Abstract This research aims to explore the functional characteristics of Augmented Reality (AR) supported teaching in secondary chemistry learning, based on the theoretical framework of AR research in science education proposed by (JSET 22:449-462, 2013). Grounded theory and interview were used to construct AR related functional models. In the study, pre-service teachers were first asked to experience organic chemistry AR teaching app—“Meet Organic”, and then semi-structured interview was conducted. Secondly, Nvivo12 was used to encode the interview results and finally relevant functional models were obtained. According to the results, AR education functions in science education are mainly divided into three parts: “Visual Presentation” , “Concept Understanding” and “Thinking Construction” , and each function is divided into 2–3 sub-functions based on the interview depth coding. Exploring the functional differences of AR between male and female meanwhile. The results of this research are of great importance to promote the in-depth understanding of the essence of augmented reality supported teaching, and the deep integration of AR teaching with science education. It also provides suggestions for future teachers to design teaching methods based on AR.
Hand Gesture Control Robot

In recent years, there has been a growing interest in developing human-machine interfaces that allow users to interact with robots in a more intuitive and natural way. Hand gesture control has emerged as a promising modality for such interfaces, offering a hands-free and immersive interaction experience. This paper presents the design, development, and evaluation of a hand gesture-controlled robot system. The proposed system utilizes computer vision techniques and machine learning algorithms to recognize and interpret hand gestures, enabling users to control the robot's movements and actions seamlessly. The robot is equipped with sensors and actuators, allowing it to respond to the detected gestures in real-time. The goal is to create a user-friendly and efficient interface that enhances the ease of interaction between humans and robots. The findings pave the way for the development of more sophisticated and user-friendly robotic interfaces, fostering the integration of robots into daily human activities. The proposed system's adaptability and versatility make it a promising candidate for further exploration and deployment in real-world scenarios, marking a significant step towards the advancement of gesture-based control in robotics
Unraveling the Potential of Immersive Virtual Environments for Behavior Mapping in the Built Environment: A Mapping Review

Introduction/Purpose. Behavior mapping is a crucial practice to capture precise data on human activities. Over the years, technological advancements have improved reliable data collection over the conventional “paper and pen”-based methods. Likewise, researchers increasingly utilize IVE (immersive virtual environment) technology in the built environment for its experiential capabilities. The aim of this research is to comprehensively investigate the most recent research and technological trends, data collection methods, and challenges and opportunities associated with IVE adoption for behavior mapping. Design/Methodology/Approach. This study investigates the current status of behavior mapping in the built environment through a mixed qualitative–quantitative study. A systematic background review was conducted on 353 articles, and a bibliometric analysis was performed on 123 journal articles retrieved from Scopus. The analysis generated science maps, unveiling technological advancements, data types, collection methods, research trends, and potential directions to address the research questions at hand. Findings. A thorough qualitative review presented deeper insight into major transitions and technological contributions, whereas the quantitative study revealed current research trends and future directions. Furthermore, it discussed the contribution, opportunities, and challenges of IVE technology for data collection. Research Limitations. The review and bibliometric study findings are circumscribed by literature data retrieved from Scopus data only, and potential research directions, opportunities, and obstacles are proposed based on analysis of review results, which might cause an incomprehensive nature. Practical Implications. This study highlights the importance of behavioral mapping research and the potential for technology to assist it. The paper also identifies current shortcomings and constraints in the literature, emphasizing the need for interventions in IVE technology.
Improvements to DQN

The success story of the DQN agent, which achieved human-level performance in playing Atari games, marked a major breakthrough in the field of artificial intelligence and reinforcement learning. Since then, extensive research and development efforts have been dedicated to building upon the foundations of DQN. Notably, these endeavors have focused on enhancing the network architecture and refining techniques for optimal utilization of experience replay samples.
Problems with Continuous Action Space

In the previous chapters, we covered reinforcement learning problems with discrete action spaces, where actions are typically represented using discrete numbers, such as integers. However, in the real world, many problems require continuous actions that cannot be represented by integers. Continuous action spaces are common in real-world applications such as robotics, where precise and continuous control is essential.
Advanced Policy Gradient Methods

One of the primary challenges associated with policy gradient methods is their instability and sensitivity to hyperparameters, such as the learning rate. This can lead to oscillations in the agent’s performance, resulting in slow convergence or even divergence. Furthermore, these methods often suffer from high variance in gradient estimates, which hampers convergence speed. Moreover, standard policy gradient methods exhibit poor sample efficiency, as they utilize training samples only once for learning purposes.
Nonlinear Value Function Approximation

In this chapter, we will explore the use of nonlinear methods to approximate value functions in reinforcement learning. Value functions play a crucial role in reinforcement learning as they estimate the expected reward an agent can obtain in a given state and follow the policy afterwards. In the previous chapter, we discussed linear methods that rely on constructing a feature vector and computing a weighted combination of features as the state or state-action value. However, these methods have limitations when applied to complex and challenging domains. In such cases, a simple linear combination may fail to capture all the relevant features and information. Additionally, determining the appropriate features that capture the necessary information for the task at hand requires substantial domain knowledge and experience.
Temporal Difference Learning

In the previous chapter (Chap. 4 ), we introduced Monte Carlo (MC) methods for reinforcement learning, which allow an agent to learn from its own experience without relying on a model of the environment. However, MC methods are limited to episodic reinforcement learning problems, where the agent interacts with the environment in discrete episodes. In this chapter, we introduce temporal difference (TD) learning, a class of algorithms that generalize MC methods to support both episodic and continuing reinforcement learning problems. TD learning is based on the idea of updating value estimates using a combination of current rewards and estimated future rewards, making it a more versatile and efficient approach than MC methods for many types of reinforcement learning problems. We’ll explore the theory and implementation of various TD learning algorithms in this chapter.
Linear Value Function Approximation

Part I of this book focuses on using tabular methods to represent the value functions of Markov decision processes (MDPs). However, this approach is limited to small-scale MDPs. In many real-world problems, the state space is too large for a table-based approach to be practical. There are two main reasons for this: first, the sheer number of states may require a large amount of memory to store the values; second, the learned value functions may not generalize well to new states.
Policy Gradient Methods

In this chapter, we will explore policy-based methods, another family of reinforcement learning algorithms. In previous chapters, we focused on value-based methods, which estimate the optimal state-action value function. Value-based methods can become computationally expensive for large state spaces, and they can struggle with environments where rewards are sparse.
Curiosity-Driven Exploration

Reinforcement learning is a type of machine learning in which an agent learns by taking actions in an environment and receiving feedback in the form of reward signals. The objective of the agent is to maximize the cumulative reward over time. However, there are certain challenges that can hinder the learning process. These challenges include difficult-to-explore environments, sparse or uncertain rewards.
Distributed Reinforcement Learning

This chapter explores the use of distributed reinforcement learning, which involves multiple agents running in parallel to interact with the environment to generate sample trajectories or transitions, and use samples to train the agent (e.g., to learn the optimal policy or value function). This approach offers several benefits over single-agent architectures, including faster convergence, better exploration, improved robustness, and increased scalability.
Introduction

Artificial intelligence has made impressive progress in recent years, with breakthroughs achieved in areas such as image recognition, natural language processing, and playing games. In particular, reinforcement learning, a type of machine learning that focuses on learning by interacting with an environment, has led to remarkable achievements in the field.
Planning with a Model: AlphaZero

Throughout this book, we have discussed various model-free reinforcement learning algorithms that have proven effective in solving simple reinforcement learning problems such as classic control tasks and Atari video games. However, for more complex problems, these algorithms may not perform well, even with the application of advanced techniques like distributed training and curiosity-driven exploration. Therefore, in this final chapter, we will explore how to use model-based reinforcement learning to tackle more challenging tasks.
RM-STC: Row-Merge Dataflow Inspired GPU Sparse Tensor Core for Energy-Efficient Sparse Acceleration

This paper proposes RM-STC, a novel GPU tensor core architecture designed for sparse Deep Neural Networks (DNNs) with two key innovations: (1) native support for both training and inference and (2) high efficiency for all sparsity degrees. To achieve the first goal, RM-STC employs a uniform sparse encoding scheme that natively supports all operations holistically in forward and backward passes, thereby eliminating the need for costly sparse encoding transformation in between. For the second goal, RM-STC takes inspiration from the row-merge dataflow and combines the input-gathering and output-scattering hardware features to minimize the energy overhead. Experiments show that RM-STC achieves significant speedups and energy efficiency improvements over dense tensor cores and previous sparse tensor cores.
δLTA: Decoupling Camera Sampling from Processing to Avoid Redundant Computations in the Vision Pipeline

Continuous Vision (CV) systems are essential for emerging applications like Autonomous Driving (AD) and Augmented/Virtual Reality (AR/VR). A standard CV System-on-a-Chip (SoC) pipeline includes a frontend for image capture and a backend for executing vision algorithms. The frontend typically captures successive similar images with gradual positional and orientational variations. As a result, many regions between consecutive frames yield nearly identical results when processed in the backend. Despite this, current systems process every image region at the camera’s sampling rate, overlooking the fact that the actual rate of change in these regions could be significantly lower.
Strix: An End-to-End Streaming Architecture with Two-Level Ciphertext Batching for Fully Homomorphic Encryption with Programmable Bootstrapping

Homomorphic encryption (HE) is a type of cryptography that allows computations to be performed on encrypted data. The technique relies on learning with errors problem, where data is hidden under noise for security. To avoid excessive noise, bootstrapping is used to reset the noise level in the ciphertext, but it requires a large key and is computationally expensive. The fully homomorphic encryption over the torus (TFHE) scheme offers a faster and programmable bootstrapping (PBS) algorithm, which is crucial for many privacy-focused applications. Nonetheless, the current TFHE scheme does not support ciphertext packing, resulting in low-throughput performance. To the best of our knowledge, this is the first work that thoroughly analyzes TFHE bootstrapping, identifies the TFHE acceleration bottleneck in GPUs, and proposes a hardware TFHE accelerator to solve the bottleneck.