Reviewerly/exHarmony huggingface dataset:
Emotion Tracking in Virtual Reality Fashion Shows

Virtual Reality (VR) technologies are increasingly being adopted in fashion marketing to create innovative new channels for sales, advertising, brand communication, and strengthening relationships with consumers. In addition, virtual technologies can be useful for personalizing the offer, gathering insights, and user engagement. However, there seem to be few occurrences where the potential of virtual technologies on User eXperience (UX) and engagement is explored, evaluating the advantages compared to real-life experiences. Analyzing these aspects for virtual fashion experiences can be challenging. For measuring the UX, quantitative methods such as sensors capable of recording and examining the body's responses and automatic systems used to recognize emotions by tracking facial movements and expressions have been increasingly employed. In the context of fashion marketing, they can help analyze the implications that virtual experiences have, also considering content personalization and company promotion. This paper presents the design and development of a VR application that includes emotion-tracking technology that can be used to evaluate users’ emotional reactions to virtual experiences and compare them with real ones. The final aim of the study is to increase the degree of user engagement and the quality of the virtual fashion show UX of the Italian clothing brand Bacon. Preliminary test sessions were organized to evaluate users’ emotional reactions to the virtual experience and verify their effectiveness in relation to the objectives of the study.
Evaluating Virtual Reality as Immersive Medium for Enhancing Music Skills

The VR interfaces have become more affordable than ever before. Because of their multimodal interaction capabilities, these interfaces provide a medium to virtually participate in various learning processes that can engage both sensory and motor senses. In this paper, we present an ongoing study to evaluate a VR game designed to enhance four aspects of rhythmic perceptual skills in elementary school children. The game developed is based on a well-known tower defense scenario in which the user controls units with different rhythmic patterns and tempo changes. Currently, 22 students are participating in the study, which shows promising results. We report the preliminary results and discuss possible applications of e-learning in virtual environments in music theory education.
Application of Knowledge Representation and Reasoning to Building Indoor Air Quality

The importance of indoor air quality (IAQ) has been recognized for centuries, with increased interest in the relationship of IAQ to energy efficiency and other building performance issues in the last several decades. Managing IAQ requires an understanding of the complex interactions between buildings, systems, occupants and environmental conditions as well as the interactions of IAQ and related performance issues. Most efforts to address IAQ are specific to the affected buildings, spaces and occupants, and those tasked with solving IAQ problems generally need to consider a wide range of information that can be difficult to access and manage. Knowledge Representation and Reasoning (KRR) techniques are being developed for buildings to organize such information and ultimately enhance building performance. Specifically, Semantic Web technologies have been used to create models representing building system components and their relationships. Such models can improve building analytics and enterprise knowledge management to access the contextual information necessary for automated fault detection and diagnostics, control system configuration to optimize energy use, building commissioning and other applications. This paper describes how KRR can be used to address IAQ with discussion of an example using measured indoor CO2 concentrations to evaluate ventilation rates.
Visualization of Large Datasets in Virtual Reality Systems

This article explores the technology of large data files and the possibilities of their visualization in virtual reality. The three-dimensional unlimited scene, perception of perspective, freedom of movement, and interaction using natural gestures are all unique properties of virtual reality that can significantly change the way we receive, control, and evaluate visualization. Individual elements are discussed in detail, and their positive and negative aspects are described along with the potential applications. The knowledge gained from exploring extensive data files and virtual reality is used to develop two interactive demonstrations. The first virtual scene deals with the visualization of data from the smart city of Aarhus. The second demonstration works with the statistical data pertaining to the Czech Republic. The benefits and findings are then evaluated and summarized. The work also describes other possible uses of this technology and directions for further development.
Effortlessly Populating Immersive Training Simulations with Background Characters

Immersive training simulations engage users in compelling environments with interactive and pedagogically meaningful activities. Integrating virtual characters in these simulations has proven highly beneficial, especially in technical contexts where social interactions are key in the training. Although most of the research on the topic endeavours to model tutoring agents endowed with natural interaction, cognitive, or even emotional abilities, there is a more modest yet widespread need for background characters with lower capabilities but higher ease of integration on the developer’s behalf. In this paper, we demonstrate how informed environments, which already constitute an appropriate framework for designing interactive environments, can be leveraged for simulating the realistic behaviour of background characters, even allowing for scripted events and scenario control during the training session. The model presented in the paper, called CRITS (CRowds for Immersive Training Simulations), has been applied to a large-scale crisis management scenario and has produced satisfying results in terms of the behaviours obtained and ease of integration.
A Comprehensive Approach to the Analysis of VR Training Progress of Industry 4.0 Employees

Innovative workforce training solutions are essential as Industry 4.0 transforms the manufacturing and automation landscape. The Virtual Reality (VR) training system described in this paper uses multiple visualization techniques to track employees’ progress. To accommodate a range of user preferences and training objectives, the system makes use of various visualization techniques. PDF printouts offer a concise, portable summary of individual performance metrics, while 2D maps provide an overview of spatial navigation and task completion within the VR environment. 3D scenes allow for a more immersive, interactive visualization of employee progress, allowing users to better understand complex scenarios and relationships. The proposed system seeks to improve user engagement, comprehension, and retention of training outcomes by incorporating these visualization techniques.
An Effective CNN-Based Super Resolution Method for Video Coding

The reference picture resampling (RPR) technique in Versatile Video Coding allows the input video to be down-sampled before encoding, and then the decoded video will be up-sampled. Compared with the handcrafted up-sampling interpolation filter, the convolutional neural network based super resolution (SR) method has better restoration effect. In this paper, we introduce the RPR technique into the SR method, to effectively restore the details lost in the down-sampling and encoding process. Meanwhile, the normalized attention module in the network is designed to adaptively determine the weight of the extracted features. Moreover, the prediction picture generated during the codec process is fed into the network as side information, which can provide the directional and texture information of the original picture, and implicitly derive the lost details together with the reconstruction picture. Compared with VTM-11.0-NNVC, the experimental results show that the proposed method achieves 12.79% and 10.53% BD-rate reductions, under all intra and random access configurations, respectively.
Unsupervised Video Anomaly Detection with Diffusion Models Conditioned on Compact Motion Representations

This paper aims to address the unsupervised video anomaly detection (VAD) problem, which involves classifying each frame in a video as normal or abnormal, without any access to labels. To accomplish this, the proposed method employs conditional diffusion models, where the input data is the spatiotemporal features extracted from a pre-trained network, and the condition is the features extracted from compact motion representations that summarize a given video segment in terms of its motion and appearance. Our method utilizes a data-driven threshold and considers a high reconstruction error as an indicator of anomalous events. This study is the first to utilize compact motion representations for VAD and the experiments conducted on two large-scale VAD benchmarks demonstrate that they supply relevant information to the diffusion model, and consequently improve VAD performances w.r.t the prior art. Importantly, our method exhibits better generalization performance across different datasets, notably outperforming both the state-of-the-art and baseline methods. The code of our method is available HERE .
Exploring Audio Compression as Image Completion in Time-Frequency Domain

Audio compression is usually achieved with algorithms that exploit spectral properties of the given signal such as frequency or temporal masking. In this paper we propose to tackle such a problem from a different point of view, considering the time-frequency domain of an audio signal as an intensity map to be reconstructed via a data-driven approach. The compression stage removes some selected input values from the time-frequency representation of the original signal. Then, decompression works by reconstructing the missing samples as an image completion task. Our method is divided into two main parts: first, we analyse the feasibility of a data-driven audio reconstruction with missing samples in its time-frequency representation. To do so, we exploit an existing CNN model designed for depth completion, involving a sequence of sparse convolutions to deal with absent values. Second, we propose a method to select the values to be removed at compression stage, maximizing the perceived audio quality of the decompressed signal. In the experimental section we validate the proposed technique on some standard audio datasets and provide an extensive study on the quality of the reconstructed signal under different conditions.
Budget-Aware Pruning for Multi-domain Learning

Deep learning has achieved state-of-the-art performance on several computer vision tasks and domains. Nevertheless, it still has a high computational cost and demands a significant amount of parameters. Such requirements hinder the use in resource-limited environments and demand both software and hardware optimization. Another limitation is that deep models are usually specialized into a single domain or task, requiring them to learn and store new parameters for each new one. Multi-Domain Learning (MDL) attempts to solve this problem by learning a single model that is capable of performing well in multiple domains. Nevertheless, the models are usually larger than the baseline for a single domain. This work tackles both of these problems: our objective is to prune models capable of handling multiple domains according to a user defined budget, making them more computationally affordable while keeping a similar classification performance. We achieve this by encouraging all domains to use a similar subset of filters from the baseline model, up to the amount defined by the user's budget. Then, filters that are not used by any domain are pruned from the network. The proposed approach innovates by better adapting to resource-limited devices while, to our knowledge, being the only work that is capable of handling multiple domains at test time with fewer parameters and lower computational complexity than the baseline model for a single domain.
Self-Similarity Block for Deep Image Denoising

Non-Local Self-Similarity (NLSS) is a widely exploited prior in image denoising algorithms. The first deep Convolutional Neural Networks (CNNs) for image denoising ignored NLSS and were made of a sequence of convolutional layers trained to suppress noise. The first denoising CNNs leveraging NLSS prior were performing non-learnable operations outside the network. Then, pre-defined similarity measures were introduced and finally learnable, but scalar, similarity scores were adopted inside the network. We propose the Self-Similarity Block (SSB), a novel differentiable building block for CNN denoisers to promote the NLSS prior. The SSB is trained in an end-to-end manner within convolutional layers and learns a multivariate similarity score to improve image denoising by combining similar vectors in an activation map. We test SSB on additive white Gaussian noise suppression, and we show it is particularly beneficial when the noise level is high. Remarkably, SSB is mostly effective in image regions presenting repeated patterns, which most benefit from the NLSS prior.
Usability Evaluation of Mixed Reality Applications in VET Training

This paper presents a usability study of wearable and handheld Mixed Reality and hypervideo technologies to support procedural learning in the vocational education and training context. As a case study an application based on the Tangram game was developed for HoloLens 2, Android smartphones and hypervideo. The app is structured as a procedural learning activity with incremental levels of scaffolding. The usability of these applications was evaluated in a user testing involving 90 students. No remarkable difference has emerged among the three applications in terms of completion rate during the learning process, but in the retention and transfer tasks HoloLens 2 users performed worse than users of the other devices. Considering the time on task metric, HoloLens 2 users time was aligned with hypervideo, while smartphone users generally took more time, probably because smartphones are not hand-free devices. An expert review was also conducted, providing a number of suggestions for improvements, mainly concerning the interaction design, and the positioning of holograms in the physical space. In the comparison among the devices, none was clearly preferred over the other: HoloLens 2 presented higher attractiveness, motivation of use, sense of immersion, but also greater cognitive load; handheld devices resulted easy to use but required considerable physical effort in activities where they needed to be hold for a long time. Hypervideo was easy to access, with an intuitive interaction, but less attractive.
Depth Camera Face Recognition by Normalized Fractal Encodings

Face recognition is a thriving topic in biometrics literature. In most cases, it is performed on RGB images using deep learning approaches. However, due to the wider scenarios in which face recognition can be applied, it is necessary to operate when illumination conditions are not in favor of the RGB image analysis. To overcome this problem, we present DLIF (Depth Landmark Identity by Fractal), a new depth face recognition method that uses fractal encoding to treat faces as depth features. A facial landmark predictor is developed to detect and extract the face, making DLIF completely in-depth. Comparisons of the fractal-encoded faces are obtained by the Canberra distance, which makes DLIF simple but effective. Statistical analysis is performed to improve DLIF, resulting in DLIF+, which takes into account the source of the image. DLIF and DLIF+ are tested on four challenging datasets and on a fifth dataset, named BIPS, with 72 subjects. The results obtained show that DLIF and DLIF+ are competitive with the state of the art and enhance the robustness of this method in relation to the number of subjects considered.
Not with My Name! Inferring Artists’ Names of Input Strings Employed by Diffusion Models

Diffusion Models (DM) are highly effective at generating realistic, high-quality images. However, these models lack creativity and merely compose outputs based on their training data, guided by a textual input provided at creation time. Is it acceptable to generate images reminiscent of an artist, employing his name as input? This imply that if the DM is able to replicate an artist's work then it was trained on some or all of his artworks thus violating copyright. In this paper, a preliminary study to infer the probability of use of an artist's name in the input string of a generated image is presented. To this aim we focused only on images generated by the famous DALL-E 2 and collected images (both original and generated) of five renowned artists. Finally, a dedicated Siamese Neural Network was employed to have a first kind of probability. Experimental results demonstrate that our approach is an optimal starting point and can be employed as a prior for predicting a complete input string of an investigated image. Dataset and code are available at: https://github.com/ictlab-unict/not-with-my-name .
Developing Thermostat User Mental Models to Inform Energy-Saving Design

This paper presents a study investigating the user mental models (UMMs) of ten Canadians, related to home heating systems and smart thermostat (ST) usage. The work contrasts previous investigation (Revell and Stanton in Appl Ergon 45:363–378, 2014), shifting the focus towards STs. Interviews evaluated self-reported behaviour and generated diagrams of UMMs. The research was conducted remotely, using video-calls and an online whiteboard tool. Overall, participants’ conceptualizations echoed a UMM previously observed in literature, recognized as feedback theory. This paper presents novel investigation of ST users, employing comparable methods with previous work. It contrasts results with literature and demonstrates a more uniform ideology among ST users. Furthermore, it utilizes novel methods that support the collection of qualitative data, that can be applied to future studies.
A Deep Learning Based Approach for Synthesizing Realistic Depth Maps

This paper presents a novel cycle generative adversarial network (CycleGAN) architecture for synthesizing high-quality depth maps from a given monocular image. The proposed architecture uses multiple loss functions, including cycle consistency, contrastive, identity, and least square losses, to enable the generation of realistic and high-fidelity depth maps. The proposed approach addresses this challenge by synthesizing depth maps from RGB images without requiring paired training data. Comparisons with several state-of-the-art approaches are provided showing the proposed approach overcome other approaches both in terms of quantitative metrics and visual quality.
Spatial Transformer Generative Adversarial Network for Image Super-Resolution

High-resolution images play an essential role in the performance of image analysis and pattern recognition methods. However, the expensive setup required to generate them and the inherent limitations of the sensors in optics manufacturing technology leads to the restricted availability of these images. In this work, we exploit the information retrieved in feature maps using the notable VGG networks and apply a transformer network to address spatial rigid affine transformation invariances, such as translation, scaling, and rotation. To evaluate and compare the performance of the model, three publicly available datasets were used. The model achieved very gratifying and accurate performance in terms of image PSNR and SSIM metrics against the baseline method.
A Differentiable Entropy Model for Learned Image Compression

In an end-to-end learned image compression framework, an encoder projects the image on a low-dimensional, quantized, latent space while a decoder recovers the original image. The encoder and decoder are jointly trained with standard gradient backpropagation to minimize a rate-distortion (RD) cost function accounting for both distortions between the original and reconstructed image and the quantized latent space rate. State-of-the-art methods rely on an auxiliary neural network to estimate the rate R of the latent space. We propose a non-parametric entropy model that estimates the statistical frequencies of the quantized latent space during training. The proposed model is differentiable, so it can be plugged into the cost function to be minimized as a rate proxy and can be adapted to a given context without retraining. Our experiments show comparable performance with a learned rate estimator and better performance when is adapted over a temporal context.
SCENE-pathy: Capturing the Visual Selective Attention of People Towards Scene Elements

We present SCENE-pathy, a dataset and a set of baselines to study the visual selective attention (VSA) of people towards the 3D scene in which they are located. In practice, VSA allows to discover which parts of the scene are most attractive for an individual. Capturing VSA is of primary importance in the fields of marketing, retail management, surveillance, and many others. So far, VSA analysis focused on very simple scenarios: a mall shelf or a tiny room, usually with a single subject involved. Our dataset, instead, considers a multi-person and much more complex 3D scenario, specifically a high-tech fair showroom presenting machines of an Industry 4.0 production line, where 25 subjects have been captured for 2 min each when moving, observing the scene, and having social interactions. Also, the subjects filled out a questionnaire indicating which part of the scene was most interesting for them. Data acquisition was performed using Hololens 2 devices, which allowed us to get ground-truth data related to people's tracklets and gaze trajectories. Our proposed baselines capture VSA from the mere RGB video data and a 3D scene model, providing interpretable 3D heatmaps. In total, there are more than 100K RGB frames with, for each person, the annotated 3D head positions and the 3D gaze vectors. The dataset is available here: https://intelligolabs.github.io/scene-pathy .
Image Retrieval in Semiconductor Manufacturing

Content-Based Image Retrieval has a lot of applications in the industry, where large collections of data from manufacturing need to be automatically queried e.g. for quality inspection purposes. In this work we design an image retrieval solution over IMAGO, a dataset of Transmission Electron Microscopy (TEM) images of nano-sized silicon structures collected in the production site of STMicroelectronics, in Agrate Brianza, Italy. Image retrieval in imago is challenging because: i) only a limited portion of images are provided with labels, namely type of semiconductor structure, ii) most images refer to unseen classes that are not represented in the training set, and iii) images of the same class can be acquired at different magnification levels of the electronic microscope. Our main contribution is the design of a deep-learning based image retrieval system that leverages a training procedure that alternates between siamese loss, assessed on annotated samples, and reconstruction loss, assessed on unlabelled samples. Our solution exploits the whole information in the IMAGO dataset, and our experiments confirm we can successfully retrieve images of unseen classes that exhibit the same structure of the query ones. Our solution is currently deployed in STMicroelectronics production sites.