Reviewerly/exHarmony huggingface dataset:
Patient smart home monitoring using vision neural network transformers

Image captioning is a task that involves generating natural language descriptions of the content of an image, and has the potential to support healthcare providers in monitoring patient conditions and routines at home. The ability to remotely monitor patients can provide valuable information to healthcare providers, allowing them to identify changes in patient behavior and facilitate timely interventions. In this study, we examine the usability of transformer neural networks for image caption generation from surveillance camera footage taken at regular intervals of one minute. Our objective is to develop and evaluate a transformer neural network model for generating captions of patient behavior, trained and evaluated on the Common Objects in Context (COCO) dataset. Our study provides a proof-of-concept for the potential of transformer neural networks in image captioning for remote monitoring of patient behavior. By generating natural language descriptions of patient behavior, healthcare providers can obtain valuable insights into patient routines and conditions, allowing them to monitor patients remotely and identify changes in behavior that may require intervention. Furthermore, our study highlights the potential for transformer neural networks to support healthcare providers in identifying patterns and trends in patient behavior over time.
Age-Invariant Face Recognition Using Face Feature Vectors and Embedded Prototype Subspace Classifiers

One of the major difficulties in face recognition while comparing photographs of individuals of different ages is the influence of age progression on their facial features. As a person ages, the face undergoes many changes, such as geometrical changes, changes in facial hair, and the presence of glasses, among others. Although biometric markers like computed face feature vectors should ideally remain unchanged by such factors, face recognition becomes less reliable as the age range increases. Therefore, this investigation was carried out to examine how the use of Embedded Prototype Subspace Classifiers could improve face recognition accuracy when dealing with age-related variations using face feature vectors only.
A Contrario Mosaic Analysis for Image Forensics

With the advent of recent technologies, image editing has become accessible even without expertise. However, this ease of manipulation has given rise to malicious manipulation of images, resulting in the creation and dissemination of visually-realistic fake images to spread disinformation online, wrongfully incriminate someone, or commit fraud. The detection of such forgeries is paramount in exposing those deceitful acts. One promising approach involves unveiling the underlying mosaic of an image, which indicates in which colour each pixel was originally sampled prior to demosaicing. As image manipulation will alter the mosaic as well, exposing the mosaic enables the detection and localization of forgeries. The recent introduction of positional learning has facilitated the identification of the image mosaic. Nevertheless, the clues leading to the mosaic are subtle and frail against common operation such as JPEG compression. The pixelwise estimation of the mosaic is thus often imprecise, and a comprehensive analysis and aggregation of the results are necessary to effectively detect and localize forged areas. In this work, we propose mimic: Mosaic Integrity Monitoring for Image a Contrario forensics. an a contrario method to analyse a pixelwise mosaic estimation. We show that despite the weakness of these traces, the sole analysis of mosaic consistency is enough to beat the state of the art in forgery detection and localization on uncompressed images. Moreover, results are promising even on slightly-compressed images. The a contrario framework ensures robustness against false positives, and the complementary nature of mosaic consistency analysis to other forensic tools makes our method highly relevant for detecting forgeries in high-quality images.
Pyramid Swin Transformer for Multi-task: Expanding to More Computer Vision Tasks

We presented the Pyramid Swin Transformer, a versatile and efficient architecture tailored for object detection and image classification. This time we applied it to a wider range of tasks, such as object detection, image classification, semantic segmentation, and video recognition tasks. Our architecture adeptly captures local and global contextual information by employing more shift window operations and integrating diverse window sizes. The Pyramid Swin Transformer for Multi-task is structured in four stages, each consisting of layers with varying window sizes, facilitating a robust hierarchical representation. Different numbers of layers with distinct windows and window sizes are utilized at the same scale. Our architecture has been extensively evaluated on multiple benchmarks, including achieving 85.4% top-1 accuracy on ImageNet for image classification, 51.6 $$AP^{box}$$ with Mask R-CNN and 54.3 $$AP^{box}$$ with Cascade Mask R-CNN on COCO for object detection, 49.0 mIoU on ADE20K for semantic segmentation, and 83.4% top-1 accuracy on Kinetics-400 for video recognition. The Pyramid Swin Transformer for Multi-task outperforms state-of-the-art models in all tasks, demonstrating its effectiveness, adaptability, and scalability across various vision tasks. This breakthrough in multi-task learning architecture opens the door to new research and applications in the field.
A SandBox to Sandbox: A Tangible Terrain Generator for Prototyping Game Worlds

Designing the game world is foundational for creating games. However, existing software solutions for creating game worlds such as terrain generators often require a significant amount of time and experience to be mastered. Particularly for beginners, this poses a challenge and slows down the progress in development. To address this issue, we propose a tangible terrain generator named SandBox composed of a small sandbox enabling users to create a terrain landscape using sand. Leveraging the engaging and familiar activity of playing in a sandbox and creating simple models, we hypothesize that SandBox offers easy-to-learn and intuitive landscape creation for games. We evaluated SandBox in a study with eight participants and compared it to a common terrain builder using the 3D-modeling software Blender. Qualitative insights indicate that SandBox was well-received. Neither usability nor task completion time was significantly different. We discuss the use of our tangible terrain generator and propose further features for future work.
HabitAR: Motivating Outdoor Walk with Animal-Watching Activities

Outdoor exercise has been indicated to have physical and mental benefits over indoor exercise and tends to take place in more engaging and attractive settings. Increasing motivation is a critical factor in behavior change. We envision that the interaction between users’ walking data and components of their walking environment can be a potential method to motivate physical activity and offer novel opportunities. In this study, we introduce HabitAR, a blog-community-based application connecting the walk experience with the animal watching activities that provide different routes in which the types of animals that can be watched are proportional to walking distance. Following the research-through-design approach, we explored how this concept works on users’ motivation by building an emotional connection between users and animals with social blogging. We present the results of the pilot study and the insights gained from this study.
Theatre of Tomorrow - A Virtual Exhibition and Performing Arts Platform Created by Digital Game Technology

In recent years, the rapid development of gaming technology has made it possible to apply it to exhibitions and performances. Theater of Tomorrow is a virtual exhibition and performance platform integrating artificial intelligence, digital games, and performing arts. The platform provides an exhibition space that breaks the limits of time, space, interactive communication, and theatrical experience. Through the internet, the platform has the function of playing both exhibition and performance arts. It is also a social media where professionals can have a specific virtual community to create performing arts and learn from each other. In addition, this platform can display the Intangible Cultural Heritage, preserving the traditional performing arts. Theater of Tomorrow has significant social value, especially during the epidemic. Of course, it has some things that could be improved, such as visual effects limited by the accuracy of the model and the need for more weight and fluidity in the movements of the digital dancers. Nowadays, physical performances are re-emerging, but virtual online platforms are relevant and valuable as a powerful complement to offline versions, which continue to improve with technology development.
Calm Digital Artwork for Connectedness: A Case Study

Connectedness is increasingly recognized as an important aspect of life and wellbeing. The technology-driven solutions aimed at fostering connectedness can also introduce negative pressure or influence when too much precise and real-time information is involved. There are also growing concerns regarding information security and privacy associated with these solutions. This study aims to address these challenges by proposing a novel approach that combines the principles of calm technology with the expressivity of Mondrian style digital artwork to create connectedness between loved ones apart. Using a co-constructing story approach, we conducted a user study involving five participant groups to stimulate their imaginations and visions. The results showed that combining wearable data with Mondrian style artwork can provide a unique and meaningful way to strengthen emotional bonds among distant loved ones. This study provides a fresh perspective on connectedness, suggesting a novel engaging solution. Future research could explore wider applications of this approach and further refine the user experience and practical implementation.
Person Activity Classification from an Aerial Sensor Based on a Multi-level Deep Features

The intelligent surveillance system is considered as an important research field since it provides continuous personal security solution. In fact, this surveillance system support security guards by warning them in suspicious situations such as recognize the abnormal person activity. In this context, we introduced a new method for person activity classification that includes offline and inference phases. Based on convolutional neural networks, the offline phase aims to generate the person activity model. Whereas, the inference phase relies on the generate model to classify the person’s activity. The main contribution of the proposed method is to introduce a multi-level deep features to handle inter- and intra-class variation. Through a comparative study, performed on the UCF-ARG dataset, we assessed the performance of our method compared to the state-of-the-art works.
Research and Practice of Hierarchical Painting-Based Digital Design Method

As digital systems and information become an important part of people’s daily lives and social relationships, new insights into how technology interacts with traditional painting practices are crucial in education. To study how digital painting can be promoted in education in the future, we combine virtual digital painting with traditional painting in an educational environment to explore innovative experiences of new forms of painting, supporting new attempts in art education and inspiring people’s enthusiasm for art. We attempted to combine three interactive methods: 1) painting robot technology for creating texture, 2) computer software for generating painting styles, and 3) UV printing for producing colors. 20 students from an art collage participated in our user study. Through this approach, we created a new painting experience, triggering discussions on how technology can support existing painting practices. The discussions revealed that participants hoped to use technology-assisted methods to handle the levels of their paintings, involving human-machine collaboration in the creative process, and to enhance their understanding of traditional painting’s use of color, texture, and layers. Most participants expressed interest in integrating digital art into reality. The results of this study can serve as considerations for guiding future painting education initiatives.
Less-than-One Shot 3D Segmentation Hijacking a Pre-trained Space-Time Memory Network

In this paper, we propose a semi-supervised setting for semantic segmentation of a whole volume from only a tiny portion of one slice annotated using a memory-aware network pre-trained on video object segmentation without additional fine-tuning. The network is modified to transfer annotations of one partially annotated slice to the whole slice, then to the whole volume. This method discards the need for training the model. Applied to Electron Tomography, where manual annotations are time-consuming, it achieves good segmentation results considering a labeled area of only a few percent of a single slice. The source code is available at https://github.com/licyril1403/hijacked-STM .
i-Cube, an (In)tangible Interface for Multi-user Generative Art Creation

Generative art is a highly innovative and dynamic art form that uses mathematical algorithms and computer programs to create intricate patterns, shapes, and designs. What sets it apart from traditional art forms is the level of interaction between people and the medium of creation, as most generative art requires collaboration between the artist, participants, and technology to produce visually engaging and thought-provoking outputs. This study aims to explore the use of intangible interfaces for human interaction and bring individuals to the diverse world of generative art. To achieve this, we developed the intangible cube (iCube), a wireless portable device that allows up to 20 people to interact with the same generative art using gesture and voice. Through collecting data from iCube, users’ feedback will be integrated and visualized in real time. Rather than get a visually appealing result, this study focuses on how individuals interact with the device and create generative art through collaboration.
Empirical Studies on the Conditions of Achieving Interactive Mindfulness Through Virtual Reality Single User Experience in Macao

Virtual reality, a computer-generated 3D environment, allows one to navigate and possibly interact, resulting in real-time simulation of one or more of the user’s five senses (Gutierrez et al., 2008; Vince, 2004). Virtual tours and places have swiftly become popular in education, professional training, arts, exhibitions, and medication and rehabilitation. The empirical studies derived from the PhD thesis research aim to identify the conditions for Macao’s single-user experience to achieve mindfulness in virtual reality through immersion and interactivity. The writers describe three spectrums on the levels of immersion, interactivity, and mindfulness-based on multiple definitions. For the empirical study, immersion and interaction are controlled experiment constants. As an international city in Southern China, we suggest four Macao experiment settings, addressing spiritual memory, historical memory, aesthetic appreciation, and idea meditation, collect qualitative and quantitative survey data. The collected data are analysed to improve virtual reality conditions for local mindfulness.
Quality Assessment for High Dynamic Range Stereoscopic Omnidirectional Image System

This paper focuses on visual experience of high dynamic range (HDR) stereoscopic omnidirectional image (HSOI) system, which includes such as HSOI generation, encoding/decoding, tone mapping (TM) and terminal visualization. From the perspective of quantifying coding distortion and TM distortion in HSOI system, a “no-reference (NR) plus reduced-reference (RR)” HSOI quality assessment method is proposed by combining Retinex theory and two-layer distortion simulation of HSOI system. The NR module quantizes coding distortion for HDR images only with coding distortion. The RR module mainly measures the effect of TM operator based on the HDR image only with coding distortion and the mixed distorted image after TM. Experimental results show that the objective prediction of the proposed method is better compared some representative method and more consistent with users’ visual perception.
A Single Image Neuro-Geometric Depth Estimation

This paper introduces a novel neuro-geometric methodology for single image object depth estimation, abbreviated as NGDE. The proposed methodology can be described as a hybrid methodology since it combines a geometrical and a deep learning component. NGDE leverages the geometric camera model to initially estimate a set of probable depth values between the camera and the object. Then, these probable depth values along with the pixel coordinates that define the boundaries of an object, are propagated to a deep learning component, appropriately trained to output the final object depth estimation. Unlike previous approaches, NGDE does not require any prior information about the scene, such as the horizon line or reference objects. Instead, NGDE uses a virtual 3D point cloud projected to the 2D image plane that is used to estimate probable depth values indicated by 3D-2D point correspondences. Then by leveraging a multilayer perceptron (MLP) that considers both the probable depth values and the 2D bounding box of the object, NGDE is capable of accurately estimating the depth of an object. A major advantage of NGDE over the state-of-the-art deep learning-based methods is that it utilizes a simple MLP instead of computationally complex Convolutional Neural Networks (CNNs). The evaluation of NGDE on KITTI indicates its advantageous performance over relevant state-of-the-art approaches.
Prov-Replay: A Qualitative Analysis Framework for Gameplay Sessions Using Provenance and Replay

The usage of game data to analyze relevant information and possibilities about the game has been widely explored by academia and industry. Hence, numerous techniques exist for collecting data with distinct objectives. Among the strategies employed for analyzing gameplay sessions, the collection and analysis of provenance data emerge as prominent. In this paper, we improve provenance approaches through a Prov-Replay conceptual framework. Our framework uses provenance data providing an interactive provenance graph visualization synced with a replay to help to understand the context of gameplay events. To validate Prov-Replay, we applied this framework in an under-development commercial game, collecting real game session data and performing a test with the game developers.
Design of Collaborative Learning Experience with AR–AR Collaboration Game for Children at Museum

We are currently developing an AR–AR collaboration game to support children’s learning in museums. This system provides an AR–AR collaboration experience between multiple mobile PC devices for a collaborative exhibition viewing experience through joint physical actions in the game. A prototype of an exploration game for one person using physical actions that can be played on a mobile PC device has been developed as the first stage of our project. We evaluated the operability of the physical actions and the appropriateness of the gameplay style for child visitors in a preliminary experiment. On this basis, we developed an AR–AR collaboration game that can be played by two people using joint physical actions. This paper describes the preliminary experiments, evaluation results, and current system.
A Hybrid Quantum-Classical Segment-Based Stereo Matching Algorithm

Our contribution introduces a hybrid quantum-classical Stereo Matching algorithm that demonstrates the potential of using Quantum Annealing in Computer Vision in the future once quantum processors have enough qubits to support practical, real-world Computer Vision applications. The classical component of our approach involves dividing the input image into homogeneous color segments and using a local Stereo Matching technique to estimate their respective initial disparity planes. In the quantum component, we assign a label to each segment by the estimated disparity planes. Such a labeling problem is classically intractable. The outcomes of our experiments on the D-Wave quantum computer indicate that our method produces results that compare well with the ground truth. Nonetheless, the precision of our approach is greatly influenced by the quality of the initial image segmentation, which is a common challenge for all classical Stereo Matching methods that rely on segmentation-based techniques.
Exploring the Influence of Collision Anxiety on Player Experience in XR Games

Extended Reality (XR) applications are becoming increasingly widespread due to technical advances, but also have their own challenges regarding the user’s real-world orientation. Especially during virtual reality use at home, users are at risk of collisions with the environment due to restricted play space. Users might also experience discomfort due to this risk, even if no collisions occur. We propose a novel definition for the feeling of disorientation and fear of colliding with real objects in XR, called Collision Anxiety (CA). To measure the collision anxiety, we developed and tested a not yet validated questionnaire (Collision Anxiety Questionnaire) and used it to examine the influence of CA on immersion and general player experience. In three studies, a total of 149 participants played different variations of an augmented (AR) or virtual reality (VR) version of a game. Results suggest that while an AR game can provide a similar player experience compared to its VR equivalent, differences regarding CA exist.
A Bayesian latent class model for integrating multi-source longitudinal data: application to the CHILD cohort study

Abstract Multi-source longitudinal data have become increasingly common. This type of data refers to longitudinal datasets collected from multiple sources describing the same set of individuals. Representing distinct features of the individuals, each data source may consist of multiple longitudinal markers of distinct types and measurement frequencies. Motivated by the CHILD cohort study, we develop a model for joint clustering multi-source longitudinal data. The proposed model allows each data source to follow source-specific clustering, and they are aggregated to yield a global clustering. The proposed model is demonstrated through real-data analysis and simulation study.
Simulation of Intelligent Image Processing Model Based on Machine Learning Algorithm

Medical image segmentation is an important link in medical image processing and analysis, and it is also a very arduous task. In the process of image processing and image analysis, image segmentation is the most basic step, and the accuracy of segmentation directly affects the accuracy of subsequent work, which is the premise of medical image analysis, understanding, description and three-dimensional reconstruction. The topic selected in this paper is the simulation of intelligent image processing model based on machine learning algorithm. A novel MRI (magnetic resonance imaging) image segmentation algorithm is obtained by combining machine learning with fuzzy theory. The spatial filter provides strong noise filtering performance. By guiding the clustering process of FCM (Fuzzy C-means) through the filtered results, we can expect to get stronger anti-noise ability than the standard FCM. The simulation results show that the improved FCM algorithm proposed in this paper has high diagnostic accuracy, high segmentation accuracy, stable algorithm and strong robustness. The improved FCM algorithm improves the anti-noise ability by fusing spatial filter, and this method has achieved good segmentation and anti-noise effect.