# Physiological Single trial decoding of art interest

Marc Welter, Axel Bouneau, Martinez Jesus Casal, Redmond Erin, Jonathan

Baum, Tomás Ward, Fabien Lotte

► To cite this version:

Marc Welter, Axel Bouneau, Martinez Jesus Casal, Redmond Erin, Jonathan Baum, et al. Physiological Single trial decoding of art interest. NAT 2025 - NeuroAdaptive Technology Conference, Apr 2025, Berlin, Germany. hal-05449184

HAL Id: hal-05449184

https://hal.science/hal-05449184v1

Submitted on 8 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Physiological Single trial decoding of art interest

Marc Welter¹, Axel Bouneau¹, Jesus Casal Martinez², Erin Redmond³, Jonathan Baum¹, Tomas Ward³, Fabien Lotte¹

1. Inria Centre at the University of Bordeaux / LaBRI, France

2. Valencia Polytechnic University, Spain

3. Dublin City University, Ireland

Keywords: Physiological Computing, Mental State Decoding

## Abstract

Both well-being and aesthetic experience are correlated with physiology. Thus, optimizing art presentation to evoke desirable mental states in virtual environments based on physiological states could have beneficial effects on user experience and well-being. However, single trial aesthetic experience decoding from physiological signals has not been well studied. We tested a Support Vector Machine classifier with cardiac and electrodermal features to decode art interest. Although, average performance was poor (54%), the model yielded high accuracy beyond chance level for a few participants. This shows that art interest can, at least for some individuals, be decoded from single trial physiological data.

## Introduction

Research in empirical aesthetics suggested that engaging with visual artworks can positively affect health, for example by reducing stress [1]. Aesthetic experience has been correlated to physiological measures such as heart beat and skin conductance features [2]. The same features have also been related to well-being [3]. It seems plausible that such features could be used to improve user experience in virtual museum by personalizing art presentation based on physiological responses. However, single trial decoding approaches of aesthetic experience from physiology have been lacking. In this work, we report single trial classification results of art interest from electrodermal activity (EDA) and photoplethysmography (PPG).

## Materials And Methods

We collected PPG and EDA data from 13 healthy adults while they watched 120 high quality digital reproductions of diverse visual artworks in a virtual art museum. The experiment consisted of 6 blocks of 20 artwork presentation and rating trials. At the start of each trial, a blank screen was displayed for a random interval between 0.5 and 0.8s. After that, a fixation cross appeared for 5s, in order to washout potential emotions evoked by previous artworks and to measure a stable baseline. Then, an artwork was presented for 10s. Finally, the participants reported their Interest evoked by artworks on a scale from 0-100. These ratings served as ground truth for subsequent aesthetic Interest classification. After each run, the participants were allowed to rest for a minute. PPG and EDA data were recorded at 32 Hz using a Shimmer3 GSR+ (Shimmer, Dublin, Ireland). EDA electrodes were placed on the palmar surfaces of the participants left medial and distal phalanges. The PPG sensor was put on the left earlobe. Balanced Interest/Disinterest classes were created by selecting the 45% highest and the 45% lowest rated artworks respectively. This partitioning resulted in ca. 55-58 samples per class per subject. Thus, we defined aesthetic Interest decoding as a subject-specific 2-class supervised classification problem with an estimated chance level (alpha=0.01) of 60.9% [4]. In order
to decode aesthetic interest for visual art from the physiological signals, the following pipeline was
used and validated with a 10-fold shuffled and stratified cross-validation: First, we manually inspected
the data to reject bad channels. This resulted in the rejection of the EDA channels of subjects
5,7,10,11,12 and 13. Then, we applied a 3rd order 0.5-8 Hz bandpass Butterworth filter to remove
noise for PPG and a 4th order 3 Hz Low-pass Butterworth filter for EDA. Then, trial epochs were
extracted from 1-10 s (t=0s being the start of the artwork display) during stimulus presentation and
baseline corrected from -4s to -0.2s.

We extracted the following features from each epoch for classification with Support Vector Machines (SVM) [5]: For PPG, number of Peaks, heart rate, mean of beat-to-beat interval, standard deviation of intervals and square root of the mean of the squared successive differences between adjacent intervals were computed over the whole epoch. For EDA, the following skin conductance response features were calculated on 3s windows with an overlap of 1s: number of peaks, mean response height and maximum height. In order to improve separability, the vector of concatenated normalized features was then transformed using supervised Uniform Manifold Approximation and Projection (UMAP) learned from the training data [6]. For each cross-validation fold, optimal hyperparameters (UMAP dimension, SVM C and Gamma) were selected using Bayesian hyperparameter search with a 10-fold shuffled and stratified cross-validation (inner cross-validation) on the training set data of that fold (from the outer cross-validation). The selected features were then fed into the classifier and aesthetic Interest decoding performance was evaluated by computing the mean test accuracy over outer cross-validation folds for each subject.

Results

Our aesthetic interest classification model performed quite poorly with a mean accuracy of 54±12%
(see Figure 1). Only two subjects yielded results above random chance level (70% and 88%
respectively).

Conclusion

Although, average art interest decoding was bad, the high performance well above chance level for two participants seems promising. Further post-hoc analysis with feature elimination showed that EDA features contributed strongly to the good classification results. This could be due to individual preferences, i.e. some subjects might be interested in arousing or calming art, whereas the interests of others could run along different axes, e.g aesthetic understanding. Future research should include more phenomenological methods in order to identify these preferences. Unfortunately, EDA was very noisy for many participants which limits its applicability, especially for more ecologically valid applications outside the lab.

Regarding cardiac features, it might very well be that 10s trials are too short to obtain discriminable information from PPG. Although informative cardiac features from ultra-short time segments have been reported with Electrocardiography, the less robust and reliable PPG usually requires longer data [7].

Furthermore, aesthetic experience might be too complex of a psychological construct for physiological decoding applications. Instead, one could for example ask users at the beginning if they want to have a calming experience and then adapt art presentation based on physiological correlates of relaxation. Finally, our experimental protocol included a subjective rating task and forced participants to look at all artworks for the same time interval. This resulted in an unaturalistic and constrained experience compared to art gazing in real museums which limits the interpretability of the results. Therefore, more ecologically valid protocols need to be developed and tested with larger sample sizes.
<figure><img src="image_2.png" /><figcaption>Figure 1: The figure shows the mean aesthetic interest decoding accuracy for all subjects. The horizontal blue line indicates random chance level for 55 trials per class.</figcaption></figure>

## References

[1] Fancourt, D. & Finn, S. What is the Evidence on the Role of the Arts in Improving Health and Well-being? A Scoping Review. Health Evidence Network Synthesis Report, 67.

[2] Tschacher, W. et al. (2011). Physiological Correlates of Aesthetic Perception of Artworks in a Museum. Psychology of Aesthetics, Creativity and the Arts, 6(1), 96-103.

[3] Schmidt, P., Reiss, A., Dürichen, R., & Van Laerhoven, K. (2019). Wearable-Based Affect Recognition—A Review. Sensors, 19.

[4] Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of neuroscience methods, 250, 126-136.

[5] Cortes, C. & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297.

[6] Healy, J. & McInnes, L. (2024). *Uniform manifold approximation and projection*. Nat Rev Methods Primers 4(82).

[7] Mejía-Mejía, E. & Kyriacou, P.A. (2023). Duration of photoplethysmographic signals for the extraction of Pulse Rate Variability Indices. Biomedical Signal Processing and Control, 80(1).