# Remote Facial Expression and Heart Rate Measurements to Assess Human Reactions in Glass Structures

Authors: Chiara Bedon; Silvana Mattei
Journal: Advances in Civil Engineering
 (2021)

---

## Abstract

In engineering applications, human comfort fulfillment is challenging because it depends on several aspects that can be mathematically controlled and optimized, like in case of structural, energy, or thermal issues, and others. Major troubles can indeed derive from combined human reactions, which are related to a multitude of aspects. The so-called “emotional architecture” and its nervous feelings are part of the issue. The interaction of objective and subjective parameters can thus make the “optimal” building design complex. This paper presents a pilot experimental investigation developed remotely to quantify the reactions and nervous states of 10 volunteers exposed to structural glass environments. As known, intrinsic material features (transparency, brittleness, etc.) require specific engineering knowledge for safe mechanical design but can in any case evoke severe subjective feelings for customers, thus affecting their psychological comfort and hence behaviour and movements. This study takes advantage of static/dynamic Virtual Reality (VR) environments and facial expression analyses, with Artificial Intelligence tools that are used to measure both Action Units (AUs) of facial microexpressions and optical heart rate (HR) acquisitions of volunteers exposed to VR scenarios. As shown, within the limits of collected records, the postprocessing analysis of measured signals proves that a rather good correlation can be found for measured AUs, HR data trends, and emotions under various glazing stimuli. Such a remote experimental approach could be thus exploited to support the early design stage of structural glass members and assemblies in buildings.


---

## Body

## 1. Introduction

It is generally recognized that human comfort in the built environment is a target for a multitude of aspects [1,2]. Engineering tools that can be used to optimize comfort are able to account for a great number of factors and parameters which can affect comfort levels severely. Typical applications are focused on thermal comfort, indoor air quality, visual comfort, noise nuisance, ergonomics, vibrations, and others. In addition, it is also recognized that other aspects (like, for example, personal factors, nervous states, and architectural parameters) should be also properly considered in the list of influencing indicators for comfort. There are no doubts about the correlation of built environment characteristics and its impact on the occupants’ emotions, behaviours, and physical well-being [2]. The same behaviours can also affect typical engineering issues and indirectly influence the building design and detailing [3–6].Structurally speaking, vibration serviceability issues and the consequent risk of user discomfort are, for example, usually addressed in terms of acceleration peaks and recommended limit parameters. Their satisfaction ensures possible annoyance for customers or suggests a design retrofit/modification [7–12]. As a matter of fact, however, there are no doubts that human reactions to vibrations strongly depend on the physiological perception of frequency and amplitude of vibrations. The operational context and the nervous state of users have thus a severe impact on the degree of human tolerance of vibration issues [9]. Most importantly, such a set of feelings can be maximized in presence of uncomfortable architectures and building components.The well-known psychological effect of architecture can have both positive and negative effects on users and thus can evoke subjective feelings that could possibly interact with mechanical choices and technical detailing of structural design [13]. This is typical of architects that evoke nervous states in the so-called “emotional buildings” [14–19]. Pilot literature studies, in this regard, proved that experimental measures can be carried out to quantify users emotions [20]. Smart sensors for real or Virtual Reality (VR) walks proved also to represent useful tools in support of the quantitative analysis of human reactions and emotions to building stimuli [21,22]. In this paper, a special care is spent for structural glass applications in buildings. Known as versatile but vulnerable constructional material, glass transparency and capacity to adapt to various setup configurations make it a largely used solution [23]. The high aesthetic impact of glass structures can be thus sometimes in contrast with the need of more efficient feeling of protection for the occupants, as it could be for extreme accidents, pedestrian systems, or uncomfortable configurations (Figure 1). Glass walkways are in fact often known as “architectures of vertigo” [26], where transparent structures are conceived as spaces of visceral thrills and intense psychophysiological stimuli with deep sensory experience and sociospatial implications. Additionally, glass pedestrian systems can be highly demanding in engineering terms, due to the mutual interaction of mechanical dynamic parameters, complex Human-Structure Interaction (HSI) phenomena, and even severe subjective reactions [9]. The analytical analysis and quantitative measure of emotional states for customers can thus represent a useful tool in the whole design process.Figure 1
Selection of glass built scenarios: (a) cracks and shards in glass walls and windows under hazard (reproduced from [24] with permission from Elsevier®, copyright license agreement n. 5042381253637, April 2021); (b) indoor pedestrian system (walkway in Aquileia, IT; reproduced from [9] under the terms and conditions of CC-BY license); (c) Cliffside skyway, CN (reproduced from [25] under the terms and conditions of CC-BY license).
(a)(b)(c)


## 2. Research Goals and Experimental Strategy

A virtual experimental study is presented and assessed in this paper, in support of structural glass design. The goal of the proposed method is to quantify the prevailing human reactions and comfort levels for occupants that are exposed to different virtual built environments made of/with a primary role of glass. The measured reactions are then compared to address the psychological comfort level of volunteers under imposed visual stimuli. In accordance with Figure2, the process takes advantage of a quantitative analysis of emotions based on the analysis of facial microexpressions (Action Units (AUs) of nervous states) and also heart rate (HR) parameters for volunteers exposed to several glass built scenarios.Figure 2
Example of (a) glass structure used as input stimulus (apartment floor in Paris, FR, ©Jerry Jacobs Design) and (b) remote facial expression analysis.
(a)(b)While the use of glass material in buildings is largely increasing and further demanded in the post-Covid-19 stage [27,28], its impact on the psychological comfort of customers can be severely affected by transparency, intrinsic brittleness, lack of technical knowledge on its structural performance, vulnerability risk, etc. [29,30].Glass material for structural applications is in fact known to require specific engineering knowledge for optimal and safe mechanical design. Besides, discomfort for building in occupants can still arise and thus affect their behaviour and movements inside/onto a glass system. Typical examples can be related to glass pedestrian systems or structural glass barriers with risk of shards/fall.In this regard, under the limits of remote experimental measures, and based on the herein presented research outcomes, it is expected that building structural design could take advantage of subjective measurements in support of classical mathematical tools and engineering models. Besides, real in situ experimental procedures should be also carried out for a further extension of the methodology.More in detail, the pilot experimental investigation was carried out with the active contribution of 10 participants and the FaceReaderTM automatic facial expression recognition software [31]. The number of volunteers was chosen based on the availability of individual participants and on the number of experimental records to process for each participant/stimulus/time interval. The group of participants was selected from a class of volunteers preliminary involved in measurement trials discussed in [25]. The class of volunteers included 60% females and 40% males, with an average age of 28 years. No preliminary technical knowledge on structural glass in constructions was required at the time of the experiment.The remote experimental procedure was planned as in Figure3, as an adaptation and extension of [25]. Two different stimuli were designed to address nervous states and human reactions of volunteers, namely, a VR set of “static” items and a “dynamic” video clip of virtual walks. A special attention was paid for the selection of stimuli, so as to capture different facial microexpressions and emotions. For this reason, a preliminary Computer Assisted Web Interviewing (CAWI) survey was also distributed in Winter 2020 and used to explore the prevailing subjective reactions for the selected participants asked to virtually deal with buildings or open spaces characterized by a prevailing role of structural glass. At the end of the remote experiments, moreover, the participants were asked to comment about their feelings.Figure 3
Flowchart of the virtual experimental analysis.There are several literature studies which confirm the potential of human emotions as a guide for designers in different research fields [32,33]. Besides, such a result can be achieved in different ways. Facial microexpressions are, for example, considered essential for the quantitative analysis of basic emotions, because human faces provide useful information about feelings and the inner individual states. At the same time, there is a large number of literature experiments that confirm the strict correlation between human reactions and emotions with HR data and their variations over time [34–36], as well as the useful feedback that can be derived from biosignals [37–39]. In the present study, both facial expressions and optical HR data are recorded for volunteers exposed to virtual glass structures.As shown in Sections3–5, the analysis of experimental results confirms that the use of glass in buildings is still largely affected by scattered human reactions. For selected stimuli, a rather close match can be observed for most of participants. It is worth to be noted that a good trend in facial microexpressions/emotions and HR data can be found for the imposed scenarios. Such an outcome, consequently, suggests the comparative analysis of remote facial measures with possible in situ experimental records.


## 3. Virtual Experimental Analysis

### 3.1. Remote Testing

The procedure schematized in Figure4 was repeated twice for each volunteer, first with a static VR input and later with a VR video clip of virtual walks. All the experiments were carried out remotely, under Covid-19 restrictions to mobility, with the support of laptops, a shared screen for the input stimuli, and a web cam for recording the face of participants.Figure 4
Flowchart for the virtual experimental investigation.The invited subjects were preliminary informed about how the experiment would be conducted. To this aim, the setup was properly checked, so as to avoid disturbing effects for the record acquisition and for the postprocessing analysis. A special attention was given to ensure an optimal and mostly uniform ambient illumination and face exposure for the participants. The position of the screen for each participant was also checked, so as to avoid distortions or introduce misleading inclination values for the experimental acquisition. Finally, the presence of participants with glasses or beard was allowed but separately noted, for a more refined analysis of collected data. Once the operational conditions were verified, the input source was shared to all the participants. During the presentations, audio communication was allowed to provide additional comfort to the participants. No audio source was indeed introduced in addition to the visual input. During each one of the presentations, the webcam of controlling machine was used to record the facial expressions of the participants, while looking at the shared screen. For this purpose, a preliminary tentative to capture the best cam resolution and sampling rate was carried out. The final choice resulted in high quality records with minimum 1280 × 720 (or 1920 × 1028) video resolution and a total of frames in the range of 4300 fo each signal (30–60, the range of frame rate; 12432 kbits/s, the average bitrate). The software analysis was based on all the collected frames for each record, thus requiring an average of 10 minutes/setup.


### 3.2. Measurement of Raw Signals

The analysis and interpretation of facial expressions and heart rate data for the participants exposed to VR stimuli was carried out with a hybrid approach. A fundamental role was assigned to the automatic analysis offered by FaceReaderTM, for the quantification of basic human reactions, emotional states, and HR data as in Figure 2. Furthermore, the manual elaboration of software results represented a key step of analysis.The selected commercial software was used because it is based on Artificial Intelligence tools and can efficiently detect subjective emotions, human reactions, and HR parameters from input video records. Based on 500 key points for the description of facial movements, the output classifications ofN = 7 nervous states based on facial expressions include “happy”, “sad”, “scared”, “disgusted”, “surprised”, “angry”, and “neutral” feelings. These states are normalized and quantified in terms of instantaneous evaluation of Action Units (AU) for basic emotions. Among others, an additional intrinsic advantage of the used software is represented by the optical measure of HR of subjects, based on complex remote photoplethysmography (rPPG) optical techniques. The rPPG module analyzes HR data and HR variability from input videos of participants, by quantifying the amount of light that is reflected by their face. The latter relates to cardiac cycles and changes in blood volume based on video captured by the camera [40]. Different literature studies proved that the remote HR analysis can be powerful and efficient [41–44]. The limit requirement is that setup conditions for the experiments are properly addressed. Possible movements of participants (i.e., while walking, talking, etc.) may result in overestimates or underestimates for real HR data [45,46].In the present study, the implicit advantage was represented by constant ambient conditions for all the participants and by their mostly fixed position during the whole experimental analysis (sitting on a chair). Moreover, a preliminary check of setup features resulted in input video records for the software generally classified as “high quality” video sources for remote analysis (Figure5). An example of resulting AUs and HR records is shown in Figure 6 for one of the involved participants. The experimental measurements for each participant/virtual experiment/inner state were in fact collected in the form of AU charts of single basic emotions (in the range from 0 to 1) and HR data (bpm), as a function of time.Figure 5
Quality level for the input video records of each participant, under static or dynamic VR input stimuli.Figure 6
Example of AU (0–1 range) records of facial expressions and HR data (bmp), over the time of the experimental analysis, as obtained for one of the participants under VR stimuli.


### 3.3. Postprocessing Strategy for Measured Raw Signals

A set of charts agreeing with Figure6 was used for the interpretation of human reactions and for the analysis of their possible correlations with the imposed VR stimuli. To this aim, postprocessing feedback from volunteers was also used to double-check the emotional states. It is worth to be noted that the chart in Figure 6 omits the first instants of records, where a preliminary discussion was carried out with each participant to provide instructions and create a comfortable condition.


#### 3.3.1. Action Units and Emotions

Given that multiple AU records were collected for each participant/state/VR stimulus, the first postprocessing step consisted in the detailed analysis of single AUs. The first effort was spent for the normalization of AU records in time, so as to offer a more independent analysis and comparison of AU records for multiple participants. This was measured as(1)AUnormt=AUoriginalt−AUinitial,avgt.with 0 ≤ t ≤ 120 s and AUnorm(t), the normalized record for each participant, emotion, and VR scenario, at the time instant t; AUoriginal(t), the original signal record for each participant, emotion, and VR scenario, at the time instant t; AUinitial,avg the mean value of each emotion, participant, and VR scenario (preliminary stage).Following equation (1), the individual records of basic emotions, AUnorm(t), were successively grouped in basic pos∗(t) and neg∗(t) plots. In the pos∗(t) set, the AU data in time for feelings marked as “happy” were considered only:(2)pos∗t=∑i=1N−1AUnormt=fhappy.Similarly, the neg∗(t) set included the input data from feelings marked as “sad”, “angry”, “scared”, and “disgusted”:(3)neg∗t=∑i=1N−1AUnormt=fsad,angry,scared,disgusted.The “surprised” AU data (“SUR(t)”) was disregarded in the so-calculated pos∗(t) and neg∗(t) signals but analyzed separately. Depending on the individual subject, context, and stimulus, “surprised” states can be in fact typically associated with both positive or negative feelings [25]. For the same reason, “neutral” states were omitted from the study, due to the virtual remote setup. The final marking of emotional data was based on the analysis of prevailing AUs at a given time instant t, where the SUR(t) input was superimposed as(4)POS∗t=pos∗t+SURtifpos∗t>neg∗t,pos∗tifpos∗t<neg∗t,or(5)NEG∗t=neg∗t+SURtifneg∗t>pos∗t,neg∗tifneg∗t<pos∗t.In conclusion, the final detection of comfort trends for a given visual stimulus was carried out based on the percentage variation of elaborated signals, over the time of analysis. At each incrementt of the experiment, the emotional variation was calculated for both POS∗(t) and NEG∗(t) data as(6)Δ%=100Ytn−Ytn−1Ytn−1,with Y = POS∗(t) or NEG∗(t), respectively, as in the example of Figure 7.Figure 7
Example of calculated (a) POS∗ and (b) NEG∗ measurements of AU signals, as obtained for one of the participants under the VR input source.
(a)(b)The prevailing emotional state (and thus comfort trend) was established analytically, in terms of absolute maximum values of nervous feelings and max/min percentage variations. Each time instantt was classified with a final POS or NEG mark, uniquely detected from data elaboration. A unitary “comfort weight” value C(t) = POS or NEG (based on the prevailing nervous state) was assigned to each participant/instant/scenario, to facilitate the comparative analysis of multiple signals and stimuli. More precisely,for static VR, the primary/prevailing emotion based on AUs was calculated for each one of the 27 input items andfor dynamic VR input, the attention was still focused on primary emotions based on AUs, but for specific frames only that were detected and selected from absolute AU peaks and building configurations.


#### 3.3.2. Heart Rate and Emotions

The elaboration of HR data derived from rPPG techniques was based on the analysis of single measures over the time of the experiment. Studies of literature (see, for example, [21] and Figure 8) demonstrated that HR measures can offer useful feedback for the quantification of emotions and human feelings. In the present study, based also on the relatively short duration of stimuli, the attention was focused on the comparison of HR data trends with facial expression measurements, for a given participant/stimulus. In the present study, the HR variation in time was labelled as HR+(t) or HR-(t), respectively (with 0 ≤ t ≤ 120s), with(7a)HR+tHRt>HRt−1,or(7b)HR−tHRt<HRt−1.Figure 8
Heart rate analysis over the time of VR experiment: (a) procedure presented in [21] (figure reproduced under the terms and conditions of a CC-BY license) and (b) current study.
(a)(b)As in Section3.3.1, a unitary “comfort weight” C(t) was assigned to the HR+ and HR− parameters, so that each instant t could be quantified as in equation (7) with a POS or NEG reaction. Care was paid especially for HR+ data, so as to detect possible discomfort/NEG feelings of volunteers, while the HR decrease (HR−) was related to comfortable situations. Furthermore, a null comfort weight was assigned to the limit condition HR(t) = HR(t-1).


#### 3.3.3. Correlation of Heart Rate and Facial Expression Measurements

The combined analysis of AUs and HR outcomes for the comfort quantification of participants was carried out by comparison of normalized weights over the time of the experiment. In this manner, the attention was focusedOn the analysis of single elaborated signals for a given participant and stimulus (HR or facial AU)On the synchronized analysis of HR and facial expression data for a single participant, over the time of experiment; in this case, the attention was focused on the potential agreement of AU and HR data and trends at a given time instanttOn the analysis of reaction trends from the whole group of volunteers


## 3.1. Remote Testing

The procedure schematized in Figure4 was repeated twice for each volunteer, first with a static VR input and later with a VR video clip of virtual walks. All the experiments were carried out remotely, under Covid-19 restrictions to mobility, with the support of laptops, a shared screen for the input stimuli, and a web cam for recording the face of participants.Figure 4
Flowchart for the virtual experimental investigation.The invited subjects were preliminary informed about how the experiment would be conducted. To this aim, the setup was properly checked, so as to avoid disturbing effects for the record acquisition and for the postprocessing analysis. A special attention was given to ensure an optimal and mostly uniform ambient illumination and face exposure for the participants. The position of the screen for each participant was also checked, so as to avoid distortions or introduce misleading inclination values for the experimental acquisition. Finally, the presence of participants with glasses or beard was allowed but separately noted, for a more refined analysis of collected data. Once the operational conditions were verified, the input source was shared to all the participants. During the presentations, audio communication was allowed to provide additional comfort to the participants. No audio source was indeed introduced in addition to the visual input. During each one of the presentations, the webcam of controlling machine was used to record the facial expressions of the participants, while looking at the shared screen. For this purpose, a preliminary tentative to capture the best cam resolution and sampling rate was carried out. The final choice resulted in high quality records with minimum 1280 × 720 (or 1920 × 1028) video resolution and a total of frames in the range of 4300 fo each signal (30–60, the range of frame rate; 12432 kbits/s, the average bitrate). The software analysis was based on all the collected frames for each record, thus requiring an average of 10 minutes/setup.


## 3.2. Measurement of Raw Signals

The analysis and interpretation of facial expressions and heart rate data for the participants exposed to VR stimuli was carried out with a hybrid approach. A fundamental role was assigned to the automatic analysis offered by FaceReaderTM, for the quantification of basic human reactions, emotional states, and HR data as in Figure 2. Furthermore, the manual elaboration of software results represented a key step of analysis.The selected commercial software was used because it is based on Artificial Intelligence tools and can efficiently detect subjective emotions, human reactions, and HR parameters from input video records. Based on 500 key points for the description of facial movements, the output classifications ofN = 7 nervous states based on facial expressions include “happy”, “sad”, “scared”, “disgusted”, “surprised”, “angry”, and “neutral” feelings. These states are normalized and quantified in terms of instantaneous evaluation of Action Units (AU) for basic emotions. Among others, an additional intrinsic advantage of the used software is represented by the optical measure of HR of subjects, based on complex remote photoplethysmography (rPPG) optical techniques. The rPPG module analyzes HR data and HR variability from input videos of participants, by quantifying the amount of light that is reflected by their face. The latter relates to cardiac cycles and changes in blood volume based on video captured by the camera [40]. Different literature studies proved that the remote HR analysis can be powerful and efficient [41–44]. The limit requirement is that setup conditions for the experiments are properly addressed. Possible movements of participants (i.e., while walking, talking, etc.) may result in overestimates or underestimates for real HR data [45,46].In the present study, the implicit advantage was represented by constant ambient conditions for all the participants and by their mostly fixed position during the whole experimental analysis (sitting on a chair). Moreover, a preliminary check of setup features resulted in input video records for the software generally classified as “high quality” video sources for remote analysis (Figure5). An example of resulting AUs and HR records is shown in Figure 6 for one of the involved participants. The experimental measurements for each participant/virtual experiment/inner state were in fact collected in the form of AU charts of single basic emotions (in the range from 0 to 1) and HR data (bpm), as a function of time.Figure 5
Quality level for the input video records of each participant, under static or dynamic VR input stimuli.Figure 6
Example of AU (0–1 range) records of facial expressions and HR data (bmp), over the time of the experimental analysis, as obtained for one of the participants under VR stimuli.


## 3.3. Postprocessing Strategy for Measured Raw Signals

A set of charts agreeing with Figure6 was used for the interpretation of human reactions and for the analysis of their possible correlations with the imposed VR stimuli. To this aim, postprocessing feedback from volunteers was also used to double-check the emotional states. It is worth to be noted that the chart in Figure 6 omits the first instants of records, where a preliminary discussion was carried out with each participant to provide instructions and create a comfortable condition.


### 3.3.1. Action Units and Emotions

Given that multiple AU records were collected for each participant/state/VR stimulus, the first postprocessing step consisted in the detailed analysis of single AUs. The first effort was spent for the normalization of AU records in time, so as to offer a more independent analysis and comparison of AU records for multiple participants. This was measured as(1)AUnormt=AUoriginalt−AUinitial,avgt.with 0 ≤ t ≤ 120 s and AUnorm(t), the normalized record for each participant, emotion, and VR scenario, at the time instant t; AUoriginal(t), the original signal record for each participant, emotion, and VR scenario, at the time instant t; AUinitial,avg the mean value of each emotion, participant, and VR scenario (preliminary stage).Following equation (1), the individual records of basic emotions, AUnorm(t), were successively grouped in basic pos∗(t) and neg∗(t) plots. In the pos∗(t) set, the AU data in time for feelings marked as “happy” were considered only:(2)pos∗t=∑i=1N−1AUnormt=fhappy.Similarly, the neg∗(t) set included the input data from feelings marked as “sad”, “angry”, “scared”, and “disgusted”:(3)neg∗t=∑i=1N−1AUnormt=fsad,angry,scared,disgusted.The “surprised” AU data (“SUR(t)”) was disregarded in the so-calculated pos∗(t) and neg∗(t) signals but analyzed separately. Depending on the individual subject, context, and stimulus, “surprised” states can be in fact typically associated with both positive or negative feelings [25]. For the same reason, “neutral” states were omitted from the study, due to the virtual remote setup. The final marking of emotional data was based on the analysis of prevailing AUs at a given time instant t, where the SUR(t) input was superimposed as(4)POS∗t=pos∗t+SURtifpos∗t>neg∗t,pos∗tifpos∗t<neg∗t,or(5)NEG∗t=neg∗t+SURtifneg∗t>pos∗t,neg∗tifneg∗t<pos∗t.In conclusion, the final detection of comfort trends for a given visual stimulus was carried out based on the percentage variation of elaborated signals, over the time of analysis. At each incrementt of the experiment, the emotional variation was calculated for both POS∗(t) and NEG∗(t) data as(6)Δ%=100Ytn−Ytn−1Ytn−1,with Y = POS∗(t) or NEG∗(t), respectively, as in the example of Figure 7.Figure 7
Example of calculated (a) POS∗ and (b) NEG∗ measurements of AU signals, as obtained for one of the participants under the VR input source.
(a)(b)The prevailing emotional state (and thus comfort trend) was established analytically, in terms of absolute maximum values of nervous feelings and max/min percentage variations. Each time instantt was classified with a final POS or NEG mark, uniquely detected from data elaboration. A unitary “comfort weight” value C(t) = POS or NEG (based on the prevailing nervous state) was assigned to each participant/instant/scenario, to facilitate the comparative analysis of multiple signals and stimuli. More precisely,for static VR, the primary/prevailing emotion based on AUs was calculated for each one of the 27 input items andfor dynamic VR input, the attention was still focused on primary emotions based on AUs, but for specific frames only that were detected and selected from absolute AU peaks and building configurations.


### 3.3.2. Heart Rate and Emotions

The elaboration of HR data derived from rPPG techniques was based on the analysis of single measures over the time of the experiment. Studies of literature (see, for example, [21] and Figure 8) demonstrated that HR measures can offer useful feedback for the quantification of emotions and human feelings. In the present study, based also on the relatively short duration of stimuli, the attention was focused on the comparison of HR data trends with facial expression measurements, for a given participant/stimulus. In the present study, the HR variation in time was labelled as HR+(t) or HR-(t), respectively (with 0 ≤ t ≤ 120s), with(7a)HR+tHRt>HRt−1,or(7b)HR−tHRt<HRt−1.Figure 8
Heart rate analysis over the time of VR experiment: (a) procedure presented in [21] (figure reproduced under the terms and conditions of a CC-BY license) and (b) current study.
(a)(b)As in Section3.3.1, a unitary “comfort weight” C(t) was assigned to the HR+ and HR− parameters, so that each instant t could be quantified as in equation (7) with a POS or NEG reaction. Care was paid especially for HR+ data, so as to detect possible discomfort/NEG feelings of volunteers, while the HR decrease (HR−) was related to comfortable situations. Furthermore, a null comfort weight was assigned to the limit condition HR(t) = HR(t-1).


### 3.3.3. Correlation of Heart Rate and Facial Expression Measurements

The combined analysis of AUs and HR outcomes for the comfort quantification of participants was carried out by comparison of normalized weights over the time of the experiment. In this manner, the attention was focusedOn the analysis of single elaborated signals for a given participant and stimulus (HR or facial AU)On the synchronized analysis of HR and facial expression data for a single participant, over the time of experiment; in this case, the attention was focused on the potential agreement of AU and HR data and trends at a given time instanttOn the analysis of reaction trends from the whole group of volunteers


## 3.3.1. Action Units and Emotions

Given that multiple AU records were collected for each participant/state/VR stimulus, the first postprocessing step consisted in the detailed analysis of single AUs. The first effort was spent for the normalization of AU records in time, so as to offer a more independent analysis and comparison of AU records for multiple participants. This was measured as(1)AUnormt=AUoriginalt−AUinitial,avgt.with 0 ≤ t ≤ 120 s and AUnorm(t), the normalized record for each participant, emotion, and VR scenario, at the time instant t; AUoriginal(t), the original signal record for each participant, emotion, and VR scenario, at the time instant t; AUinitial,avg the mean value of each emotion, participant, and VR scenario (preliminary stage).Following equation (1), the individual records of basic emotions, AUnorm(t), were successively grouped in basic pos∗(t) and neg∗(t) plots. In the pos∗(t) set, the AU data in time for feelings marked as “happy” were considered only:(2)pos∗t=∑i=1N−1AUnormt=fhappy.Similarly, the neg∗(t) set included the input data from feelings marked as “sad”, “angry”, “scared”, and “disgusted”:(3)neg∗t=∑i=1N−1AUnormt=fsad,angry,scared,disgusted.The “surprised” AU data (“SUR(t)”) was disregarded in the so-calculated pos∗(t) and neg∗(t) signals but analyzed separately. Depending on the individual subject, context, and stimulus, “surprised” states can be in fact typically associated with both positive or negative feelings [25]. For the same reason, “neutral” states were omitted from the study, due to the virtual remote setup. The final marking of emotional data was based on the analysis of prevailing AUs at a given time instant t, where the SUR(t) input was superimposed as(4)POS∗t=pos∗t+SURtifpos∗t>neg∗t,pos∗tifpos∗t<neg∗t,or(5)NEG∗t=neg∗t+SURtifneg∗t>pos∗t,neg∗tifneg∗t<pos∗t.In conclusion, the final detection of comfort trends for a given visual stimulus was carried out based on the percentage variation of elaborated signals, over the time of analysis. At each incrementt of the experiment, the emotional variation was calculated for both POS∗(t) and NEG∗(t) data as(6)Δ%=100Ytn−Ytn−1Ytn−1,with Y = POS∗(t) or NEG∗(t), respectively, as in the example of Figure 7.Figure 7
Example of calculated (a) POS∗ and (b) NEG∗ measurements of AU signals, as obtained for one of the participants under the VR input source.
(a)(b)The prevailing emotional state (and thus comfort trend) was established analytically, in terms of absolute maximum values of nervous feelings and max/min percentage variations. Each time instantt was classified with a final POS or NEG mark, uniquely detected from data elaboration. A unitary “comfort weight” value C(t) = POS or NEG (based on the prevailing nervous state) was assigned to each participant/instant/scenario, to facilitate the comparative analysis of multiple signals and stimuli. More precisely,for static VR, the primary/prevailing emotion based on AUs was calculated for each one of the 27 input items andfor dynamic VR input, the attention was still focused on primary emotions based on AUs, but for specific frames only that were detected and selected from absolute AU peaks and building configurations.


## 3.3.2. Heart Rate and Emotions

The elaboration of HR data derived from rPPG techniques was based on the analysis of single measures over the time of the experiment. Studies of literature (see, for example, [21] and Figure 8) demonstrated that HR measures can offer useful feedback for the quantification of emotions and human feelings. In the present study, based also on the relatively short duration of stimuli, the attention was focused on the comparison of HR data trends with facial expression measurements, for a given participant/stimulus. In the present study, the HR variation in time was labelled as HR+(t) or HR-(t), respectively (with 0 ≤ t ≤ 120s), with(7a)HR+tHRt>HRt−1,or(7b)HR−tHRt<HRt−1.Figure 8
Heart rate analysis over the time of VR experiment: (a) procedure presented in [21] (figure reproduced under the terms and conditions of a CC-BY license) and (b) current study.
(a)(b)As in Section3.3.1, a unitary “comfort weight” C(t) was assigned to the HR+ and HR− parameters, so that each instant t could be quantified as in equation (7) with a POS or NEG reaction. Care was paid especially for HR+ data, so as to detect possible discomfort/NEG feelings of volunteers, while the HR decrease (HR−) was related to comfortable situations. Furthermore, a null comfort weight was assigned to the limit condition HR(t) = HR(t-1).


## 3.3.3. Correlation of Heart Rate and Facial Expression Measurements

The combined analysis of AUs and HR outcomes for the comfort quantification of participants was carried out by comparison of normalized weights over the time of the experiment. In this manner, the attention was focusedOn the analysis of single elaborated signals for a given participant and stimulus (HR or facial AU)On the synchronized analysis of HR and facial expression data for a single participant, over the time of experiment; in this case, the attention was focused on the potential agreement of AU and HR data and trends at a given time instanttOn the analysis of reaction trends from the whole group of volunteers


## 4. Discussion of Stimuli and Results

### 4.1. Static VR Input

The participants were subjected to a selection of 27 pictures, equally spaced at time intervals of 5 seconds. Figure9 shows some examples, where labels #n denote the order of items. For each subject, the stimuli sequence was kept fixed. The items were selected from magazines, scientific journals, construction companies webpages, and newspapers. Major advantage for the selection of items was taken from the preliminary CAWI survey (Step 1 in Figure 3).Figure 9
Selection of static input sources for the static VR experiment: (a) private room (Off-grid itHouse, Pioneertown, USA; © Airbnb); (b) Amsterdam RAI Hotel, NL (courtesy of © A Bakker); (c) Hongyagu bridge, Hebei, CN (© REUTERS/Stringer); (d) windows under blast hazard; (e) stair (Apple Cube, New York, USA, © Sedak GmbH & Co KG); (f) facade and walls; (g) window under vehicle impact; (h) Space Needle tower (Seattle, WA, USA); (i) skyscraper (Willis Tower, Chicago, USA). (a) Item #1; (b) item #2; (c) item #4; (d) item #6; (e) item #8; (f) item #11; (g) item #12; (h) item #15; (i) item #20.
(a)(b)(c)(d)(e)(f)(g)(h)(i)


### 4.2. Analysis of Prevailing Reactions Based on AUs and HR Data

Average POS and NEG reactions were first calculated for each static item and participant. Figure10 shows the prevailing reactions in the group of volunteers, giving emphasis on measured (a) discomfort (NEG and HR+ data) or (b) comfort (POS and HR−), respectively.Figure 10
Detection of prevailing reactions (for the whole group of participants), as a function of the time of analysis, based on facial expression or heart rate data. In evidence, the measured (a) discomfort or (b) comfort of participants.
(a)(b)As far as a different time instant is selected in Figure10, it is worth of interest to note the variation of POS/NEG and HR+/HR− data for volunteers. The chart shows a clear variation of emotions and human reactions for an assigned stimulus, with marked fluctuations of POS and NEG data. Rather balanced POS/NEG conditions can be noticed for some instants of the experiment. Moreover (at least for some intervals), a good correlation of NEG/HR+ trends can be observed, in the same way of POS/HR− variations in time. Within the limitations of the study, such a finding confirms that HR increases corresponding to discomfort and negative feelings for participants, while the HR decrease can be related to a more comfortable situation and nervous state.A more concise analysis of measured AUs and HR data for each stimulus can be indeed carried out based on Figure11, where average experimental outcomes are proposed for the 27 items (5 seconds each). The good match of NEG and HR+ data for most of the pictures is worth to be noted. Rather close agreement (but for a limited number of items) can be observed also for POS and HR− trends, thus enforcing the potential of the procedure.Figure 11
Detection of prevailing reactions (for the whole group of participants), as a function of the input stimulus, based on facial expression or heart rate data. In evidence, the discomfort of participants.In this regard, some first conclusions about the experimental method can be derived when global results are grouped and analyzed as for items characterized byprevailing NEG reactions (#14, #19, #25, #27),prevailing POS reactions (#1, #10, #20, #21),mostly “balanced” NEG and POS reactions (#2, #3, #5, #15, #16),prevailing NEG values with a good match of HR+ reactions, to denote discomfort (#19, #25), andprevailing POS values with a good match of HR+ reactions (#10, #20, #21, #24).Selected items with marked NEG and POS reactions are proposed in Figures12 and 13, respectively.Figure 12
Static input items with prevailing (a)-(b) NEG, (c)-(d) POS reactions, or (e)-(f) balanced NEG-POS reactions: (a) Zhangjiajie bridge, Hunan, CN (© Getty Images); (b) cliff concept boutique hotel (© Hayri Atak); (c) mock-up of glass floor system (© Vitroplena bvba, BE); (d) walls (Amsterdam, NL (© A’DAM Lookout)); (e) walls (Glass Pavilion Rheinbach, DE (from [47] (© (F) Wellershoff)); (f) roof (Botanical Garden, Amsterdam, NL). (a) Item #25; (b) item #27; (c) item #10; (d) item #21; (e) item #3; (f) item #16.
(a)(b)(c)(d)(e)(f)Figure 13
Static input items with good match of NEG (or POS) and HR variation: (a) Markthal, Rotterdam, NL (© MVRDV); (b) Hubertus Hotel (Valdaora, IT, © Design & Contract). (a) Item #19; (b) item #24.
(a)(b)


### 4.3. Analysis of Reactions Based on the Subject of Items

A special care was paid for the analysis of experimental measurements for grouped static pictures. The attention was focused on glass structures and components representative of major categories of items and built environments, namely:Group A: pedestrian glass systems and structures or glass elements characterized by risk of fall for the occupants (items #4, #8, #10, #13, #15, #17, #18, #20, #22, #23, and #27).Group B: glass systems or elements with damage and/or under hazard (items #6, #9, #12, and #25).Group C: others (#1, #2, #3, #5, #7, #11, #14, #16, #19, #21, #24, and #26).Figure14 reports the residual static items from the full set of 27 pictures, while Table 1 presents a summary of prevailing reactions based on facial expressions and heart rate parameters (% of prevailing measurements for grouped items).Figure 14
Static input items: (a) private room (Off-grid itHouse, Pioneertown, USA; © Airbnb); (b) glass bricks (Chanel Amsterdam Store, NL, © MVRDV); (c) delamination (reproduced from [48] under the terms and permissions of a CC-BY license); (d) skywalk (Mahanakhon, Bangkok, THA, © Tripadvisor); (e) roof (30 St Mary Axe Tower, London, UK, © ArchDaily); (f) footbridge (glass bridge, Lisbon, PT, © Schlaich Bergermann Partner, DE); (g) skywalk (Jasper National Park, CA, © Getty Images); (h) walkway (Cliffside skyway, CN, © Global Times); (i) observation deck (360 Chicago Tilt, Chicago, USA, © Tripster); (j) underwater restaurant (N) © Dezeen. (a) Item #5; (b) item #7; (c) item #9; (d) item #13; (e) item #14; (f) item #17; (g) item #18; (h) item #22; (i) item #23; (j) item #26.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Table 1
Analysis of prevailing reactions for static VR items by subject (% values given for the items).
Prevailing reaction (facial expression) (%)Prevailing reaction (heart rate) (%)POSNEGHR-HR+Group A (11 items)18823763Group B (4 items)01005050Group C (12 items)48527030It is worth to be noted that there is a marked NEG response for groups A and B that are characterized by a major risk for the participants. In particular, the presence of damage (B) evokes high discomfort in the majority of the involved volunteers. Besides, for the set of pictures in group C, a more balanced response was measured. For the pictures in group A, the experimental outcome suggests that pedestrian glass systems involve the highest human reaction and intrinsic discomfort, compared to other glazing solutions in buildings. The effect can be justified by the fact that the human interaction with the structure becomes predominant. It is important to notice the POS effect of occupants in the pictures, as, for example, in the case of item #10.Even under the limitations of the experimental study herein presented, the collected results suggest that as far as the structural design of the glass system is optimally designed to preserve robustness, stability, and overall protective feedback for the occupants, the first human reaction is slightly on the side of POS feelings. In this regard, it is necessarily required to explore further this kind of scenario and extend the VR experimental stage with field experiments, so as to capture the human reactions of active pedestrians and combine these emotional feelings with the structure vibrations and other mechanical parameters of primary interest for the dynamic characterization of the structure itself under the effects of walking occupants.In terms of HR+ and HR− correlations in Table 1, finally, the collected results suggest some agreement with facial expression outcomes but also recommend the need of a more extended analysis in this direction (i.e., number of volunteers/stimuli).


## 4.1. Static VR Input

The participants were subjected to a selection of 27 pictures, equally spaced at time intervals of 5 seconds. Figure9 shows some examples, where labels #n denote the order of items. For each subject, the stimuli sequence was kept fixed. The items were selected from magazines, scientific journals, construction companies webpages, and newspapers. Major advantage for the selection of items was taken from the preliminary CAWI survey (Step 1 in Figure 3).Figure 9
Selection of static input sources for the static VR experiment: (a) private room (Off-grid itHouse, Pioneertown, USA; © Airbnb); (b) Amsterdam RAI Hotel, NL (courtesy of © A Bakker); (c) Hongyagu bridge, Hebei, CN (© REUTERS/Stringer); (d) windows under blast hazard; (e) stair (Apple Cube, New York, USA, © Sedak GmbH & Co KG); (f) facade and walls; (g) window under vehicle impact; (h) Space Needle tower (Seattle, WA, USA); (i) skyscraper (Willis Tower, Chicago, USA). (a) Item #1; (b) item #2; (c) item #4; (d) item #6; (e) item #8; (f) item #11; (g) item #12; (h) item #15; (i) item #20.
(a)(b)(c)(d)(e)(f)(g)(h)(i)


## 4.2. Analysis of Prevailing Reactions Based on AUs and HR Data

Average POS and NEG reactions were first calculated for each static item and participant. Figure10 shows the prevailing reactions in the group of volunteers, giving emphasis on measured (a) discomfort (NEG and HR+ data) or (b) comfort (POS and HR−), respectively.Figure 10
Detection of prevailing reactions (for the whole group of participants), as a function of the time of analysis, based on facial expression or heart rate data. In evidence, the measured (a) discomfort or (b) comfort of participants.
(a)(b)As far as a different time instant is selected in Figure10, it is worth of interest to note the variation of POS/NEG and HR+/HR− data for volunteers. The chart shows a clear variation of emotions and human reactions for an assigned stimulus, with marked fluctuations of POS and NEG data. Rather balanced POS/NEG conditions can be noticed for some instants of the experiment. Moreover (at least for some intervals), a good correlation of NEG/HR+ trends can be observed, in the same way of POS/HR− variations in time. Within the limitations of the study, such a finding confirms that HR increases corresponding to discomfort and negative feelings for participants, while the HR decrease can be related to a more comfortable situation and nervous state.A more concise analysis of measured AUs and HR data for each stimulus can be indeed carried out based on Figure11, where average experimental outcomes are proposed for the 27 items (5 seconds each). The good match of NEG and HR+ data for most of the pictures is worth to be noted. Rather close agreement (but for a limited number of items) can be observed also for POS and HR− trends, thus enforcing the potential of the procedure.Figure 11
Detection of prevailing reactions (for the whole group of participants), as a function of the input stimulus, based on facial expression or heart rate data. In evidence, the discomfort of participants.In this regard, some first conclusions about the experimental method can be derived when global results are grouped and analyzed as for items characterized byprevailing NEG reactions (#14, #19, #25, #27),prevailing POS reactions (#1, #10, #20, #21),mostly “balanced” NEG and POS reactions (#2, #3, #5, #15, #16),prevailing NEG values with a good match of HR+ reactions, to denote discomfort (#19, #25), andprevailing POS values with a good match of HR+ reactions (#10, #20, #21, #24).Selected items with marked NEG and POS reactions are proposed in Figures12 and 13, respectively.Figure 12
Static input items with prevailing (a)-(b) NEG, (c)-(d) POS reactions, or (e)-(f) balanced NEG-POS reactions: (a) Zhangjiajie bridge, Hunan, CN (© Getty Images); (b) cliff concept boutique hotel (© Hayri Atak); (c) mock-up of glass floor system (© Vitroplena bvba, BE); (d) walls (Amsterdam, NL (© A’DAM Lookout)); (e) walls (Glass Pavilion Rheinbach, DE (from [47] (© (F) Wellershoff)); (f) roof (Botanical Garden, Amsterdam, NL). (a) Item #25; (b) item #27; (c) item #10; (d) item #21; (e) item #3; (f) item #16.
(a)(b)(c)(d)(e)(f)Figure 13
Static input items with good match of NEG (or POS) and HR variation: (a) Markthal, Rotterdam, NL (© MVRDV); (b) Hubertus Hotel (Valdaora, IT, © Design & Contract). (a) Item #19; (b) item #24.
(a)(b)


## 4.3. Analysis of Reactions Based on the Subject of Items

A special care was paid for the analysis of experimental measurements for grouped static pictures. The attention was focused on glass structures and components representative of major categories of items and built environments, namely:Group A: pedestrian glass systems and structures or glass elements characterized by risk of fall for the occupants (items #4, #8, #10, #13, #15, #17, #18, #20, #22, #23, and #27).Group B: glass systems or elements with damage and/or under hazard (items #6, #9, #12, and #25).Group C: others (#1, #2, #3, #5, #7, #11, #14, #16, #19, #21, #24, and #26).Figure14 reports the residual static items from the full set of 27 pictures, while Table 1 presents a summary of prevailing reactions based on facial expressions and heart rate parameters (% of prevailing measurements for grouped items).Figure 14
Static input items: (a) private room (Off-grid itHouse, Pioneertown, USA; © Airbnb); (b) glass bricks (Chanel Amsterdam Store, NL, © MVRDV); (c) delamination (reproduced from [48] under the terms and permissions of a CC-BY license); (d) skywalk (Mahanakhon, Bangkok, THA, © Tripadvisor); (e) roof (30 St Mary Axe Tower, London, UK, © ArchDaily); (f) footbridge (glass bridge, Lisbon, PT, © Schlaich Bergermann Partner, DE); (g) skywalk (Jasper National Park, CA, © Getty Images); (h) walkway (Cliffside skyway, CN, © Global Times); (i) observation deck (360 Chicago Tilt, Chicago, USA, © Tripster); (j) underwater restaurant (N) © Dezeen. (a) Item #5; (b) item #7; (c) item #9; (d) item #13; (e) item #14; (f) item #17; (g) item #18; (h) item #22; (i) item #23; (j) item #26.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Table 1
Analysis of prevailing reactions for static VR items by subject (% values given for the items).
Prevailing reaction (facial expression) (%)Prevailing reaction (heart rate) (%)POSNEGHR-HR+Group A (11 items)18823763Group B (4 items)01005050Group C (12 items)48527030It is worth to be noted that there is a marked NEG response for groups A and B that are characterized by a major risk for the participants. In particular, the presence of damage (B) evokes high discomfort in the majority of the involved volunteers. Besides, for the set of pictures in group C, a more balanced response was measured. For the pictures in group A, the experimental outcome suggests that pedestrian glass systems involve the highest human reaction and intrinsic discomfort, compared to other glazing solutions in buildings. The effect can be justified by the fact that the human interaction with the structure becomes predominant. It is important to notice the POS effect of occupants in the pictures, as, for example, in the case of item #10.Even under the limitations of the experimental study herein presented, the collected results suggest that as far as the structural design of the glass system is optimally designed to preserve robustness, stability, and overall protective feedback for the occupants, the first human reaction is slightly on the side of POS feelings. In this regard, it is necessarily required to explore further this kind of scenario and extend the VR experimental stage with field experiments, so as to capture the human reactions of active pedestrians and combine these emotional feelings with the structure vibrations and other mechanical parameters of primary interest for the dynamic characterization of the structure itself under the effects of walking occupants.In terms of HR+ and HR− correlations in Table 1, finally, the collected results suggest some agreement with facial expression outcomes but also recommend the need of a more extended analysis in this direction (i.e., number of volunteers/stimuli).


## 5. Discussion of Results from Dynamic VR Experiment

The second stage of remote investigation was carried out with the same group of participants exposed to a virtual walk in a glazing environment. Major benefit was taken from the adaptation of a VR clip representative of a real case-study building (Generali Real Estate French Branch) located in Paris. The building, as shown in Figure15, is currently under renovation and is characterized by a large amount of structural glass in facades, roofs, and floors. In order to preserve a total duration of 120 seconds for the whole experimental analysis, the selection of clips for the case-study building was divided into three different virtual walks (40 seconds/each), realized at the roof/terrace level (W1), at the ground level, from the main entrance of the building (W2), and at the first level (W3), within the internal open space. More in detail, the walks were characterized by the following:W1 (outdoor): risk of fall (glass balustrades) and presence of glass facades.W2 (indoor): risk of fall (glass floor), but also glass roof on the top, glass walls, and internal partitions.W3 (indoor): risk of falls (glass balustrades) and presence of a glass floor (visual stimulus only).Figure 15
Selection of screenshot frames from the VR video clip (courtesy of © Generali Real Estate French Branch, adapted fromhttp://www.helloworldparis.com) inspired by the case-study palace in Paris, FR: (a) view of the building (from Rue Réaumur); (b) internal open space; (c) top terrace; (d) glass conical envelope/facade; (e) glass floor (from the top/ground level); (f) glass roof (from the bottom/ground level).
(a)(b)(c)(d)(e)(f)Figure16 shows the analysis of POS, NEG, and HR+ data for the group of volunteers. The dynamic VR scenario gave evidence of scattered emotional responses from the volunteers. Furthermore, the HR+ trend for the group of volunteers was observed to have close correlation with NEG data, thus confirming and enforcing, under the limitations of the present setup, the experimental findings from Section 4.Figure 16
Detection of prevailing reactions (for the whole group of participants), as a function of the time of analysis for the dynamic VR scenario.Worth of interest in Figure16 is the local and global fluctuation of POS or NEG peaks (and thus HR+ data) and their correlation with visual stimuli. Major NEG/HR+ peaks of discomfort were in fact detected for conditions characterized by the presence ofoutdoor or indoor balustrades (risk of fall) andfloors (risk of fall),while major comfortable responses were calculated for scenarios withfacades/walls (no direct contact) androof components (no direct contact),thus suggesting again an agreement with the experimental outcomes from the static VR scenario.On the other side, a more scattered and weak evolution of HR+ data with NEG values was noticed in Figure 16, compared to Section 4. A reason for such an outcome could be found in the dynamic nature of the stimulus itself [20,21].Certainly, a direct comparison of human reactions under static or dynamic VR stimuli is not possible. The context features as well as the subjective response of participants in a given remote scenario could be affected by several aspects. In Figure17, to this aim, the analysis of heart rate parameters is shown for all the volunteers exposed to the two experimental conditions. A mostly uniform response under static or dynamic stimuli can be observed only for a few participants.Figure 17
HR data (bpm) for the involved participants under static or dynamic VR setup: (a) average and (b) standard deviation.
(a)(b)In this regard, future investigations will be carried out to quantify more in detail the subjective reaction of occupants in glass building scenarios, as well as the correlation of design parameters and quantitative data of human reactions. Most importantly, instrumented experimental configurations will be examined under field scenarios with real walks from the participants and a new calibration of their subjective feelings.


## 6. Conclusions

The analysis, quantification, and optimization of human comfort in the built environment is a target for several design fields and depends on several engineering aspects that can be mathematically controlled to satisfy specific performance indicators. Typical examples relate to structural, energy, and thermal design issues. On the other side, human reactions are also sensitive to a multitude of parameters that find origin in subjective feelings of users, in the same way in which the so-called “emotional architecture” aims at evoking positive responses of building occupants.In this paper, an experimental analysis was presented and addressed in support of the quantitative measure and analysis of emotional and nervous states of users for selected built scenarios, towards the definition of enhanced design strategies that could combine engineering targets/methods with architectural concepts, technological solutions, and evocation of subjective feelings of customers. In doing so, major advantage was taken from the use of virtual experimental techniques and a facial expression commercial software able to capture both microexpressions and nervous states of participants (based on facial Action Units (AUs)), as well as heart rate (HR) parameters. A group of 10 volunteers was in fact exposed to visual stimuli of static and dynamic glazing environments.By designing an input stimulus characterized by a prevailing role of structural glass elements (i.e., roofs, floors, balustrades, etc.), the attention was focused on the quantification of the herein called “positive” (POS) or “negative” (NEG) feelings of participants based on their facial microexpressions and feedback, as well as to address the possible correlation of heart rate modifications with the so-detected POS and NEG emotions. As shown, the analysis of remote experimental measurements gave evidence of some good correlations of facial expression parameters, HR trends, and input stimuli, and this was found in agreement with preliminary expectations at the time of the experimental planning. Postprocessing feedback also suggested the use of remote tools and facial measurements in support of comfort analysis, as a direct quantitative measure of nervous states evoked by virtual stimuli. The remote measurement of HR variations for participants was then addressed to explore their potential correlation with selected input stimuli and corresponding facial expression/emotional state. Well promising outcomes were observed through the postprocessing analysis, with general HR trends in correlation with facial microexpressions outcomes, for most of the stimuli. On the other side, while proving the efficiency of the approach, the same experimental outcomes highlighted scattered human reactions for some of the stimuli, thus enforcing the need of additional in situ experimental procedures to address more in detail the emotional state and trend of glazing customers. [49].



---

*Source: 1978111-2021-11-05.xml*