Indoor environment control method for improving
well-being using human thermal stress estimated by
Yuragi learning
Keita Hara

Risa Yoshida

Daichi Kominami

Graduate School of Information Science Graduate School of Information Science Graduate School of Information Scienceand Technology, Osaka University
and Technology, Osaka University
and Technology, Osaka University
Osaka, Japan
Osaka, Japan
Osaka, Japank-hara@ist.osaka-u.ac.jp
r-yoshida@ist.osaka-u.ac.jp
d-kominami@ist.osaka-u.ac.jp

Yuichi Ohsita

Michiko Kaihotsu

Masayuki Murata

Cybermedia Center, Osaka University
Osaka, Japanyuichi.ohsita.cmc@osaka-u.ac.jp

Technology Innovation Center,
Daikin Industries, Ltd.
Osaka, Japanmichiko.kaihotsu@daikin.co.jp

Graduate School of Information Scienceand Technology, Osaka University
Osaka, Japan
murata@ist.osaka-u.ac.jp

Abstract—“Workstyle Reform” is promoted in Japan to improve the work environment surrounding workers. However, it
is pointed out that the reduction of working hours by improving
marginal productivity may increase workers’ stress on a perunit hour basis. The realization of a system that quantiﬁes stressand improves it accordingly will lead to our well-being, and in
this paper, we propose methods that estimate the stress state ofeach individual and control the indoor environment in real time
based on the estimated stress state to maintain or improve a wellbeing space. Estimation of the individual stress state is based on“Yuragi learning,” which is a method of making decisions frominformation including noise, based on a model of the cognitive
process of a human brain. We implemented Yuragi learning torealize real-time stress estimation based on data streaming frombiometric sensor devices. We also implemented the method tocontrol devices that act in an indoor space on the basis of theestimation. In our experiment, we prepared two room conditions
with different temperatures and humidity levels and conﬁrmedthat our method estimated the stress state of a subject andcontrolled the room with an actuator that directs the air ﬂow to
the subject in a real-time manner.
Index Terms—Bayesian attractor model, biometric information, multi-modal integration.

I. INTRODUCTION
In many countries, including Japan, the working environment surrounding workers is undergoing a reevaluation due
to the decline in the working-age population caused by thefalling birth rate and the aging population, as well as thediversiﬁcation of working needs, such as balancing workwith childcare and nursing care. As seen in the “Workstyle
Reform” of Japan, long working hours are being rectiﬁed andin response to the spread of the new coronavirus, work athome and other work styles that accommodate an individual’s
work-life balance are being recommended [1].

In terms of changes in workplace and work-related circumstances, the authors of [2] argue that the dominant models within human resource management theory and researchmainly focus on ways to improve productivity, therefore risk
affecting work-related well-being, and have detrimental consequences for employees and, in some cases, the organization.
There is still room to consider ways of working that increase
productivity while also taking into account personal wellbeing [3]. Hence, quantitative methods are required to assess
individual stress for this, but even in the ﬁeld of mental health,which is highly related to human stress, assessment methods
have been developed for depression and other mental healthproblems.
Reference [4] takes the approach of providing an appropriate
working environment to the challenge of increasing productivity. This study reported that workers who felt drowsy were
awakened by air and light, and their productivity increased.
If human stress can be estimated quantitatively, it would bepossible to improve overall productivity from various perspectives, such as alleviating stress by manipulating the work space
based on the estimation or directly encouraging people to takea break. The authors of [5] provide a survey of internal ofﬁce
spaces and well-being, and point out that while internal ofﬁce
spaces that provide workers with well-being are important, thescientiﬁc basis for such spaces needs further analysis.
It has been reported that it is possible to estimate anindividual’s stress state using biometric information such as
skin potential activity, skin temperature, and pulse wave [6].
By estimating a person’s stress state based on biometric information and manipulating the indoor environment accordingly,
it may be possible to improve the stress of people spendingtime in a room. However, there are a few issues that need tobe resolved. First, the biological information obtained from

streaming data. Before describing the stress state estimation
method, BAM and Yuragi learning are explained in detail(Sect. II-B and Sect. II-C, respectively). The computer performs stress state estimation using Yuragi learning (Sect. II-D).
Based on estimated stress information, the actuator acts on theindoor space (Sect. II-E).
A. Biometric information and its acquisition

Fig. 1. System overview

wearable sensors contains noise. Furthermore, the impact ofstress on biological information differs from person to person.
In this paper, we propose a method that estimates the stressstate of each individual. Regarding the noise issue, we utilize“Yuragi learning,” which is based on the cognitive mechanismof the brain to make decisions from observed information that
contains noise. Our research group has applied it successfullyin various studies so far [7]. Observation and decision-making
processes in Yuragi learning are based on the Bayesian attractor model (BAM) [8]. The issue of individual differences
is solved by implementing a personalized Yuragi learning
training process and a multimodal integration process. We
also propose a method to control the indoor environment
based on human stress estimated by the method to maintainor improve a well-being space in which each individual canspend time comfortably and willingly for work. Our proposal
is to sequentially estimate the stress state using biologicalinformation as input and control the indoor environment, andcan be applied in various ways by changing the devices.
We conduct experiments to verify the series of operations
in which our methods estimate the thermal stress state usinga wristband capable of measuring biometric information andcontrol a circulator based on the stress state. If the results
indicate that a person is under thermal stress, the circulator
control is applied to eliminate the cause of thermal stress or
improve well-being.
II. P ROPOSAL
Well-being is a subjective indicator and varies from person
to person. Therefore, in this study, we will use a method thatsenses various biological information using biometric sensors
and estimates a person’s psychological state from the obtained
biometric information. Based on the estimation results, we willintervene in the space by controlling actuators and control the
space to relieve the stress state to realize a well-being space
where each individual can work spontaneously without feeling
discomfort.
Here, we describe Fig. 1 and the structure of this section.
First, biometric information is acquired (Sect. II-A). The
biometric information obtained here is sent to the computer as

Schmidt et al. provided a publicly available multimodal
dataset for the researchers as WESAD (Wearable Stress andAffect Detection) [6]. This multimodal data set contains blood
volume pulse (BVP), electrocardiogram (ECG), electrodermal
activity (EDA), electromyogram (EMG), respiration (RESP),
skin temperature (TMP), and 3-axis accelerometer (ACC)
recorded from wrist and chest mounted devices on 15 subjects,
as well as self-reported emotional states for three different
emotional states (neutral, stress, amusement). Self-reported
values for three different emotional states (neutral, stress,
and amusement) were included. Based on these data, a large
number of features (extracted from physiological and motion
signals) and general machine learning methods (decision trees,
random forests, AdaBoost, linear discriminant analysis, knearest neighbor methods) were used to create benchmarks.
Using this information, the three-class (normal, stressed, andagitated) or two-class (stressed and unstressed) estimation
problem yielded high-level estimation results, indicating that
it is possible to estimate psychological states from multiple
sources of biometric information.
B. Bayesian attractor model
In [8], the task of decision-making is to select one of multiple choices based on observed information, and a probabilistic
framework is proposed in which the decision-making process
is formulated by Bayesian inference. To model the decisionmaking process in the brain, a dynamics model is deﬁned inwhich the variables representing the decision have attractors,
and each of the aforementioned choices is associated with an
attractor. Bayesian attractor model (BAM) uses Bayesian estimation with external stimuli as input to determine which of thepreviously prepared choices the observed target corresponds
to.
In BAM, the probability distribution of the decision state zt
is updated using Bayesian inference when BAM receives input
xt . The posterior probability distribution P (zt |xt ) reﬂects theambiguity and uncertainty of brain states. In this Bayesianinference, the following generative model is assumed.
√zt − zt−∆t = ∆tf (zt−∆t ) + ∆tWt
(1)
xt = M σ(zt ) + vt .

(2)

Here, f (z) represents Hopﬁeld dynamics, which is one of
the attractor models. f is designed to have N attractors, denoted by Φ = {ϕ1 , . . . , ϕN }, and N corresponds to the number of choices to be stored in the model. M = [µ1 , . . . , µn ] isa matrix of feature values corresponding to each stored choice.
σ is a multidimensional sigmoid function whose value range

2q
is 0 to 1. Wt and vt are noise terms, Wt ∼ N (0, ∆tI) and2
vt ∼ N (0, r I) respectively (I is the unit matrix). N means
normal distribution. Since q and r determine the magnitudeof uncertainty of the dynamics and the observation in thegenerative model, q is called dynamics uncertainty and r iscalled sensory uncertainty.
By estimating the aforementioned generative model in thereverse direction by Bayes’ theorem, a model of decision
making is obtained. To account for the non-linearity of thegenerative model, an approximate calculation is performed
using the unscented Kalman Filter (UKF). This estimation of
the state yields the posterior probability distribution P (zt |xt )of zt . Therefore, the attractor to which the internal state of
the brain is close to is determined based on the size of theprobability density P (zt = ϕn |xt ). P (zt = ϕn |xt ) is called“conﬁdence” for the nth choice and decisions are made basedon the magnitude of conﬁdence. Even if the observed values
contain noise, the accumulation of observations can increase
conﬁdence and enable appropriate decision-making.

C. Yuragi learning
Our research group has proposed a machine learning methodfor decision making based on BAM as Yuragi learning, and has
successfully applied it to various applications [7]. The humanbrain is capable of performing complex cognitive and task
functions simultaneously. The secret of this brain is believed tolie in the ﬂuctuations (called “Yuragi” in Japanese) that exist
in all living systems, including the human brain. Biologicalsystems can have a high degree of freedom by utilizing noise
rather than eliminating it. BAM is one of the models ofhuman decision-making proposed to explain reaction times,
correct response rates, and changes in decision making indecision-making experiments. Yuragi learning is a framework
for engineering applications of BAM, providing design ofobservations (preprocessing), design of attractors (training),
and ties between decision-making and various estimation
problems.
D. Stress estimation by Yuragi learning
Figure 2 provides an overview of our stress estimation
method in which we create discriminators using Yuragi learning. We extract features from the information observed in each
modality. We denote the observation of the ith modality by
(i)
(i)
(i)
(i)
ot and the features extracted from ot by xt . xt is used
as input to the corresponding discriminator based on Yuragi(i)learning. Each discriminator updates its cognitive state zt(i)(i)
using feature inputs xt for the ith modality. Fi = P (zt =(i)Φ|xt ) is the output of the discriminator, and our method
makes the ﬁnal decision by integrating Fi for all modalities.
Yuragi learning is a method that makes decisions after accumulating a sufﬁcient amount of observations, and is characterized by being less susceptible to temporary noise. Therefore,
we address the noise in biometric information using Yuragi
learning. We also take into account individual differencesby conﬁguring the discriminators for each person. Training
in Yuragi learning simply sets the features corresponding to

Fig. 2. Stress estimation model of the proposed method

stress/non-stress states to its feature matrix M . Therefore,
it is possible to easily train a classiﬁer for each individual.
Additionally, the useful modalities are also distinct for each
person. Therefore, we also select the useful modalities foreach person using the conﬁdence level of Yuragi learning. The
estimation results with high conﬁdence level are used for theﬁnal decision. By doing so, we can avoid using results thatcannot differentiate the stress state in the current situation.
1) Training: Training dataset requires biometric information labeled to represent various states. In this paper, we use
the label “stress” to represent the state of stress and the label“non-stress” to represent the state of not feeling stress.
a) Feature selection: This section explains the processof selecting features. The candidate selection process begins
after collecting some observations. We denote the set of timeslots T train in which observed data can be used for training.
Candidate features are selected by the following steps.
To begin, we include all the features in a candidate featureset.
• Normalize the features by•

xt,j =

xcand
− µj
t,j
σj

(3)

where xcand
is the jth feature in a candidate feature sett,j
at time t. µj is the mean, and σj is the standard deviationof the jth feature where the label is “non-stress.”• Calculate x̄j , which is the average of xt,j in the case ofstress state,∑{t∈T train ,L(t)=stress} xt,j(4)x̄j = ∑{t∈T train ,L(t)=stress} 1•

where L(t) is the label of the dataset at time t.
Check if the magnitude of each element of x̄j is greaterthan the threshold λ. If |x̄j | is greater, keep the jthfeature as a candidate feature. Otherwise, the jth feature
is excluded.

For each individual, the same process is repeated for each
modality. Note that if |x̄j | is smaller than the threshold for all
j in a modality, the modality itself is excluded. Through thesesteps, we can identify features that can differentiate between
a stress and a non-stress state.

b) Setting of attractor: Discriminators based on Yuragi
learning can be trained by assigning typical features of labels
to attractors. We set the typical features of the non-stress state
to ϕ0 and those of the stress state to ϕ1 .
2) Multimodal stress estimation: The initial normalizedfeatures xt are derived from the observation information using
Eq. (3). Subsequently, the decision state is estimated using a
Bayesian approach based on the generative model expressedin Eqs. (1) and (2), and then the conﬁdence level P (zt |xt )is calculated. As shown in Fig.2, there is a discriminator foreach modality, each of which estimates the stress state. The
results of each discriminator are then combined to make theﬁnal decision through the following steps.
• Creation of conﬁdence vectors
We construct a conﬁdence vector that includes the conﬁdence calculated from each discriminator. The conﬁdence
vector for the ith discriminator is obtained by
(i)

(i)

Fi = P (zt = Φ|xt )Exclusion of results with low conﬁdence
Some discriminators may not be able to distinguishthe stress state. In such cases, the conﬁdence of thediscriminator becomes low. In our method, we exclude
the results of discriminators whose conﬁdences are lessthan a pre-deﬁned threshold.
• Integration of conﬁdence vectors
We integrate the conﬁdence vectors of the selected discriminators. The method to calculate the weighted sumof conﬁdence vectors is used.
Σi∈M wi F̄iΣi w i(6)where M is the number of modalities, F is the ﬁnal
result after integration, and wi is the weight for the
ith modality. M is determined by the process excluding low-conﬁdence modalities and the process describedin Sect. II-D1. F̄i is the normalized conﬁdence vector
obtained by
FiF̄i = ∑
(7)j∈M FjNote that wi needs to be deﬁned in advance. In this paper,
we assume that wi is trained using the data of the subjects
so as to maximize the accuracy of the ﬁnal decisions F .
Then, common wi can be used for all subjects.
The estimation of stress state using biometric data has
been tested on 31 subjects to conﬁrm its validity in our
preliminary experiment. This paper presents the realization ofa framework for automatic control of the indoor environment
using biometric data, and for reasons of space limitation, we
omit the detail of the result.
E. Control of actuator
Various approaches targeting a person’s ﬁve senses can beconsidered to control actuators to maintain a sense of wellbeing in the indoor space. For example, music could be played

Fig. 4. Daikin assist circulator [10]to encourage a break when strong stress is estimated, or an
aroma diffuser could be used to spread the person’s preferredscent in the space. In this paper, we implement a speciﬁc
example of stress reduction in a thermal environment throughthe use of air conditioning equipment. Details are given in thenext section.

(5)

•

F =

Fig. 3. E4 wristband [9]III. E VALUATION
A. Implementation
In the experiment, we consider the discomfort felt when
the temperature and humidity in a room are set to values
that would be perceived as uncomfortable, and show thatthe proposed system can estimate such stress and control theactuators in real time.
1) Devices: We use the E4 wristband (Fig. 3) [9] toacquire biometric information, which is also used in [6],manufactured by Empatica. The E4 wristband can acquire 3axial acceleration (ACC), blood volume pulse (BVP), electrodermal response (EDA), heartbeat interval (IBI), and skintemperature (TMP).
In order to relay the biometric information obtained from theE4 wristband to the computer that performs Yuragi learning,
we installed a streaming server provided by Empatica. The
streaming server software is installed on a laptop computer andconnected to the E4 wristband via Bluetooth. The streaming
server is capable of acquiring biometric data from multiple E4
wristbands and transmitting the data over the network using theAPIs provided by Empatica. Note that observed data are also
stored on the wristbands, and if communication is temporarily
interrupted, it has the ability to resend the data it was unable
to send when reconnected.
For an actuator to control an indoor room, we use a Daikin
assisted circulator (Fig. 4) [10]. In addition to the airﬂow adjustment function, this device has the function of adjusting the
airﬂow direction. The assist circulator is connected to a smallLinux-based computer (we used a Raspberry Pi3 model B,
hereinafter abbreviated as PI3) via serial communication. We
have made hardware modiﬁcations to the assisted circulator
so that its operation can be changed by serial commands sentfrom the PI3. In the implementation in this experiment, theestimation of the stress state by Yuragi learning is performed
on a computer that runs the streaming server, so this computer
sends control commands to the assist circulator via the PI3.

2) Feature extraction: Stress state estimation using Yuragi
learning is performed on the computer where the aforementioned streaming server is installed. The stress estimation software using Yuragi learning and the streaming server softwarecommunicate with each other through socket communication
using TCP. In this implementation, the Yuragi learning software connects to localhost with a speciﬁed port. By specifying
in advance the type of biological data to be acquired by thestreaming server, the biological data arrive at the socket as
streaming information.
In this implementation, we obtain four types of biometric
information, TMP, EDA, BVP, and IBI from an Empatica E4
wristband. That is, the number of modalities is four. In addition
to the raw EDA observations recorded by Empatica E4, we
extract three key values from the recorded EDA by using
cvxEDA [11], phasicEDA, SNMAphasicEDA and tonicEDA.
A brief description follows.
• phasicEDA: Signal R, which is extracted only from thepart of EDA that changed abruptly in a short period oftime• SNMAphasicEDA: Changes in the calculated periodicityof R• tonicEDA: Signal with slowly changing trend taken out
For each of the above total of seven observations, the mean,
standard deviation, minimum value, maximum value, range,and slope are calculated for a ﬁxed time. These 7 × 6 values
are used as feature values. We used a sliding window with awindow size of 60 seconds and a window shift of 0.25 seconds.
3) Yuragi learning: Although the software of Yuragi learning is available in [12], multimodal processing functions havenot been implemented, and from the viewpoint of real-time
calculation, we implemented it in the C++ language. The basic
feature of Yuragi learning written in the C++ language ispublished in [13].
We set the threshold λ to 1.1 for the selection of features.
We also set the threshold value for the conﬁdence obtainedby Yuragi learning to 0.001. The sensory uncertainty r is set
to 0.4 and the dynamics uncertainty q is set to 0.5. These
values were selected from those that were highly effective inthe pre-training data.
B. Experimental setup
To conduct experiments for the validation of the proposedsystem, we prepared comfortable and uncomfortable indoor
rooms based on PMV (Predicted Mean Vote) as deﬁned in
ISO 7730 [14]. PMV is a famous index intended to predict
the average value of a group of occupants’ votes on a sevenpoint thermal sensation scale (Hot (+3) to Cold (−3)). In thisexperiment, the PMV of the comfortable room was set to 0
and that of the uncomfortable room to +2.
Training data is acquired in each room for one subject.
When acquiring the data, an interval of 10 minutes is set
to avoid the inﬂuence of the previous environment, and thenthe data is acquired for 10 minutes. After the learning iscomplete, the subject moves to a room with an uncomfortableenvironment and stays for 10 minutes. Yuragi learning is(a) EDA(b) TMP(c) BVP(d) IBI

Fig. 5. Observed raw data from E4 wristband

performed for each biological information every second. The
results of the estimation are checked every 10 seconds, and if
the stress state is estimated as an estimation result more than acertain number of times, the room environment is changed byan actuator (assisted circulator). This experiment is designed
to show that it is possible to demonstrate that the implemented
system can estimate discomfort due to temperature and humidity and control the equipment to remedy the situation in realtime.
C. Result
We have conﬁrmed that control occurs as expected. The
input during training in each room is shown in Fig. 5, and theresults of feature selection based on these inputs are shown in
Table I for the subject. The blue and orange lines represent
the measured values for 10 minutes spent in the comfortableand uncomfortable rooms, respectively, where the number ofsamples that can be obtained per unit of time differs according
to the Empatica E4 speciﬁcations.
The estimation results for each biometric information are
shown in Fig. 6, and the integrated result is shown in Fig. 7.
Note that the actual output of Eq. (6) is the conﬁdence levelin two attractors (ϕ0 corresponding to the non-stressed stateand ϕ1 corresponding to the stressed state), but the estimatedstate of each time step is shown in the ﬁgure. The solid green
line is the result of the estimation.
We have succeeded in estimating the stress state of thesubject in the uncomfortable room, as shown in Fig. 7.
The air blowing of the circulator continued during the stressstate. Although subjective comfort was somewhat improvedby the airﬂow provided by the circulator, this was not seen
as a biological response. In the future, further experimental
evaluations will be conducted with different environmental
settings and actuators.

TABLE I
S ELECTED FEATURES
EDA
phasicEDA
SNMAphasicEDA
tonicEDA
TMP
BVP
IBI
(a) EDA

mean

std

mean
mean

std
std

min

max

range

slope

min
min

max
max
max

range
range

slope

(b) TMP(c) BVP(d) IBI

Fig. 6. Stress estimation result (0: ‘non-stressed state’ and 1: ‘stressed state’)Fig. 7. Multimodal Integration of stress estimation

IV. C ONCLUSION
We proposed a method for estimating stress by integrating
multiple types of biometric information using Yuragi learning.
In our method, we conﬁgure multiple discriminators based on
Yuragi learning. Each discriminator makes decisions based on
the corresponding biometric information. Then, by integrating
the decisions of all discriminators, our method makes the ﬁnal
decision. Furthermore, our method also excludes decisionsof Yuragi learning whose output conﬁdence is low to avoid
estimation of stress from inaccurate information. Experiments
have shown that the system can operate in real time fromobservation of actual biological data to control of actuators.
A future work will be to clarify how to control the actuators
to completely remove stress and how to estimate and control
them for different people.
R EFERENCES[1] “Abenomics: For future growth, for future generations, and for a
future japan,” https://www.japan.go.jp/abenomics/ userdata/abenomics/
pdf/1901 abenomics.pdf (Accessed on Nov. 1, 2023).

[2] D. E. Guest, “Human resource management and employee well-being:
Towards a new analytic framework,” Human resource management
journal, vol. 27, no. 1, pp. 22–38, 2017.
[3] A. Sakuraya, A. Shimazu, H. Eguchi, K. Kamiyama, Y. Hara, K. Namba,
and N. Kawakami, “Job crafting, work engagement, and psychologicaldistress among japanese employees: A cross-sectional study,” BioPsychoSocial medicine, vol. 11, no. 1, pp. 1–7, 2017.
[4] T. Kogo, M. Tsujikawa, Y. Kiuchi, A. Nishino, and S. Hashimoto,“Model predictive control of shallow drowsiness: Improving productivity
of ofﬁce workers,” in Proceedings of Annual International Conferenceof the IEEE Engineering in Medicine and Biology Society, Jul. 2019,
pp. 2459–2465.
[5] S. Colenberg, T. Jylhä, and M. Arkesteijn, “The relationship between
interior ofﬁce space and employee health and well-being–a literature
review,” Building Research & Information, vol. 49, no. 3, pp. 352–366,
2021.
[6] P. Schmidt, A. Reiss, R. Duerichen, C. Marberger, and K. Van Laerhoven, “Introducing wesad, a multimodal dataset for wearable stressand affect detection,” in Proceedings of the 20th ACM international
conference on multimodal interaction, 2018, pp. 400–408.
[7] M. Murata and K. Leibnitz, Fluctuation-Induced Network Control andLearning: Applying the Yuragi Principle of Brain and Biological Systems. Springer, 2021.
[8] S. Bitzer, J. Bruineberg, and S. J. Kiebel, “A Bayesian attractor model
for perceptual decision making,” PLoS Computational Biology, vol. 11,
no. 8, p. e1004442, 2015.
[9] “E4 wristband,” https://www.empatica.com/en-int/research/e4/, (Accessed on Nov. 1, 2023).
[10] “Daikin assisted circulator,” https://launch.ac.daikin.co.jp/onlineshop/
detail03, (Accessed on Nov. 1, 2023).
[11] A. Greco, G. Valenza, A. Lanata, E. P. Scilingo, and L. Citi, “cvxEDA:
A convex optimization approach to electrodermal activity processing,”
IEEE transactions on biomedical engineering, vol. 63, no. 4, pp. 797–
804, 2015.
[12] “YLPF: Yuragi Learning Platform Core Module,” https://github.com/
nbic-ist-osaka-u-ac-jp/YLPF-Core (Accessed on Nov. 1, 2023).
[13] “bayesian-attractor-model-python-cpp-,”https://github.com/
d-kominami/bayesian-attractor-model-python-cpp(AccessedonNov. 1, 2023).
[14] “ISO 7730:2005,” https://www.iso.org/standard/39155.html (Accessedon Nov. 1, 2023).