2023 27th International Conference Information Visualisation (IV)

Hierarchical data visualization of GenderDifference: Application to Feeling of Temperature
Yuki Nakai

Takayuki Itoh

Hidekazu Takahashi

Satoshi Nakashima

Tetsu Yamamoto

Ochanomizu University
Tokyo, Japan
g1820524@is.ocha.ac.jp

Ochanomizu University
Tokyo, Japan
itot@is.ocha.ac.jp

FujitsuTokyo, Japan

FujitsuTokyo, Japan

FujitsuTokyo, Japanloans”. This bias caused by multiple attributes and factors iscalled intersectional bias.
Such biases include qualitative and subjective issues, andtherefore, a quantitative determination is not always a sufﬁcient solution. In addition, human decision-making is necessary to ﬁnd solutions to biases. In other words, it is usefulthat the analysts of the data bias understand the data anddiscover biases in the data by humans, rather than applying
fully automatic techniques to ﬁnd the biases hidden in thedata. Visualization techniques have been actively studied asa technique to help us understand the data, and are usefulfor the purpose of discovering biases in data. In particular,
to discover biases in speciﬁc parts of datasets, visualization
techniques that allow both an overview of the entire dataset
and the discovery of a speciﬁc subset of the dataset is useful.
From this perspective, we aimed to develop a visualization
method that satisﬁes the following requirements. From thisperspective, we aimed to develop a visualization method thatmeets the following requirements.

Abstract—New social problems caused by data bias have been
an important issue in recent years. Data bias is sometimes difﬁcult
to determine using only quantitative methods, and it requires
human decision-making to resolve. Visualization technology canhelp humans understand data bias and assist with this issue. This
paper proposes a visualization method for detecting intersectional
bias caused by multiple attributes. The method generates ahierarchical structure by classifying data items according tomultiple attributes, and applies a hierarchical data visualization
method equipped with band charts. It provides a visualizationthat gives an overview of the entire data and makes it easy
to detect biases in a particular subset of the data. This paper
introduces an application to the gender difference in the feelingof temperature with air conditioning systems, and presents theresults of participant evaluations to verify the effectiveness of the
proposed visualization over baseline implementation.
Index Terms—Hierarchical data visualization, Gender difference, Feeling of temperature

I. I NTRODUCTION
Data bias has caused new social problems with the spreadof machine learning and data science. For example, biases
in training data may cause biases in the results of machinelearning. Examples of such social problems include the disadvantageous results for women when machine learning is
introduced into the hiring process of a company, and thedifference in the percentage of men and women who areapproved for credit cards and mortgage loans.
Eliminating disadvantages caused by speciﬁc attributes isvery essential to establish a highly equitable society. In particular, the beneﬁt imbalance caused by gender has been of
great importance in recent years. A new keyword, Gendered
Innovation, is attracting attention as an effort to solve thisproblem.
Data bias is not always found in the entire dataset. Rather,bias is often found in a subset of the data that has certain
personal attributes. Therefore, it is important to observe thedifferences in the numerical distributions of each personal
attribute in the dataset while ﬁnding the data bias. The personal
attributes here include, for example, gender, region, race, andgeneration. In other words, bias is often found in a subset ofthe dataset that is composed of individuals who have speciﬁc
personal attributes and satisfy other factors. For example,
speciﬁcally, it can happen that “Women whose occupationis zero have a higher percentage of disapproval of mortgage

2375-0138/23/$31.00 ©2023 IEEE
DOI 10.1109/IV60283.2023.00039

1:
2:
3:

Enable to overview of the entire dataset.
Assist the discovery of bias in a subset divided basedon personal attributes.
Archive a visual representation that makes bias easy
to detect.

This paper proposes a method to visualize the bias ofnumerical distribution in a dataset consisting of a large numberof persons. This study assumes that a person is a data element
and each person has various attribute values. The proposed
method divides these data elements hierarchically by attributesand visualizes them using our hierarchical data visualization
method [7]. Here, it ﬁlls a rectangular region representing
a hierarchy with multiple band charts. By representing thenumerical distribution of a certain attribute value in its band
charts, it visualizes differences in numerical distribution due tospeciﬁc attributes. For example, to visualize the bias between
men and women, two rows of band charts for men and women
are ﬁlled in. The hierarchical data visualization applied in thisstudy satisﬁes Requirement 1 because it aims to provide an
overview of the entire hierarchical dataset, and Requirement 2
because it classiﬁes and displays data hierarchically based on
the attributes of the data elements. It also satisﬁes Requirement
3 because it makes it easy to ﬁnd bias among attributes by178

C. Hierarchical data visualization

using a visual representation of a series of band charts.
This paper introduces an application of the presented visualization method to the gender difference in the feeling oftemperature with air conditioning systems, and discusses theeffectiveness of this method.

Itoh et al. [6], [7] proposed a hierarchical data visualization
method. It represents the leaf nodes of hierarchical data as
rectangular icons, branch nodes as rectangular frames, and thehierarchical structure as a nested structure of two-dimensional
rectangular groups. Then, by arranging these in the smallest
possible screen space, it displays the entire hierarchical dataon a single screen. Figure 1 shows an example of visualization
using this method.

II. R ELATED WORK
A. Visualization of data bias
Visualization of data bias is a well-focused issue over thelast several years. One of the motivations of this study isthat the fairness of machine learning has been focused on
as a major social issue and data bias is its big factor. In
addition, it is useful to solve this problem through active
human intervention in the machine learning process. The
following are examples of recent studies on the visualizationof data bias.
Cabrera et al. [5] presented FairVis, a visual analytics
system to prevent discrimination and unequal learning outcomes by grouping compositely sensitive attributes such as
race and gender and focusing on the intersectional bias that
occurs between groups. Ahn et al. [2] presented FairSight thatvisualizes the three phases, data processing (Data), selection oflearning models (Model), and generation of rankings as learning outcomes (Outcome), to achieve the visualization focusing
on fairness. Wang et al. [12] presented DiscriLens which
supports comprehensive visual analytics of bias in machinelearning by combining a discrimination detection module witha visualization module. Tochigi et al. [11] visualized thebias of machine learning in the recommendation system bydisplaying the difference between the viewing history andthe recommendation results in the machine learning movie
recommendation system data.
The visual representation in these systems consists of multiple views such as simple bar charts, line charts, and scatterplots. Thus, users need to search for biases in the data throughrepetitive operations through multiple views. On the other
hand, most information visualization systems are designed
according to the operation scenario proposed by Schneiderman[10], “Overview ﬁrst, zoom and ﬁlter, then details on demand”.
The ﬁrst overview is an important component guiding the next
appropriate operations. Based on this idea, this study differs
signiﬁcantly from the previous studies mentioned above in that
we focus on letting users ﬁnd the bias in a particular part ofthe ﬁrst overview.

Fig. 1. An example of visualization using a hierarchical data visualization
method proposed by Itoh et al [7].

This visualization method ﬁlls the screen with rectangularareas and icons. This type of visualization method that ﬁlls
the screen with visual elements such as rectangles and icons iscalled “space-ﬁlling”. Treemaps [4], [8] are representative examples of space-ﬁlling hierarchical data visualization methods.
Experimental results [7] show that it is superior to Treemaps
in terms of the shape of the rectangular regions generated by
the algorithm presented by Itoh et al. In this respect, it is moresuitable for the purpose of this study than Treemaps.
Voronoi Treemap [3] is an example of a type of visualization
method that ﬁlls the screen with non-rectangular shape regions.
On the other hand, this method needs to ﬁll the screen withrectangular regions because it represents the data bias in a band
chart, as described below. In this sense, this method uses themethod proposed by Itoh et al.
III. V ISUALIZATION OF BIAS AS HIERARCHICAL DATA

B. Intersectional bias

A. Supposed input dataset

Bias is not always present in the entire dataset. Rather,bias is often found in a subset of the data. In particular, thebias found in a subset that satisﬁes multiple speciﬁc attributes
is called an intersectional bias. Intersectional bias is easy
to overlook and not easy to mitigate. As an example of acountermeasure to this problem, Kobayashi et al. [9] proposeda method called One-vs-One Mitigation. This method attempts
to improve the fairness of binary classiﬁcation by machinelearning applied comparison processing between subset pairs.

The proposed method assumes that the following dataset isgiven. Here, A represents the entire dataset consisting of a setof persons, ai represents the i-th person, and n represents thenumber of persons in the dataset.
A = {a1 , a2 , . . . , an }Here, ai corresponding to the i-th person has the following
variables. ei is the real value to be visualized, gi is the gender179

of the i-th person, rij is the attribute value of the j-th real typevariable, and cik is the attribute value of the k-th categoricaltype variable.

while the right side shows the numerical distribution of ei for
women in this example. The proposed method uses the HSI
color system to calculate the color of each region of the band
chart, and calculates the color of each class of the numerical
distribution according to the following principles.
• Hue: Assign a value to each attribute. In the example in
Figure 3, the hue of men is blue while the hue of womenis red.
• Saturation: Assign the lower value if a value is closer tothe median of each class to the overall mean. Or, assign
the higher value if a value is closer to the median of eachclass to the maximum/minimum value.
• Intensity: Assign the higher value, the higher the medianvalue for each class.
In the example shown in Figure 3, the band chart showsthat the total number of men is larger than that of women,that the proportion of men who feel hot is higher, and that the
proportion of women who feel cold is higher.

ai = {ei , gi , ri j, . . . , cik , . . .}B. Generation of hierarchical structures and display of band
chartsThe proposed method hierarchically classiﬁes a group of
persons using multiple attribute values selected by the useramong the attribute values rij or cik , and forms a tree
structure. If the dataset contains the intersectional bias of realvalue ei caused by the multiple attribute values selected by
the user, it is visualized under a particular node in this tree
structure. The proposed method visualizes this tree structurewith the aforementioned hierarchical data visualization method[7]. Figure 2 shows an example of visualization. The proposed
method generates a tree structure by iterating the process of
selecting one attribute at a time and classifying the person
based on that attribute. In the visualization of the tree structure,
the outer rectangular region corresponds to the classiﬁcation
result with the ﬁrst selected attribute, and the inner rectangularregion corresponds to the classiﬁcation result with the laterselected attribute.

Fig. 3. Representation of bias among attributes by multiple band charts.

C. Interaction
Our implementation features radio buttons for selecting
multiple (three in the current implementation) attributes before
generating the hierarchical structure. The hierarchical structure
is regenerated and the display is updated each time the three
attributes are selected and the “refresh” button is pressed.
Moreover, hovering the cursor over the band chart display thecorresponding three attributes in the text.

Fig. 2. Tree-structured representation of person group dataset based on
multiple attributes.

The visualization method originally represents the leaf
nodes by square icons, whereas the method represents the
distribution of ei of a group of persons corresponding toa group of leaf nodes by multiple band charts. Speciﬁcally,
the proposed method classiﬁes a group of persons according
to certain attributes, divides the real number ei into several
classes for each attribute, and tabulates the number of persons
in each class. Then, it displays the results for each class foreach attribute in band charts. In our current implementation,
the number of classes is ﬁxed at 7.
Figure 3 shows an example of a band chart, using thevisualization of bias between men and women as an example.
The left side shows the numerical distribution of ei for men

IV. A PPLICATION EXAMPLES ON THE FEELING OFTEMPERATURE WITH AIR CONDITIONING SYSTEM DATAThis paper presents an application example of the open
dataset [1] of the feeling of temperature with air conditioning
systems. We extracted the following attribute values for 32,373
subjects from this dataset.
• TS: Evaluation value for temperature. 0 is just right, “1,2, 3” are hot, “-1, -2, -3” are cold.

180

Fig. 4. Visualization example (1). The persons were classiﬁed in order of Building, Season, and Cloth.

The thickness of the band charts indicates that women are
more likely to wear light clothing in summer, while men are
more likely to wear thick clothing. Moreover, the color ratioshows that men judged it to be “hot” because of the large light
blue area, while women judged it to be “cold” because of thelarge dark red area, especially in (3) and (4). This indicatesthat men and women tend to differ in their judgments. On theother hand, in the band chart inside the classroom frame in thelower left of the visualization results, there is no signiﬁcantdifference in the area of light blue and orange, and dark blueand dark red. So this indicates that the judgements between
men and women in terms of the feeling of temperature in
the classroom are not much different. However, in the fallcorresponding to the inner ﬂame (b), we found a complex
discrepancy in the judgment of the feeling of temperature
between men and women, depending on the thickness of theirclothing.

Sex: Gender in the biological sense. Persons who answered other than “man” or “woman” were excluded
from this example.
• Age: Age.
• Cloth: Real value for the thickness of the clothing. The
larger the value, the thicker.
• Metab: Real value for metabolic rate.
• Season: Four category values: spring, summer, fall, andwinter.
• Building: Five category values: ofﬁce, classroom, residential, senior citizen facility, and others.
• Strategy: Three category values for air conditioning,
ventilation, and mixing.
In summary, TS corresponds to ei , Sex to gi , Age, Cloth,
Metab to rij , and Season, Building, Strategy to cij .
•A. Visualization example (1)Figure 4 shows the visualization result applying the personclassiﬁcation with the attribute values of Building, Season,and Cloth, in that order. Outer rectangular frames in thevisualization result correspond to the ﬁve category values of
Building, and inner them are rectangular frames corresponding
to the four category values of Spring, Summer, Fall, andWinter. Further inside are rectangular regions corresponding
to the four intervals of Metab. The band charts inside theofﬁce frame at the top of the visualization result show thatdark red occupies a larger area than dark blue. This indicatesthat more women than men judged it to be “cold”. The inner
frame (a) corresponds to summer, and the four regions inside
it indicate that (1) is the lightest and (4) the thickest clothing.

B. Visualization example (2)Figure 5 shows the visualization results applying the personclassiﬁcation by using the attribute values of Strategy, Season,
Building, in that order. The light blue and orange areas in (a)corresponding to summer are larger than the dark blue anddark red areas inside the ventilation ﬂame at the upper leftof the visualization screen. This indicates that many peopleof both men and women judged it to be “hot”. Conversely,
the dark blue and dark red areas are larger than the light blue
and orange areas in (b), which corresponds to winter. This
indicates that many people of both men and women judged it
to be “cold”. The inner area of the air conditioner frame in181

Fig. 5. Visualization example (2). The persons were classiﬁed in order of Strategy, Season, and Building.

the lower left of the visualization screen shows that the dark
red area is larger than the dark blue area in almost all band
charts, indicating that women are more likely to judge it as“cold” regardless of the season or location. Inside the mix at
the lower right of the visualization screen, the dark red area
is larger than the dark blue area in (c), which corresponds toautumn, and in (d), which corresponds to spring, indicatingthat women are more likely to judge it to be “cold”. On
the other hand, the difference in judgment between men andwomen is smaller in summer and winter, suggesting that a
mixture of air conditioning and ventilation is desirable insummer and winter.

We used the dataset of the temperature feeling with air
conditioning presented in the previous section to generatevisualization examples.
The participants were presented with the three visualization
results (P, B1, and B2) in a random order, and were asked
to answer the following questions Q1-Q8 on a 7-point Likert
scale (1: not at all agree to 7: strongly agree) for each visualization result. Q1 and Q2 are questions to verify Requirement
3, Q3 and Q4 are questions to verify Requirement 2, and Q5 isa question to verify Requirement 1. Q6-Q8 are questions takenfrom the NASA Task Load Index (NASA-TLX). Q6 to Q8 aremarked with a superscript ”-” because a higher response value
indicates a negative result.

V. E VALUATION BY PARTICIPANT EXPERIMENT
We conducted a participant experiment to verify the effectiveness of the proposed method. The subjects were 12 students majoring in computer science. This experiment prepared,in addition to the visualization results by the proposed method(hereinafter referred to as ”P”), the visualization results by thefollowing two baseline implementations (hereinafter referred
to as ”B1” and ”B2”).
B1: Visualization results simply applying the method
proposed by Itoh et al [7]. Band charts are notdisplayed; the icons corresponding to each of thepeople in the dataset are displayed.
B2: Visualization results that display only a set of band
charts, without any rectangular ﬂames representing
the hierarchical structure.

Q1:
Q2:

Q3:

Q4:

Q5:

182

Easy to ﬁnd which combination of the three attributes
is “more male” or “more female”.
Easy to ﬁnd which combination of the three attributes
is “more men feeling hot/cold” or “more womenfeeling hot/cold”.
Easier to understand which single of the three attributes is responsible for the difference in responses
between men and women.
Easier to understand which “combination of multiple
attributes” among the three attributes is responsible
for the difference in responses between men andwomen.
Satisfactory as a visualization for overview and analysis of gender differences in the temperature feeling.

Q6− : Perceptual activities such as contemplation and memory are necessary when using this visualization.
Q7− : Great effort is needed to understand this visualization.
Q8− : Feel frustrated while looking at this visualization.

dataset with a very large number of attributes. Therefore,
we would like to develop a method to automatically selectcombinations of attributes that are worth visualizing.
Next, we would like to have case studies for various datasets
other than the temperature feeling with air conditioning. Also,
we would like to visualize biases in numerical distributions
caused by attributes other than gender (e.g., generation, region,
race, occupation). Furthermore, we would like to utilize theproposed method as a tool for correcting biases in machinelearning training datasets, and verify its effectiveness in theﬁeld of machine learning.
The participant experiments presented in this paper used
still images of the visualization results and do not show theactual operation of the visualization system. Therefore, wewould like to conduct task-based participant experiments while
actually operating the visualization system to measure the levelof achievements of the tasks.

TABLE IR ESULTS OF PARTICIPANT EXPERIMENTS FOR COMPARISON WITHBASELINE .
Q1
Q2
Q3
Q4
Q5
Q6−
Q7−
Q8−

P
6.33
5.17
4.92
4.42
5.67
4.33
3.92
2.92

B1
3.25∗
2.33∗
2.25∗
2.00∗
2.50∗
4.83
5.33
5.50∗

B2
5.42∗
4.50∗
3.33∗
3.33∗
3.92∗
4.42
4.83
4.00

Table V shows the averages of the responses as an interval
scale. Wilcoxon’s rank-sum test was applied to “P and B1”and “P and B2”, and results with p < 0.05 were marked with“*”.
The results from Q1 to Q5 all show that the proposed
method (P) is signiﬁcantly higher than the baseline (B1,
B2). This suggests that the combination of “representationof hierarchical structure” and “band chart” is effective insatisfying requirements 1, 2, and 3.
On the other hand, the results of Q6 to Q8 were all rated
higher than the baseline for the proposed method, but theresults are not statistically signiﬁcant. Since this study mainly
targets users who are engaged in data analysis in their daily
work, in this sense, we do not consider that “perceptual activity
is necessary” or “effort is required” to be a major issue. On theother hand, if we suppose this visualization is ever used forpresenting widely to the general public, such as consumers,
the need for perceptual activity and effort may hinder theirdissemination. In other words, more ingenuity in visualization
design is necessary when expanding the target users to include
general consumers.

R EFERENCES[1] Ashrae global thermal comfort database ii. https://www.kaggle.com/
datasets/claytonmiller/ashrae-global-thermal-comfort-database-ii.
[2] Yongsu Ahn and Yu-Ru Lin. Fairsight: Visual analytics for fairness
in decision making. IEEE transactions on visualization and computer
graphics, 26(1):1086–1095, 2019.
[3] Michael Balzer and Oliver Deussen. Voronoi treemaps. In IEEE
Symposium on Information Visualization, pages 49–56. IEEE, 2005.
[4] Benjamin B Bederson, Ben Shneiderman, and Martin Wattenberg.
Ordered and quantum treemaps: Making effective use of 2d space todisplay hierarchies. AcM Transactions on Graphics (TOG), 21(4):833–
854, 2002.
[5] Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng,
Jamie Morgenstern, and Duen Horng Chau. Fairvis: Visual analytics
for discovering intersectional bias in machine learning. In 2019 IEEE
Conference on Visual Analytics Science and Technology (VAST), pages
46–56. IEEE, 2019.
[6] Takayuki Itoh, Hiroki Takakura, Atsushi Sawada, and Koji Koyamada.
Hierarchical visualization of network intrusion detection data in the ip
address space. IEEE Computer Graphics and Applications, 26(2):40–47,
2006.
[7] Takayuki Itoh, Yumi Yamaguchi, Yuko Ikehata, and Yasumasa Kajinaga.
Hierarchical data visualization using a fast rectangle-packing algorithm.
IEEE Transactions on Visualization and Computer Graphics, 10(3):302–
313, 2004.
[8] B Johonson and B Scheneiderman. Tree-maps: A space ﬁlling approachto the visualization of hierarchical information space. In IEEE Visualization, volume 91, pages 275–282, 1991.
[9] Kenji Kobayashi and Yuri Nakao. One-vs.-one mitigation of intersectional bias: A general method for extending fairness-aware binaryclassiﬁcation. In International Conference on Disruptive Technologies,
Tech Ethics and Artiﬁcial Intelligence, pages 43–54. Springer, 2021.
[10] Ben Shneiderman. The eyes have it: A task by data type taxonomy
for information visualizations. In The craft of information visualization,
pages 364–371. Elsevier, 2003.
[11] Ami Tochigi, Takayuki Itoh, and Xiting Wang. Visualization of bias ofmachine learning for content recommendation. IEEE VIS Posters, 2021.
[12] Qianwen Wang, Zhenhua Xu, Zhutian Chen, Yong Wang, Shixia Liu,and Huamin Qu. Visual analysis of discrimination in machine learning. IEEE Transactions on Visualization and Computer Graphics,
27(2):1470–1480, 2020.

VI. C ONCLUSION
This paper proposed a method to visualize intersectional
bias in a dataset of a large number of persons as hierarchicaldata. This method classiﬁes groups of persons composing thedataset hierarchically based on user-selected attributes, andvisualizes them using our own hierarchical data visualization
method. This method represents the bias of the numericalvalue distribution using multiple band charts to represent thenumerical values that a group of persons corresponding to a
speciﬁc node in the hierarchical structure. This paper presented
the visualization results using the dataset of the temperature
feeling with air conditioning and the results of participant
experiments.
One of the future issues is to deal with the dataset with avery large number of attributes. The proposed method requires
an attribute selection operation to generate a hierarchicalstructure. This operation becomes very complicated for the183