2020 24th International Conference Information Visualisation (IV)

Spatial and Temporal Visualization of Pedestrians
Based on Walking States
Natsumi Tsuchida

Yuri Miyagi

Masaki Onishi

Grad. School of Humanities and Sciences
Ochanomizu University
Tokyo, Japantsuchida.natsumi@is.ocha.ac.jp

Artificial Intelligence Research Center
National Institute of Advanced
Industrial Science and Technology
Ibaraki, Japanmiyagi@itolab.is.ocha.ac.jp

Artificial Intelligence Research Center
National Institute of Advanced
Industrial Science and Technology
Ibaraki, Japan
onishi@ni.aist.go.jp

Takayuki Itoh
Grad. School of Humanities and Sciences
Ochanomizu University
Tokyo, Japan
itot@is.ocha.ac.jp
Abstract—People flow has useful knowledge in various fields
including traffic, disaster prevention, and marketing. Such information can be obtained by looking for the movement patterns ofeach person and the features of each place.
We have proposed a visualization method that summarizes
the paths of a large number of pedestrians in order to discover
their useful information. However, we have not been working
on visualization focusing on walking states whether people aremoving rapidly or slowly. This paper presents a visualization
method based on the walking states of a large number ofpedestrian paths measured by using cameras. The visualization
tool allows users to inspect walking people based on walking
states from spatial and temporal views. The spatial view applies
a heatmap and the temporal view adopts a stacked line graph.
In addition, the user interface has a function to simultaneously
visualize changes in the walking state distribution over time andwalking paths.
Index Terms—visualization, people flow, walking state

I. I NTRODUCTION
Walking people can be measured by various devices suchas GPS data and video. Among them, camera-based measurements of people flow lines are expected to be widelyused in the future because pedestrians do not have to haveany devices. Also, camera-based measurements can record
relatively accurate positions of walking people. People flowline data have been utilized for various fields because the data
has much useful knowledge. It is worth analyzing the videos
to discover movement patterns and features of the people,
since we can discover fruitful knowledge to solve many social
problems. For example, we can establish better evacuationroutes, find causes of traffic jams ,and design product displaysthat attract more customers by means of the analysis of people
flow data.
Here, we need to get people flow data recorded for a longtime and in large space in order to realize such analysis.
However, it is often a very complicated and bothering task to
discover useful knowledge from the people flow data because a2375-0138/20/$31.00 ©2020 IEEE
DOI 10.1109/IV51561.2020.00099

large number of people flow lines may be contained. Moreover,data owners who have experienced at the spot do not always
have enough time and advanced knowledge for human flow
analysis. Comprehensive visualization of people flow data
enables many users who do not have the expert knowledge
to analyze the data efficiently.
We found that visualizing and summarizing the walking
paths is not enough just to understand the intention ofpedestrians. It is important to consider the walking status ofeach pedestrian, especially in a space where various types ofpedestrians are mixed. Here, walking status means a variety
of semantics and movements of the pedestrians in this paper.
Typical examples of the walking status include walking while
glancing at something, walking slowly while staring at something, or walking straight and quickly forward a destination.
Our goal is to propose a visualization tool that represents
the statistics of walking states since most of the existingvisualization techniques just represent a set of walking paths
by drawing a set of lines.
We have proposed a visualization method [1] that summarizes the walking paths after compressing the trajectories of alarge number of pedestrians. However, our previous method
did not show walking states whether people are walking
rapidly or slowly.
This paper presents a technique that classifies the flow data
based on the walking states of a large number of pedestrianpaths before visualizing them. Here, our current study simply
defines the walking states by walking speeds.
Our goal is the development of a visualization system
satisfying the following tasks:
T1: Visualize the spatial distributions of walking states.
T2: Visualize the temporal changes of the distributionsof walking states.
To realize the above tasks, the presented technique firstly
divides the whole area taken by cameras into small rectangularsubareas and aggregates the number of pedestrians who pass448

through each subarea for each walking state. We define the
distribution of the number of walking people by each subarea
and walking state as walking state distribution in this paper.
The technique realizes T1 by drawing the distributions ofwalking states of each subarea as a set of stacked bar charts
instead of drawing all the walking routes. This representation
solves the problem of conventional line-drawing-based visualization techniques that stationary pedestrians are displayed as
dots and therefore they are difficult to be found. In addition,
the technique applies a stacked polyline chart to realize T2.
This representation makes us observing how the distribution ofwalking states changes and picking up at noticeable time periods. These visualization components can represent pedestrians
based on walking states including stationary states. It could be
useful for people who want to stop and know what time andplace they are paying attention to, such as convenience storesand museums.
II. R ELATED W ORKThis section introduces existing studies to analyze andvisualize trajectories of pedestrians and non-pedestrians.
Many existing methods classify and visualize spatiotemporal people flow data recorded as real values. Also, there
have been many existing visualization methods to display the
space where the walking paths are measured and to drawlines (or figures) representing the walking paths directly in adisplay space. As an example, Yabushita et al. [2] proposed atechnique that summarizes similarly shaped walking routes ofa large number of people who pass through particular subareasat open spaces where there are no prescribed roads. This pathdrawing method realizes better comprehensibility for many
trajectories recorded in a place where pedestrians can movefreely; however, this method does not visualize focusing on
the stationary states because stasis people are drawn by notlines but dots.
There have been a variety of visualization methods otherthan directly drawing lines of walking routes. Matsumoto etal. [3] proposed that the six items in the people flow line
analysis be visualized simultaneously by drawing points, lines,
and planes on a three-dimensional map. This tool enables
easy comparison of each walking status in different places
at the same time. Wang et al. [4] presented a visualization
method using a map and a string graph that connects thestarting and arriving points of taxis in urban areas in orderto analyze the people flow in such areas and to estimate thetravel time. This technique realizes better comprehensibility
by using not a chord graph but a large number of trajectories
on a map. Liu et al. [5] proposed a visualization techniquethat represents the stationary states of people by combining a
Voronoi diagram and a heatmap to analyze the distribution ofpeople entering immovable spaces such as trains and elevators.
However, it is difficult to observe the details of walking records
by using these methods that are specialized in summarizing
and representing walking paths, since these methods do not
draw human flow itself. Also, all of these are visualization
techniques only for pedestrians in a particular moment, anddo not represent temporal changes for a long time. According
to the above discussion, we found a solution that combines
the above three approaches to representing a large number ofwalking paths and their temporal changes.
Linking multiple views is an effective approach to visualize
complex datasets. Visualization systems linking multiple visualization elements are called ”Linked View” and have alreadybeen applied to various application domains of visualization
techniques. Guo et al. [6] proposed a simultaneous visualization method that the traffic lines of driving cars on the road are
visualized from three viewpoints including space, time, andmulti-dimensional attribute values. Miyazaki [7] proposed amethod to visualize the people flow by calculating probability
density and vector fields at the same time. This method enables
users to overlay two types of visualization and analyze walking
data from multiple viewpoints. These methods are common to
our study since they apply linked views. Meanwhile, our study
differs from the above two methods in targeting walking pathdata in open spaces that fixed paths are not specified.
Minemoto et al. [8] classified the walking state into three
types: direct, route search and goal search, and proposed amethod for drawing walking paths that represent the walking
states by a variety of touches of lines. Although this method is
a visualization method that enables users to easily understandwalking states because walking people are represented only
as lines, it does not represent stationary states effectively.
Meanwhile, we are developing a visualization method that
focuses on both pedestrians’ movement and stationary statesin this study.
Services for visualizing people flow for general users havealso been released recently. As an example, A service called
Ekishi-Vision [9] has been released by Tokyu Corporation.
This system displays the degree of congestion on the platform
and near the ticket gates in real-time, distributing from Tokyu
Line App. However, such services differ from our study in
specializing in visualizing real-time human flow.
III. P RESENTED V ISUALIZATION T OOL
This section presents the processing flow and detailed
description of the technical components of the presented
technique. We adopt not 3D- but 2D-visualization because
2D-visualization is convenient to use without specifying appropriate viewpoints. We applied the environment of recording
people flow data as reported by Miyagi et al. [1]A. Recording People Flow Data
We define that a record of people flow data includes thefollowing information:• Timestamp that the position of a pedestrian is measured.
• ID of the pedestrian.
• Position of the pedestrian in a 2D space (x, y).
We can form a trajectory of a particular pedestrian bycollecting the records which have the particular ID corresponding to the pedestrian, and then chronologically ordering thecollected records. Our current implementation uses a motion
capture device Xtion to measure the position of the head of

449

1

2

3

4

5

7

8

9

10 11 12

The four-level walking state values indicate stationary / slow
walking / normal walking / fast walking. The thresholds ofthe walking state values are defined based on the researchby Sasaki et al. [10] Also, we assign unique colors to thewalking state values by applying the color map shown in the
TABLE I. Specifically, people whose walking speed is slow
are represented in red or yellow, and those who are fast are
represented in green or blue.

6

13 14 15 16 17 18
19 20 21 22 23 24

Fig. 1. Symbolic map for visualization.

TABLE IR ELATIONSHIP BETWEEN WALKING S TATE VALUE AND C OLOR

the pedestrian in a 3D space and records them converting
into a 2D space. This implementation assigns a particular
ID to each pedestrian, and measures positions of heads ofpedestrians every one millisecond. Remark that it regards as
another pedestrian when he/she exits the camera frame andthen gets into the frame again; therefore, the same person may
be counted as multiple persons. We deal with the trajectories
in a 2D space corresponding to the floor of the real spacebecause changes in heights of the pedestrians’ heads do notaffect while determining the walking states.
B. Visualizing the Distribution of Walking States in All Subarea
The technique classifies the walking states of the pedestrians
and then visualizes the statistics of the walking states for each
subarea dividing the floor taken by the camera. Our current
implementation determines the walking states simply based on
the walking speeds.
We firstly symbolize and divide the floor taken by a camera
to construct a symbolic map (for example as shown in Fig.
1) and aggregate the pedestrians who have moved forward,
backward, left and right from each subarea of the floor. Here,the setting of an appropriate number of the subareas depends
on the shape and size of the floor and the resolution of thedisplay used for the visualization. We preferred to divide thefloor so that subareas are shaped square and appropriately
sized. Fig. 1 illustrates that the floor is divided into 24
subareas. Therefore, we aggregated the number of pedestrians
for 96 types of movements since the 24 subareas have fourmovement directions including forward, backward, left andright.
In the current implementation, the walking states are simply
classified based on the walking speeds. Specifically, we apply
the following processes.
1) Calculate the speed of each pedestrian in each subarea.
2) Calculate the average speed per second if the walking
person is in the same subarea as one second before.
3) Continue this calculation until the pedestrian moves toa different area.
4) Calculate the speed and direction of the movement ineach subarea if a pedestrian moves to different subareas
every second.
5) Classify the speed of the person into one of the fourlevels and specify as the walking state value.

Color
■ Red
■ Yellow■ Green■ BlueWalking State Value
12
3
4

Speed[m/s]〜 0.1
0.1 〜 0.45
0.45 〜 0.75
0.75 〜

Next, we count the number of people for each walking statevalue in each subarea. The technique visualizes the statistics
of pedestrians for each walking state and movement directionin each subarea.
Our representation of the distribution of walking states is
as follows. Gray squares are placed at the centers of subareas,
and four-color stacked bar charts are drawn in the upper, lower,
left and right side of the squares. The stacked bar chart in eachdirection is drawn in the colors assigned to the walking statevalues. The sizes of the charts depict the number of pedestrians
in each walking state.
C. Visualization of Temporal Change of Walking State in All
Subareas
In addition to the representation described in Section III-B,
we visualize the temporal change of the number of pedestriansof each walking state in all subareas as a stacked line chart.
This visualization represents the total number of walking
people of each walking state in each period (one minute in
the current implementation) for all subareas as shown in Fig.
1. Our implementation counts the number of pedestrians by
the number of unique pedestrian IDs. The total number ofcounted IDs may be larger than the actual total number ofparticipants since multiple IDs may be assigned to the same
person as explained in Section III-A. The color assignmentis based on the color map shown in TABLE I, as applied byanother visualization described in section III-B.
D. Visualization of Temporal Change of Walking State in theUser-Selected Subarea
Our implementation prepares walking route files for each
subarea and records the walking routes that pass throughthe subareas to the corresponding files. The implementationloads the file corresponding to a subarea when a user selects
it and displays the walking routes described in the opened
file. As an example, when it is divided into 24 subareas asshown in Fig. 1, the implementation creates 24 files, anddifferent files are opened in accordance with the selected area.
It means 24 different drawing results can be shown. Our
implementation represents walking paths as individual lines450

= poster= camera

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

-6.15

0

3.0(m)
Fig. 3. Visualization of walking state distribution in all areas.

0-3.0
6.15(m)Fig. 2. Visualization map in poster presentation venue.

without summarizing as a small number of bundles. It does
not distinguish the moving directions in order to keep simpler
representations.
E. Visualization of Walking Paths After Passing through theUser-Selected Subarea
We count the change in the number of pedestrians of eachwalking state for each subarea (24 places in total in theexample of Fig. 1). The movement direction is not taken
into account while counting. The color assignment is basedon the color map shown in TABLE I, as applied by anothervisualization described in section III-B.
IV. V ISUALIZATION E XAMPLES
A. Experiment Settings
We visualized a people flow dataset recorded at the poster
presentation venue of a joint laboratory meeting held in 2017using the proposed methods. Fig. 2 shows the layout of thevenue where the dataset was recorded. We divided into 24
subareas as shown in Fig. 2 as a result of our own trial anderror. Red points in this figure depict the locations where
Xtion is installed. Green rectangles at both sides of the room
wall (upper and lower ends in Fig. 2) depict positions ofposters. The central parts of this room are passages. A poster
session at the meeting consisted of eight presenters, and fivesessions every one hour were held. We recorded this datasetwith 40 participants without stopping for five hours. The
dataset was divided into five files corresponding to one hour.
This section introduces the visualization results of particular
temporal periods.
B. Visualization Examples
Fig. 3 shows an example of visualizing the distribution ofwalking states and temporal change in all subareas by applying
the methods described in Sections III-B and III-C. The left sideof Fig. 3 shows the spatial distribution of walking states in all

subareas and the right side of Fig. 3 shows the temporal changeof the distribution of walking states in all subareas. The color
assignment is based on the color map shown in TABLE I. The
larger the drawing subarea is, the larger the number of passed
pedestrians in the subarea is.
Fig. 3 illustrates that walking speeds of pedestrians aretotally slow. We estimate the reason is that a large numberof people participated in the poster session in a narrow space
where it was difficult to pass each other. Meanwhile, the left
side in Fig. 3 illustrates that almost everyone was stationary
because many people listened to poster presentations on thewall sides. On the other hand, the visualization result illustrates
that many people walked more speedy at the central passage.
Visualization of four walking directions makes users easy tounderstand not only the speeds of pedestrians but also their
moving direction intuitively. The sizes of the stacked bar charts
indicate that the numbers of pedestrians vary depending on thepositions of posters. For example, in the subarea surrounded by
the gray circle in Fig. 3 (left), many people stop at subareasin the lower side of the visualization result because the redbars of the stacked bar charts are larger than at subareas in
the upper side. We estimate the reason is that there was adoorway at the lower right side in the left side of Fig. 3 anda larger number of people saw the poster presentation in thesubareas at the lower side.
Lastly, the black circles at the right side in Fig. 3 illustratethat walking states have changed significantly. Specifically,
the number of stationary participants indicated by red bars
decreased while the number of other participants indicated byyellow, green and blue bars increased about every five minutes.
This suggests that poster presentations were over and many
participants moved to another poster about every five minutes.
Fig. 4 shows an example of selecting a particular subarea
and simultaneously visualizing walking paths. Also, the figure
shows the temporal changes in walking states in the subarea
by applying the representations described in Sections III-D
and III-E. The stacked line graph in Fig. 4 (right) illustrates
the temporal change of walking states over time in theselected subarea and many lines shown in the left side ofFig. 4 represents paths of pedestrians after passing throughthe selected subarea. These visualizations illustrate that theratios of the walking states changed over time. Simultaneous
visualization of temporal change of walking states and walking
routes makes it possible to compare the change of the number

451

(a) Select area 4.

(b) Select area 22.
Fig. 4. Simultaneous visualization of time change of walking states andwalking routes in selected area.

of people over time and the routes of pedestrians who pass
the selected subarea.
Fig. 4 (a) and Fig. 4 (b) show the visualization results
when a user selected the subareas enclosed by the blue circles
in Fig. 3. Fig. 4 (a) is a visualization when a user selected
the upper blue circle (area 4 of figure 2). Fig. 3 (b) is a
visualization when the lower blue circle (area 22 in Fig. 2)
is selected. Two subareas encircled by blue lines in Fig. 3
are almost symmetry; however, the walking routes in the two
subareas in Fig. 4 differ greatly. Specifically, many pedestrians
who have passed through the subarea 4 shown in Fig. 4(a) move backward (downward in the visualization result),
whereas a lot of pedestrians who have passed through thesubarea 22 shown in Fig. 4 (b) move not only in the forward(upward in the visualization result) direction but also in thelateral directions. Drawing such walking paths can promote
to understand these detailed movements of pedestrians. Also,
the stacked line graph in Fig. 4 (right) shows that a larger
number of pedestrians passed the particular subarea as shownin Figure 4 (b) rather than another time period shown in Fig.
4 (a).
C. Discussion
One of the essential issues is the design for drawing thewalking routes. The current implementation assigns colors andpositions of polygonal lines randomly in order to prevent thepolygonal lines representing multiple walking paths to reduce
the overlaps of lines that may worsen the comprehensibility ofvisualization results. This drawing method has a problem thatthe visualization results may fluctuate. We would like to adopt
a method for drawing walking routes presented by Minemoto
et al. [8]Another issue is that we need to adjust the amount ofwalking paths to be drawn. The current implementation fordrawing walking route drawing displays all the routes stored
as walking data. It may cause a serious amount of overlapsof the lines that may worsen the comprehensibility. We could
like to cluster walking paths and display only characteristic
walking path clusters to solve this problem.
The division of movement directions at each subarea is also
an important factor for our visualization. Many studies on theanalysis of people flow have classified the moving directionsof pedestrians into eight directions including not only forward,
backward, left and right but also oblique directions. The
movements to the diagonally adjacent subareas are deleted
from the visualization result in the current implementation;
however, drawing walking routes in selected subareas includes
the diagonal direction. Actually, the visualization results illustrate that some pedestrians directly moved to diagonallyadjacent subareas. This fact causes inconsistency between thetwo visualization results. The current implementation has fourbasic directions to avoid the complication of the visualization
views. Therefore, we would like to visualize the number ofpedestrians in eight directions including the diagonal directions while visualizing the number of pedestrians for eachwalking state for all subareas.
Finally, it is essential to support the visualization of longterm walking datasets as one of the goals of this research.
Specifically, we would like to support a user interface thatallows users to select a particular period or subarea that he/shewants to observe in detail from long-term walking datasets.
V. C ONCLUSIONS AND F UTURE W ORK
In this paper, we proposed a technique to visualize a large
amount of walking data in a wide area and for a long time
captured by multiple cameras. We classify the walking states
based on the walking speeds of pedestrians and visualize the
distribution of walking states of the pedestrians in the region
taken by the cameras. Furthermore, a linked visualization of
the temporal change of walking states and walking routes
enables the observation of the change of walking states.
This method enables us to understand the overview ofthe walking states from the distribution of walking statesin all subareas. Walking states of all pedestrians for each
subarea could be represented exhaustively by applying stacked
bar charts to visualize the existence of stationary pedestrians
that cannot be represented only by walking paths.
Also,
the visualization of temporal change of the distribution ofwalking states and walking routes in the user-selected subareas
enables to observe temporal changes of walking states atparticular locations. We represent the walking routes after
passing through each subarea because they are not represented
by the stacked bar charts that draw the distribution of walking
states.
It is not always sufficient and appropriate to classify thewalking states only based on walking speeds. Taking the poster
presentation hall as an example, we cannot determine whether
pedestrians are watching the poster presentations or they are
just chatting, even they have the same stationary state. This is alimitation of our current implementation that classifies walking452

states only from the coordinate values of the walking routes.
We would like to analyze the input movies to appropriately
determine the walking states in the future.
R EFERENCES[1] Y. Miyagi, M. Onishi, C. Watanabe, T. Itoh, M. Takatsuka, “Classification and Visualization for Symbolic People Flow Data”, Journal of
Visual Languages and Computing, Vol. 43, pp. 91–102, 2017.
[2] H. Yabushita, T. Itoh, “Summarization and Visualization of Pedestrian
Tracking Data”, 15th International Conference on Information Visualisation(IV2011), pp. 537–542, 2011.
[3] T. Matsumoto, Y. Nakamura, Y. Shiraishi, O. Takahashi, “Simultaneous
Visualization of Analysis Items for Human Mobility Analysis”, In The
19th IPSJ Workshop On Multimedia Communication and Distributed
Processing(DPSWS2011), pp. 63–69, 2011.
[4] H. Wang, H. Huang, X. Ni, W. Zeng, “Revealing Spatial-Temporal
Characteristics and Patterns of Urban Travel: A Large-Scale Analysis
and Visualization Study with Taxi GPS Data”, International Journal of
Geo-Information(ISPRS), pp. 257–278, 2019.
[5] X. Liu, W. Song, L. Fu, H. Zhang, “Pedestrian Inflow Process Under
Normal and Special Situations”, Proceeding of Pedestrian and Evacuation Dynamics 2016, pp. 136–143, 2016.
[6] H. Guo, Z. Wang, B. Yu, H. Zhao, X. Yuan, “TripVista: Triple Perspective Visual Trajectory Analytics and Its Application on Microscopic
Traffic Data at a Road Intersection”, 2011 IEEE Pacific Visualization
Symposium, pp. 163–170, 2011.
[7] S. Miyazaki, “Visualization of Customer Shopping-Path Using REID
Techniques”, Architectural Institute of Japan(AIJ) Journal of Technology
and Design, Vol. 18, pp. 1033–1037, 2012.
[8] T. Minemoto, M. Doi, X. Zhang, F. Akiyama, K. Ono, M. Watanabe,“Visualization Method for People Flow in Train Station Area ―Examinations of Effective Use of Individual People Flow Data (2)”, Bulletinof Japanese Society for Science of Design, Vol. 63, pp. 33–42, 2016.
[9] K. Yoneji, D. Matsubara, N. Akira, M. Iida, T. Murakami, N. Nukaga,“Congestion Degree Visualization Technology Using a Monitor Camerain the Station”, Journal of Digital Practices, Vol. 8, No. 2, pp. 152–159,
2017.
[10] M. Sasaki, H. Watanabe, “A Study on the Change of Walking Speedin Viewing Space”, Journal of Man-Environment Research Association,
Vol. 6, No. 2, pp. 33, 2000.

453