# Geospatial Intelligence for Health and Productivity Management in Japanese Restaurants and Other Industries

Takeshi Kurata

► To cite this version:

Takeshi Kurata. Geospatial Intelligence for Health and Productivity Management in Japanese Restaurants and Other Industries. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2021, Nantes, France. pp.206-214, 10.1007/978-3-030-85906-0_23 . hal-04022110

HAL Id: hal-04022110

https://inria.hal.science/hal-04022110

Submitted on 9 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Geospatial Intelligence for Health and Productivity Management in Japanese Restaurants and Other Industries

Takeshi Kurata¹²

¹ Human Augmentation Research Center, AIST, Japan

² University of Tsukuba, Japan
t.kurata@aist.go.jp

**Abstract.** Health and Productivity Management (HPM) requires simultaneous improvement of labor productivity and Quality of Working (QoW), which consists of health, workability, and rewarding. In order to deal with a wide range of issues for HPM, engineering approaches are much more effective rather than just relying on experience and intuition. First, this paper outlines Geospatial Intelligence (GSI) as a tool for such engineering approaches, which supports problem solving by linking geospatial data with other data. Next, we summarize use cases of GSI in service and manufacturing sites, including Japanese restaurants, which have addressed labor productivity and QoW. Finally, we extract the metrics regarding labor productivity and QoW used in those use cases.

**Keywords:** Health and productivity management (HPM), Geospatial intelligence (GSI), Quality of working (QoW), Indoor positioning

## 1 Introduction

In 2015, SDGs were adopted by the United Nations, and in the same year, the Ministry of Economy, Trade and Industry (METI) and the Tokyo Stock Exchange (TSE) started to select HPM stocks in Japan. HPM requires the simultaneous improvement of labor productivity and QoW [1]. As shown in Fig. 1, QoW consists of health, workability, and rewarding, which is closely related to occupational safety and health, QWL, and decent work, as well as social capital. To improve both labor productivity and QoW in a well-balanced manner, it is necessary to deal with a wide range of issues. Therefore, there is limitation in taking action by experience and intuition, and engineering approaches are essential.

## 2 Geospatial Intelligence

It is natural to incorporate ideas such as digital twin and DX as engineering approaches that contribute to HPM. An effective tool for this purpose is GSI, which supports problem solving by linking geospatial data, including positioning data, with other
sources of data. This is often described as GEOINT, but it has a strong national security nuance, so here we call it GSI. The conceptual diagram of GSI is shown in Fig. 2 [2].

GSI makes organic use of both (1) digital twin/IoT/IoH technologies and methodologies and (2) social technologies and methodologies cultivated in service engineering, IE, etc. to provide a comprehensive picture of the current situation in service and manufacturing sites while acquiring 6M data there. Furthermore, GSI can support process modelling, improvement activities, and preliminary evaluation (simulation). It has been reported that about 60-80% of data is related to location [3], and that humans spend about 90% of their time indoors [4]. In light of these facts, we believe that GSI, especially Indoor GSI is an effective way to promote DX (Fig. 1).

<figure><img src="image_3.png" /><figcaption>Fig. 1. Health and Productivity management (Labor productivity and QoW).</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2. Conceptual diagram of GSI.</figcaption></figure>

## 3 Use cases of GSI

Tables 1 and 2 aggregate the use cases of GSI in service and manufacturing sites, including Japanese restaurants, that AIST has been involved in or investigated. As
shown in the tables, both of productivity and QoW have been addressed in most of the
cases. Although we have started applied GSI with a clear awareness of QoW and HPM
only since 2015, it is confirmed that both productivity and QoW have been targeted
since 2010 when we started applying GSI for ‘Kaizen (improvement)’. Table 1 lists the
objectives, methods, effects and results, positioning technologies used, joint research
partners, investigation sites, and references for each case study, and each row of Table
2 presents the evaluation items, indicators, and contents related to labor productivity
and QoW used for each case in the same row of Table 1.

<figure><img src="image_5.png" /><figcaption>Fig. 3. Example of CQSCC with GSI in a Japanese restaurant in 2012.</figcaption></figure>

The sites and job categories covered include helpers and nurses at nursing homes [6], guest room attendants at Japanese inns [7], customer service assistants and catering staff at Japanese restaurants [8-11], flight attendants on passenger planes [12], building maintenance workers [13], pickers at distribution warehouses [14], office workers, and manufacturing workers at cable factories [2][15]. Over the years, especially through practical experiments in Japanese restaurants, GSI-related techniques and methodologies have been developed and improved, and knowledge has been accumulated [8-11]. In addition, in the past few years, we have confirmed that the technologies, methodologies, and knowledge can be transferred from the service sites to the manufacturing sites [2][15].

As an example of GSI utilization, we briefly present a case study of a Japanese
restaurant in 2012 [9] along with each item in Tables 1 and 2 (Fig. 3). The purpose of
using GSI in this case was to reduce the physical workload of the employees while
maintaining hospitality, which had been improved in the previous year [8]. In order to
achieve this goal, a CSQCC (Computer Supported Quality Control Circle) activity [8] was conducted by measuring flow lines, analyzing descriptive statistics of each indicator, and visualizing them. For measuring flow lines, the CSQCC installed an integrated indoor positioning system based on PDR, active RFID, and map matching. It was proposed spontaneously by the CSQCC participants to adopt an area-in-charge system, and it resulted in a shorter walking distance while maintaining hospitality.

In addition, as a spinoff effect of adopting the area-in-charge system, skill indicators for individuals and teams were designed as in the dedication and defense ratios (precision and recall in [9]) regarding the area in charge. These indicators were derived from POS (Point of Sales) data, and it allowed us to visualize the distribution of the skills of each wait staff. Thus, it can be seen that productivity (waste), health (physical load), rewarding (hospitality skills, spontaneity), and workability (information sharing, transparency of work conditions) were efficiently and simultaneously handled by GSI in this case study.

# 4 Summary of use cases

The purposes of each case have progressed from understanding the current (As-Is) situation alone to supporting improvement and pre-assessing (To-Be) hypotheses by simulations. In addition, the specific objectives have become broader such as skill analysis, training support, simultaneous support for productivity improvement, safety management, and health management, effect verification of introducing robots, and feasibility verification of employment support for the visually impaired (Fig. 4).

In some cases, existing indicators can be applied, while in other cases, the process starts with designing indicators. The evaluation items {indicators} for labor productivity include those that can be obtained directly from geospatial data, such as wasted movement {flow lines}, traffic congestion {heat maps indicating duration of stay}, and deviation from code of operation {duration of stay in each area}; those that can be applied indirectly from geospatial data, such as workload and working hours; and other items (number of additional orders {POS}, service quality {customer complaints}, added value {operating profit (rate)}, and labor productivity {sales per hour}) are used (Fig. 5).

As described above, QoW consists of health, workability, and rewarding. Regarding health, the walking distance is used for both the negative aspect of physical load and the health-promoting aspect of exercise intensity, and geospatial data is also applied to the evaluation of exercise space usage rate {duration of stay in the area} and mental load {relaxation/contingency allowance}. As for rewarding, individual and team skills are evaluated based on the flow lines, movement speed, duration of stay, working hours, or the dedication/defense ratio in charge. For qualitative evaluation of skills by retrospective interviews, pseudo FPV (First-Person View) videos generated with 3D models of work sites and flow lines [5] are sometimes used as visual geospatial contents.

And with respect to interpersonal relations and organization out of workability, it is confirmed whether information sharing, transparency of work conditions, and
visualization of the 6M data have been carried out, and geospatial data such as flow lines and heat maps are used as contents for this purpose. Other evaluation items for interpersonal relations and organization include fairness {unevenness of workload among workers}, descriptive statistics of communication {flow lines and duration of stay}, and Employee Satisfaction (ES) {questionnaire}. Flow lines and heat maps are also utilized to validate the safety when working alone and the intersection situation between workers and vehicles. There is also a diversity-related case in which the feasibility of work support of low-vision and blind people is evaluated with working hours.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Purposes and specific objectives extracted and summarized from Table 1.</figcaption></figure>

## 5 Conclusion

In this paper, a wide range of actual cases are summarized to contribute to the study of how to utilize GSI to promote DX for the purpose of HPM. By dealing with multiple items related to labor productivity and QoW at the same time, the organizations will be able to maintain a high balance between them. We summarized the indicators used in each of the cases listed in Table 2 as shown in Fig. 5. It demonstrates that there are diverse but common indicators obtained directly or indirectly from geospatial data, and that they can be easily applicable by GSI. In fact, in the specific case of the Japanese restaurant introduced in section 3, the combination of geospatial data such as flow lines and POS data was able to simultaneously handle multiple indicators associated with productivity, health, rewarding, and workability for HPM. In addition, the organizations also receive the advantage of being able to disperse the initial/running costs of GSI over several purposes rather than just for a single purpose [15].

DX means digitizing 6M information, including geospatial information, and transforming products, services, business processes, and ultimately the organization. However, it takes a long time to see results after digitization, and not a few organizations find it difficult to gain understanding of GSI implementation from both the executive labor sides. The only way to achieve this drastically is to spread the concept of DX throughout society, but making it easier to introduce, operate, and maintain the GSI infrastructure and improving interoperability in groups such as supply chains using 6M data will also contribute to promoting the application of GSI at the
practical level. In addition, geospatial artificial intelligence (GeoAI) technologies [16] based on machine learning are likely to attract more and more attention from service and manufacturing industries from now on. We will continue our research and development on how GeoAI can combine the evaluation indicators summarized in Fig. 5 and utilize them for the purpose summarized in Fig. 4, or whether new metrics can be acquired by GeoAI.

<figure><img src="image_7.png" /><figcaption>Fig. 5. Evaluation items {indicators/contents} extracted and summarized from Table 2.</figcaption></figure>
References

1. QoW, http://www.cocn.jp/report/thema91-L.pdf (accessed 2021-03-21)

2. Kurata, T., et al., Towards Realization of 6M Visualization in Manufacturing Sites, The First IEEE VR Workshop on Smart Work Technologies, 4 pages (2019)

3. Hahmann, S., Burghardt D., How much information is geospatially referenced? Networks and cognition, International Journal of Geographical Information Science, 27:6, 1171-1189 (2013)

4. Klepeis, N. E., et al., The National Human Activity Pattern Survey (NHAPS): A Resource for Assessing Exposure to Environmental Pollutants, Lawrence Berkeley National Laboratory (2001)

5. Ishikawa, T., et al., Interactive 3-D Indoor Modeler for Virtualizing Service Fields, Virtual Reality, 17(2), pp.89-109 (2013)

6. Motomura, Y., et al., Community-Based Participatory Service Engineering: Case Studies and Technologies, Kwan, S. K., Spohrer, J. C., Sawatani, Y., eds., Global Perspectives on Service Science: Japan, Springer-Verlag New York, pp.63-78 (2016)

7. Nakajima, M., et al., Cognitive Chrono-Ethnography Lite, Work: A Journal of Prevention, Assessment and Rehabilitation (IEA 2012: 18th World congress on Ergonomics - Designing a sustainable future), Vol.41, pp.617-622 (2012)

8. Fukuhara, T., et al., Improving Service Processes based on Visualization of Human-behavior and POS data: a Case Study in a Japanese Restaurant, Proc. ICServ 2013, pp.1-8 (2013)

9. Fukuhara, T., et al., Estimating skills of waiting staff of a restaurant based on behavior sensing and POS data analysis: A case study in a Japanese cuisine restaurant, Proc. AHFE2014, pp.4287-4299 (2014)

10. Okuma, T., et al., Role of Servicing Activity Visualization in Quality Control Circle, Proc. ICServ 2015, 6pp. (2015)

11. Shinmura, T., et al., Service robot introduction to a restaurant enhances both labor productivity and service quality, CIRP ICME '19 (2019)

12. Tachioka, K., et al., Behavioral Measurements of Cabin Attendants Together with Observations and an Analysis of Their Tasks by Using Service Process Model, Proceedings of the 4th International Conference on Serviceology (ICServ2016), pp.271-277 (2016)

13. Ichikari, R., et al., T., A Case Study of Building Maintenance Service Based on Stakeholders' Perspectives in the Service Triangle, Proc. ICServ 2018, pp.87-94 (2018)

14. Myokan, T., et al., Pre-evaluation of Kaizen plan considering efficiency and employee satisfaction by simulation using data assimilation -Toward constructing Kaizen support framework-, Proc. ICServ 2016, 7pp. (2016)

15. Kurata, T., et al., IoH Technologies into Indoor Manufacturing Sites, IFIP International Conference on Advances in Production Management Systems (APMS), pp.372-380 (2019)

16. Openshaw, S., and Openshaw, C., Artificial Intelligence in Geography, Chichester, UK: John Wiley & Sons (1997)
<table><caption>**Table 1.** Use cases using GSI: Purposes, objectives, methods, effects and results, positioning techs, joint research partners, and references. (PDR: Pedestrian Dead Reckoning, RFID: Radio Frequency Identification, BLE: Bluetooth Low Energy, MAP: Map matching, QC: Quality Control, OJT: On-the-Job Training).</caption><thead><tr><th>Site [Job category]</th><th>Year</th><th>Purpose</th><th>Method</th><th>Effect and result</th><th>Positioning technologies</th><th>Joint research partners/ investigation sites</th><th>References</th></tr></thead><tbody><tr><td>Nursing home [ helpers, nurses ]</td><td>2010</td><td>As-Is comprehension of care work flow</td><td>Flow-line measurement, descriptive statistics, interview</td><td>Process modeling, visualization of issues</td><td rowspan="2">PDR+ Active RFID +MAP</td><td>SUPER COURT</td><td>[6]</td></tr><tr><td>Japanese inn [guest room attendants]</td><td>2011</td><td>Skill analysis</td><td>Flow-line measurement, descriptive statistics, retrospective interview</td><td>As-Is comprehension of skill</td><td>Kinosaki hot springs</td><td>[7]</td></tr><tr><td rowspan="2">Japanese restaurant 1 [Customer service assistants and catering staff]</td><td rowspan="2">2011</td><td>Hospitality improvement</td><td rowspan="2">Flow-line measurement, descriptive statistics, QC activity</td><td>Increase in duration of stay in guest areas and additional orders</td><td rowspan="2">PDR+ Active RFID +MAP</td><td rowspan="2">GANKO FOOD SERVICE Co.,Ltd.</td><td>[8]</td></tr><tr><td>Physical load reduction while hospitality improvement</td><td>(In addition to the above) reduction of walking distance, skill modeling</td><td>[9]</td></tr><tr><td>Japanese restaurant 2 [Customer service assistants and catering staff]</td><td>2012</td><td>Support of new restaurant launch</td><td>Flow-line measurement, descriptive statistics, QC activity</td><td>visualization of issues. Kaizen effect check, rapid launch</td><td></td><td></td><td>[10]</td></tr><tr><td>Japanese restaurant 3 [Customer service assistants and catering staff]</td><td>2018 -19</td><td>Effectiveness verification of robot installation</td><td></td><td>Increase in sales per man hour and dedication in charge</td><td>PDR+BLE +MAP</td><td>University of Tokyo, ANA Strategic Research Institute Co., Ltd.</td><td>[11]</td></tr><tr><td>Passenger plane [Flight attendant]</td><td>2015</td><td>Constructing training support environments to facilitate awareness learning</td><td>Flow-line measurement, descriptive statistics, process modeling, retrospective interview</td><td>Skill modeling in serving drink, training support</td><td>PDR+BLE +MAP</td><td></td><td>[12]</td></tr><tr><td>Office [office worker]</td><td>2016</td><td>Improving office environments (Communication/exercice space installation)</td><td>Flow-line measurement, descriptive statistics</td><td>Increase in usage rate of communication and exercise space</td><td>RGB-D camera</td><td>Fujikura Ltd.</td><td>—</td></tr><tr><td>Building [Building maintenance worker]</td><td>2017</td><td>Investigating effects and impacts of work restructuring</td><td>Flow-line measurement, descriptive statistics, Questionnaire, interview</td><td>Clarification of effects and issues before and after the restructuring</td><td>PDR+BLE +MAP</td><td>KAJIMA TATEMONO SOGO KANRI CO.,LTD.</td><td>[13]</td></tr><tr><td>Distribution warehouse 1 [Picker]</td><td>2014</td><td>As-Is comprehension of picking work flow, Kaizen simulation</td><td>Flow-line measurement, descriptive and inferential statistics, process modeling, interview</td><td>Process modeling, To-Be simulation to compare Kaizen plans</td><td>Visible Light Communication</td><td>Framework, Inc., TRUSCO NAKAYAMA Corporation</td><td>[14]</td></tr><tr><td>Distribution warehouse 2 [low-vision/blind subject]</td><td>2018 -19</td><td>Effectiveness check of work support of low-vision and blind workers</td><td>Picking operation instructions based on positioning</td><td>Feasibility confirmed</td><td>Passive RFID</td><td>GOV Co.,Ltd., TRUSCO NAKAYAMA Corporation</td><td>—</td></tr><tr><td>Cable factory 1 [Manufacturing worker]</td><td>2018 -19</td><td>Workload leveling</td><td>Flow-line measurement, descriptive statistics, information sharing, OJT, retrospective interview</td><td>Workload leveling, Clarification of issues</td><td>BLE</td><td></td><td>—</td></tr><tr><td>Cable factory 2 [Manufacturing worker]</td><td>2018 -19</td><td>Simultaneous support for productivity improvement, safety management, and health management</td><td>Flow-line measurement, descriptive statistics, information sharing, POC</td><td>—</td><td>PDR+BLE +MAP</td><td>Sumitomo Electric Industries, Ltd.</td><td>[2][15]</td></tr></tbody></table>
<table><caption>**Table 2.** Use cases using GSI: Evaluation items on labor productivity and QoW (outside {}): evaluation items, inside {}: evaluation indicators and contents used, bold: indirectly related to geospatial data, bold underlined: directly related to geospatial data).</caption><thead><tr><th rowspan="3">Site<br>[Job category]</th><th colspan="2">Labor productivity</th><th colspan="8">Evaluation item [indicators and contents]</th></tr><tr><th rowspan="2">Output</th><th rowspan="2">Input</th><th colspan="4">Health</th><th colspan="4">QoW</th><th colspan="3">Workability</th></tr><tr><th>Physical</th><th>Mental</th><th colspan="2">Skill<br>Hospitality</th><th colspan="2">Rewarding<br>Spontaneity</th><th>Interpersonal<br>relations/<br>Organization</th><th>Safety</th><th>Diversity</th></tr></thead><tbody><tr><td>Nursing home<br>[助养人, 护士]</td><td colspan="2">Wasted<br>movement<br>[Flow lines]</td><td></td><td></td><td></td><td></td><td>Individual and<br>team skills<br>[Flow lines,<br>duration of<br>stay]</td><td></td><td></td><td></td><td></td></tr><tr><td>Japanese inn<br>[ guest room<br>attendants]</td><td colspan="2"></td><td></td><td></td><td>Subjective<br>skill<br>description<br>[Pseudo FPV<br>video]</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Japanese restaurant 1<br>[Customer<br>service<br>assistants and<br>catering staff]</td><td>Number of<br>additional orders<br>[POS]</td><td>Wasted<br>movement<br>[Flow lines]</td><td>Physical load<br>[Walking<br>distance]</td><td>Customer<br>service time<br>[Heat maps,<br>Duration of<br>stay in guest<br>areas]</td><td>Individual and<br>team skills<br>[POS,<br>Dedication/d<br>effort rate in<br>charge]</td><td></td><td></td><td>Keizen<br>proposal</td><td>Information sharing,<br>transparency of<br>work conditions<br>[Geospatial data]</td><td></td><td></td></tr><tr><td>Japanese restaurant 2<br>[Customer<br>service<br>assistants and<br>catering staff]</td><td rowspan="2">Labor productivity<br>[Sales per man hour]</td><td></td><td></td><td></td><td></td><td>Dedication [Heat maps]</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Japanese restaurant 3<br>[Customer<br>service<br>assistants and<br>catering staff]</td><td></td><td></td><td></td><td></td><td>Individual and team skills<br>[Dedication rate in charge]</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Passenger plane<br>[Flight<br>attendant]</td><td></td><td></td><td></td><td></td><td>Interpersonal<br>difference in<br>serving drink<br>[Flow lines]</td><td></td><td>Look-back</td><td></td><td></td><td></td><td></td></tr><tr><td>Office<br>[office worker]</td><td>Amount of added<br>value<br>(Operating profit<br>(rate))</td><td></td><td>Exercise<br>space usage<br>rate [Duration<br>of stay]</td><td></td><td></td><td></td><td></td><td></td><td>descriptive<br>statistics of<br>communication<br>[Flow lines,<br>duration of stay]</td><td></td><td></td></tr><tr><td>Building<br>[Building<br>maintenance<br>worker]</td><td>Service quality<br>(Customer<br>complaints)</td><td>Workload in<br>office, Wasted<br>movement<br>[Flow lines]</td><td>Physical load<br>[Walking<br>distance]</td><td>Mental load<br>[Relaxation/c<br>ontingency<br>allowance]</td><td>Objective skill<br>[movement<br>speed,<br>working<br>hours]</td><td></td><td></td><td>ES<br>[Questionnaire]</td><td></td><td></td><td></td></tr><tr><td>Distribution<br>warehouse 1<br>[Packer]</td><td></td><td>Traffic<br>congestion<br>[Heat maps],<br>Workload per<br>man hour,<br>working hours<br>per team</td><td></td><td></td><td></td><td></td><td></td><td>Fairness<br>[Unevenness of<br>workload among<br>workers]</td><td></td><td></td><td></td></tr><tr><td>Distribution<br>warehouse 2<br>[low-<br>vision/blind<br>subject]</td><td></td><td>Comparison<br>between sighted<br>and low-<br>vision/blind<br>subjects [working<br>hours]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Feasibility of<br>work support of<br>low-vision and<br>blind people<br>[Working hours]</td><td></td></tr><tr><td>Cable factory 1<br>[Manufacturing<br>worker]</td><td></td><td>Deviation from<br>code of operation<br>[Duration of<br>stay]</td><td></td><td></td><td></td><td>Interpersonal<br>difference<br>[Duration of<br>stay]</td><td></td><td></td><td>Information sharing,<br>transparency of<br>work conditions<br>[Geospatial data]</td><td></td><td></td></tr><tr><td>Cable factory 2<br>[Manufacturing<br>worker]</td><td></td><td>Wasted<br>movement<br>[Flow lines]</td><td>Physical<br>load/Exercise<br>intensity<br>[Walking<br>distance]</td><td></td><td></td><td></td><td></td><td></td><td>Information sharing,<br>transparency of<br>work conditions, EM<br>visualization<br>[Geospatial data]</td><td>Working<br>alone/Intersec<br>tion situation<br>between<br>workers and<br>vehicles<br>[Flow lines,<br>heat maps]</td><td></td></tr></tbody></table>