# Analyzing Operations on a Manufacturing Line Using Geospatial Intelligence Technologies

Takeshi Kurata, Munenori Harada, Katsuko Nakahira, Takashi Maehata, Yoshinori Ito, Hideki Aso

► To cite this version:

Takeshi Kurata, Munenori Harada, Katsuko Nakahira, Takashi Maehata, Yoshinori Ito, et al. Analyzing Operations on a Manufacturing Line Using Geospatial Intelligence Technologies. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2022, Gyeongju, South Korea. pp.69-76, 10.1007/978-3-031-16407-1_9. hal-05386010

HAL Id: hal-05386010

https://inria.hal.science/hal-05386010v1

Submitted on 27 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Analyzing Operations on a Manufacturing Line Using Geospatial Intelligence Technologies

Takeshi Kurata¹(✉), Munenori Harada², Katsuko Nakahira², Takashi Maehata³, Yoshinori Ito⁴, and Hideki Aso⁴

¹ Human Augmentation Research Center, AIST, Chiba, Japan
t.kurata@aist.go.jp

² Faculty of Engineering, Nagaoka University of Technology, Nagano, Japan

³ IoT R&D Center, Sumitomo Electric Industries, Ltd., Osaka, Japan

⁴ IoT Acceleration Lab, J-Power Systems Co. Ltd., Ibaraki, Japan

**Abstract.** This paper reports on a case study of work analysis using geospatial intelligence technologies conducted on a manufacturing line at a J-Power Systems (JPS) plant in FY2020. First, an overview of the workplace, the purpose of the analysis, and the types of data used in the analysis is presented. Next, we describe the data processing methods developed in the preparatory phase of the analysis, and finally, we report the results and discussion of the work analysis.

**Keywords:** Geospatial intelligence (GSI) · Indoor positioning · Machine learning · Manufacturing line · Work analysis

## 1 Introduction

Health and productivity management (HPM) [1–3] is realized by simultaneously improving labor productivity and Quality of Working (QoW) in a well-balanced manner. Since it is necessary to deal with a wide range of issues for that, engineering and Digital Transformation (DX) approaches are essential. On the other hand, it has been reported that 60–80% of information is related to location information [4] and that humans spend approximately 90% of their time indoors [5]. Therefore, geospatial intelligence (GSI), especially indoor GSI, which supports problem solving by linking geospatial data with other data, is expected to be an effective means to promote DX and HPM [6, 7]. This paper reports on a case study of work analysis using indoor GSI technologies conducted on manufacturing lines (MLs) at a JPS plant.

The area was about 1,800 m² rectangle in size. An indoor positioning system [8] and a PLC were installed to collect data to determine the position of each worker and the state of each equipment (Fig. 1). There are three MLs with three workers in charge of one line each. The leader of the MLs manages the entire work area, and sometimes works in cooperation with other workers as needed.

The main objective of this work analysis was to investigate the feasibility of (1) improving the operation rate, and it also included feasibility studies on (2) automating
work records and (3) visualizing work transitions, respectively, and (4) confirming the value of work skills. From the perspective of HPM, (1) to (4) are all related to productivity, and (2) to (4) are related to QoW (physical/mental load reduction, transparency of work status, and skill improvement).

<figure><img src="image_3.png" /><figcaption>Fig. 1. Workplace overview and the indoor GSI.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2. Types of time-series work classification data (left), breakdown of work classifications, as well as the GT values and work-shift overlaps (right).</figcaption></figure>

## 2 Basics of This Work Analysis Case Study

In the actual case study presented in this paper, work analysis was conducted in the area containing the work-in-progress (WIP) manufacturing process at a JPS plant [7].

We use four types of time-series work classification data for different purposes as shown in Fig. 2-left, which are along the “ML axis” or the “worker axis” where The ML axis indicates the time axis of each ML, and the worker axis indicates the time axis of each worker.

The “ground-truth (GT) data” were prepared manually by visual inspection using recorded video for three days of work. Only the GT data have both the ML axis and the worker axis. The handwritten stoppage-factor records (hereinafter referred to as “HS records”) are daily handwritten records kept by workers in the workplace. The “quasi-GT data” are generated by integrating the HS records and PLC data using the method described in Sect. 3.1. Since both HS records and PLC data are constantly obtained
onsite, they can be easily scaled up. The “auto-estimated data” are the estimation results of a method based on machine learning (LSTM: Long short-term memory) [9] using PLC data and positioning data as described in Sect. 3.2. All three types of data are along the ML axis. Figure 2-right shows the breakdown of work classifications, as well as the GT values and work shift overlaps for workers #1, #2, and #3.

<figure><img src="image_5.png" /><figcaption>Fig. 3. MR detection process (left), Travel distance map for position-based features (right).</figcaption></figure>

# 3 Methodology of Work Analysis on ML Axes

## 3.1 Quasi-Ground-Truth Data Generation

The HS records contain no missing data along the ML axis. However, there are two significant issues for investigating the feasibility of improving the operation rate. One is that the start and end times of each record are rounded off, and the other is that “material replenishment (MR)” works are not included in the records even if they are supposed to be one of major stoppage factors for the MLs.

Meanwhile, because IoT transition is still underway in the workplace, PLC data that can be sent to the server and used for analysis are limited.

In this case study, we developed a method to generate “quasi-GT data” that are more accurate in time and include MR works by integrating HS records and PLC data. The method mainly consists of a process to make time recording more accurate using PLC data and an MR detection process. This section outlines the latter, which is also used to extract one of PLC-based features for LSTM.

With PLC data, it is possible to grasp each status of “out of material” and “open/closed status of the equipment cover”. We developed a method for detecting MR works by identifying the state transitions caused by their combinations, as shown in Fig. 3-left. Note that “arrangement in internal setup #1 (with MR)”, “container replacement for WIP input”, and “arrangement in internal setup #1 (without MR),” which are S<sub>A</sub>, S<sub>C</sub>, and S<sub>D</sub> respectively, are related setups that are difficult to separate in the machine learning described below, so we deal with them collectively in this case study.

The accuracy of the quasi-GT data generation method, including the MR detection process, was evaluated by comparing the HS records, GT data, and quasi-GT data, respectively. Comparison of the GT and quasi-GT data (Table 1-bottom-left) confirmed
that the quasi-GT data can be generated with an accuracy of approximately 92%, that it is possible to obtain data closer to the GT data than HS records (Table 1-top-left), and that it is possible to automatically record operation (OP) and MR with the high accuracy of 96% and 93% respectively by using PLC data.

<table><caption>**Table 1.** Comparisons between HS records, GT data, Quasi-GT data, and Auto-estimated data.</caption><thead><tr><th colspan="6">HS Records vs GT data: 76%</th><th colspan="6">HS Records vs Quasi-GT data: 80%</th></tr><tr><th></th><th>recall</th><th>precision</th><th>f1</th><th>time ratio</th><th>time (sec.)</th><th></th><th>recall</th><th>precision</th><th>f1</th><th>time ratio</th><th>time (sec.)</th></tr></thead><tbody><tr><td>OTHER</td><td>41.3%</td><td>66.8%</td><td>51.0%</td><td>61.9%</td><td>11,096</td><td>OTHER</td><td>40.5%</td><td>100.0%</td><td>57.7%</td><td>40.5%</td><td>16,948</td></tr><tr><td>OP</td><td>97.6%</td><td>66.5%</td><td>79.1%</td><td>146.7%</td><td>63,148</td><td>OP</td><td>100.0%</td><td>65.3%</td><td>79.0%</td><td>153.2%</td><td>60,486</td></tr><tr><td>MR</td><td>0.0%</td><td></td><td></td><td>0.0%</td><td>23,575</td><td>MR</td><td>0.0%</td><td></td><td></td><td>0.0%</td><td>22,072</td></tr><tr><td>S<sub>A</sub> S<sub>C</sub> S<sub>D</sub></td><td>88.2%</td><td>95.6%</td><td>91.8%</td><td>92.2%</td><td>25,696</td><td>S<sub>A</sub> S<sub>C</sub> S<sub>D</sub></td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>23,700</td></tr><tr><td>S<sub>B</sub></td><td>100.0%</td><td>65.2%</td><td>78.9%</td><td>153.5%</td><td>2,346</td><td>S<sub>B</sub></td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>3,600</td></tr><tr><td>S<sub>E</sub></td><td>81.1%</td><td>90.5%</td><td>85.6%</td><td>89.6%</td><td>11,045</td><td>S<sub>E</sub></td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>9,900</td></tr><tr><td>TB</td><td>96.8%</td><td>95.9%</td><td>96.3%</td><td>100.9%</td><td>21,400</td><td>TB</td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td><td>21,600</td></tr></tbody></table><table><thead><tr><th colspan="6">Quasi-GT data vs GT data: 92%</th><th colspan="6">Auto-estimated data vs Quasi-GT data: 91%</th></tr><tr><th></th><th>recall</th><th>precision</th><th>f1</th><th>time ratio</th><th>time (sec.)</th><th></th><th>recall</th><th>precision</th><th>f1</th><th>time ratio</th><th>time (sec.)</th></tr></thead><tbody><tr><td>OTHER</td><td>88.8%</td><td>58.2%</td><td>70.3%</td><td>152.7%</td><td>11,096</td><td>OTHER TB</td><td>88.2%</td><td>80.2%</td><td>84.0%</td><td>110.0%</td><td>24,130</td></tr><tr><td>OP</td><td>93.6%</td><td>97.7%</td><td>95.6%</td><td>95.8%</td><td>63,148</td><td>OP</td><td>95.4%</td><td>96.7%</td><td>96.0%</td><td>98.6%</td><td>54,808</td></tr><tr><td>MR</td><td>90.4%</td><td>96.6%</td><td>93.4%</td><td>93.6%</td><td>23,575</td><td>MR</td><td>94.6%</td><td>95.9%</td><td>95.2%</td><td>98.6%</td><td>20,982</td></tr><tr><td>S<sub>A</sub> S<sub>C</sub> S<sub>D</sub></td><td>88.2%</td><td>95.6%</td><td>91.8%</td><td>92.2%</td><td>25,696</td><td>S<sub>A</sub> S<sub>C</sub> S<sub>D</sub></td><td>73.6%</td><td>77.5%</td><td>75.5%</td><td>94.9%</td><td>11,671</td></tr><tr><td>S<sub>B</sub></td><td>100.0%</td><td>65.2%</td><td>78.9%</td><td>153.5%</td><td>2,346</td><td>S<sub>B</sub></td><td></td><td></td><td></td><td>0</td><td></td></tr><tr><td>S<sub>E</sub></td><td>81.1%</td><td>90.5%</td><td>85.6%</td><td>89.6%</td><td>11,045</td><td>S<sub>E</sub></td><td>76.6%</td><td>86.4%</td><td>81.2%</td><td>88.7%</td><td>6,900</td></tr><tr><td>TB</td><td>96.8%</td><td>95.9%</td><td>96.3%</td><td>100.9%</td><td>21,400</td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

## 3.2 Automatic Recording Based on LSTM

To achieve automatic recording of work classifications in addition to OP and MR without HS records, we developed a work classification estimation method based on LTSM using 47 features derived from PLC data and 9 features derived from positioning data. Besides the features obtained using the MR detection process described above, internal-setup features were also designed based on PLC data. Although they are more coarse-grained than work classifications such as OP, MR, and S series denoted as S*, it can represent each state such as the first and second half of setup, and several types of machine operations during setup.

As for features based on positioning data, we designed 9 distance features each of which represents the travel distance for a worker to reach each target equipment. For efficiency, maps of travel distances from each equipment, as shown in Fig. 3-right, are pre-computed and stored. In addition, the time spent in each work area is also used as one of features by defining work areas that are assumed to correlate strongly with some of specific works in the work classification.

Either the GT or quasi-GT data can be used for LSTM training. Although the GT data are more accurate, they are only available for three days in this case study and are very expensive to prepare in actual operation. In contract, we can obtain the quasi-GT data continuously, and they are close enough to the GT data as shown in the previous section. For these reasons, LSTM training was conducted using the quasi-GT data.

We evaluated the accuracy of the LSTM-based method. Since the purpose of work classification estimation was to automatically generate data comparable to the quasi-GT data.
data, which contributes to elimination of manual recording works, the comparison target here was the quasi-GT data. As a result of evaluation by k-fold cross-validation (K = 3), the estimation accuracy was about 91% (Table 1-bottom-right). Note that K = 3 was used for this evaluation since there are state sequences the occurrence frequency of which is small with respect to the length of whole data.

While OP and MR are highly accurate thanks to each detection process that con-
stitutes the quasi-GT data generation method, there was room for improvement for
estimating S* each of which is internal-setup-related work classification. As shown in
Table 1-bottom-left, comparing the GT data, the accuracy of S* in the quasi-GT data is
not so high for the training data, and it may have affected the accuracy of the estimation.
One of the practical issues for improving the quality of training data is not to increase the
burden in the workplace for collecting high-quality training data. In addition, to estimate
S* involving no equipment controls, it is crucial to obtain the workers’ status because
PLC data is not available. Therefore, positional data is expected to play an important
role in identifying the status of workers.

3.3 Indoor Positioning

The integrated positioning method used in this case study consists of BLE (Bluetooth Low Energy) positioning, PDR (Pedestrian Dead Reckoning), and map matching [8]. Since the data for international indoor positioning competitions [10, 11] were collected at the same workplace as this case study, here we use the competition data for performance evaluation. As a result, CE50, a representative accuracy indicator, was 4.39 m (Reference: CE50 in AIST's method [12, 13] was 2.73 m).

At the target workplace, solar BLE beacons (Fujitsu PulsarGum) were installed for the maintenance-free advantage. The shortest advertisement transmission interval is 1.28 s, however, depending on the amount of electricity generated by solar power, the transmission interval may be longer. It may cause passing it unnoticed. Compared to warehouses where visual work is often required everywhere, in manufacturing sites, there are many areas where the luminance level is not sufficient such as aisles and other areas where visual work is not required. In such environments, the advertisement transmission interval is getting longer. Therefore, it would be possible to improve the positioning accuracy and the quality of features related to setup works by installing battery-powered beacons, which have shorter and more constant transmission intervals than solar beacons, in critical areas where PLC data is not available for work classification.

4 Work Analysis on Manufacturing Line Axis

We conducted work analysis using the quasi-GT data and GT data instead of using the
results of the work classification estimation by machine learning since it was still in
the feasibility study phase during this case study. This section reports on the quasi-GT
data-based analysis on the ML axis.

Figure 4-left shows a state transition diagram of the work classification, that is, the work process model. The transitions from OP clearly identify that the major stoppage factor is MR and that the main setup workflow after OP is “SC to {SA or SD} to SE.” Since
it is possible to perform simulations based on such models as in Ref. [14], preliminary evaluations with the model are expected for efficient ‘Kaizen’ of these MLs and the similar MLs in the future.

There are 3,230 OPs in the quasi-GT data with 406 operation cycle. Note that we define one operation cycle as a cycle consisting of (1) setting a full WIP container on the input side to the ML, (2) running the ML, (3) consuming the WIP, and (4) removing the empty WIP container. As shown in Fig. 4-right, there was a weak positive correlation between the operation rate and the continuous operation time (operation time per run) within each cycle. These results implies that the longer the continuous operation time is, the higher the operation rate is.

As each ML requires a brief stop for MR, we also analyzed MR and the approaching actions to the equipment for MR. Comparing the HS records with the quasi-GT data confirmed that 34% of time for OP in the HS records were MR (Fig. 6-middle-right). It was also found that the approaching time obtained by the MR work detection process (Fig. 3-left) accounted for 36% of the MR time. Based on these results, if all approaching times that take longer than the median were replaced by the median, the operation rate could be improved by 3%.

We also compared differences in operation rates depending on whether MR was passive or active. A passive cycle is one in which all approaching actions are carried out after the material runs out, and an active cycle is one that includes at least one MR started before the material runs out. The analysis confirmed that the median operation rates for passive and active cycles are 47% and 49%, respectively, and that the variation in operation rates is greater for passive cycles (Fig. 6-top-right).

<figure><img src="image_6.png" /><figcaption>Fig. 4. Quasi-GT data-based transition diagram on the ML axis (left), Operation rate and continuous operation time within each cycle (right).</figcaption></figure>

# 5 Work Analysis on Manufacturing-Line and Worker Axes

Obtaining the correspondence matrix between each work classification on the ML and worker axes from the GT data, we confirmed that “WIP output adjustment (WA)” (adjustment of the position of WIP output in the container) occupies about half of the time in OP, the ratio of “manual work record (MWR)” in OP to MWR except in OP is 2:1, and about half of “cleaning and garbage disposal (CL)” is done during internal setup.
The percentage of time for WA in OP averaged 47%, the lowest 35%, and the highest 60% (Fig. 5). For example, if the ratio is kept not to exceed 40%, the reduction in WA is equivalent to 9% of time for OP. By allocating the reduction in OP to other works (such as HR, CL, advance execution of internal setup, etc.), downtime is expected to be reduced. Improving work skills contributes to the reduction of time for WA, and it is expected to lead to higher operation rates and more rewarding work. In addition, although automation of WA is not currently underway, the effect is not small.

If MWR and CL were performed in OP in Shifts #1, #2, #3, and #5 as the same rate as the high-skilled worker in Shift #4, the average time for the increase of MWR and CL would be equivalent to 8% of time for OP. Although the 9% estimated above for WA reduction in OP is only an average, the increase of MWR and CL is expected to be within the range of the WA reduction.

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** WA (WIP output adjustment) and MWR (manual work record) in OP (left), WA and CL (cleaning and garbage disposal) in OP (right).</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Candidate metrics related to the operation rate and work skills.</figcaption></figure>

# 6 Conclusion

This paper reported on the case study of work analysis using GSI technologies on MLs at the JPS plant. First, regarding the improvement of the operation rate, we were able to
extract candidates of metrics related to the operation rate and work skills as shown in Fig. 6. Next, regarding the automation of MWR, it was shown that although sufficient accuracy was obtained in estimating OP and MR, there is room for improvement in the accuracy of estimating S*. In particular, it was found that improving the accuracy of indoor positioning is required to obtain the workers' status in the actual workplace.

The feasibility of visualizing work transitions was demonstrated by generating the work process model (Fig. 4-left) using the quasi-GT data. In addition, it was confirmed that the “active MR metrics” and replacement of WA in Figs. 5 and 6 are deeply related to work skills which contribute to QoW.

## References

1. Goetzel, R.Z., et al.: Health and productivity management: establishing key performance measures, benchmarks, and best practices. J. Occup. Environ. Med. 43(1), 10–17 (2001)

2. Takahashi, H.: Association of organizational factors with knowledge of effectiveness indicators and participation in corporate health and productivity management programs. J. Occup. Health 63(1), e12205 (2021). https://doi.org/10.1002/1348-9585.12205

3. Mori, K., et al.: Development, success factors, and challenges of government-led health and productivity management initiatives in Japan. J. Occupa. Environ. Med. 63(1), 18–26 (2021). https://doi.org/10.1097/JOM.0000000000002002

4. Hahmann, S., Burghardt, D.: How much information is geospatially referenced? networks and cognition. Int. J. Geogr. Inf. Sci. 27(6), 1171–1189 (2013). https://doi.org/10.1080/13658816.2012.743664

5. Klepeis, N.E., et al.: The National Human Activity Pattern Survey (NHAPS): A Resource for Assessing Exposure to Environmental Pollutants, Lawrence Berkeley National Laboratory (2001)

6. Kurata, T.: Geospatial intelligence for health and productivity management in Japanese restaurants and other industries. In: Dolgui, A., Bernard, A., Lemoine, D., von Cieminski, G., Romero, D. (eds.) APMS 2021. IAICT, vol. 632, pp. 206–214. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85906-0_23

7. Kurata, T., et al.: IoH technologies into indoor manufacturing sites. In: Ameri, F., Stecke, K.E., von Cieminski, G., Kiritsis, D. (eds.) APMS 2019. IAICT, vol. 567, pp. 372–380. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29996-5_43

8. Maehata, T., et al.: Sensor data fusion for indoor positioning system. IEICE Tech. Rep 120(204), RCS2020–106, pp.75–79 (2020)

9. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

10. xDR Challenge in industrial Scenarios 2019 (2019). https://unit.aist.go.jp/harc/xDR-Challenge-2019/, Accessed 24 Mar 2022

11. Potorti, F., et al.: Off-line evaluation of indoor positioning systems in different scenarios: The experiences from IPIN 2020 competition. IEEE Sens. J. (2021). https://doi.org/10.1109/ISMN.2021.3083149

12. Kourogi, M., Kurata, T.: Personal positioning based on walking locomotion analysis with self-contained sensors and a wearable camera. In: Proceedings of ISMAR, pp. 103–112 (2003). https://doi.org/10.1109/ISMAR.2003.1240693

13. Ichikari, R., et al., Complementary integration of PDR with absolute positioning methods based on time-series consistency. In: Proceedings of IPIN, Work-in-Progress, pp. 4–7 (2016)

14. Myokan, T., et al.: Pre-evaluation of Kaizen plan considering efficiency and employee satisfaction by simulation using data assimilation toward constructing Kaizen support framework. In: Proceedings of ICServ, p. 7 (2016)