Assembly Issue Resolution System Using Machine
Learning in Aero Engine Manufacturing

Jörg Brünnhäuser, Sonika Gogineni, Jonas Nickel, Heiko Witte, Rainer Stark

► To cite this version:

Jörg Brünnhäuser, Sonika Gogineni, Jonas Nickel, Heiko Witte, Rainer Stark. Assembly Issue Resolution System Using Machine Learning in Aero Engine Manufacturing. IFIP International Conference on Advances in Production Management Systems (APMS), Aug 2020, Novi Sad, Serbia. pp.149-157, 10.1007/978-3-030-57993-7_18 . hal-03630891

HAL Id: hal-03630891

https://inria.hal.science/hal-03630891

Submitted on 5 Apr 2022

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Assembly issue resolution system using machine learning
in aero engine manufacturing

Jörg Brünnhäußer¹, Sonika Gogineni¹, Jonas Nickel²,
Heiko Witte² and Prof. Rainer Stark¹,³

¹ Fraunhofer IPK, Pascalstraße 8-9, 10587 Berlin, Germany

² Rolls-Royce Deutschland, Eschenweg 11, 15827 Blankenfelde-Mahlow, Germany

³ Technische Universität Berlin, Pascalstraße 8-9, 10587 Berlin, Germany
joerg.bruennhaeusser@ipk.fraunhofer.de

**Abstract.** Companies are progressively gathering data within the digitalization of production processes. By actively using these production data sets operational processes can be improved, hence empowering businesses to compete with other corporations. One way to achieve this is to use data from production processes to develop and offer smart services that enable companies to continuously improve and to become more efficient. In this paper, the authors present an industrial use case of how machine learning can be implemented into smart services in production processes to decrease the time for resolving upcoming issues in manufacturing. The implementation is carried out by using an assistance system that aids a team which attends to problems in the assembling of turbines. Therefore, the authors have analyzed the assembly problems from an issue management system that the team had to resolve. Subsequently three different approaches based upon natural language processing, regression and clustering were selected. This paper also presents the development and evaluation of the assistance system.

**Keywords:** Machine learning, data-driven service, issue resolution.

# 1 Introduction

An increasing amount of data is stored within enterprise systems and databases of many companies. By introducing Internet of Things (IoT) concepts more and more data throughout the whole product lifecycle are made available every second. This includes the production process where due to the advancing digitalization almost every step within that process leaves data behind. A key aspect for many manufacturing companies is to take advantage of these data to increase competitiveness through machine learning (ML), Smart Services and Data-Driven Applications [1].

The application of ML on existing data is one approach to improve manufacturing processes. This paper shows on an example how one can set up a Smart Service with Machine Learning based on already available data and put it into practice. In the best case it can save time and increase the efficiency, hence saving costs. This is also the goal of the COCKPIT 4.0 project (funded by the Berlin Senate, the State of Branden-
burg and the European Regional Development Fund – ERDF 1.8/03) in which a con-
sortium of Rolls-Royce Germany, Fraunhofer IPK and the BTU Cottbus are working
together to address different challenges in the context of digitalization. This paper is
about one work package of the COCKPIT 4.0 project which aims to improve the effi-
ciency of the gas turbine assembly at the Rolls-Royce site in Dahlewitz, Germany. The
goal of the work package is to connect various data sources with semantic technologies
[2] and support the resulting semantic network with machine leaning techniques to as-
sist in solving production issues efficiently. The learned models enable the creation of
an assembly issue resolution system which will aid the fitters to fix manufacturing prob-
lems faster. The resulting assistance system improves problem identification, resolution
and prevention by the fitters, shop floor management and assembly manufacturing en-
gineering.

## 2 Problem description

Modern aerospace products are not only highly complex but are also supplied by an
equally complex and globally distributed supply chain. While this approach enables
both a cost-effective and flexible product development and production, it also increases
the risk of supply chain disruptions and part supply issues for the engine assembly pro-
cess [3]. Even though many of these risks can be managed upstream of the final engine
assembly, it is necessary to quickly react to missing parts and other assembly issues in
the build line in order to avoid build stops and delivery delays.

Rolls-Royce Germany has established a small, dedicated “Voice of the Fitter” (VoF) team that focuses on resolving any assembly issues as quickly as possible. As the issues might arise from a number of sources, e.g. supplier production issues, delays in logistics or customs, transport or handling damage, the team needs to access seven data sources and uses different strategies to resolve the issues. Hence, a lot of time is spent on identifying the right data source, accessing the data source, finding the right information, assessing the data, taking a decision and implementing suitable action. In such cases, contextualized data sets assist the users in addressing some of the above tasks such as identifying, accessing, finding and assessing the right data [2]. In addition, ML can be used in enhancing those tasks and also in supporting decision making. This results in saving time for the VoF team.

In this paper, the creation of a smart, ML enabled assistance system that connects all
data sources in one location and provides context-related information and resolution
strategies to the VoF team is developed. The resulting system can subsequently be ex-
panded to support related tasks in the company, e.g. concession assessments for non-
conforming parts.

## 3 State of the art

ML is an emerging technology which enables powerful applications and handling of
complex data. Production systems are providing a lot of use cases for ML. Wuest et al.
[4] outlined the reasons for the increasing importance of ML and mentioned the challenges as well. However, some advantages mentioned are the capability “to handle high-dimensional, multi-variate data, and the ability to extract implicit relationships within large data-sets in a complex and dynamic, often even chaotic environment” [4]. There are already an abundance of applications such as the estimation of the manufacturing costs of jet engine components [5], the prediction of basic parameters for turbomachinery design based on the load level [6] or approaches to do predictive maintenance [7]. There are also several papers for different systems which support production tasks. In this work [8] an assistance system for operational workers is created which uses simulation and Proper Generalized Decomposition algorithms to provide decision support. Additionally another decision support system [9] uses Case Based Reasoning for scheduling tasks. However, there seems to be no suitable approach which addresses the resolution of single and diverse assembly problem in combination with ML.

## 4 Solution definition

The VoF use case was one of several use cases collected in the business understanding phase of the Cockpit 4.0 project. The use case was further detailed by interviews with stakeholders involved in VoF. Opportunities for machine learning were identified as part of the requirements capturing during the interviews. Followed by the next phase of the CRISP-DM [10] process the actual data was fetched and reviewed. The source data for this use case consists of two tables.

<table><caption>**Table 1:** Excerpt of VoF table with an exemplary problem case</caption><thead><tr><th>vof_case_desc<br>id</th><th>vof_case_desc<br>ruption</th><th>vof_case<br>state</th><th>vof_o-<br>pen_date</th><th>vof_close<br>_date</th><th>vof_custo<br>mer</th><th>vof_cat-<br>gory</th><th>vof_priority</th></tr></thead><tbody><tr><td>41</td><td>LAUNCH STOP<br>due to short-<br>age blades</td><td>Closed</td><td>03.04.2017<br>07:42</td><td>10.04.2017<br>12:25</td><td>x0036</td><td>Delivery</td><td>Build Stop</td></tr></tbody></table>

<table><caption>**Table 2:** Excerpt of VoF comments table with comments from VoF team</caption><thead><tr><th>vof_en-<br>gine_type</th><th>vof_part_num-<br>ber</th><th>vof prob-<br>lem_ident</th><th>vof_part_descrip-<br>tion</th><th>vof_organisa-<br>tion</th><th>vof_respon-<br>sible_id</th></tr></thead><tbody><tr><td>BR725</td><td>FW85439</td><td>Missing com-<br>ponent</td><td>BLADE,HPT 2</td><td>Production</td><td>83</td></tr></tbody></table>

<table><thead><tr><th>vof_case_id</th><th>vof_comment</th><th>vof_com-<br>ment_up-<br>date_time</th><th>vof_com-<br>ment_up-<br>date_user</th><th>vof_com-<br>ment_type</th><th>id</th></tr></thead><tbody><tr><td>1041</td><td>11off in transit, expected<br>on dock today 05/04<br>around noontime</td><td>05.04.2017<br>11:23</td><td>x0030</td><td>Comment</td><td>41</td></tr></tbody></table>

One table stores the recent VoF cases (see **Table 1**). The second table consists of comments (see **Table 2**) which are linked to the first table. One case can have zero to
many comments. These comments describe different things like if a part needs to be reworked, if a part which was missing is ordered or is stuck in delivery. Furthermore, it is noted in the comment table if a VoF case is escalated to a higher level of management. In order to aid the Voice of the Fitter team the authors considered three approaches based on the available data as useful:

1. **Natural language processing (NLP) based approach:** The engineering goal was to identify cases with similar descriptions and similar comments, which might be useful for investigating new VoF case. The available data is not labeled which prevents supervised learning. Therefore, NLP could be applied to show similar cases.

2. **Case length prediction:** The goal here was to generate a model that could predict how long an incoming case will take to be resolved. This information could be helpful to judge the priority. For instance, if a single case would take very long, measures could be taken to reduce the time span like escalating the case or getting support from other departments. The start and end date of each case is available so the duration of every case can be calculated and supervised learning can be applied.

3. **Clustering:** Unsupervised learning could be used to separate the data into clusters. Those clusters could provide the user with 'similar' cases which can be taken as starting point for their research on how to resolve the current issue

# 5 Design and development of the prototype

The approaches discussed in the previous sectors are further detailed in this section. This is done by explaining the data science aspects of the approaches followed by the implementation in the prototypes architecture.

## 5.1 Data science

**NLP based approach.** To select an optimal algorithm to find similar cases, three different algorithms were tested. The first one chosen was the commonly used Jaccard coefficient [11], which is an index for the similarity of two sets by the technique known as shingling [12]. This approach does not take into account the meaning of the words. The second approach was based on word2vec [13]. Word2vec is trained [14] on a large data set like the Google News corpus and calculates vector representations for each word. The vectors for semantic similar words should also be similar. In contrast the self-trained model was trained with a data set of the VoF case descriptions in order to cast the engineering vocabulary and abbreviations into the model. The calculation of the distance was done by the word mover's distance [15].

All models were tested with the same data set and subsequently some examples have been evaluated by an expert. The results provide information about the performance of the approaches to find a similar VoF cases. The evaluation revealed that the self-trained model performed best. This seems reasonable because the data set contains many individual words that are not in common usage. Jaccard delivered the second best result, so the pre-trained model from Google was the least accurate. However, this evaluation was just a first test and further research needs to be conducted.
**Case length prediction.** Initially the label ‘duration in hours’ was calculated from the start and the end date to be able to train a regression model. After the data preparation of the two tables mentioned in the previous chapter the data was split into a training and a test data set. The evaluation makes it necessary to test the models with unknown data which was not involved in the training. For this purpose the test data set was used. The training data set was used for the training of several different models like linear regression, lasso regression and support vector regression which was taken from the scikitlearn library [16]. Additionally, a model tree (a decision tree with different models on each leaf [17]) was trained. The best model type for the leaf models have been estimated with grid search.

<table><caption>**Table 3:** Different trained models are compared regarding the root-mean-squared error</caption><thead><tr><td>Model type</td><td>RMSE [in hours]</td></tr></thead><tbody><tr><td>Linear Regression</td><td>46.02</td></tr><tr><td>Lasso Regression</td><td>45.83</td></tr><tr><td>Support Vector Regression (SVR)</td><td>45.15</td></tr><tr><td>Model Tree with (SVR)</td><td>43.54</td></tr></tbody></table>

For the evaluation the model made predictions of the VoF case length based on the test data set as input. Those predictions can be compared to the actual values of the duration and the root-mean-squared error can be calculated which gives a metric to evaluate the models (Table 3). In this case the Model Tree approach (with SVR models on each leaf) scored best.

**Clustering.** The authors compared several combinations of feature reduction methods and clustering algorithms. A subset of six configurations as shown in Table 4 was chosen from the best performing algorithms based on several scores. The subset includes centroid based k-means clustering and a density based DBSCAN clustering. Additionally HDBSCAN was tested as proposed by [18], a combined approach between hierarchical and density based clustering.

The best results were reached with a k-means clustering. The best performing dimensionality reduction method was UMAP which is used for non-linear dimensionality reduction introduced by McInnes et al. [19]. The Silhouette score was proposed in [20]. It measures the separateness and tightness of the clusters found by a clustering and can take values between -1 and 1, whereby a value close to -1 implies that the points are misclassified as, the distance towards the centers of the clusters are larger than the distances to the center of the closest neighboring cluster. A score around 1 implies that the result is very well-classified. The Calinski-Harabasz score was proposed in [21]. It minimizes within-cluster distances and maximizes between-cluster distances, therefore, compact, well-separated clusters get a better score. The larger the Calinski-Harabasz score of a clustering is, the better is the clustering [21]. The Davies-Bouldin score was presented in [22], unlike the other scores it is not maximized, but minimized. It calculates the average similarity of the rated clusters with their most similar cluster [22].

For the application the Kmeans1-clustering was chosen based on the performance metrics introduced above and the intended use. Kmeans2 had the best Silhouette score and Davies-Bouldin score, but the three resulting clusters were not practical for the
considered use case. The performance of Hdbscan1, Hdbscan2, Hdbscan3 and Dbscan was worse than that of Kmeans1. After examining whether the contents of the clusters produced by Kmeans1 are sound and realistic it was chosen as the final clustering approach. Furthermore, seven clusters were indeed the number expected by experts. For UMAP the implementation by [23] and for HDBSCAN the one by [24] was used. All other implementations were based on [16].

<table><caption>**Table 4:** Table shows the evaluation metrics of the different clustering. One of the HDBSCAN cluster always consist out of outliers.</caption><thead><tr><th></th><th>Clustering</th><th>Dimension Reduction</th><th>Silhouette</th><th>Calinski-Harabasz</th><th>Distances</th><th>Number of Clusters</th></tr></thead><tbody><tr><td>Hdbscan1</td><td>HDBSCAN</td><td>UMAP</td><td>0.6</td><td>3472</td><td>1.24</td><td>7</td></tr><tr><td>Hdbscan2</td><td>HDBSCAN</td><td>UMAP</td><td>0.55</td><td>3907</td><td>1.02</td><td>11</td></tr><tr><td>Hdbscan3</td><td>HDBSCAN</td><td>UMAP</td><td>0.62</td><td>9937</td><td>1.02</td><td>7</td></tr><tr><td>Dbscan</td><td>DBSCAN</td><td>UMAP</td><td>0.62</td><td>4308</td><td>0.36</td><td>8</td></tr><tr><td>Kmeans1</td><td>K-Means</td><td>UMAP</td><td>0.64</td><td>13732</td><td>0.39</td><td>7</td></tr><tr><td>Kmeans2</td><td>K-Means</td><td>UMAP</td><td>0.7</td><td>9542</td><td>0.33</td><td>3</td></tr></tbody></table>

## 5.2 Implementation of models into a Prototype

The prototype for visualizing the ML results to the VoF team was designed based on a microservice architecture. The advantage is the separation of every functionality into a single, atomic web service. The technology stack consists of the Java Spring Boot framework for the backend and vue.js for the frontend. The gateway-service hosts the frontend and serves as controller between the user and the different services. These services take care of the database access or the filtering of the data.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Screenshot of the current version of the prototype. 1./3. shows the position of the NLP and the clustering. The duration prediction is shown on position 2.</figcaption></figure>
For the voice of the fitter use case the user can see open and closed cases on a 3D model of an engine (see Fig. 1), furthermore, filter options are available in order to handle a larger set of cases. If a case is selected a box pops up with information about the case. If it is an open case the first ML model comes into play which predicts the duration. This gives the user an estimate of how long such a case usually takes, and therefore how critical this case might get. The clustering appears on the right hand side after a click on the “Clustering” button. The individual clusters consist of at least 100 cases. In order to show the user the most suitable case where the NLP approach is used. The cases from the same cluster are sorted by the textual similarity of their case description.

# 6 Conclusion and outlook

This paper briefly summarizes the approach and a use case to use ML and generate smart services, which helps reduce time of issue resolution in manufacturing. The significance of this work is also to show how to improve production processes with Machine Learning based on existing data in practice. VoF team and their assembly issue resolution was selected as the initial use case to be enhanced with a smart service and machine learning. Three different approaches of the application of ML were investigated: a NLP approach for text similarity, a prediction of the case duration with regression and a clustering of all the cases. This was further integrated into a prototype system to assist the team. The ML models and the prototype were presented and discussed with the experts from Rolls-Royce Germany. This prototype can be used as an example of how existing data can be turned into a smart service which can improve certain processes. The results show the potential for time savings in solving assembly problems. However, the exact time savings still need to be investigated in further work. In addition more data from enterprise systems and databases will be added to the prototype which will allow new approaches for ML. Finally the usability and the usefulness of the prototype and the ML models will be evaluated to assess how those models are assisting in resolving assembly issues.

# References

1. Stich, V. et al.: Digitale Dienstleistungsinnovationen. Smart Services agil und kundenorientiert entwickeln (2019).

2. Gogineni, S., Exner, K., Stark, R., Nickel, J., Oeler, M., Witte, H.: Semantic Assistance System for Providing Smart Services and Reasoning in Aero-Engine Manufacturing. In: Emmanuel Garoufallou, Francesca Fallucchi und Ernesto William De Luca (Hg.): Metadata and Semantic Research, Bd. 1057. Cham: Springer International Publishing (Communications in Computer and Information Science), S. 90–102 (2019).

3. Treuner, F., Hübner, D., Baur, S., Wagner, S. M.: A survey of Disruptions in Aviation and Aerospace supply chains and Recommendations for Increasing Resilience. Supply Chain Management, 14(3), 7-12 (2014).

4. Wuest, T., Weimer, D., Irgens, C., Thoben, K.-D.: Machine learning in manufacturing: advantages, challenges, and applications. In: Production & Manufacturing Research 4 (1), S. 23–45 (2016).
5. Loyer, J.-L., Henriques, E., Fontul, M., Wiseall, S.: Comparison of Machine Learning methods applied to the estimation of manufacturing cost of jet engine components. In: International Journal of Production Economics 178, S. 109–119 (2016).

6. Azzam, M., Haag, J.-C., Jeschcke, P.: Application concept of artificial neural networks for turbomachinery design. Computer Assisted Methods in Engineering and Science, [S.l.], v. 16, n. 2, p. 143-160 (2017).

7. Susto, G. A., Schirru, A., Pampuri, S., McLoone, S., Beghi, A.: Machine Learning for Predictive Maintenance: A Multiple Classifier Approach. In: IEEE Trans. Ind. Inf. 11 (3), S. 812–820 (2015).

8. Belkadi, F., Dhuieb, M.A., Aguado, J.V., Laroche, F., Bernard, A., Chinesta, F.: Intelligent assistant system as a context-aware decision-making support for the workers of the future. Computers & Industrial Engineering 139:105732 (2020).

9. Kocsis, T., Negny, S., Floquet, P., Meyer, X., Rév, E.: Case-Based Reasoning system for mathematical modelling options and resolution methods for production scheduling problems: Case representation, acquisition and retrieval. Computers & Industrial Engineering 77:46–64 (2014).

10. Wirth, R., Jochen H.: Crisp-dm: towards a standard process modell for data mining (2000).

11. Niwattanakul, S., Singthongchai, J., Naenudorn, E., Wanapu, S.: Using of Jaccard Coefficient for Keywords Similarity (2013).

12. Manna, M., Abdulameer, G.: Web Documents Similarity Using K-Shingle Tokens and Min-Hash Technique. In: Journal of Engineering and Applied Sciences 13, S. 1499–1505. (2018).

13. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed Representations of Words and Phrases and their Compositionality, 16.10.2013. (2013).

14. Somani, A. K., Shekhawat, R. S., Mundra, A., Srivastava, S., Verma, V. K. (Hg.): Smart systems and IoT: innovations in computing. Proceeding of SSIC 2019. Singapore: Springer (Smart innovation, systems and technologies, volume 141) (2020).

15. Kusner, M. J., Sun, Y., Kolkin, N. I., Weinberger, K. Q.: From Word Embeddings to Document Distances. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37: JMLR.org (ICML’15), S. 957–966 (2015).

16. Buitinck et al.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp. 108-122 (2013)

17. Github Model Tree project, https://github.com/ankonzoid/LearningX/tree/master/advanced_ML/model_tree, last accessed 2020/02/21

18. Campello, R. J. G. B., Moulavi, D., Sander, J.: Density-Based Clustering Based on Hierarchical Density Estimates. In: Advances in Knowledge Discovery and Data Mining, vol. 7819. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 160–172 (2013).

19. McInnes, L., Healy, J., Melville, J.: UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. (2018).

20. Rousseeuw, P. J.: Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. In: Journal of Computational and Applied Mathematics 20, pp. 53–65. (1987).

21. Calinski, T., Harabasz, J.: A dendrite method for cluster analysis. In: Comm. in Stats. - Theory & Methods 3 (1), pp. 1–27 (1974).

22. Davies, D. L., Bouldin, D. W.: A Cluster Separation Measure. In: IEEE Trans. Pattern Anal. Mach. Intell. PAMI-1 (2), pp. 224–227. (1979).

23. McInnes, L., Healy, J., Saul, N., Großberger, L.: UMAP: Uniform Manifold Approximation and Projection. In: JOSS 3 (29), p. 861 (2018b).

24. McInnes, L., Healy, J., Astels, S.: hdbscan: Hierarchical density based clustering. In: JOSS 2 (11), p. 205 (2017).