# Cross Industrial PLM Benchmarking Using Maturity Models

Philipp Steck, Felix Nyffenegger, Helen Vogt, Roman Hänggi

► To cite this version:

Philipp Steck, Felix Nyffenegger, Helen Vogt, Roman Hänggi. Cross Industrial PLM Benchmarking Using Maturity Models. 17th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2020, Rapperswil, Switzerland. pp.538-552, 10.1007/978-3-030-62807-9_43. hal-03753136

HAL Id: hal-03753136

https://inria.hal.science/hal-03753136v1

Submitted on 17 Aug 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Cross Industrial PLM Benchmarking Using Maturity Models

Philipp Steck<fnref n="1" />*, Felix Nyffenegger<fnref n="2" />, Helen Vogt<fnref n="3" />, Roman Hänggi<fnref n="4" />

Fachhochschule OST, Rapperswil, Switzerland

Zürcher Hochschule für Angewandte Wissenschaften (ZHAW), Winterthur, Switzerland

**Abstract.** Maturity models are commonly used to assess a company's position on a roadmap towards a defined mature PLM environment. The models focus mainly on an internal point of view, comparing a current state to a possible future state. The high level of adjustments in these models to the individual companies needed, make it difficult to compare companies across industries using existing maturity models.

The aim of this paper is to introduce a generic, cross-industrial maturity model suitable for the benchmarking of companies. The model uses an ability-based approach, similar to a case form report.

Using the model, a first benchmarking study has been conducted among ten Swiss companies. This initial study allowed to verify and discuss the suitability of the developed model. Furthermore, the actual results of benchmarking lead to interesting insights into potential success factors to achieve higher PLM maturity. This paper discusses both the maturity model and the actual results.

**Keywords:** PLM, Maturity Model, Benchmarking.

## 1 Introduction

Product lifecycle management (PLM) is a widely accepted practice in today's companies. In many places it has become a strategic initiative to support the overall goal of an enterprise. However, due to its complexity, it is hard to clearly explain the impact and value created by the implementation of PLM. Accordingly, arguing for investment into new PLM initiatives on top management level can be hard. Maturity models help to assess a company's current state and are able to show the potential next steps towards a higher maturity level. However, this is not enough to get a clear view on the added value to the company by this next maturity level. To better qualify such an investment, it would be interesting to do an industry wide comparison of the impact of maturity levels on a company's performance. Since companies in a particular economic system differentiate in very dimension (product, organization, business model, tools, ...) the

<footnote n="1">¹philipp.steck@ost.ch*</footnote>

<footnote n="2">²felix.nyffenegger@ost.ch</footnote>

<footnote n="4">⁴roman.haenggi@ost.ch</footnote>

<footnote n="3">³vogh@zhaw.ch</footnote>
only way to achieve this is cross industrial benchmark. The initial question of the pre-
sented work was to evaluate weather maturity models can help to do a cross industrial
benchmark and extract common success factors of PLM. As a result of this analysis a
new assessment model was created and tested.

## 2 Related Works

The main goal of a PLM maturity assessment tool is to improve the PLM implementa-
tion process. This poses a great challenge for many companies [1]. PLM maturity mod-
els have been developed and used to assess the PLM implementation situations and
determine the relative position of the enterprise by comparing PLM maturity levels with
other enterprises [2]. Various frameworks for PLM maturity models have been de-
scribed in literature [3] and been benchmarked [4]. As Vezzetti has shown, several im-
portant maturity models are worth analyzing: The Capability Maturity Model Integra-
tion (CMMI) is widely recognized within the PLM community [5] and has developed
into an established model in the field of information systems development [6]. It is
composed of five maturity levels: Initial, Repeatable, Defined, Managed and Opti-
mized. Batenburg's model [7] focusses on the assessment of PLM implementations.
The model applies four maturity levels: ad-hoc, departmental, organizational and in-
terorganizational. Sääksvuori [8] determines the maturity of a large international cor-
poration for a corporate-wide PLM development program and describes business and
PLM related issues on the product lifecycle management. The origin of the model lies
in the idea of phases or stages, which a company usually goes through as it adapts to
new cultural issues, processes, management practices, business concepts and modes of
operation [4]. In contrast to the previous models, which mainly focus on internal com-
pany processes, Kärkkäinnen [9] proposes a model that focuses on the customer aspects
of PLM maturity. The authors distinguish between the following main levels, namely
Chaotic, Conscientious, Managed, Advanced and Integration stages, and use elements
such as level of proactivity, extent of coordination, extent of integration and quality and
type of customer knowledge to characterize and measure these levels.

Most of these models have a clear scope on manufacturing companies and mainly focus
on an as-is or a future state analysis. Hence, the models have primarily an intra-com-
pany view. They do not aim at comparing or benchmarking different companies. Fur-
thermore, most models empathize the organizational perspective rather than a technical
or functional point of view. The study carried out by Kärkkäinnen and Silventoinen
analysed a broad spectrum of different models. Base on their analysis, it can be said
that most maturity models do not have an indepth description of quantifiable factors
that differentiate the different levels [10]. Finally, the analyzed models evaluate ma-
turity using open-ended questions or ask the participants to rank their own maturity
based on a description of functionalities or of a use-case. The approach of using closed
yes or no questions to verify whether a PLM system can fulfill a functionality is a more
objective way of evaluating. It does not depend as much on the interviewee's personal
opinion and in this way overcomes the typical limitations of a scale based questioning
survey as described by Franzen [11].
The proposed model differs from the commonly used ones in the following points:

* Maturity is defined in this study as the capability of a company's PLM environment to fulfill a certain task. This allows a cross-industrial comparison of capabilities. Whether or not a certain capability is beneficial for a company depends on the market context.

* Main target of this model is the comparison of different companies for benchmarking purposes. Hereby the model helps companies to assess their level of maturity and compare themselves across industries. The models described in Chapter 2, on the other hand, have a mainly internal perspective and generally do not aim to compare companies.

* Industrial focus: The proposed model is applicable across industries. The models analysed all clearly focus on a limited number of industrial sectors.

* Survey format: The proposed closed yes or no questions focusing on PLM abilities and enable the exact analysis of the available functionalities within a PLM environment. The analysed models use scale based ratings and ask the companies to rank their abilities in different areas.

This makes the proposed model and its application new and unique. It could help to quantify the influential factors for a successful PLM environment.

## 3 Method

To develop a cross-industry PLM maturity model, which is suitable for the benchmarking of companies, the methodology as proposed in the SPICE model has been used. SPICE is an international framework for assessment of software processes developed jointly by the ISO and the International Electrotechnical Commission (IEC). According to the literature search of Wangenheim et. al it is also one of the most commonly used approaches to develop maturity models for software products [12]. It describes the pre-conditions needed to conduct process analyses [13]. Additionally, it has been adapted in other industries (construction) and for other applications (earnings) [14][15]. The model was developed and validated along the process shown in figure Fig. 1.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Chosen methodology for establishing the new PLM maturity model</figcaption></figure>
## 3.1 Maturity Model Development

Maturity models can be characterized by the number of dimensions (such as the 'process areas' in CMM), the number of levels, a descriptor for each level (such as the CMM's differentiation between initial, repeatable, defined, managed, and optimizing processes), a generic description or summary of the characteristics of each level as a whole, a number of elements or activities for each dimension, and a description of each element or activity as it might be performed at each level of maturity [16]. The introduced model distinguishes five levels of maturity (Initial, Low, Intermediate, Mature and Best Practice) as shown in Table 1. The five-level approach has been successfully used in similar application by Hchicha et al [17]. Further is it also a recommended approach to assess processes on the base of ISO/IEC15504 [13]. For each of the five levels, Table 2 provides a brief description of the expected traits and abilities. The description is based on an interview with a PLM expert and on a literature review [18].

<table><caption>Table 1. Description of traits for the different levels in the maturity model</caption><tr><td>Level 1</td><td>initial</td><td><ul><li>No data standards</li><li>Reactive approach</li><li>No master data plan</li><li>No clear strategy</li><li>Processes are not clearly defined</li><li>Processes are not transparent</li><li>Processes are not supervised</li><li>Process automation is not existing</li><li>All interfaces are handled manually</li></ul></td></tr><tr><td>Level 2</td><td>low</td><td><ul><li>Some processes are clearly defined</li><li>Processes are not transparent to all stakeholders</li><li>There is no systematic KPI tracking for key processes in place</li><li>Process automation is only implemented for a few processes/task</li><li>Processes are partially supervised</li><li>Some automated interfaces</li><li>No cross-supply-chain integration</li></ul></td></tr><tr><td>Level 3</td><td>intermediate</td><td><ul><li>Nominal data governance implemented</li><li>Some sort of MDM in place</li><li>Some KPI tracking regarding data quality in place</li><li>Most processes are clearly defined</li><li>Processes are not transparent to all stakeholders</li><li>There is no systematic KPI tracking for key processes in place</li><li>Processes are only partly supervised by tools</li><li>Multitool processes are not implemented</li><li>Automated interfaces widely used</li><li>Some cross-supply-chain integration</li></ul></td></tr><tr><td>Level 4</td><td>Mature</td><td><ul><li>MDM managing all companies meta data</li><li>data layers implemented</li><li>clearly defined data governance</li><li>Data models enable a smooth and quick exchange</li><li>All processes are clearly defined</li><li>Processes are transparent to operators and managers</li><li>There is systematic KPI tracking for key processes in place</li><li>Processes are supervised by tools</li><li>Multitool processes are implemented</li><li>Process engines are implemented for key processes</li><li>Automated interfaces widely used</li><li>Cross-supply-chain integration is standards</li></ul></td></tr></table>
<table><thead><tr><th>Level 5</th><th>Best-practice</th></tr></thead><tbody><tr><td></td><td><ul><li>Scalable and easy to adjust data models</li><li>Data simplifies processes and is highly adjustable to different tools</li><li>Data is stored in a central database</li><li>Data objects are minimized</li><li>All processes are clearly defined</li><li>Processes are transparent to all stakeholders</li><li>There is systematic KPI tracking for all processes in place</li><li>Processes are supervised by tools</li><li>Processes enable a seamless workflow among the enitre supply chain</li><li>Multitool processes are implemented</li><li>Process engines are implemented for all processes</li><li>Processes interact directly with customers and suppliers</li></ul></td></tr></tbody></table>

## 3.2 Development of the Survey

For the creation of the survey, the decision was taken to create closed questions in binary form (yes/no). This approach is commonly used in medical examinations where a patient's symptoms are evaluated using case report forms. These forms mostly use closed yes or no questions. This approach helps to objectively evaluate the pattern of a possible illness based on the patient's description [19]. Transferring this to the field of PLM maturity, a questionnaire with a total of 64 closed binary questions was created. The questionair was created using a five step approach.

1. Creation of a description of tasks for every phase of the lifecycle. Hereby the phases of beginning, mid and end of life defined by S. Terzi et al were utilized [20].

2. For every task in each phase a description of functionalities that support or enable the task was created.

3. The functionalities were transformed into closed yes or no questions.

4. For validation of the questions, the model was pretested with a selected partner company.

5. The questionnaire was reviewed by PLM experts (more than 15 years of experience). The goal of this step was to ensure that all questions are unique and do not correlate with each other.

Table 2 shows the complete questionnaire. On the left hand side the current life cycle phase of the product is shown.
<table><caption>Table 2. Questionaire and categoristion</caption><thead><tr><th>ID</th><th>Question</th><th>Lifecycle phase</th><th>Question</th><th>Lifecycle phase</th></tr></thead><tbody><tr><td>1</td><td>Requirements for new products are collected systematically and transferred to R&amp;D via a clearly defined process</td><td></td><td>Bill of material can be generated automatically using configurations</td><td></td></tr><tr><td>2</td><td>Requirements are recorded directly in a process engine.</td><td></td><td>On the base of bills of material the system can automatically generate parametrical CAD models.</td><td></td></tr><tr><td>3</td><td>Information about the current portfolio is provided systematically and can be sorted by properties.</td><td></td><td>There system enables the management of adjusted bills of materials for each end user according to the respective requirements.</td><td></td></tr><tr><td>4</td><td>Within the process, resources are allocated and the planning is supported by an automated tool</td><td></td><td>The bill of material for service purposes is generated automatically.</td><td></td></tr><tr><td>5</td><td>Findings from previous development projects are systematically incorporated into the definition of new development orders.</td><td></td><td>CAD models of assemblies are automatically checked for errors (e.g. collision check).</td><td></td></tr><tr><td>6</td><td>Currently running development projects are tracked within a tool that enables setting according to defined requirements.</td><td></td><td>The system checks the stored data for duplicates and completeness.</td><td></td></tr><tr><td>7</td><td>Initial concepts to fulfill the requirements can be clearly assigned to requirements within the system.</td><td></td><td>The design review is carried out in a system-led process.</td><td></td></tr><tr><td>8</td><td>The results of the product conception are systematically recorded and always according to the same procedure.</td><td></td><td>The system supports different roles and thus enables the systematic control of engineering data (example: developer and reviewer).</td><td></td></tr><tr><td>9</td><td>Concepts and initial drafts are handed over to the development teams via a clearly defined release process.</td><td></td><td>If errors are detected during the review, any comments can be entered directly into the system and can be clearly assigned to a responsible.</td><td></td></tr><tr><td>10</td><td>Documentation on fixed drafts can be clearly assigned to the final article in the system.</td><td></td><td>CAD data can be extracted directly from the system in a neutral CAD format.</td><td></td></tr><tr><td>11</td><td>The system allows the search for modules that fulfill similar functions and requirements.</td><td></td><td>If CAD data is shared with a stakeholder, this happens in a clearly structured form (in terms of content and following a process based on clear guidelines).</td><td></td></tr><tr><td>12</td><td>Article numbers are assigned automatically by the system.</td><td></td><td>In the event of a change, a is clear to the executing engineer whether the data has been shared externally or not.</td><td></td></tr><tr><td>13</td><td>Numbering sequences are clearly defined for standard and purchased parts.</td><td>Beginning of life</td><td>Data is always exchanged via secure databases in which access can be monitored.</td><td></td></tr><tr><td>14</td><td>The same article number always used for all processes throughout the whole life cycle.</td><td></td><td>All production relevant data for individual parts can be found in the CAD model.</td><td></td></tr><tr><td>15</td><td>While working in the CAD the engineer has access to information on tools / machines that are used in manufacturing.</td><td></td><td>Manufacturing information is transferred to the production sites in a defined form and according to a clearly defined process.</td><td></td></tr><tr><td>16</td><td>Models are enriched with a defined set of metadata.</td><td></td><td>Manufacturing information is released fully automatically for the defined manufacturing locations and is automatically available after release.</td><td></td></tr><tr><td>17</td><td>CAD models of individual parts are automatically checked for errors (e.g. incomplete definitions)</td><td></td><td>Production resources such as tools and machines are stored in the system and are accessible to the engineer during all development stages.</td><td></td></tr><tr><td>18</td><td>Faulty CAD models cannot be transferred to the next process instance.</td><td></td><td>CAD data is clearly to the actual CAD model and is considered a part of the change process.</td><td></td></tr><tr><td>19</td><td>CAD models are the single source of truth and contain all relevant information for the whole life cycle (example: surface condition, material data, tolerances, surface treatment, law material)</td><td></td><td>All manufacturing information regarding a product is stored in one consolidated database.</td><td></td></tr><tr><td>20</td><td>The system enables the search for parts using technical traits of components.</td><td></td><td>As built bills of materials are clearly assigned to the orders and can be called up at any time.</td><td></td></tr><tr><td>21</td><td>Within the system one can systematically search for possible, suitable standard parts.</td><td></td><td>Visualizations can be generated automatically using the existing product parameters.</td><td></td></tr><tr><td>22</td><td>While selecting suitable standard parts the engineer gets immediate information about cost and availability.</td><td></td><td>Change requests are systematically recorded and clearly assigned to the articles.</td><td></td></tr><tr><td>23</td><td>It is a new standard part would have to be introduced, the system allows a systematic check for possible alternatives.</td><td></td><td>Change requests get systematically reviewed and categorized with system support.</td><td></td></tr><tr><td>24</td><td>The system enables the systematic search for suitable purchased parts on the basis of technical parameters.</td><td></td><td>A part of a change order the required changes can highlighted directly in the CAD model.</td><td></td></tr><tr><td>25</td><td>On the base of selected technical attributes and description the system is able to propose standard suppliers for certain parts.</td><td></td><td>Change orders are clearly assigned to a responsible.</td><td></td></tr><tr><td>26</td><td>The creation of functional descriptions for tenders is supported by the system.</td><td></td><td>In case of an index change, an overview of all affected articles is automatically generated.</td><td></td></tr><tr><td>27</td><td>Simulation results and models are directly linked to all other product data.</td><td></td><td>Communication with stakeholders regarding changes is conducted through a clearly structured process.</td><td></td></tr><tr><td>28</td><td>Transfer of the data to any special departments (example: simulation) is carried out in a clearly structured process, following clear guidelines.</td><td></td><td>The system supports the developer in deciding whether a new part number is necessary or not.</td><td></td></tr><tr><td>29</td><td>Simulation results are sent back to the responsible development department via a clearly structured process.</td><td>Middle of life</td><td>If an article is discontinued, all data is automatically set to invalid.</td><td></td></tr><tr><td>30</td><td>Bills of material are treated in a clearly defined process. Following clear guidelines.</td><td></td><td>If a product is discontinued the system is able to deliver an overview of all interdependencies.</td><td></td></tr><tr><td>31</td><td>Bills of material are delivered directly from the CAD data.</td><td></td><td>The system supports communication on product.</td><td></td></tr><tr><td>32</td><td>Bills of material are transferred to ERP without manual intervention.</td><td></td><td>Any successor products can be recorded within the system.</td><td></td></tr></tbody></table>
The following data can be extracted from the assessment questions:

* Sum of questions answered with yes

* Overall maturity level reflecting on the number of yes-answers

* Number of times a specific question got answered with yes

* Additional descriptive data about the participants and additional commentary and qualitative information

Using the sum of yes-answers the data can be used to create a benchmark of the companies and rank them. On this base the different PLM environments of the companies can be compared. On the base of the impact different factors can be estimated. Based on these factors, possible success factors for a mature PLM environment can be assessed using the qualitative data.

## 3.3 Crossindustrial Benchmarking Study

Using the questionnaire, a benchmarking study was conducted among potential best practice companies. The companies have been selected based on their potentially mature PLM-environment rather than industrial criteria. The following selection criteria were used:

i. Swiss headquarter

ii. Comparable product strategy

iii. Possible mature PLM-environment

iv. Similar manufacturing depth

v. Available during the required time period

Thus, the study does not represent an industrial benchmark. An overview of the different traits of all the participating companies can be found in the Table 3. Due to agreements with the participants, the data is displayed without any explicit description of the companies. The given categorical data will be used for further statistical analysis.

All interviews were conducted with company employees. The interviewees hold positions with direct responsibilities for the improvement of the IT-infrastructure and/or the PLM-environment in general. The interviews took place in the different locations. The minimum timeframe per interview was two hours. The interview followed hereby the following agenda:

1. Brief introduction of the research project (personal interview)

2. General questions about the PLM strategy and personal estimations (personal interview)

3. Interview using the assessment questions as shown in chapter 3.3 (personal interview)

4. Immediate discussion of the results to gain a deeper understanding of the case at hand (personal interview)

5. Extended data analysis

6. Review of the findings by the participants

7. Creation of the final report
<table><caption>**Table 3.** Participating companies of the benchmarking study</caption><thead><tr><th>ID</th><th>Industries:</th><th>Size</th><th>Last PLM change</th><th>Product strategy</th><th>Changes currently planned:</th><th>Manufacturing locations</th><th>Engineering locations</th><th>Company age</th></tr></thead><tbody><tr><td>A</td><td>Energy</td><td>medium</td><td>2004</td><td>ATO</td><td>yes</td><td>multiple</td><td>single</td><td>25 to 50 years</td></tr><tr><td>B</td><td>Energy</td><td>medium</td><td>2019</td><td>ATO, ETO, MTS</td><td>yes</td><td>multiple</td><td>multiple</td><td>50 to 100 years</td></tr><tr><td>C</td><td>Mechanical Engineering</td><td>small</td><td>2019</td><td>ETO</td><td>yes</td><td>single</td><td>single</td><td>> 100 years</td></tr><tr><td>D</td><td>Building industries</td><td>large</td><td>2017</td><td>ATO, ETO</td><td>yes</td><td>multiple</td><td>multiple</td><td>> 100 years</td></tr><tr><td>E</td><td>Building industries</td><td>large</td><td>2014</td><td>MTS, ATO</td><td>yes</td><td>multiple</td><td>single</td><td>50 to 100 years</td></tr><tr><td>F</td><td>Mechanical Engineering</td><td>small</td><td>2019</td><td>ATO</td><td>yes</td><td>single</td><td>single</td><td>< 25 years</td></tr><tr><td>G</td><td>Mechanical Engineering</td><td>large</td><td>2018</td><td>MTS, ATO</td><td>yes</td><td>multiple</td><td>multiple</td><td>25 to 50 years</td></tr><tr><td>H</td><td>Mechanical Engineering</td><td>small</td><td>2019</td><td>ETO</td><td>yes</td><td>multiple</td><td>single</td><td>< 25 years</td></tr><tr><td>I</td><td>Mechanical Engineering</td><td>small</td><td>2010</td><td>ATO</td><td>yes</td><td>single</td><td>single</td><td>25 to 50 years</td></tr><tr><td>J</td><td>Building industries</td><td>large</td><td>2019</td><td>MTS, ATO</td><td>no</td><td>multiple</td><td>multiple</td><td>< 25 years</td></tr></tbody></table>

# 4 Results

## 4.1 Overall Benchmarking Results

Using the developed questionnaire, the participating companies have been interviewed. The overall scores (amount of yes-answers) have been used to rank them. Figure 2 shows the results. The barplot on top shows the total sum of yes answers and the corresponding maturity levels. The matrix shows the individual answers of the participants of every question. The histogram on the right shows the amount of times a question has been answered with yes. The data is sorted in an ascending order using the sum of yes- answers from the lowest scoring to the highest one. For the further analysis, the overall PLM capability of a company is being considered. This means that only the sum of yes- answers is considered relevant, thus the answer to a single questions does not reflect the maturity of a company.
<figure><img src="image_4.png" /><figcaption>Fig. 2. Results Benchmarking overall</figcaption></figure>

## 4.2 Additional Insights from Top Performing Companies

Based on the results, the three top-scoring companies have been asked to describe organizational aspects that they felt were most critical in their opinion to the success of their PLM strategy and implementation. The following success factors were identified:

* Established a long-term road map regarding the system-landscape and the goals

* Running smaller, but more frequent process improvement projects.

* Having designated teams working on the improvement of the PLM-environment.
* Frequent exchange with users on problems, possible ideas and upcoming projects

* Frequent exchange with other companies and field experts

* Implementing small changes on their own – this includes programming

* PLM trainings are coordinated by designated specialists

* Special tools implemented for PLM-users to propose changes

## 4.3 Analysis of Influencing Factors

The further analysis focuses on the total number of yes-answers (scores) achieved by the companies. Table 4 shows the number of yes-answers given by the different companies and a summary of the basic statistical values.

<table><caption>Table 4. Number of yes-answers per company</caption><thead><tr><th>ID</th><th>A</th><th>B</th><th>C</th><th>D</th><th>E</th><th>F</th><th>G</th><th>H</th><th>I</th><th>J</th></tr></thead><tbody><tr><th>Score</th><td>24</td><td>24</td><td>33</td><td>42</td><td>36</td><td>38</td><td>23</td><td>29</td><td>19</td><td>34</td></tr><tr><th>Mean</th><td colspan="2">Median</td><td colspan="2">Variance</td><td colspan="4">Standard deviation</td></tr><tr><th>30.2</th><td colspan="2">31</td><td colspan="2">6.84</td><td colspan="4">7.54</td></tr></tbody></table>

A factor comparison of the mean values has been conducted. The analysed factors are shown in Table 5.

<table><caption>Table 5. Analysed factors</caption><thead><tr><th>ID</th><th>Factor</th><th>Factor levels</th></tr></thead><tbody><tr><td>1</td><td>Company Size</td><td>Small, Medium, Large</td></tr><tr><td>2</td><td>Main Product strategy</td><td>ATO, ETO, Mixed</td></tr><tr><td>3</td><td>Industries</td><td>Building, Energy, Machine</td></tr><tr><td>4</td><td>Manufacturing locations</td><td>Single, Multiple</td></tr><tr><td>5</td><td>Engineering locations</td><td>Single, Multiple</td></tr><tr><td>6</td><td>Company age</td><td>&lt;25 years, 25 to 50 years, 50 to 100 years, &gt; 100 years</td></tr><tr><td>7</td><td>Termination of the last PLM Project</td><td>&lt;1 year, &gt;1 year</td></tr></tbody></table>

Fig. shows the results. The mean values are visualized as horizontal lines within the diagrams. All differences in the mean values that are greater than the standard deviation of the overall data set (6.84) were considered relevant.

This analysis shows that the following factors could be relevant for a high level of maturity:

* Company age

* Company size

* Industries

* Termination of the last PLM Project

Further analysis using a regression model would require a larger sample size.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Factorwise analysis</figcaption></figure>

## 4.4 Sample size calculation

To gain representative results a larger sample group would be required. The minimum required sample size has been calculated on the base of the formula defined by Cochran [21]. The required number of participants is 247. For the calculation the following values have been used:

* Relevant population = 1956. The relevant population size has been defined as all European companies with at least 250 employees working in the sector of machine engineering [22, 23].

* Standard deviation relative to median = 0.24. This value has been transferred from the current sample

* Margin of error of 5% and a z-value of 1.96 in accordance of a expected confidence interval of 95%.
# 5 Discussion

The overall results show a broad spectrum, ranging from the lowest score of 19 yes-answers to the highest value of 42. The maturity of five companies has been classified as intermediate (level 3). Four have been classified as low (level 2) and only one has achieved the level 4, which is described as mature. First, this distribution can be intrep-reted as a validation of the assement questions. This is due to the fact that the broad spectrum can only be achieved with a set of questions that is able to differentiate the companies on the base of their abilities. If the questions would be less generalistic, the expected distribution of the results would be either polarized or very strongly concen-trated on a single score. Second, the distribution can be interpreted as a sign for possibly huge differences among the supported PLM functionalities within the companies. Some potential influential factors have been detected. These will be discussed in chapter 5.1 and 5.2. Some of the questions never got answered with yes (19, 26 and 61). This may have two possible reasons. First, the questions simply may not reflect a relevant func-tionality of a PLM environment. Second, the participants may not be mature enough to achieve the required functionality. To ensure that the questions are relevant and realis-tic, a second round of review of the current questions may be needed.

## 5.1 Additional Insights from Top Performers

The measurements and approaches presented are mainly based on the discussion with the interviewees. Their influence on the actual PLM maturity has not been investigated as part of this study. At this point, the mentioned measurements can be interpreted as fac-tors that may contribute to a higher level of maturity.

However, the data has only been collected for the top three and not for all participants. For further research, a number of structured additional questions, targeting the chosen strategy and PLM tools utilized by these companies, would help to gain further insights. This data should definitely be collected for all participants, since the analysis of the chosen approaches of the low scoring companies could be just as insightfull as shown by Cooper [24].

## 5.2 Influencing Factors

The factor by factor by factor comparison of means clearly shows that certain factors lead to a larger difference among the means than others. Using the 6.84 threshold, the following relevant factors have been selected:

- Company size

Large and small companies scored higher than medium sized companies. The higher scores of the smaller companies can be explained by the expected lower level of organ-izational complexity. In contrast, the higher scores of the large companies would be due to the larger amount of ressources. The middle sized companies low scores could be caused by the increasingly complex set up without the right means to overcome this complexity.
* **Industries:** Companies from the building industry achieved significantly better results than companies from the machine engineering and energy industry. All three companies active in the building industry have a broad product portfolio with mixed product strategies. This is a clear sign of the comparatively high complexity of the overall PLM environment. This increased complexity may have led to a selection process within the industries, leaving only the mature companies.

* **Change:** The highest score was achieved by the companies that recently changed their environment. This is a clear sign, that PLM projects overall lead to an improvement in PLM maturity. Companies are capable to conduct such projects and gain benefits from them even if the change was only recently.

* **Companies age:** The highest mean was achieved by the oldest companies. The second highest by the youngest. These values are comparable close to one another and the difference to the companies older than 50 years is rather small. The companies older than 25 but younger than 50 years lag considerably behind. The root causes of this cannot be evaluated at this point. However, all companies scoring low in this category have the commonality that their last change in the PLM environment took place more than one year ago.

All the discussed results are based on a small sample group, the results are not statistically significant and the selected group is not representative for their corresponding industries. Additionally, the set of factors currently used is not comprehensive. As part of further research additional evaluation of factors would be required.

## 5.3 Sample Size Calculation

The calculated value of 247 European companies is based on the assumption, that the standard deviation of the current sample is transferable to a larger case group. Not all of the current participants would meet the criteria for the proposed study. This leads to the conclusion that the calculation has to be regarded as an estimation and a first indicator. Furthermore, any change in the assessment questions or the overall model would require a recalculation of the standard deviation on the base of a new study.

## 6 Conclusion

The chosen methodology can deliver meaningful results. Hereby, the frequent validation by experts and the review of the companies of the results have been proven to be crucial elements. The developed model is able to differentiate the companies and delivers a broad spectrum of results. Within these results certain patterns can be detected. These patterns lead to factors that may influence the maturity of a PLM environment of a company. The additional discussion of the results with the high scoring companies has been a useful approach to learn about measurements and approaches which may be beneficial for the maturity of the PLM environment. On the base of Cooper additional analysis of the low scoring companies would be beneficial. Conducting an additional study with a larger sample group would allow the data to be analyzed using statistical means and to quantify the effects of different factors. As part of this expanded study an
expansion of the current set of questions could be beneficial to cover additional aspects
of the PLM environment and get an even more holistic point of view.

References

1. Silventoinen AS*, Pels HJ, Kärkkäinen H et al. (2013) PLM maturity assessment as a tool for PLM implementation process. In: Thoben K-D (ed) Product lifecycle management: Proceedings of the PLM10 Conference held at the University of Bremen, Germany, 12-14 July 2010. Inderscience Enterprises Ltd, Genève, pp 369–379

2. Zhang H, Bouras A, Sekhari,, Aicha et al. (2014) A PLM components monitoring framework for SMEs based on a PLM maturity model and FAHP methodology. Journal of Modern Project Management(2): 108–119

3. Paavel M, Karjust K, Majak J (2017) PLM Maturity Model Development and Implementation in SME. Procedia CIRP 63: 651–657. doi: 10.1016/j.procir.2017.03.144

4. Vezzetti E, Violante MG, Marcolin F (2014) A benchmarking framework for product lifecycle management (PLM) maturity models. Int J Adv Manuf Technol 71(5-8): 899–918. doi: 10.1007/s00170-013-5529-1

5. Wendler R (2012) The maturity of maturity model research: A systematic mapping study. Information and Software Technology 54(12): 1317–1339. doi: 10.1016/j.infsof.2012.07.007

6. Fraser P, Moultrie J, Gregory M (2002) The use of maturity models/grids as a tool in assessing product development capability. In: 2002 IEEE International Engineering Management Conference, pp 244–249

7. Batenburg R, Helms RW, Versendaal J (2006) PLM roadmap: stepwise PLM implementation based on the concepts of maturity and alignment. IJPLM 1(4): 331–351. doi: 10.1504/IJPLM.2006.011053

8. Saaksvuori A, Immonen A (2008) Product Lifecycle Management. Springer-Verlag Berlin Heidelberg, Berlin, Heidelberg

9. Kärkkäinen H, Pels HJ, Silventoinen A (2012) Defining the Customer Dimension of PLM Maturity. In: Rivest L, Bouras A, Louhichi B (eds) Product Lifecycle Management. Towards Knowledge-Rich Enterprises, vol 388. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 623–634

10. Kärkkäinen H, Silventoinen A (2016) Different Approaches of the PLM Maturity Concept and Their Use Domains – Analysis of the State of the Art. In: Bouras A, Eynard B, Foufou S et al. (eds) Product Lifecycle Management in the Era of Internet of Things, vol 467. Springer International Publishing, Cham, pp 89–102

11. Axel Franzen (2014) Antwortskalen in standardisierten Befragungen. In: Baur N, Blasius J (eds) Handbuch Methoden der empirischen Sozialforschung. Springer VS, Wiesbaden, pp 665–675

12. García-Mireles GA, Ángeles Moraga M, García F (2012) Development of maturity models: a systematic literature review. In: 16th International Conference
on Evaluation & Assessment in Software Engineering (EASE 2012): 14-15 May 2012. IEEE, Piscataway, NJ, pp 279-283

13. Wagner KW, Dürr W (2008) Reifegrad nach ISO/IEC 15504 (SPiCE) ermitteln. Carl Hanser Verlag GmbH & Co. KG, München

14. Marshall S, Mitchell G (2010) Applying SPICE to e-Learning: An e-Learning Maturity Model? In: Lister R (ed) Computing education 2004: Proceedings of the Sixth Australasian Computing Education Conference, Dunedin, New Zealand, January 2004, pp 185-191

15. Goulding, Haigh, Finnemore et al. (1999) SPICE: Is a capability model applicable in the construction industry? In: Lacasse MA (ed) Information technology in construction: CIB W78 workshop. NRC Research Press, Ottawa

16. Mettler T (2011) Maturity assessment models: a design science research approach. IJSSS 3(1/2): 81. doi: 10.1504/IJSSS.2011.038934

17. Hachicha M, Moalla N, Fahad M et al. (2016) A Maturity Model to Promote the Performance of Collaborative Business Processes. In: Bouras A, Eynard B, Foufou S et al. (eds) Product Lifecycle Management in the Era of Internet of Things, vol 467. Springer International Publishing, Cham, pp 112-124

18. Stark J (2015) Product Lifecycle Management: Volume 1: 21st Century Paradigm for Product Realisation, 3rd ed. 2015. Decision Engineering. Springer International Publishing, Cham

19. Bellary S, Krishnankutty B, Latha MS (2014) Basics of case report form designing in clinical research. Perspect Clin Res 5(4): 159-166. doi: 10.4103/2229-3485.140555

20. Terzi S, Bouras A, Dutta D et al. (2010) Product lifecycle management - from its history to its new role. IJPLM 4(4): 360. doi: 10.1504/IJPLM.2010.036489

21. Cochran WG ((1953)) Sampling techniques. Charles E. Tuttle, Tokyo

22. Bundesamt für Statistik (2019) Bestand aktiver Unternehmen. je-d-06.02.02.01.01. https://www.bfs.admin.ch/bfs/de/home/statistiken/industrie-dienstleistungen/unternehmen-beschaeftigte/unternehmensdemografieassetdetail.10687098.html. Accessed 27 Mar 2020

23. EuroStat (2017) Industrie nach Beschäftigtengrößenklassen (NACE Rev. 2, B-E). https://appsso.eurostat.ec.europa.eu/nui/submitViewTableAction.do. Accessed 27 Mar 2020

24. Cooper RG (1979) The Dimensions of Industrial New Product Success and Failure. Journal of Marketing 43(3): 93. doi: 10.2307/1250151