<figure><img src="image_1.png" /></figure>

HAL
open science

# CLIMB Model: Toward a Maturity Assessment Model for Product Development

Monica Rossi, Sergio Terzi

► To cite this version:

Monica Rossi, Sergio Terzi. CLIMB Model: Toward a Maturity Assessment Model for Product Development. 12th IFIP International Conference on Product Lifecycle Management (PLM), Oct 2015, Doha, Qatar. pp.103-111, 10.1007/978-3-319-33111-9_10 . hal-01377432

HAL Id: hal-01377432

https://inria.hal.science/hal-01377432

Submitted on 7 Oct 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# CLIMB Model: Toward a Maturity Assessment Model for Product Development

Monica Rossi¹, Sergio Terzi¹

¹Politecnico di Milano, Department of Management, Economics and Industrial Engineering Piazza Leonardo da Vinci, 20133, Milano, Italy

monica.rossi@polimi.it

sergio.terzi@polimi.it

**Abstract.** Product development (PD) becomes crucial for the competitiveness, survival and prosperity of any organization. In order to deliver products successfully, companies can choose between a vast amount of best practices to apply in their innovation processes. However PD processes are still wasteful in practice. With the aim of (i) creating awareness between practitioners on the meaning of PD best practices, (ii) understanding how to measure the maturity in the use of such best practices and in order to (iii) understand the real level of application of these practices, the paper propose CLIMB: a maturity assessment model based on prevalent PD best practices in literature able to measure the maturity of companies in their PD activities. Also the paper proposes the results of an empirical data collection in 2012-2013 within the GeCo Observatory initiative in Italy, which gathered data through face-to-face interviews from more than 100 companies using the CLIMB model. The results is that the tool is effective and that more researches are needed to understand which circumstances lead the choice of certain PD best practices over others.

**Keywords:** Product Development (PD), Product Development Assessment, Product Development Maturity, Assessment Tool, Benchmarking, Maturity Model, Best Practice, Product Development Best Practices, CLIMB Model.

## 1 Introduction: the Need of an Assessment Model for Product Development

Product development (PD) is the mean by which companies innovate and introduce new product to the marketplace; nowadays PD is becoming more and more crucial for companies competitiveness, prosperity, and survival [1]–[6]. The success or failure of innovation processes is drastically affected by the choice of
engineering and design practices to be implemented during the product develop-
ment phase. In literature a large number of such engineering and design practices
(i.e. tools, methods, techniques) has been explored and studied. Between those,
some practices are recognized to foster effectiveness and efficiency of PD and are
acknowledged as best practices [7]–[14]. Both in literature and practice, there is a
constant and challenging research of those kinds of best practices and many efforts
have already identified a conspicuous number of practices able to lead companies
toward successful results [5].

However, despite nearly 40 years of scientific research focused on improving
PD through the promotion of PD best practices, recent results reveal that these
attempts have failed to materialize as expected in practice [15], [16]. Several gaps
are still open. Between those: it is not always clear if practitioners are aware of the
meaning of PD best practices [7]; it is not known the level of diffusion of those
best practices within industries [5], [7]; and it is in doubt if practitioners are able
to identify which practices they could implement in their organizations [7], [13].

These open issues drive the rational of this paper, that aims at covering the ex-
isting gaps by providing as a first extent a best practice framework able to create
consciousness of what constitutes a best practice in PD (in section 3). The need of
a framework is given by the fact that PD is multi-dimensional, and any attempt in
literature done to understand the complexity in the variety of PD best practices
tries to identify categories of best practices at first. Moreover, basing on this
framework, this paper proposes an assessment model (named CLIMB) to be used
to assess the maturity of companies in the use of the identified PD best practices
(in section 3). The purpose is double: at the first place the aim is to cover the liter-
ature gap of understanding the as-is situation in the diffusion of the identified PD
best practices in the industrial context. In the second place, as a managerial impli-
cation, the authors want to provide companies with a simple and visual assessment
tool, to be used both for benchmarking and for self-assessment purposes within
companies. PD practitioners are keen to benchmark PD practices because identify-
ing any practice that is able to more efficiently and/or effectively deliver a new
product could represent the difference between success and failure [13]. The self-
assessment leads to the identification of PD weak areas where to direct improve-
ment efforts. The ultimate purpose of the CLIMB model is to concretely support
top management, project managers, and decision makers to identify and select
which PD best practices to implement with the hope that companies will manifest
and sustain these to expand their PD efforts.

The paper starts from an in depth literature review that, together with several
focus groups with experts, have served to build the PD best practice framework
and the CLIMB model. Moreover the so-developed maturity assessment model
has been used to assess 103 companies in Italy and the results of the level of diffu-
sion of the proposed PD best practices is reported in session 4 of the paper, fol-
lowed by further thoughts and on-going and future researches in the final session.
## 2 State of the Art: Classifications of Best Practices in Product Development

Any practice whether a technique, a method, a process, or an activity that enables to deliver more efficiency and/or effectiveness than any other manner can be considered as a *best practice* [13], [17]. Vice versa, we can define a *poor practice*.

Product Development is a multidimensional process, constitute of several different but intercorrelated elements across multiple layers and facets. In literature more than 100 PD best practices have been identified, such as the adoption of multifunctional teams, the use of modularization and standardization for parts and components, the use of design for x techniques, the use of the PLM systems to support the data management through the whole life cycle of a product, for example.

Given the high number of these practices and the different level they operate, few authors tried the effort of classifying PD best practices across different PD dimensions [1], [10]–[13], however a unique classification is missing. Some scholars report different grouping of practices at different levels of PD. Under the product development literature field, for example, Barczak et al. (2009) propose 8 different classes [14]: *The new PD process, The fuzzy front end (FFE), Portfolio management, Organizing for new product development, Market research tools, Engineering design tools, and Technology & organizational tools supporting new PD*. Similarly to this classification, as a consequence of complementary research activities run within the PDMA (Product Development & Management Association), another arrangements is suggested as follow [7], [13]: *Strategy, Research, Commercialization, Process, Project Climate, Company Culture, and Metrics & Performance Measurement*.

Another attempt of classify principles and practices in product development across dimension, has been given by lean PD literature. The most acknowledged classification is from Morgan and Liker (2006) and consists of the following three areas [1]: *Skilled People, Process, and Tools & Technology*.

Despite those different dimensions, both streams acknowledge similar –and sometimes complementary- practices to foster successful product development that inspired the development of the proposed best practice framework at the basis of the CLIMB maturity assessment model.

## 3 The CLIMB Maturity Assessment Model

The CLIMB maturity assessment model aims at covering the above-mentioned gaps (see §1) by:

(i) Creating awareness on the existing best practices in product development, thanks to the PD best practice framework;

(ii) Providing a useful tool both for scholars -by giving an as-is picture of the current usage and diffusion of PD best practices- and for practitioners -
giving them a powerful tool to evaluate their current situation and identifying possible improvements actions based on the benchmarking with what is believed best in literature and eventually with other industrial cases.

The CLIMB maturity assessment model starts from the properly developed PD best practice framework. Then, it builds upon the categories of this framework a maturity model that evaluates 5 different level of accomplishment of the considered categories of best practice. Practically, CLIMB model is composed of (1) a PD Best Practice framework, (2) a questionnaire, (3) a maturity evaluation scale, and (4) a radar chart for a visual representation. All the components are described in the next sections.

## 3.1 The PD Best Practice Framework

Within this study, the authors have identified more than 100 prevalent best practices proposed in literature by different scholars and basing on that, and on a series of focus groups conducted with experts, they propose a most update framework to collect and categorize PD best practices. The focus groups were constituted by the members of the advisory board of GeCo Observatory - an Italian research initiative created in the frame of the Observatories of the Business School of Politecnico di Milano (http://www.osservatori.net/progettazione_plm). In the specific, 25 practitioners have been consulted together three times during the development and refinement of the framework, and their experience's based suggestions and feedback have been used to develop the final version of the framework.

The framework categorizes 107 PD best practices, through 8 areas, (i) *Activities & Flow*, (ii) *Decision Making*, (iii) *Training*, (iv) *Roles and Collaboration*, (v) *Knowledge Management Process*, (vi) *Knowledge Management Techniques*, (vii) *Methods*, (viii) *Computerization and Software*, respectively grouped into 4 dimensions: *Process*, *People*, *Knowledge Management* and *Tools*, as summarized in Table (Table 1).

<table><caption>**Table 1.** PD Best Practice Framework, list and number of best practices (#BP) for each area</caption><thead><tr><th colspan="2">Area: PEOPLE</th></tr><tr><th>Sub-Area: Roles & Collaboration</th><th># BP</th></tr></thead><tbody><tr><td><ul><li>Cross-functional team</li><li>All actors are involved in the project team, even when globally distributed</li><li>Clear definition of roles and responsibilities for each individual</li><li>High flexibility on task execution</li><li>There is an overall responsible (PM) with technical background</li><li>Full customer involvement in development</li><li>Involvement of experienced designers from the earliest stages of the projects</li></ul></td><td>7</td></tr><tr><th>Sub-Area: Training</th><th># BP</th></tr><tr><td><ul><li>Formal programs to support multidisciplinary skills development</li><li>One-to-one tutoring</li></ul></td><td>3</td></tr></tbody></table>
* KPIs to assess training outcomes

## Area: PROCESS

### Sub-Area: Activities & Flow # BP

* Formal NPD model, properly followed and documented by the various actors involved

* Strongly collaborative development process

* Complex set of KPIs to measure NPD performance

* Frontloading the PD process

* Continuous Improvement Initiatives

* Many solutions are designed and inferior solutions are progressively discarded when new information becomes available

* Complete focus on customer value

* Formalized process for analyze competitors (Reverse Engineering)

8

### Sub-Area: Decision Making # BP

* Lifecycle perspective vision. Consideration of the whole product life phases during PD (10 phases)

* Basing decision making process on strategic factors (12 factors)

22

## Area: KNOWLEDGE MANAGEMENT

### Sub-Area: KM Process # BP

* Previous knowledge is retrieved by individuals at different PD stages (5 stages)

* Formal overall knowledge management plan

* Main source of knowledge is coming from formal means, such as design rules defined by the company/stakeholders for ensuring the strategic factors are considered in the PD process (12 factors)

* Formal sources of knowledge are continuously update and reviewed (3 formal sources)

* Rely on previous knowledge for PD projects

### Sub-Area: KM Techniques # BP

* Structured Tools and techniques formally used to capture, share and reuse knowledge (11 different techniques)

11

## Area: TOOLS

### Sub-Area: Methods # BP

* Formal engineering/design methods (11 methods)

11

### Sub-Area: Computerization & Software # BP

* Product Development is strongly supported by software platforms (22 softwares)

23

Total number of best practices 107

## 3.2 The CLIMB Maturity Model

After developing the proposed best practice framework, the authors developed a maturity model able to associate to each of the practice and category of practice of the framework, a level of accomplishment reached by the respondent.

The maturity model is made of a questionnaire, an evaluation scale made of 5 maturity levels, and a radar chart. The questionnaire is completely based on the
proposed PD best practice framework and each of the questions investigates one of the best practices. The number of questions corresponds to the number of the investigated best practices (Table 1). Each question, scored through a 5 points scale, is structured as in the following example (taken from the area Training and Competencies):

Example: How does the company support skills' development?

a. Everyone is personally responsible for developing and maintaining his/her own skills (1)

b. A situation between a and c (3)

c. The company gives training on the job (5)

d. A situation between c and e (7)

e. The company promotes multidisciplinary skills with formal programs (i.e. training plans, rotation between project teams) (9)

For each of the more 107 practice five different levels of accomplishment can be selected by the respondent: he/she can choose whether his/her company states at a poor practice level, at a best practice level, or somewhere in between. Those levels can assume a score of 1 3 5 7 9, as reported in the blanks above. The lowest level of accomplishment (a), scored with 1, corresponds to a poor practice in opposition to the the higher level (e), which corresponds to a best practice, and it is scored with 9. Additionally there are three middle levels, whose intermediate circumstance (c), scored with 5, is described in order to facilitate the respondent to address his choice.

A group of one or more questions concurs to describe each of the 8 areas of the framework (Table 1). The score of a single area is calculated as an additive scale (summing the single scores of the questions describing the area) then normalized in %. The following formula defines how the score for each generic area (Ai) is calculated:

$$a_i = \frac{\sum_{1}^{m_i} q_{ij}}{8 * m_i}$$

Where:

ai is the score corresponding to i-th area, expressed in %

i=1...8, is the indicator for the areas

qij is the score of the answer to the question j, belonging to the i-th area

j=1...mi, is the indicator for the questions, depending on the area the number of questions changes

mi, is the number of questions of the i-th area

8*mi is the maximum score the area can assume in the case the respondent declares to always reach the best practice level -scored with 9- for all the j practices investigated within the i-th area.

Each of the eight areas expressed in %, and 5 possible stages of accomplishment of a best practice condition are defined toward the i-th area. The 5 levels are
20% width intervals in the scale from 0 to 100% and are namely: Chaos (0%-20%), Low (21%-40%), Intermediate (41%-60%), Mature (61%-80%), and Best Practice (81%-100%) (Fig.1). From here the name CLIMB.

<figure><img src="image_3.png" /><figcaption>Fig. 1. The 5 Maturity Levels in the CLIMB Model</figcaption></figure>

The level of accomplishment achieved within each of the 8 areas can be then represented in a radar chart (Fig.2). The radar chart gives an immediate and effective picture of the level of implementation of the considered practices along the eight areas of the framework (Table 1) and displays the positioning of the company within one of the 5 CLIMB stages (Fig.1). The proposed model could serve as basis for empirical investigations, as the one run in 2012/2013 in Italy and described in the next section.

<figure><img src="image_4.png" /><figcaption>Fig. 2. The Radar Chart of the CLIMB Model</figcaption></figure>

## 4 The Diffusion of PD Best Practices in Italy: the Empirical Research of the GeCo Observatory

In order to evaluate the level of diffusion of the identified PD best practices within industry, and to understand the level of maturity reached by companies in their PD activities, the authors have run an empirical research in Italy, from March 2012 to February 2013. The study has been conducted within the above-mentioned GeCo Observatory initiative on 103 Italian and multinational companies, with at least one product development site in Italy. Each interview involved a project
manager, a technical director, and/or a team of engineers working in PD. An average of 2.5 hours have been spent in each company for each face-to-face interview.

The sample is constituted of both small and medium enterprises (SMEs), and big enterprises. Details of the size of the sample are in Table 2. Companies belong to different sectors, grouped into 4: Mechanics, Electrics, Electronics and Other Sectors (such as Fashion, Chemical and Food). Table 3 summarizes the distribution of the sample across the sectors.

<table><caption>**Table 2.** Sample: Size</caption><thead><tr><th>Size (number of employees)</th><th>N° of companies</th><th>Class</th><th>N° of companies</th></tr></thead><tbody><tr><td>Micro (&lt;10)</td><td>4</td><td></td><td></td></tr><tr><td>Small (10&gt;employees&lt;50)</td><td>13</td><td>SMEs</td><td>38</td></tr><tr><td>Medium (50&gt;employees&lt;250)</td><td>21</td><td></td><td></td></tr><tr><td>Big (250&gt;employees&lt;1000)</td><td>29</td><td>LARGE</td><td>65</td></tr><tr><td>Macro (&gt;1000)</td><td>36</td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 3.** Sample: Sector</caption><thead><tr><th>Sector</th><th>N° of companies</th></tr></thead><tbody><tr><td>Mechanics</td><td>44</td></tr><tr><td>Electrics</td><td>27</td></tr><tr><td>Electronics</td><td>18</td></tr><tr><td>Other</td><td>14</td></tr></tbody></table>

Fig.3 depicts the radar charts resulting from the empirical research, according respectively to the sectors and the size of the companies belonging to the sample. Despite from one from one could expect, there are not significant differences in behaviours between BIG and SMEs or between sectors. Areas such as computerization & software, methods, and knowledge management techniques present the lowest level of maturity within the interviewed sample.

## 5 Conclusion & Further Research

The paper aimed to develop a maturity assessment model, named CLIMB, able to cover the identified gaps of creating awareness on the meaning of PD best practice and creating a way to depict the as-is situation both for scholars and practitioners. The model results effective for the purpose it was created for. Clarity on the meaning of best practice in PD, together with a list and classification of PD best practices are given through the PD best practice framework. The CLIMB model, based on a 5-levels maturity scale, is a powerful tool, useful not only to gather data from companies in the field, but also as a self-assessment tool for managers. The managerial implications of this piece of work are extremely rele-
vant. From one side respondents become aware of the gap existing between the
real level of application of certain best practices in certain areas of PD, compared
to ideal practices available to be used. Also thanks to the GeCo Observatory re-
search, companies could benchmark themselves, not only with the best case from
literature, but also with the “rest of the world”. This is to be considered a first
important step toward consciousness on where direct PD improvement efforts.

However, despite the data collected so far don’t seems to demonstrate that size
or sectors affect the use of best practices in PD, it can’t be stated that all the 107
best practices are suitable or ideal to be used in every circumstances and in every
company. Logics behind the use of proper set of PD best practices driven by con-
tingency variables require higher attention. Further studies should be taken in
order to understand if the use of PD best practices is context depended. The GeCo
Observatory is going in this direction.

<figure><img src="image_5.png" /><figcaption>Fig. 3. Current situation of the sample by sector (left) and size (right)</figcaption></figure>

**Acknowledgments.** This work was partly funded by the European Commission through DIVERSITY (GA_636692) and Manutelligence (GA_636951) Projects, as well as by the GeCo Observatory. The authors wish to acknowledge their gratitude to all the partners for their contributions during the development of concepts presented in this paper.

References

[1] J. M. Morgan and J. K. Liker, *The Toyota product development system*. New York: Productivity Press, 2006.
[2] J. P. Womack, D. T. Jones, and D. Roos, *The machine that changed the world: The story of lean production, Toyota's secret weapon in the global car wars that is revolutionizing world industry (New Ed.)*. Simon & Schuster, 2007.

[3] J. P. Womack and D. T. Jones, *Lean thinking: banish waste and create wealth in your corporation*. New York: Simon & Schuster, 1996.

[4] B. Bayus, "Are product life cycles really getting shorter?," *J. Prod. Innov. Manag.*, vol. 11, no. 4, pp. 300-308, 1994.

[5] A. Griffin, "PDMA research on new product development practices: updating trends and benchmarking best practices.," *J. Prod. Innov. Manag.*, vol. 14, no. 6, pp. 429-458, 1997.

[6] H. Chesbrough and A. K. Crowther, "Beyond high tech: early adopters of open innovation in other industries.," *R&D Manag.*, vol. 36, no. 3, pp. 229-236, 2006.

[7] K. B. Kahn, G. Barczak, J. Nicholas, A. Ledwith, and H. Perks, "An Examination of New Product Development Best Practice," *J. Prod. Innov. Manag.*, vol. 29, no. 2, pp. 180-192, Mar. 2012.

[8] G. Szulanski, "Exploring internal stickiness: Impediments to the transfer of best practice within the firm," *Strateg. Manag. J.*, vol. 17, no. S2, pp. 27-43, Dec. 1996.

[9] P. S. Goodman, *Critical issues in doing research that contribute to theory and practice*. In Doing r. Lanham, MD: Lexington Books., 1985.

[10] R. G. Cooper, S. J. Edgett, and E. J. Kleinschmidt, "Benchmarking best npd practices-I," *Res. Technol. Manag.*, vol. 47, no. 1, p. 31, 2004.

[11] R. G. Cooper, S. J. Edgett, and E. J. Kleinschmidt, "BENCHMARKING BEST NPD PRACTICES-II," *Res. Technol. Manag.*, vol. 47, no. 3, 2004.

[12] R. G. Cooper, S. J. Edgett, and E. J. Kleinschmidt, "BENCHMARKING BEST NPD PRACTICES - III," *Res. Technol. Manag.*, vol. 47, no. 6, pp. 43-55, 2004.

[13] G. Barczak and K. B. Kahn, "Identifying new product development best practice," *Bus. Horiz.*, vol. 55, no. 3, pp. 293-305, May 2012.

[14] G. Barczak, A. Griffin, and K. B. Kahn, "PERSPECTIVE: Trends and Drivers of Success in NPD Practices: Results of the 2003 PDMA Best Practices Study," *J. Prod. Innov. Manag.*, vol. 26, no. 1, pp. 3-23, Jan. 2009.

[15] D. J. Flint, "Compressing new product success-to-success cycle time Deep customer value understanding and idea generation," *Ind. Mark. Manag.*, vol. 31, pp. 305-315, 2002.

[16] R. G. Cooper, "The Invisible Success Factors in Product Innovation," *J. Prod. Innov. Manag.*, vol. 16, no. 2, pp. 115-133, Mar. 1999.

[17] R. 1989. Camp, *Benchmarking: The search for industry best practices that lead to superior performance*. Milwaukee, WI: ASQ Quality Press, 1989.