# Designing a Performance Measurement System for Materials Management Under Engineering Change Situations in ETO Environment

Pavan Kumar Sriram, Bjørn Andersen, Erlend Alfnes

<figure><img src="image_1.png" /></figure>

## To cite this version:

Pavan Kumar Sriram, Bjørn Andersen, Erlend Alfnes. Designing a Performance Measurement System for Materials Management Under Engineering Change Situations in ETO Environment. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2015, Tokyo, Japan. pp.263-270, 10.1007/978-3-319-22759-7_31. hal-01431105

HAL Id: hal-01431105

https://inria.hal.science/hal-01431105

Submitted on 10 Jan 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Designing a performance measurement system for materials management under engineering change situations in ETO environment

Pavan K Sriram¹, Bjørn Andersen¹, and Erlend Alfnes¹

¹Norwegian University of Science and Technology, Trondheim, Norway
pavan.sriram@ntnu.no, bjorn.andersen@ntnu.no, erlend.alfnes@ntnu.no

**Abstract.** In this study we aim to understand how performance indicators should be designed and implemented across various phases and levels in materials management in the order fulfilment process under engineering changes. The paper address questions such as how can such systems help managers to handle and manage materials management under engineering change situations? How do we convince potential users and obtain their support when starting to develop such a system? How can we aggregate performance indicators? How do we present results? Then using the literature review and the results of the empirical study from a Norwegian company operating in ETO product delivery strategy, we develop a framework.

**Keywords:** Engineer-to-order, engineering change, performance measurement, case studies

## 1 Introduction

Materials management has always been one of the most important and critical processes within production planning and control at companies with ETO supply chains (Persona et al., 2004). Managing the material flow in an effective and efficient way facilitates achieving success as it provides availability of materials with the right type, in the right quantities and at the right time to different phases of the supply chain. In a situation that material-planning activity does not perform well, supply chain faces problems such as work stoppages, delays of end product, losing responsiveness and hence losing valuable customer satisfaction.

Having a supply chain with large number of suppliers creates even greater need for an effective and efficient management of material flow, as it is required to take into account deliveries of numerous components from different suppliers. Moreover, there are many challenges in ETO production environment that affect performance of material planning; e.g. engineering changes (Wänström and Jonsson, 2006) and uncertainty of product specification, mix and volume (Bertrand and Muntslag, 1993). In addition material and production requirements vary from project to project which makes plan-adfa, p. 1, 2011.
ning and control more difficult (Stevenson * et al., 2005). Therefore, it is vital for ETO companies to control and monitor the performance of materials management under engineering change situation precisely. Measurement and assessment of performance are prerequisites for improvement (Armstrong and Baron, 2005, Otley, 1999, Douwe P. Flapper et al., 1996, Parthiban and Goh, 2011). The processes of measuring and improving the performance are gathered in an area called performance management which provides an iterative process between these two steps (Parthiban and Goh, 2011). The main concern of performance management is mostly what to measure and developing performance measures for different activities and individuals (de Leeuw and van den Berg, 2011, Gunasekaran and Kobu, 2007).

Even though a lot of literature on topic of performance measurement and performance measurement systems for supply chain exists, (Kaplan and Norton, 2001, Theeranuphattana and Tang, 2007, Otley, 1999, Lin and Shen, 2007) there is little information regarding performance measurement specifically in management of materials under engineering change situations in an ETO environment. The main result of this paper is to design a performance measurement system for materials management under engineering change situations in ETO environment.

# 2 Literature Review

## 2.1 Materials Management in ETO environment

ETO environment are characterized mostly by large and complex products, which are designed and produced by customers' requirements (Hicks and Pongcharoen, 2006, Bertrand and Muntslag, 1993, Gelders, 1991). Products in this type of supply chain are required in low quantities and sometimes in medium volumes, but generally they contain a diversity of components in a complex combination (Hicks and Braiden, 2000). Each component should be assigned to specific operation in the production (Jiao et al., 2007).

Overall task of material planning process is to ensure material availability at the right stage of production and at the right time. To do so material planning uses bill of materials, inventory data, and data from master production schedule in order to determine time-phased plans for all components and raw materials required for production (Vollmann et al., 2005). For conventional material planning accurate lead times and safety stocks are the most critical characteristics to determine the performance of this process (Jonsson and Mattsson, 2008)

Material planning process includes broad set of tasks and activities like planning required materials, supplier selection, purchasing, inventory management and forecasting. Therefore, this process is not only simple computer calculations but also it includes effective communication mechanisms, education activities and training programs (Bell and Stukhart, 1986). The material planning process starts when the order is received, materials specifications and materials coding systems are established and bill of materials is created (Bell and Stukhart, 1986). In order to link bill of materials with process structure, each component in the bill of materials should be assigned to specific operations in the production (Jiao et al., 2007).
## 2.2 Performance management

In ETO environments the biggest investments are put into materials, it would be very costly for a company with ETO supply chain to have an ineffective material flow. It is critical to manage the performance of material planning activity and material flow throughout the supply chain. As products have deep and complex structures which lead to a wide range of assembly processes, it is necessary to consider performance associated with end products and assembly as well as performance relating to components (Hicks and Braiden, 2000). Effective performance of material planning activity in the supply chain supports the overall production by providing materials in a complete, timely and reliable manner. Material planning performance depends to a great extent on the environment it is executed in. (Jonsson and Mattsson, 2003) proved that user environment (software support available in the enterprise, quality of planning information, the function of planning activities in organizational design, education and knowledge of planner) have important impact on planning performance.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Material planning user environment based on (Jonsson and Mattsson, 2008)</figcaption></figure>

## 2.3 Research Methodology

A case study method was preferred because of two reasons, firstly the research focussed on the ‘how’ and ‘why’ areas of the topic. Secondly, the investigators had little control over the events and the focus in interface between engineering and production on the contemporary activities in the company. The article draws on information collected from interviews, formal discussions and literature review, which be- long to the six sources of evidences for a case study (Yin, 2009).

A literature review on the topic aimed at providing a brief yet comprehensive under- standing of the existing academic research in the area was carried out. For the inter- views semi-structured style was preferred as it provides more flexibility to both the interviewer and interviewee develop ideas and questions more widely on the issues raised in the research more widely (Denscombe, 2014). The interview process is summarized in the table below.
<table><caption>Table 1. Interview process overview</caption><thead><tr><th>Type of Interview</th><th>Respondent position in the organization</th><th>Number of formal interviews</th><th>Number of Informal interviews</th></tr></thead><tbody><tr><td><strong>Case Company</strong></td><td></td><td></td><td></td></tr><tr><td><strong>Face to Face</strong></td><td>Engineering manager</td><td>1</td><td>-</td></tr><tr><td><strong>Face to Face</strong></td><td>Manager Planning Department</td><td>3</td><td>2</td></tr><tr><td><strong>Face to Face</strong></td><td>Senior Engineer (s)</td><td>4</td><td>2</td></tr><tr><td><strong>Face to Face</strong></td><td>Purchasing Manager</td><td>2</td><td>-</td></tr><tr><td><strong>Face to Face</strong></td><td>Sales Manager</td><td>1</td><td>1</td></tr></tbody></table>

## 3 Findings and Discussion

Requirements for engineering changes come both from customers/suppliers (external), but also from design/engineering (internally). And as planning phase situated before design stage-managing such changes become problematic for the company. The time for re-planning the required material to fulfil changes influences the total lead-time of the project. Engineering changes encompass a lot of paper work, which is time consuming to proceed. Material planning should be quite flexible in order to deal effectively with order changes in each project. Hence we design a performance management system based on the indicators below, the ISO14031 guidelines and on studies from (Andersen and Fagerhaug, 2002).

The lack of a performance measurement system in the company led to the design and develop a performance measurement system with a strong focus on material planning that would consider the indicators in Table 3, which focused on how to measure the performance of material planning and what type of measures should be taken into the system. Secondly, since the engineering changes are an unavoidable characteristic of case company, it is critical for material management to be flexible with regards to engineering changes and to be able to quickly respond to these changes. The indicators will be further developed, based on multiple case studies.

Currently the case company does not have a specific performance measurement system. The set of key performance indicators developed for reporting are presented in Table 2.

<table><caption>Table 2. Current KPI at the Case Company</caption><thead><tr><th>Department</th><th>KPI</th><th>"Formula" / What is measured</th></tr></thead><tbody><tr><td rowspan="2"><strong>Sales and development</strong></td><td rowspan="2">Project implementation</td><td>Sale price - materials - work (hours * hourly charge)</td></tr><tr><td>Measures margin calculation to the actual margin after the project is completed.</td></tr><tr><td></td><td></td><td>"Full Results": &lt;1% deviation</td></tr><tr><td></td><td></td><td>Positive deviation: 1% &lt;x &lt;10% &gt; 10% Negative deviation: 1% &lt;x &lt;10% &gt; 10%</td></tr></tbody></table>
<table><tr><td>Sales and development</td><td>Sales of projects / acceptance rate</td><td>1) Value of offers versus value of contracts actually signed over a 12 - and 24-month changing average (Krone Value)<br/>2) Contracts / number of offers (Aggregate Percentage)</td></tr><tr><td>Technical</td><td>Number of thrusters in production</td><td></td></tr><tr><td>Technical</td><td>Number of thrusters delivered</td><td></td></tr><tr><td>Technical</td><td>Number of errors detected</td><td>Grouping by type of error - (1) During production, (2) After delivery</td></tr></table>

This way of measuring will help to assess the performance of material planning during the order fulfillment process by means of tracking and monitoring the current state of each project. If any deviations from the plan occur it will be shown by measurement and further actions can be taken to fix the gaps for future projects. For this purpose a standard set of performance measures can be set in order to evaluate projects performance with the same indicators. This will give possibility to internal benchmark and further improvements. However, at the same time a problem may arise, as success criteria can be different from project to project. Standard set of measurement does not address these differences.

<table><caption>**Table 3.** Performance Indicators for Materials Management under engineering change: adapted from (Sjøbakk and Bakås, 2014)</caption><thead><tr><td>Performance indicator</td><td>Measure</td></tr></thead><tbody><tr><td>BOM complexity</td><td>BOM is complex with many levels, and items can appear at more than one level. Thus, an EC will also affect many neighbouring and parent items.</td></tr><tr><td>Engineering change costs</td><td>Actual time used for BOM change/ Calculated time used for BOM<br/>Σ Cost of engineering change orders Sales /Total cost of labour for all engineering changes made during the last period.</td></tr><tr><td>Engineering capacity (Flexibility, People)</td><td>Σ Calculated time used for engineering /Σ Available time for engineering calculated for the next period (e.g. a month)</td></tr><tr><td>Proportion of customer-specific products</td><td>All products are customer-specific and are assembled from modules. These modules are produced from both standard and special items.</td></tr><tr><td>Engineering Change urgency grade</td><td>Three EC urgency grades: special orders (for customizing products), quality ECs (carried out quickly) and regular ECs to reduce cost (planned in advance).</td></tr><tr><td>Engineering change dependency</td><td>Most EC orders include items that are dependent on other EC orders; there is thus usually a high degree of EC dependency</td></tr><tr><td>Engineering change information quality</td><td>Information quality is sufficient but the materials planners sometimes lack appropriate or correct information. The materials planners do not receive the EC information until the phase-out date is confirmed, which can be too late.</td></tr><tr><td>Incoming delivery precision (Delivery precision)</td><td>Number of incoming deliveries received on time/Total number of incoming deliveries</td></tr></tbody></table>
<table><tr><td>Incoming delivery quality (Quality)</td><td>Number of incoming deliveries containing defective parts / Total number of incoming deliveries</td></tr><tr><td>Percentage rework (Quality)</td><td>Rework hours / Total production hours</td></tr></table>

The indicators are designed based on characteristics of the case company considering its unique and non-repetitive situation; they can be generally applied to other ETO/project- based production with similar characteristics. Therefore, it is recommended to take them into account when implementing performance measurement system for material planning in the case company as well as in other companies with similar ETO supply chain. Performance measures may require adjustments to serve the actual needs of a project performance tracking and evaluation. It can be time consuming for manager to measure every time new values for each project. For managers it is more suitable to assess performance of material planning activity from an operational point of view as this helps to measure and monitor day-to-day project operations. Moreover non-financial measures are preferred at operational level to give clear picture of current state of performance of production operations. Properly chosen operational measures can help to identify weak areas of production activities on which improvement initiatives should focus. Nevertheless, the operational measures do not provide shareholders with an overall picture of company's performance (e.g. return on investments) that is so important for shareholders.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Designing a performance measurement system</figcaption></figure>
# 4 Conclusion

The study revealed the necessity to develop an integrated performance measurement system for managing engineering changes affecting materials management. The measures found in this study could help managers to address challenges incurred due to engineering changes, which affect different operations, more specifically materials. Having received little attention in academic literature, this topic is an area with much potential for further research. The researchers therefore call for more studies describing the design process itself, and the actual resulting performance indicators that should be tailored to materials management-specific and engineering change situations.

## Acknowledgement

This research is part of LIFT, EFFEKT, and SUSPRO research projects carried out at NTNU. The authors thank the partners in the projects for facilitating this research work.

# 5 References

1. Andersen, B. & Fagerhaug, T. 2002. Eight steps to a new performance measurement system. *Quality Progress*, 35, 1125.

2. Armstrong, M. & Baron, A. 2005. *Managing performance: performance management in action*, CIPD publishing.

3. Bell, L. C. & Stukhart, G. 1986. Attributes of materials management systems. *Journal of Construction Engineering and Management*, 112, 14-21.

4. Bertrand, J. W. M. & Muntslag, D. R. 1993. Production control in engineer-to-order firms. *International Journal of Production Economics*, 30-31, 3-22.

5. de Leeuw, S. & van den Berg, J. P. 2011. Improving operational performance by influencing shopfloor behavior via performance management practices. *Journal of Operations Management*, 29, 224-235.

6. Denscombe, M. 2014. *The good research guide: for small-scale social research projects*, McGraw-Hill Education (UK).

7. Douwe P. Flapper, S., Fortuin, L. & Stoop, P. P. 1996. Towards consistent performance management systems. *International Journal of Operations & Production Management*, 16, 27-37.

8. Gelders, L. F. 1991. Production control in an 'engineer-to-order' environment. *Production Planning & Control*, 2, 280-285.

9. Gunasekaran, A. & Kobu, B. 2007. Performance measures and metrics in logistics and supply chain management: a review of recent literature (1995-2004) for research and applications. *International Journal of Production Research*, 45, 2819-2840.

10. Hicks, C. & Braiden, P. M. 2000. Computer-aided production management issues in the engineer-to-order production of complex capital goods explored using a simulation approach. *International Journal of Production Research*, 38, 4783-4810.
11. Hicks, C. & Pongcharoen, P. 2006. Dispatching rules for production scheduling in the capital goods industry. *International Journal of Production Economics*, 104, 154-163.

12. Jiao, J., Zhang, L. & Pokharel, S. 2007. Process platform planning for variety coordination from design to production in mass customization manufacturing. *IEEE Transactions on Engineering Management*, 54, 112-129.

13. Jonsson, P. & Mattsson, S.-A. 2008. Inventory management practices and their implications on perceived planning performance. *International Journal of Production Research*, 46, 1787 - 1812.

14. Jonsson, P. & Mattsson, S. A. 2003. The implications of fit between planning environments and manufacturing planning and control methods. *International Journal of Operations & Production Management*, 23, 872-900.

15. Kaplan, R. S. & Norton, D. P. 2001. Transforming the balanced scorecard from performance measurement to strategic management: Part I. *Accounting horizons*, 15, 87-104.

16. Lin, G. & Shen, Q. 2007. Measuring the performance of value management studies in construction: critical review. *Journal of Management in Engineering*.

17. Otley, D. 1999. Performance management: a framework for management control systems research. *Management accounting research*, 10, 363-382.

18. Parthiban, P. & Goh, M. 2011. An integrated model for performance management of manufacturing units. *Benchmarking: An International Journal*, 18, 261-281.

19. Sjøbakk, B. & Bakås, O. 2014. Designing an engineer-to-order performance measurement system: a case study. *Advances in Production Management Systems. Innovative and Knowledge-Based Production Management in a Global-Local World*. Springer.

20. Stevenson *, M., Hendry, L. C. & Kingsman †, B. G. 2005. A review of production planning and control: the applicability of key concepts to the make-to-order industry. *International Journal of Production Research*, 43, 869-898.

21. Theeranuphattana, A. & Tang, J. C. 2007. A conceptual model of performance measurement for supply chains: alternative considerations. *Journal of Manufacturing Technology Management*, 19, 125-148.

22. Vollmann, T. E., Berry, W. L. & Whybark, D. C. 2005. *Manufacturing Planning and Control Systems for Supply Chain Management*, New York, McGraw-Hill.

23. Wänström, C. & Jonsson, P. 2006. The impact of engineering changes on materials planning. *Journal of Manufacturing Technology Management*, 17, 561-584.

24. Yin, R. K. 2009. *Case study research: design and methods*, Thousand Oaks, California, Sage Publications.