# Metrics for Assessing Product Variety Utilization

Thomas Ditlev Brunø, Kjeld Nielsen, Kaj A. Joergensen, Stig B. Taps

► To cite this version:

Thomas Ditlev Brunø, Kjeld Nielsen, Kaj A. Joergensen, Stig B. Taps. Metrics for Assessing Product Variety Utilization. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2014, Ajaccio, France. pp.328-335, 10.1007/978-3-662-44733-8_41. hal-01387265

HAL Id: hal-01387265

https://inria.hal.science/hal-01387265

Submitted on 25 Oct 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Metrics for Assessing Product Variety Utilization

Thomas Ditlev Brunoe, Kjeld Nielsen, Kaj A. Joergensen, Stig B. Taps.

Department of Mechanical and Manufacturing Engineering, Aalborg University, Denmark

tdp@m-tech.aau.dk

**Abstract.** In mass customization, offering the right variety is critical and it is therefore proposed to develop an assessment system for the capabilities critical for mass customization success. This paper proposes different metrics and methods for assessing the utilization of a company's product variety. Two different methods are selected for further testing which is done on historical data for three different product families from three different companies. It is concluded that different metrics and methods may be relevant for different products, reflecting variety and complexity. However, in general, monitoring utilization of variety has potential to improve business for mass customization companies.

**Keywords:** Mass Customization, Assessment, Variety, KPI, Metrics

## 1 Introduction

In any company, it is essential to offer products, which match the needs and desires of customers to achieve sales and profit. This is true for mass producers as well as mass customizers; however, in mass customization this issue is somewhat more complex than mass production due to a much higher variety and a more complex product structure. As pointed out by Salvador et al.[9], mass customizers need three fundamental capabilities to be successful: 1) Solution Space Development – Identifying the attributes along which customer needs diverge, 2) Robust Process Design – Reusing or recombining existing organizational and value chain resources to fulfill a stream of differentiated customer needs and 3) Choice Navigation – Supporting customers in identifying their own solutions while minimizing complexity and the burden of choice [4], [9].

In order for companies to be able to establish themselves as mass customizers or for existing mass customizers to improve performance, we proposed that an assessment system for Mass Customization performance is established [5], [6].

An essential element in mass customization is product variety. Variety is what differentiates mass customization from other business strategies, however the variety offered to the customer, the solution space, must be carefully designed, since a too high variety will imply higher costs and a too low variety will imply lost sales, since customers will not be able to buy the product matching their individual requirements [8]. Hence when developing an assessment system for MC, metrics addressing the size of the solution space and whether this size fits with the demand for variety is essential,
since variety is a major driver for both cost and revenue. A number of metrics for this purpose have been proposed all related to solution space development [3]. These metrics however cover a broad area of capabilities related to solution space development, including the solution space at a certain point in time as well as dynamics of solution space development.

The metric “Used variety” (UV) introduced by Piller [7], referenced from [1] addresses how well the solution space is utilized by the customers, i.e. how much variety is offered vs. how much does actually make sense compared to the customers’ requirements. The metric is calculated as indicated in the formula below:

$$UV = \frac{\text{Number of perceived variants}}{\text{Number of all possible variants}} \quad (1)$$

Using this metric may be difficult in practice, since the number of perceived variants is not readily available, as well as the number of all possible variants may be difficult to determine for various reasons, which will be addressed below.

The research question of this paper is: *How can the utilization of a mass customizer's solution space be measured?* The research question will be addressed with a basis on the “Used Variety” metric presented above, however due to certain practical issues in implementing this metric directly, further considerations need to go into defining the metric more specifically, which is done applying the method outlined below. The reasons for assessing the solution space include enabling mass customizers to optimize their solution space as well as their configuration systems, which both influence the performance in relation to solution space utilization.

## 2 Method

To answer the research question, a number of sub questions will need to be answered. In order to do this, the following methodical approach has been applied:

1. Clarify the purpose of the metric and based on this evaluate the feasibility of the originally proposed form of the metric.

2. Identify variables, which should be used to define and calculate the metric.

3. Evaluate the availability of data necessary for calculating a value for the metric.

4. Based on findings above, propose alternative methods to calculate the metric.

5. Verification of feasibility of calculating the metric by applying the proposed methods on different data sets containing actual historic product data.

6. Evaluate results from verification and identify the most suitable method in relation to the purpose identified in step 1.

## 3 Purpose and evaluation of existing metric

The basic purpose of the metric UV is to evaluate the utilization of a company’s solution space. The reason to do so is that variety comes at a cost, and variety, which is not being sold, does not generate income and may imply economic loss. The reasons
for variety not being utilized can be very different but relating to the fundamental capabilities of MC, we expect low utilization to be due to either lack in capability for solution space development or choice navigation. Low utilization due to problems within solution space development will imply that a company has developed variety, which is simply not demanded by customers, i.e. development of the wrong products. On the other hand, utilization issues due to poor choice navigation imply that the company may have exactly the right variety, but is not able to guide the customer towards buying the product that matches the specific requirements. Either way, the result is that companies have made investments in introducing variety, which is not paying off.

Hence the metric should be able to assess to what extent the variety offered to customers is actually being utilized, i.e. sold, in order to enable companies to react to low utilization. The metric should be applicable as a KPI and should thus be able to track utilization over time. In order to track performance over time, the KPI should be calculated on a regular basis and therefore it should be possible to calculate based on readily available data from the company's IT systems and not require expensive market surveys or qualitative assessments from the company.

The way of assessing the utilization of variety proposed by Piller [7] using the metric Used Variety implies identifying the *number of perceived variants* and dividing this by the *number of all possible variants*. In general, assessing variety based on the measure “number of variants” is intuitively an obvious choice since this is based on a simple count. However, in many cases this will be impractical. One of the reasons for this is that in mass customization, the number of theoretical variants easily becomes astronomical. For example, when configuring a Mini Cooper online the configuration choices presented to the customer will result in a number of possible variants well above a 20-digit figure. Another issue that may arise when determining the number of possible variants is how to count continuous configuration variables, e.g. the height and width of a window, which can be any value within a predefined interval or custom printed T-shirts, which can have any image printed on them. In these cases, it makes little sense to determine the number of possible variants, since this figure is literally infinite. This is also the case for products composed by predefined modules or components, which can be combined in various ways to configure a product, where the number of modules included in the configuration is not bound. In this case, the number of possible variants will also literally be infinite. Hence, when determining the value of the UV metric the value will in many cases approach a zero value, as the denominator will be virtually or literally infinite. Furthermore considering an example where a given solution space is expanded with one extra binary option, this will effectively double the number of possible variants (given this option is offered for all products). This again implies that the value of the UV metric will be halved. Considering again the Mini Cooper example, adding an option to include floor mats will double the number of possible variants, thereby halving the value of the UV metric, even though the actual change in product variety would clearly not imply that the utilization of variety is reduced by 50%.

The numerator of the UV metric – number of perceived variants – may be determined in a number of different ways. The term “perceived” implies that the number should rely on the perception of the customer, which naturally is difficult to determine quantitatively. However, a reasonable way of measuring number of perceived variants is to
assume this is equal to the number of different products (configurations) that have been sold over a period. If assuming this, it will be feasible in many cases to calculate an actual value for this metric, however as indicated above the value may not be a very good representation of the actual utilization of the solution space.

# 4 Variables & calculation methods

As described by Brunoe et al. [2], the utilization of a product family's solution space can be addressed using set theory. The solution space can be described using three sets; Configurable variety, configured variety and ordered variety. The set configurable variety contains all possible product variants, whereas configured variety contains all variants that have been configured and ordered variety contains all variants that have been configured and subsequently ordered. To assess the utilization of a solution space, it must be addressed what the relationship is between the set "configurable variety" and the set "ordered variety, i.e. a small difference between the two sets implies a good utilization of the solution space. However to assess the difference between the two sets, it must be determined what are the elements of the sets.

One possibility is to define the elements in the sets as product variants, which corresponds to the approach suggested by Piller [7] for calculating the "used variety" metric. This is in some cases a simple way of measuring the utilization of the solution space, however as pointed out above, there are certain issues in other cases. Calculation of the solution space utilization (SSU₁) can be done by applying the formula below:

$$SSU_1 = \frac{\text{Number of distinct configurations}}{\text{Number of different product variants}} \qquad (2)$$

This form of the metric differs from the one defined by Piller [7], since the numerator simply counts the number of distinct configuration. The data needed to calculate the value of this metric is expected to be readily available, assuming the company has a product configurator, where the product family model defining the solution space provides the information needed for calculating the number of different product variants (simply by calculating the product of number of possible values for each configurable variable).

Another possibility for defining the elements of the sets is to let each outcome of each variable in a product configurator represent an element. E.g. if a configurator allows you to choose a product in three different sizes and three different colors, that would correspond to six elements. We define these elements as "configuration options". I.e. a configuration option corresponds to a specific value for a specific variable. Using the configuration options, the number of elements will not become astronomical when representing the solution space, but product configurators will typically contain configuration options in the order of hundreds or thousands. It is then possible, to calculate the solution space utilization based on the number of configuration options (SSU₂):

$$SSU_2 = \frac{\text{Number of different configuration options chosen in configurations}}{\text{Number of different configuration options in solution space}} \qquad (3)$$
This metric will also be able to assess the utilization of the solution space, however without the drawback of using the SSU₁ metric of e.g. reaching near zero values for larger solution spaces and being sensitive towards introduction of simple variety. On the other hand, if the sales of a product family reaches a certain level, it is expected that the value of the SSU₂ metric will approach one, as it is likely every configuration option will be chosen by a customer at least once. This leads to the issue regarding this way of measuring solution space utilization, which is that a configuration, which is chosen only one time by a customer, will count the same as a configuration option, which has been chosen thousands of times by customers. Clearly, a configuration option chosen thousands of times indicates a better utilization than a configuration option chosen only once, which this metric does not take into account.

Another way of assessing the utilization, using the configuration options, but taking the issues described above into account, would be to calculate the frequency by which each configuration variable is chosen by a customer. By doing this, it can be determined for each configuration variable, how well it is utilized. Furthermore, to assess the solution space as a whole, a distribution of the frequencies can be determined and illustrated in e.g. a histogram, which will indicate whether all configuration options are chosen similarly often, or some configuration options are chosen almost in every configuration whereas others are chosen very rarely. To represent this distribution, mean values and standard deviation can also be calculated for the frequencies for which configuration options are chosen. Although this way of assessing utilization of the solution space is not a metric in it's simplest form, i.e. a metric which can be calculated as a single figure, we include it in this paper as it gives a deeper insight than the other simpler metrics. The data used for calculating a value for this metric is identical to the SSU₁ metric

In this paper, the two methods – SSU₁ and determining the frequency and distribution of configuration options being chosen are tested on data from three different products from three different companies all producing mass customized products. The metric SSU₂ has not been included in the test because of the arguments presented above and since initial test showed that, the metric for all product families would reach the value 1.

# 5 Data

The data used for testing the different metrics and calculation approaches have been obtained from three different companies. The product families represent three entirely different product types in terms of application and complexity. Product family A is a family of mass customized capital goods equipment of high complexity. The product family is close to being Engineer to Order products and have thus a very large product variety, however they are standardized and can thus be considered mass customized products. Product Family B is a family of products, which are used as sub components of larger systems in the process industry. The products have medium variety and are customized to meet individual demands, by combining different components. The complexity of these products is relatively low. Product Family C is a family of products for
domestic utilities. The product family has medium variety but are slightly more complex than the products in family B but still much less complex compared to family A. Data obtained for the three product families consists of a product family model defining the variety, i.e. variables and their values, enabling determining the total number of variants and configuration options. Furthermore, historic data describing specific configurations which had been sold over a one to two year period was obtained for determining what variety had actually been configured.

# 6 Results

The results of testing the SSU<sub>1</sub> metric are shown in table 1. At first glance, it can be seen that all three product families have SSU<sub>1</sub> metric values very close to one, although product family A has a significantly lower value for the metric, for which the major reason is that the number of possible variants is significantly higher. Furthermore, it can be seen that the number of distinct configurations sold are not particularly high. This is due to the products being B2B products and thus not being sold in volumes comparable to consumer products. Furthermore, for families B and C it indicates that sales includes much repurchase of previously purchased configurations. I.e. one configuration is sold in larger volumes and in multiple orders.

<table><caption>**Table 1.** Results from applying metric SSU<sub>1</sub></caption><thead><tr><th>Product Family</th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><td>Distinct configuration sold</td><td>409</td><td>157</td><td>26</td></tr><tr><td>Number of possible variants</td><td>4,26E+61</td><td>1,91E08</td><td>9,17E09</td></tr><tr><td>Value of SSU<sub>1</sub> metric</td><td>9,59E-60</td><td>8,19E-07</td><td>2,83E-09</td></tr></tbody></table>

Table 2 and Figure 1 show the results of analyzing the frequencies of which configuration options are chosen by customers. In figure 1, the frequencies are divided into 20 even intervals. The bars in the histogram indicate the number of configuration options, which have been chosen the number of times, which corresponds to the frequency interval. E.g. the first bar for product family A indicates that 358 configuration options have been chosen by customers between 0 and 22 times. The number below each bar indicates the upper bound of the interval.

<table><caption>**Table 2.** Results from analysing frequency of which configuration options are chosen</caption><thead><tr><th>Product family</th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><td>Mean frequency</td><td>77,0</td><td>52,1</td><td>44,6</td></tr><tr><td>Mean frequency relative to no. of configurations</td><td>0,231</td><td>0,156</td><td>0,133</td></tr><tr><td>Std. Dev</td><td>109,6</td><td>79,28</td><td>62,48</td></tr><tr><td>Std. Dev relative to no. of configurations</td><td>0,267</td><td>0,183</td><td>0,345</td></tr></tbody></table>

Table 2 also reflects an analysis of the frequencies of configuration options being chosen, where the mean frequency and standard deviation are calculated. The mean
frequency for e.g product family A indicates that configuration options are in average
chosen 77 times by customers. The mean is also related to the number of configurations
in row two in the table, where the value 0.231 indicates that the average configuration
option is chosen in 23.1 % of the configurations. The table also shows the standard
deviation for the frequencies, where it is notable that the standard deviation in all three
product families is larger than the mean value indicating very spread frequencies, but
also a long tail of configuration options where a few configuration options are chosen
very frequently and a high number is chosen very rarely, which is also seen in figure 1.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Results from analysing frequency of which configuration options are chosen illustrated as distribution histograms. X axis labels indicate the upper bound of the interval.</figcaption></figure>
# 7 Conclusion

The basic idea of the Used Variety metric [7] is intuitively good, however insuffi-
ciently described to operationalize it. Using the SSU₁ metric a very similar measure can
be calculated, however testing on actual data showed that the method did not provide
very useful information on utilization of product variety. The metric may however be
useful in cases with low variety or in cases with a lower number of predefined product
variants. Another method for assessing variety was tested, where the frequency of
choosing a configuration option is counted, resulting in a distribution of frequencies,
which indicates if there is a large spread in the utilization of configuration options and
if there is a large part of the product variety which is rarely configured and thus poten-
tially unprofitable.

The data used to test the methods is obtained from three different companies, how-
ever all operation in a B2B setting, limiting volume. The results may have been some-
what different if data were included for consumer products as well, however in terms
of assessing the applicability of the method, we consider the testing done in this work
as sufficient to draw the conclusions stated above.

Further consideration needs to go into the practical application of the methods, e.g.
over how long time should data be sampled? Using data for 1 month will reveal a sig-
nificantly different result than data for half a year. Furthermore, not all configuration
variables may be relevant to include, since some may be redundant, if the configurator
as an example contains variables for both user requirements and product characteristics.

In general, we expect the application of product variety assessment tools as a valua-
ble tool for mass customizers in order to optimize their business in terms of determining
the right level of product variety and diagnosing the configuration process.

References

1. Blecker, T., Abdelkafi, N., Kaluza, B. et al.: Key Metrics System for Variety Steering in Mass Customization. Munich Personal RePEc Archive, (2003)

2. Brunoe, T. D., Nielsen, K., Joergensen, K. A. et al.: Describing Product Variety using Set Theory. (2014) 105-114

3. Brunoe, T. D., Nielsen, K., Joergensen, K. A.: Solution Space Assessment for Mass Customization. MCP-CE 2012, (2012) 56

4. Lyons, A. C., Mondragon, A. E. C., Piller, F. et al.: Mass Customisation: A Strategy for Customer-Centric Enterprises. Customer-Driven Supply Chains, (2012) 71-94

5. Nielsen, K., Brunoe, T. D., Joergensen, K. A. et al.: Mass Customization Measurements Metrics. (2014) 359-375

6. Nielsen, K., & Brunoe, T. D.: Mass Customisation Assessment and Measurement Framework. In: Anonymous Enabling Manufacturing Competitiveness and Economic Sustainability, pp. 165-170. Springer (2014)

7. Piller, F. T.: Logistische Kennzahlen Und Einflussgroessen Zur Performance-Bewertung Der Mass-Customization-Systeme Von Selve Und Adidas. (2002)

8. Pine, B. J.: Mass customization: The new frontier in business competition. Harvard Business School Press, Boston, Mass. (1999)

9. Salvador, F., De Holan, P. M., Piller, F.: Cracking the Code of Mass Customization. MIT Sloan Management Review, 50 (2009) 71-78