Evaluating automotive sound quality : the disconnect between market research and structured evaluations

Understanding how customers evaluate product attributes is a vital part of New Product Development (NPD). Vehicles in particular have many attributes which can contribute to a positive perception of the vehicle and its brand. The attribute of vehicle Sound Quality (SQ) is particularly important and automotive Original Equipment Manufacturers (OEMs) employ a variety of techniques to collect customers’ subjective evaluations of SQ in NPD, both during product development and after purchase. There are two main techniques for collecting customers’ subjective evaluations: structured evaluations and market research. Structured evaluations are conducted in controlled experimental conditions. This allows engineers to set targets for the design of automotive SQ for new vehicles. Market research on the other hand, is carried out in an unstructured manner, and does not inform engineers of the underlying criteria vehicle owners use to evaluate vehicle SQ. Nevertheless, market research can be influential for attracting new customers, therefore it is important for OEMs to achieve favourable ratings. However, it is currently not understood how market research methods compare to structured evaluations for automotive SQ, which leads to low confidence in interpreting customer data. A preliminary study examined customer data issues facing an automotive OEM and confirmed the need to further understand vehicle owners’ decision-making and behaviour in evaluations. Therefore, this research aims to answer the question, how do the decision-making criteria used by assessors compare between market research techniques and structured evaluations? By adopting a psychological approach, a second study was conducted to examine customers’ decisionmaking in automotive SQ structured evaluations and in a market research survey. Verbal Protocols were used as the primary data collection method. The second study identified four decision-making criteria which were used by assessors when evaluating automotive SQ in a listening room structured evaluation and a market research survey. The criteria were classified into 1) behavioural scenarios, 2) attribute criteria, 3) comparisons and 4) expectations. An Odds Ratio showed vehicle owners in a market research setting were twice as likely to use behavioural scenarios in comparison to a structured evaluation. Vehicle owners in the structured evaluation, made more comparisons to specific stimuli that were experienced and focussed on the sound stimulus presented as opposed to behavioural scenarios. A third study investigated customer decision-making in an interactive vehicle simulator, which was most representative of real-life driving. Behavioural scenarios were used by vehicle owners, in a similar frequency as those in a market research survey, which validates the usage of simulators in NPD. Simulators are therefore important tools to help experts anticipate how vehicles will be evaluated in market research. This thesis provides experts with the knowledge of how vehicle owners evaluate their vehicles in market research surveys and as a result it can inform the design of structured evaluations, which occur earlier in NPD, before the post-purchase market research has taken place. This thesis demonstrates a behavioural disconnect between structured evaluations and market research techniques. It provides knowledge on the decision-making criteria which vehicle owners use to evaluate vehicle SQ in structured evaluations and market research. This knowledge can ultimately give engineers more confidence in interpreting customer data and the ability to better anticipate customer responses, through a better understanding of how vehicle owners evaluate vehicle SQ, rather than just knowing their preferences. It also validates the use of vehicle simulators in NPD.