Discussion and conclusion - Judgement devices and the evaluation of singularities: the use of performance ratings...

Discussion and conclusion
The focus of our study was to examine how management accounting information is used in the evaluation of singularities. Following Karpik (2010), we define singularities as everyday goods and services that are unique, multidimensional, incommensurable, and of uncertain quality, and we focus specifically on the example of films, in order to address the following research question: how are ratings and narrative information implicated in choices about the value individuals ascribe to singularities?
6.1 Primacy of ratings, and the manner of their use
Our findings highlight the importance individuals ascribed to performance ratings in making decisions about which film to see. It was clear that the IMDb rating of the film was commonly weighted heavily in the decision to choose a particular film for viewing over another. This finding is consistent with research that has focused on the significance afforded to numbers in certain settings, or as Jeacle and Carter (2011: 301) describe it, “…the power of the number” (see for example, Jeacle and Carter, 2011; Miller, 2001; Porter, 1995; Robson, 1992). This body of research points to the appealing properties that quantification can provide to users of information. In the case of a user rating of hotels (within TripAdvisor), Jeacle and Carter (2011: 301) note that it “…instantly labels the perceived quality of an establishment and that number is invested with credibility all the more so because it was constructed from the experiences and seemingly honest opinions of fellow travellers.” Our findings are consistent with this assertion, in that film viewers did seem to be heavily influenced by the IMDb ratings, (which importantly, do not directly measure any one dimension of a film), and that in doing so, these film viewers relied to a considerable extent on the ‘wisdom of the masses’ in making a decision. However, our findings extend this literature in two ways, first, by explaining how numbers (in this case, ratings) are used in conjunction with other (often conflicting) judgement devices to make decisions, and second, by considering how decisions were made in circumstances where ratings did not provide a clear signal as to the quality of a singularity.
In relation to this first point, our analysis indicates that IMDb ratings were used in two main ways. The first of these was a screening tool, whereby the decision maker indicated that they would not watch films that did not achieve at least a certain rating. The second was to use the film’s rating score as a tie-breaker in the presence of other conflicting information.
Regarding the second point, despite the importance given to numbers embedded in rating scores, the use of these ratings as either an initial screening tool prior to the consideration of other judgement devices, or as a ‘tie-breaker’ in the presence of conflicting information points to the fact that users, in the main, relied on multiple judgement devices. Users acknowledged that decisions made on the basis of relying on a single judgement device often resulted in unsatisfactory outcomes. We see this findings as being analogous to a wealth of performance measurement literature that focused on the importance of relying on multiple measures, rather than a single measure of performance (see for example, Kaplan and Norton, 1992, 1996; Nørreklit, 2000).
Where our findings differ from this prior research (and indeed, from Karpik, 2010) is in relation to our finding that some film viewers avoided consulting too many judgement devices prior to watching a film in order to avoid learning too much about a film before viewing it. This finding is inconsistent with Karpik’s (2010) argument that individuals will seek as much relevant information provided by judgement devices as possible in order to reduce uncertainty about the singularity’s quality. In our setting, it was considered important by some film viewers to tolerate some level of quality uncertainty (Karpik, 2010) in order to enhance their viewing experience.19
|