# Metrics Related to Confusion Matrix as Tools for Conformity Assessment Decisions
## 1 Introduction  
The guidelines for conformity assessment of an item of interest were adopted by the Joint Committee for Guides in Metrology (JCGM) and are given by guide 106:2012 [1]. The item of interest must meet certain specifications prescribed by the norms or given externally by the manufacturer. The fulfilment of these requirements mainly refers to the fact that the value of the tested characteristic of the specific tested item must be within the set limits of the tolerance interval. The decision on the compliance of an item of interest with the given standards can be made by using a simple binary decision rule. The item of interest is conformed with the specifications if the measured value of the characteristic under test lies within the tolerance interval. Otherwise, the item of interest does not comply with the specifications. When applying a simple binary rule, apart from these two outcomes, there are no other outcomes that would indicate the state where no decision can be made for the property of interest. For a typical product obtained in a production process, the probability of rejecting a conforming product is called a specific producer's risk. The probability of accepting a non-compliant product is called specific consumer risk. If the nominal value of the measurand and measurement uncertainty for the performed measurements are known, both probabilities can be calculated using conformance probability \(p_{\mathrm{C}}\)[2]. The measurement result is complete if it comprises the measured value and the associated measurement uncertainty. Measurement uncertainty represents the quality of the measurement result. Only the result with measurement uncertainty can be used for comparing results with specifications. However, uncertainty associated with the measurement results may lead to incorrect decisions. Relying on a simple binary decision rule and measured values from a normal distribution can result in a probability of accepting a nonconforming item or rejecting a conforming item as low as 50% [3; 4; 5; 6]. This happens when the measurement uncertainty of the measured item is high, and the measured value is near the upper or lower limit of the tolerance interval (Figure 1). To reduce this type of risk, an acceptance interval is introduced in the conformity assessment procedure [7; 8; 9]. The tolerance and acceptance interval can be in different relations [10]. In the case of a simple binary decision rule, the limits of the tolerance interval and the acceptance interval coincide (Figure 2b). That decision rule is also called shared risk. To decrease the producer's risk, the acceptance interval has been set to comprise the tolerance interval (Figure 2a). If the tolerance interval comprises the acceptance interval, the intention is to minimize the consumer's risk (Figure 2c). By introducing acceptance intervals, the concept of conformity assessment is extended to more general decision problems. In this case, four different outcomes are possible. Valid acceptance occurs when the measured value is within the acceptance interval and satisfies the specifications. Valid rejection occurs when the measured value is outside the tolerance interval and outside the acceptance interval. In terms of machine learning, valid acceptance is equivalent to the term true positive (TP), and the term valid rejection is equivalent to the term true negative (TN). If an item is accepted but does not conform to specifications, this is called false acceptance (FA). False acceptance probability refers to the global consumer's risk. If the measurement result is rejected, although it conforms to a specification, this is a falsely rejected measurement (FR). The probability of this incorrect decision is called the  
Figure 1: An example of a situation where the measured value has a high measurement uncertainty whose value exceeds the lower limit of the tolerance interval [4]. The symbols \(T_{L}\) and \(T_{U}\) in figures are marks for the lower and upper limits of the tolerance interval, respectively.