# A simple procedure to assess the reliability of the best estimate thresholds (BETs): A case study on the detection threshold of tannic acid astringency

Christophe Martin, Alix Rollinat, Carole Tournier, Caroline Peltier

► To cite this version:

Christophe Martin, Alix Rollinat, Carole Tournier, Caroline Peltier. A simple procedure to assess the reliability of the best estimate thresholds (BETs): A case study on the detection threshold of tannic acid astringency. Food Quality and Preference, 2025, 127, pp.105437. 10.1016/j.foodqual.2025.105437. hal-04926205

HAL Id: hal-04926205

https://hal.inrae.fr/hal-04926205v1

Submitted on 3 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# A simple procedure to assess the reliability of the best estimate thresholds (BETs): A case study on the detection threshold of tannic acid astringency

Christophe Martin<sup>a,b,*</sup>, Alix Rollinat<sup>a,b</sup>, Carole Tournier<sup>a,b</sup>, Caroline Peltier<sup>a,b</sup>

## 1. Introduction

The detection threshold, i.e., absolute threshold, is a valuable measurement to determine whether aroma or taste-active compounds can influence the sensory properties of a product or cause a sensory defect (Lawless & Heymann, 1999). In this context, it is used to measure the substance properties (ISO, 2018), and the detection thresholds of numerous molecules responsible for aroma or taste have been evaluated in various media (Eisele & Semon, 2005; González Viñas et al., 1998; Jaeger et al., 2014; Piornos et al., 2019). Detection threshold measurements can also be used to indicate the taste or olfactory sensitivity of human subjects to specific stimuli (ISO, 2018). In this case, the detection threshold can be considered a performance index to study inter-individual variability or form groups with contrasting sensitivities (Meilgaard, 1993; Schoumacker et al., 2017).

There are several methods to measure detection thresholds, including ascending forced-choice method of limits (ASTM, 2011; ISO, 2018), adaptive methods (Harvey, 1986), rated difference from control (Lundahl et al., 1986; Chapman et al., 2006), as well as procedure derived from signal detection theory (Lee & Van Hout, 2009; Macmillan & Creelman, 1991). The principles underlying these methods have been described in various methodological manuals (Lawless & Heymann, 1999; Meilgaard et al., 2006). This diversity of methods to determine the detection threshold complicates the comparison of results across different studies. Indeed, threshold values obtained using different methods can fluctuate depending on the complexity of the task that the subjects perform (Lawless & Heymann, 1999). Therefore, the methods must be standardized. To address this issue, the American Society for Testing and Materials (ASTM) and the International Organization for Standardization (ISO) have proposed standardized methods to estimate absolute thresholds for odors and tastes (ASTM, 2011; ISO, 2018).

The rapid method proposed by ASTM is highly popular and recommended by leading methodological manuals (Lawless & Heymann, 1999; Meilgaard et al., 2006). This method, which is described in

standard E679–04 (ASTM, 2011), is the forced-choice ascending concentration series method. This method involves a series of three-alternative forced choice (3-AFC) presentations, each of which contains one test sample and two reference samples. The concentration of the test sample is increased by a constant factor (geometric increments) across the series of 3-AFC tests. To evaluate the detection threshold, participants are asked to identify the sample that differs from two other samples (forced choice). Individual best estimate threshold (BET) values are calculated from the pattern of correct and incorrect responses produced by each subject. Group thresholds are determined by averaging the logarithms of the individual BETs (ASTM, 2011).

The ASTM E679–04 method offers several advantages. It is time-efficient, since each subject receives only one presentation per concentration, so many subjects can be included (ASTM, 2011). The speed of the procedure prevents participant fatigue during prolonged testing, optimizes the feasibility of recruiting large population subgroups, and facilitates the differentiated threshold analysis (Giguère et al., 2016). However, the method has significant limitations, notably two major biases. First, the BET results can be biased due to the probability of success by chance, which is 1/3 for a single success and 1/9 and 1/27 for two and three consecutive successes, respectively (Hough et al., 2013; ISO, 2018; Lawless et al., 2010; Peng et al., 2012; Piornos et al., 2019). Second, this method carries risks because with only one presentation per subject, an above-threshold sample can be misinterpreted due to confusion or inexperience with the stimulus or test procedures (ISO, 2018). According to Lawless (2010a), a highly sensitive individual may detect low levels early in the series but can adapt, tire, or be overwhelmed by higher levels, so they can miss a target well above their threshold. This type of error is particularly detrimental to the BET accuracy when it occurs at the highest concentrations and potentially causes significant underestimation.

Several solutions have been proposed to address the bias associated with the correct answers given by chance. These solutions involve correcting the panel thresholds obtained through various modeling

<footnote><sup>a</sup> Centre des Sciences du Goût et de l'Alimentation, CNRS, INRAE, Institut Agro, Université de Bourgogne, F-21000 Dijon, France</footnote>

<footnote><sup>b</sup> INRAE, PROBE research infrastructure, ChemoSens facility, F-21000 Dijon, France</footnote>

<footnote>* Corresponding author at: Centre des Sciences du Goût et de l'Alimentation, CNRS, INRAE, Institut Agro, Université de Bourgogne, F-21000 Dijon, France. E-mail address: christophe.martin@inrae.fr (C. Martin).</footnote>
approaches, such as survival analysis (Hough et al., 2013) and simple interpolation of the correct percentage adjusted for chance (Lawless et al., 2010). Methods based on measuring the degree of certainty associated with each 3-AFC test were also proposed to minimize the bias associated with correct answers given by chance (Kim et al., 2011; Kim et al., 2020; Lawless, 2010). The principle of this correction involves increasing the BET by one dilution factor if the participant is unsure of his or her choice for the 3-AFC test to calculate the BET.

The above-mentioned methods fulfill their respective purposes. However, the accuracy of their measurements highly depends upon the initial data quality. Specifically, they cannot minimize the second bias, i. e., errors can arise from participant inattentiveness, non-compliance with the sample evaluation protocol, or adaptation phenomena during the 3-AFC test series. ASTM strongly advocates for comprehensive participant training prior to conducting the final measurements (ASTM, 2011). Repeating measurements is also recommended. However, complete and efficient panel training or repeated measurements are time-consuming and not always feasible, especially when many subjects are involved. A method to rapidly assess the quality of data collected according to the ASTM E679-04 protocol would be extremely helpful, particularly by identifying individual BETs that can be called into question. Such a method would also be particularly useful in studies that involve screening many subjects, for example, to select groups of highly sensitive or low-sensitivity individuals.

This article aims to present a simple method to provide a reliability index for thresholds estimated via procedure E679-04 (ASTM, 2011). This method enables experimenters to identify potentially biased individual BETs. After they have been identified, based on a detailed examination of the results, the experimenter can decide to discard the data from subjects whose thresholds are potentially biased or to repeat the measurements. A more automatic use of the reliability index, which is particularly useful for measurements on a large number of subjects, is also presented in this document. The proposed method should be seen as a complement to ASTM procedure E679-04, and does not exclude the possibility of applying the corrections proposed by Hough et al. (2013) and Lawless (2010a) on BETs with satisfactory reliability.

After the principle of the method has been presented, a case study is detailed, followed by a general discussion. The case study concerns the measurement of the detection threshold for tannic acid, which is responsible for the sensation of astringency. The study involved 193 subjects with no significant experience in sensory analysis.

## 2. Method

### 2.1. Principle summary

The proposed method consists in supplementing the ASTM E679-04 procedure with an evaluation of the intensity of the sample identified as different from the other two in each 3-AFC presentation. The sensation whose intensity is to be assessed corresponds to the chemical stimulus whose detection threshold is to be evaluated (e.g., astringency of tannic acid in the case study below). Then, the intensity data are used to calculate an intensity-based threshold (IBT) for each participant, which is compared with the BET calculated from the pattern of responses provided during the 3-AFC tests. The difference between individual BET and IBT determine a Data Reliability Index (DRI), which is used as a criterion to identify participants whose threshold assessment is potentially unreliable. This reasoning assumes that the individual thresholds assessed by the two different methods should be as close as possible. If so, the results can be considered reliable, where each method validates the other. In this case, the individual BETs can be retained to calculate the group threshold or create groups of subjects with contrasting sensitivities. Meanwhile, if the difference is too great for a given participant, the reliability of his/her data (BET or IBT) is questioned, and it is necessary to examine them in detail to determine whether the data for the participant can be retained.

### 2.2. Changes in the data collection process

For each 3-AFC presentation, the subjects are asked to rate the intensity of the sample that is considered different from the other two. A discrete scale from 0 to 10 is used to rate the intensity. Fig. 1 shows an example of the questionnaire used.

### 2.3. Best estimation threshold calculation

The calculation of individual BETs and group thresholds remains unchanged and strictly complies with the ASTM E679-04 standard (ASTM, 2011). Individual BETs are derived from the pattern of correct/incorrect responses of each subject. The best-estimate threshold concentration for the panelist is defined as the geometric mean of the concentration at which the last error occurred and the concentration immediately above. If a subject correctly identifies a different sample at all 3-AFC test levels, the BET is the geometric mean between the lowest concentration and the previous hypothetical lower concentration. Conversely, if a subject fails at the highest concentration, the estimate is the geometric mean between the highest concentration and the next highest hypothetical concentration, assuming that the series has been extended. The group threshold is calculated by averaging the logarithms of individual BETs.

### 2.4. Intensity-based threshold calculation

The method for determining the IBT was inspired by the Beidler taste equation. This form of the Michaelis-Menten semi-hyperbolic equation is a widely applied dose-response relationship in studies of the chemical senses (Beidler, 1954; McBride, 1987; Lawless & Heymann, 1999, chapter 6, p. 173). The curve corresponding to this equation has an S-shape, with two inflection zones when plotted as a function of log concentration. The shape of this curve indicates that the response increases slowly relative to the background, then rises sharply to enter the middle of the dynamic range of the response. This first inflection zone is an empirical definition of the threshold (Lawless & Heymann, 1999).

Based on these theoretical considerations, the principle for determining the IBT involves identifying a significant increase in intensity starting from a certain concentration within the tested range. The segment of the curve exhibiting the most pronounced intensity increase is used to define this significant change, which is assumed to indicate the threshold being surpassed. The average of the logarithms of the first concentration (the starting point of the increase) and the second concentration (the onset of the increase) is used to estimate the inflection point of the intensity curve and determine the logarithmic value of the IBT (Fig. 2).

The first step is to calculate the increase in intensity observed for each strictly increasing section of the curve, which shows the evolution of intensity as a function of the tested concentration (logarithmic value). The sections to be considered may encompass several tested concentrations if the increase in perceived intensity is continuous. The next step is to identify the section of the curve with maximal increase. This section determines the IBT, which is obtained by calculating the mean of the logarithmic values of the concentrations of the first two points in the section. This way of interpolating the detection threshold is consistent with the way BET is determined. If this analysis results in two sections of the same amplitude, the first section (with lower concentrations) is used for the IBT calculation. Thus, the logarithmic value of the threshold determined from the measured intensities can be converted into the original unit (g/L in the case study below). This three-step procedure is relatively easy to automate using a statistical analysis software. The R code to calculate individual IBTs is provided here: https://github.com/Chemosens/AFCR.git.
Taste the three solutions in the order specified on the questionnaire, and identify the astringent sample. Tick the code corresponding to the selected sample, then rate the astringency intensity of this sample on the scale provided (0 = not perceived; 10 = very intense).

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Example of the questionnaire used for each 3-AFC presentation. In this example, the aim is to measure the detection thresholds for tannic acid. The sensation whose intensity is to be noted is astringency (see the case study below).</figcaption></figure>

## 2.5. Data reliability index calculation

The DRI is calculated for each subject by subtracting the logarithmic value of the BET from the logarithm value of the IBT. This index indicates the reliability of the obtained data. As previously mentioned, since the two methods validate each other, the closer this index is to 0, the closer the two thresholds are, and therefore the more robust the data can be considered. Conversely, a DRI too far from 0 should be considered as a signal for the experimenter to examine the data in detail (intensities and results of the 3-AFC test). Comparing the results of both methods can sometimes lead to question the appropriateness of BETs. In other cases, it is the intensity measurement that may be questioned (see the case study below).

For more automatic use, minimum and maximum values that define an interval of DRIs that seem acceptable to the experimenter can be defined. Then, data from subjects whose DRIs fall outside the defined interval should be discarded. The determination of the acceptable interval depends on the data analyzed, and the experimenter must consider the percentage of subjects eliminated due to an excessively high DRI. Furthermore, after subjects whose DRIs are outside the interval have been eliminated, the correlation coefficient between BET and IBT is an interesting element to consider when defining the DRI threshold. An example is provided in the case study below.

# 3. Case study: detection threshold of tannic acid astringency

## 3.1. Ethical statement

Ethical approval for the involvement of human subjects in this study was granted by Ethics Committee for Research (CPP Ile de France I 23.01785.000230-2023-A00291-44) and was conducted according to the French law (loi Jardé) and the Declaration of Helsinki.

All participants acknowledged an informed consent statement in

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Example of determination of the intensity-based threshold (IBT). The dotted parts correspond to sections of the curve where the intensity is continuously increasing. Sections may cover more than two concentrations. The section on the right of the graph is the one used to calculate the IBT, as it shows the greatest increase in intensity (+7). The IBT is obtained by averaging the logarithms of the concentrations of the first two points in this section.</figcaption></figure>

order to participate in the study. They were financially compensated for their participation in the amount of €10.

## 3.2. Experimental conditions

All evaluations occurred in sensory tasting rooms at room temperature and under red light. Each session involved 16 volunteers on average. Each participant participated in only one session. The average duration of a session was 50 min.

## 3.3. Subjects

To participate in the study, the subjects had to meet the following criteria: 18–40 years of age; not have food allergies, chronic pathologies, olfactory or taste perception disorders; not pregnant or breastfeeding; either be nonsmokers or smoke less than once a week.

In total, 193 subjects participated in the study. The panel included 65 % women and 35 % men. The average age of the participants was 31 with a minimum age of 18 years and a maximum age of 40 years.

## 3.4. Stimuli and rinsing solutions

Tannic acid (CAS number: 1401-55-4, Sigma-Aldrich, Germany) was used to prepare the astringent samples (test samples). Eight different concentrations were used from 0.015 g/L to 1.001 g/L (Table 1). A geometric increment of 1.821 was used to construct the range. The concentration range was established to include literature data on detection thresholds for tannic acid in water. According to the literature, the threshold of tannic acid in water varies from 0.02 g/L (Valentová et al., 2001) to 0.36 g/L (Linne & Simons, 2017).

Evian® mineral water was used for all samples without tannic acid (reference samples). The same water was used as a solvent to dilute the tannic acid (test samples).

All samples had a volume of 10 mL and were placed in small white plastic cups. Each cup was coded with a three-digit code.

A mouth rinsing procedure was used to limit the persistence of astringency. This procedure involved using a 0.1 % aqueous pectin solution (E440i, Meilleur du Chef, France) and a 1 % sodium bicarbonate solution (La Patelière, France). Evian® mineral water was used as solvent for both solutions.

## 3.5. Detection threshold measurement

At the beginning of the session, the subjects were briefed on all stages of the evaluation. In particular, the principle of the 3-AFC test and rating of the astringency intensity were explained. Fig. 3 shows the complete protocol of this study.

The subjects successively performed eight 3-AFC tests. For each 3-AFC test, the order of presentation of the three samples was balanced across the panel. Each 3-AFC test included a test sample (tannic acid diluted in Evian® mineral water) and two reference samples (Evian® mineral water). The tannic acid concentration of the test sample
<table><caption>**Table 1**
Tannic acid solutions: concentration range used (geometric increment: 1.821).</caption><thead><tr><th>Order of evaluation in the 3-AFC test series</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th></tr></thead><tbody><tr><td>Tannic acid (g/L)</td><td>0.015</td><td>0.027</td><td>0.050</td><td>0.091</td><td>0.166</td><td>0.302</td><td>0.550</td><td>1.001</td></tr></tbody></table>

increased with each new 3-AFC test (see Table 1). Each sample in the series of three was compared with the other two samples. The subjects were asked to indicate which of the three samples was astringent (forced choice) and to rate the intensity of astringency perceived in the sample that was considered different from the other two samples. Intensity was rated on an 11-level discrete scale from 0 (not perceived) to 10 (very intense). Fig. 1 shows the questionnaire for this purpose.

Specific precautions adapted to the nature of the studied compound, i.e., tannic acid, were implemented to optimize the quality of the evaluations. First, for the untrained public, astringency is often confused with other perceptions, particularly bitterness. Therefore, the session began with a comprehensive introduction to the sensation of astringency. This introduction included a detailed description of the perceived sensation and examples of distinctly astringent foods. To concretely illustrate the studied sensation, participants were given a 1-g/L aqueous tannic acid solution. Then, the subjects rinsed their mouths and took a break before the measurements began. Second, since aqueous tannic acid solutions become progressively turbid as they oxidize, the time between sample preparation and serving was minimized at 1–2 min. In addition, to ensure that no visual differences were perceptible between samples with and without tannic acid, all samples were evaluated under red light. Third, the sample evaluation procedure was standardized to ensure that all subjects performed the tests under identical conditions. According to Valentova et al. (Valentová et al., 2001, Valentová et al., 2002), the astringency of tannic acid distinctly appears between 5 and 25 s after placing the solution in the mouth. In the present study, the subjects were therefore asked to put the entire solution in their mouths, hold it there for 5 s, spit it out. Then, they were asked to wait 20–30 s before tasting the next sample. Finally, between each 3-AFC test, the participants were asked to take a break of 3 min, during which they were required to follow a rinsing protocol to minimize the residual astringency induced by the previous test samples. This rinsing procedure involved sequentially taking a sip of a 0.1 % pectin solution, a sip of a 1 % sodium bicarbonate solution, and a sip of Evian® mineral water. This rinsing method has proven effective in several other studies (Munoz-Gonzalez et al., 2019; Wang et al., 2022).

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Protocol to determine detection thresholds for tannic acid astringency (adaptation of the standard procedure ASTM E679–04).</figcaption></figure>

## 3.6. Results

### 3.6.1. Best estimate threshold

Fig. 3 shows the distribution of logarithmic values of individual BETs for the 193 participants. The values ranged from -1.954 (0.011 g/L) to 0.131 (1.351 g/L) with a median of -0.910 (0.123 g/L). The 25th percentile (lower quartile) and 75th percentile (upper quartile) were equal to -1.171 (0.07 g/L) and -0.660 (0.22 g/L), respectively. The mean of the individual values, which is also the group threshold, was -0.952 (0.112 g/L).

### 3.6.2. Intensity-based threshold

The IBTs for the 193 subjects ranged from -1.696 (0.020 g/L) to -0.130 (0.742 g/L), with a median value of -0.910 (0.123 g/L). The lower and upper quartiles of the IBT distribution were identical to those of the BET distribution. The mean IBT value and consequently the group mean threshold based on intensities was -0.906 (0.124 g/L). Fig. 4 shows the distribution of IBTs. A paired-sample t-test was performed to compare the thresholds obtained with the two methods. The results indicate no significant difference between the group thresholds determined from BETs and IBTs ($t(193) = -1.146, p = 0.253$).

### 3.6.3. Data reliability index

The DRI values were -1.567 to 1.564. Half of the values were between -0.260 and 0.264. The mean and median were 0.046 and 0, respectively.

A negative DRI value indicates that the BET was above the threshold determined by the intensities (IBT). This was the case for 28.5 % of the subjects. The opposite case, i.e., a BET lower than the IBT, was more frequent (38.3 % of subjects). For 33.2 % of the subjects, the DRI was zero, i.e., the BET and IBT were equal. Finally, for 115 subjects out of 193, i.e., approximately 60 % of the panel, the thresholds determined with the two methods differed by one concentration level or less (absolute value of DRI $\le$ 0.26).

### 3.6.4. Examination of different scenarios

The data from a few subjects are examined below to provide a better understanding of the mechanism to assess the reliability of threshold results based on the obtained BETs and IBTs. The results for the 193 subjects are provided in the supplementary material, which includes

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Distribution of individual BETs and individual IBTs (logarithmic values).</figcaption></figure>
individual graphs for comparing the Best Estimate Threshold (BET) and Intensity-Based Thresholds (IBT) on a subject-by-subject basis.

### 3.6.4.1. BET and IBT equal (33.2% of the panel).
The four graphs in Fig. 5 illustrate different cases where BET and IBT are equal (DRI = 0). In general, when the thresholds obtained with the two methods were identical, the intensity curves were easy to interpret and clearly increased in intensity above the threshold estimated with the BET method. As shown in supplementary material, some of the intensity curves exhibit scoring incidents, perhaps due to a lack of concentration. If these scoring incidents are few for a given subject, they do not impair the determination of the IBT. For example, the intensity given by subject S102 for the different sample of the fifth 3-AFC test was lower than expected, considering the overall shape of the curve established over all concentrations, which could be due to a lack of concentration. However, the IBT determination was not affected by this scoring error.

For all cases with DRI = 0, the BET estimate was fully validated by the intensity measurements. Consequently, the level of confidence in the data from these subjects can be considered high.

### 3.6.4.2. BETs lower than IBTs (38.3% of the panel).
In general, the cases where the BET is lower than the IBT are easily interpreted. The four graphs in Fig. 6 clearly illustrate cases where the calculated BETs are questionable. Indeed, the intensity curves suggest that the actual thresholds of the four subjects considered might actually be higher than those indicated by the BETs. In these four cases, the difference between BET and IBT can be explained by correct answers given by chance. Indeed, the correct answer of subject S027 for the 3-AFC test involving the fifth concentration might have been given at random. The same explanation can be applied to subjects S117 (fifth and sixth concentrations), S027 (fourth and fifth concentrations), and S170 (third, fourth and fifth concentrations). The graphs provided in the supplementary material show many similar cases where the BET was potentially biased for the same reason: S027, S035, S078, S118, S039, S057, S071, S093, S125, S120, S123, S130, S147, etc.

### 3.6.4.3. BETs higher than IBTs (28.5% of the panel).
In some cases, the differences between BET and IBT can be attributed to an error of inattention during the 3-AFC tests. For example, for subject S142 (Fig. 7), the large difference between the thresholds measured with the two methods can be explained by an error of inattention or confusion during the 3-AFC test that involved the fifth lowest concentration. This error would have interrupted the series of correct responses of the subject and considerably increased his BET. The graphs included in the supplementary material show similar cases where the BET was potentially biased by an inattention error (S142, S158, S159, S081, S142, and S060).

The method to determine the IBT may also have limitations in two cases. The first case corresponds to very gradual increases in intensity as a function of the tannic acid concentration (example: subject S167, Fig. 7). In this case, no significant increase indicates that the detection threshold has been exceeded, which is clearly discernible, and the IBT values determined in this case are generally low. The second case corresponds to situations where the range of intensity notes is very small, e. g., two or three points on a scale of 0 to -10 (S033 and S007, Fig. 7). In such cases, intensity variations are so small that it is difficult to determine with certainty an increase signaling that the detection threshold has been exceeded. In some cases, this raises questions about the aptitude of the subjects for the task of rating intensities on the scale. For example, subject S007 (Fig. 7) appeared to perceive astringency in all samples at a high level (6 to -8 on a scale of 0 to -10), but increases in tannic acid concentration did not result in higher scores. In contrast, in other cases, such as subject S033 (Fig. 7), low and relatively constant intensity scores may indicate real low sensitivity. In this case, the IBT value is incorrect, and the BET, which is very high, should be preferred.

These few examples show that when the BET is greater than the IBT, it is necessary to study the results of both measurements in detail to estimate the reliability of the data.

### 3.6.5. Data selection via the data reliability index

The DRI should be considered an index to identify subjects whose

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Equal BET and IBT (DRI = 0) - Astringency intensity as a function of the tannic acid concentration (log values). BET and IBT values are shown on the x-axis. The values of 0 and 1 on the intensity curve correspond to the 3-AFC test result for the concentration of interest (1 = correct answer, 0 = wrong answer).</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>**Fig. 6.** BET lower than IBT - Astringency intensity as a function of the tannic acid concentration (log values). The BET and IBT log values are shown on the x-axis. The values of 0 and 1 on the intensity curve correspond to the 3-AFC test result for the concentration of interest (1 = correct answer, 0 = wrong answer).</figcaption></figure>

data (BET and IBT) must be examined in detail. In conjunction, the individual intensity curves and data obtained from the 3-AFC tests can be used to determine whether the gap between two thresholds is problematic. If examination of the individual data reveals that the BET can be questioned, then the measurement must be repeated or the data must be discarded.

For more automatic use, a minimum value and a maximum value that define a range of acceptable DRIs to the experimenter can be used to select the data to be retained. Data from subjects whose DRIs are outside the defined interval should be discarded. Determining this interval depends on the analyzed data, and the experimenter must consider the percentage of the value to be retained. In addition, the correlation coefficient between BET and IBT after the elimination of subjects whose DRIs are outside the interval is an interesting element to consider when defining the DRI threshold. Fig. 8 shows the evolution of Pearson's correlation coefficient, and the percentage of retained subjects is a function of the value used to define the acceptable IRD interval. For example, with 0.75, the defined interval was [-0.75, 0.75]. In this case, the percentage of retained subjects was 77 %, and the correlation coefficient between IBT and BET was 0.63 (p < 0.0001) (Fig. 8).

In the case of this study, the choice of a minimum DRI value of 0.75 (absolute value) enabled one to detect 45 subjects with questionable thresholds. The measured BETs for these 45 subjects were highly variable (Fig. 9). Eliminating the data from these 45 subjects had no major impact on the average threshold of the group. Indeed, without these 45 subjects, the calculated threshold was 0.923 (0.12 g/L), whereas it was 0.952 (0.11 g/L) for all subjects. For the 148 subjects whose data were retained, the Pearson correlation coefficient between BET and IBT was 0.63 (p < 0.0001). Meanwhile, the consequences are more significant if the data are used to form groups of subjects according to their sensitivity. Indeed, there are unselected subjects who could have been included in groups of extreme sensitivity based on their BET values (Fig. 9). For example, suppose that for this study, the selection of the most and least sensitive subjects consisted of selecting those whose BETs (log value) were greater than -0.5 and less than -1.3, respectively,

from the 193 subjects (Fig. 9). In this case, the most sensitive group would have consisted of 23 subjects, including 10 with questionable data (43.5 %). The least sensitive group would have comprised 35 subjects, including 18 with questionable data (51.4 %). An in-depth analysis of the individual data might have revealed that data from some of these subjects could be retained. However, this example shows the relevance of the proposed procedure in this document when the sensitivity groups are established on the BETs collected via the ASTM E679-04 procedure.

# 4. General discussion

The method in this document is intended to complement the procedure ASTM E679-04 by proposing a simple procedure to detect thresholds whose assessment is potentially questionable. It does not call into question the BET calculation itself or its subsequent use. The modification to the standard procedure consists of asking subjects for each 3-AFC test to rate the intensity of the sensation perceived for the sample considered to be different from the other two. The sensation to be rated corresponds to the chemical stimulus whose threshold must be determined. This additional task is simple and quick, does not alter the essence of the ASTM E679-04 procedure, particularly its speed, and is consequently suitable for large panels.

## 4.1. *Intensity rating by untrained subjects*

The 193 participants in the study in this work had no previous experience in sensory analysis. It is legitimate to ask whether untrained subjects can rate intensities on a scale, especially for a difficult-to-assess sensation such as astringency. Several elements suggest that the task consisting of rating the intensity of a characteristic on a scale is achievable by untrained subjects under specific conditions. This type of task used to be reserved for trained panels but is becoming increasingly common with consumers (Ares & Varela, 2018). For example, the rate-all-that-apply (RATA) method asks consumers to rate the intensity of all
<figure><img src="image_9.png" /><figcaption>**Fig. 7.** BET higher than IBT - Astringency intensity as a function of tannic acid concentration (log values). The BET and IBT values are shown on the x-axis. The values of 0 and 1 on the intensity curve correspond to the 3-AFC test result for the concentration of interest (1 = correct answer, 0 = wrong answer).</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Fig. 8.** Pearson correlation coefficient between IBT and BET and percentage of retained subjects as a function of the value to define the acceptable DRI interval.</figcaption></figure>

useful descriptors for product description (Ares et al., 2014). Moreover, for products that are simple from a sensory viewpoint and where the differences between products are obvious, the results of an untrained panel and the RATA method are similar to the results of a trained panel and the classic descriptive analysis (Ares et al., 2018). In addition, previous research has suggested that the concordance between the two methods can be improved by providing consumers with short training based on reference samples, definitions, and explanations of the use of the scale (Oppermann et al., 2017). The aqueous tannic acid solutions in this study can be considered relatively simple, where astringency was the only sensation perceived at a high level. Moreover, the participants were briefed at the beginning of the session on the sensation of astringency and provided a reference solution and explanations on how to use the rating scale. Although these precautions do not guarantee a measurement quality equivalent to intensive training, they provide a favorable framework for the measurement.

Although the objective of the IBT is not to replace the BET, for this

study, the average group threshold determined using intensities (0.12 g/L) was very close to that determined using the ASTM E679-04 procedure (0.11 g/L). This value is equal to the value calculated when applying the correction recommended by Lawless et al. (2010a). This result reinforces the idea that with certain precautions (explanation of the sensation to be rated, provision of a reference solution, and explanations on the use of the scale), the intensity of a single sensation generated by a chemical stimulus in a simple solution can be rated by untrained subjects.

## 4.2. Estimation of the reliability of individual BETs

The comparison of the IBT and BET enables an estimation of the reliability of the thresholds obtained via the ASTM E679-04 procedure. When both methods measure more similar thresholds, the measurement is more likely reliable. In this study, the DRI, which reflects the discrepancy between two thresholds and reliability of the data, indicated that reliability was very high for 33.2 % of the panel (DRI = 0). For 60 % of the participants, the thresholds of both methods differed by a

<figure><img src="image_11.png" /><figcaption>**Fig. 9.** BET distribution (log) for all participants, for selected subjects (absolute DRI value ≤0.75) and for subjects with questionable thresholds (absolute DRI value >0.75).</figcaption></figure>
maximum of one concentration level.

In many cases, when the BET is lower than the IBT, the difference between two thresholds can be explained by one or more correct responses given by chance during the 3-AFC test series. This bias is often mentioned concerning the ASTM E679-04 procedure (Hough et al., 2013; ISO, 2018; Lawless, 2010; Peng et al., 2012; Piornos et al., 2019). Thus, a positive DRI is useful to detect such cases. The proposed method is somewhat useful for detecting inattention errors in response to 3-AFC tests. We identified a few instances where this issue might have occurred, although these instances were fewer than the instances where correct answers appeared to have been randomly provided. These inattention errors are indicated by a negative DRI (BET > IBT), although a negative DRI is not specific to this type of error.

We observed two cases where determination of the IBT was difficult. In both cases, great care should be taken before the BET is questioned. If the intensity progressively and continuously increases as a function of the stimulus concentration for a given subject, then the IBT value is mechanically low, but no significant jump in intensity can be observed. In this case, if the BET is very high, which suggests low sensitivity, the quality of the intensity measurement can be questioned. The gradual and continuous increase in intensity can be due to anticipation bias (Lawless & Heymann, 1999). The quality of the intensity measurement can also be questioned if the intensity is relatively high for a given subject, which suggests that the astringency sensation is perceived but hardly varies with the concentration (e.g., 3 points on a scale of 0- to 10). Since the intensity hardly varies without a noticeable jump, determining the IBT is delicate, and it is challenging to use this measurement to determine the reliability of the corresponding BET.

High absolute values of DRIs can detect cases where the data require detailed examination. Then, it is up to the experimenter to analyze possible causes of the discrepancy between the thresholds measured by the two methods and decide whether to retain or discard the questionable data.

### 4.3. Subject selection based on the data reliability index

As previously mentioned, the use of the DRI alerts the experimenter to cases that require individual examination of the data. After examining the data from the intensity assessments and 3-AFC tests, the experimenter can discard, retain or repeat the measurement with subjects whose data are questionable. It is difficult to provide a DRI value that indicates a critical difference between the thresholds measured by the two methods. This value is likely specific to each dataset. Therefore, the experimenter must decide on this value according to the data and his/her knowledge of the thresholds that are generally obtained for the studied molecule.

The use of the DRI appears particularly interesting for studies designed to screen the sensitivity to a molecule in many subjects. In this case, repeating measurements to ensure data reliability is a considerable task. The use of minimum and maximum DRI values can enable automatic selection of the most reliable data. To select these minimum and maximum values, the experimenter can use the percentage of retained subjects and the correlation coefficient between two threshold measurements. This more automatic use of DRIs eliminates some data (so the initial number of subjects must be large) but avoids individual data analysis to decide whether to retain or discard them. This method of using the DRI can be considered a pre-selection stage for subjects with the most reliable data.

### 4.4. Impact of eliminating the least reliable data on thresholds determined via the ASTM E679-04 procedure

For the case study in this document, the elimination of 33 % of the subjects with the least reliable data based on individual DRIs did not have a major effect on the group threshold value (0.12 g/L without these subjects compared with 0.11 g/L with all subjects). However, it would

be interesting to know whether this is always the case for any sensation and any panel. This finding may be specific to this study.

However, we demonstrated that if the objective of the threshold measurement was to form groups of subjects that exhibited extreme (very high or very low) sensitivity, then it was important to consider individual DRIs. More specifically, in an example from this work, we showed that the data were questionable for approximately 40-50 % of the subjects selected for their higher or lower sensitivity compared with the remainder of the group.

### 4.5. Detection threshold for tannic acid astringency

The case study in this document determined the detection threshold for the astringency of tannic acid for 193 subjects. Obtaining threshold data on a large number of subjects is relatively rare, especially for astringent molecules, which emphasizes the significance of this result. The average threshold was 0.11 g/L via the ASTM E679-04 method and 0.12 g/L after excluding subjects with questionable thresholds. These findings were consistent with data obtained for tannic acid via the ASTM E679-04 procedure. For example, Nuvoli et al. (2023) demonstrated that the detection threshold for tannic acid was 0.085 g/L for a group of omnivorous subjects compared with 0.13 g/L for a group of vegan subjects. The lower and upper quartiles of the threshold distributions measured in the 193 subjects (0.07 g/L and 0.22 g/L, respectively) were carefully cited. This information can be useful for estimating the relative sensitivity level to tannic acid astringency for an individual whose threshold has been measured via the ASTM E679-04 procedure.

### 4.6. Limitations and recommendations

It is important to note that differences between the IBT and the BET may arise from biases inherent to either of the measurements. In the study presented in this paper, several differences were attributed to a biased BET, often due to correct responses occurring by chance, and occasionally due to lapses in attention during the 3-AFC tests. However, these assumptions, while probable, remain uncertain, and additional measurements would be required to confirm them. Furthermore, a limitation was identified with the IBT. When intensity ratings are either minimally variable or change very gradually without a noticeable increase, determining the IBT becomes challenging and may lead to substantial discrepancies when compared to the BET. In both cases, caution is therefore warranted when interpreting conclusions drawn from the comparison between IBT and BET. It is important to emphasize that the purpose of this method is primarily to alert experimenters to situations where uncertainty may arise, and in such cases, a detailed analysis of both 3-AFC test data and intensity measurements is necessary. If there is doubt regarding either of the values derived from these measurements, it is recommended to repeat the tests. The proposed procedure should be considered as a complement to the methods outlined in the ASTM standard. Finally, it is important to note that assessing the intensity of a stimulus by untrained panelists requires several precautions. A familiarization phase with the sensation being evaluated, along with a demonstration of the rating procedure using the scale, is essential. Whenever feasible, prior training is strongly recommended to ensure the accuracy and reliability of intensity data.

## 5. Conclusion

The method in this work was designed to complement the ASTM E679-04 procedure. Its primary aim is to alert experimenters to cases where the obtained thresholds may be biased or questionable. The results of the case study show that the proposed method can identify subjects whose sensitivity determined using the BET may be overestimated because of randomly provided correct answers. In addition, it can highlight cases where the sensitivity determined using the BET may be underestimated because of potential bias caused by inattention
errors. These two biases are often mentioned in connection with the ASTM E679-04 procedure. Certain limitations associated with the determination of thresholds based on intensities were highlighted. In all cases, after examining suspect cases, the experimenter can decide whether to exclude questionable data.

Checking the reliability of BETs using this method is of particular interest when the thresholds are measured to categorize subjects according to their sensitivity. This control does not eliminate the need to repeat measurements, which is always preferable. However, when it is difficult to perform repetitions, e.g., when the number of subjects is very large, the proposed method in this work is useful for selecting the subjects with the most reliable data. The proposed method could therefore be a valuable complementary tool, used routinely, in many real-world sensory analysis contexts where practical measurement constraints are common.

This work also contributes to the growing body of knowledge on the astringency of tannic acid.

## Funding

This study was funded by the French National Research Agency (Grant MACARON ANR-20-CE21-002). The French Ministry of Higher Education and Research provided a PhD fellowship for Alix Rollatin.

## Credit authorship contribution statement

**Christophe Martin:** Writing – original draft, Software, Methodology, Formal analysis, Conceptualization. **Alix Rollatin:** Writing – review & editing, Methodology, Investigation, Formal analysis, Conceptualization. **Carole Tournier:** Writing – review & editing, Supervision. **Caroline Peltier:** Writing – review & editing, Software, Methodology, Formal analysis, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgments

The authors would like to thank Catherine PEDRON, Betty HOF-FARTH, Jeanne CHALOYARD, Noémie HERRMANN, and Mélanie LELIEVRE for their assistance during sample preparation and data collection. The authors would like to thank Francis CANON for initiating this project.

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.foodqual.2025.105437.

## Data availability

Data will be made available on request.

## References

* Ares, G., Bruzzone, F., Vidal, L., Cadena, R. S., Giménez, A., Pineau, B., et al. (2014). Evaluation of a rating-based variant of check-all-that-apply questions: Rate-all-that-apply (RATA). *Food Quality and Preference, 36*, 87–95. https://doi.org/10.1016/j.foodqual.2014.03.006

* Ares, G., Picallo, A., Coste, B., Antúnez, L., Vidal, L., Giménez, A., et al. (2018). A comparison of RATA questions with descriptive analysis: Insights from three studies with complex/similar products. *Journal of Sensory Studies, 33*(5), Article e12458. https://doi.org/10.1111/joss.12458

* Ares, G., & Varela, P. (2018). Consumer-based methodologies for sensory characterization. In G. Ares, & P. Varela (Eds.), *Methods in consumer research. New

* approaches to classic methods* (pp. 188-209). Cambridge, UK: Woodhead Publishing. https://doi.org/10.1016/B978-0-08-102089-0-00008-X

* ASTM. (2011). *ASTM E679-04: Standard practice for determination of odor and taste thresholds by a forced-choice ascending concentration series method of limits*. Philadelphia: American Society for Testing and Materials.

* Beidler, L. M. (1954). A theory of taste stimulation. *Journal of General Physiology, 38*(2), 133-139. https://doi.org/10.1085/jgp.38.2.133

* Chapman, K. W., Whited, L. J., & Boor, K. J. (2006). Sensory threshold of light-oxidized flavor defects in Milk. *Journal of Food Science, 67*(7), 2770-2773. https://doi.org/10.1111/j.1365-2621.2002.tb08813.x

* Eisele, T. A., & Semon, M. J. (2005). Best estimated aroma and taste detection threshold for guaiacol in water and apple juice. *Journal of Food Science, 70*(4), S267-S269. https://doi.org/10.1111/j.1365-2621.2005.tb07201.x

* Giguère, J.-F., de Moura Piovesana, P., Proulx-Belhumeur, A., Doré, M., de Lemos Sampaio, K., & Gallani, M.-C. (2016). Reliability of a simple method for determining salt taste detection and recognition thresholds. *Chemical Senses, 41*(3), 205-210. https://doi.org/10.1093/chemse/bjv080

* González Viñas, M. A., Salvador, M. D., & Martin-Alvarez, P. J. (1998). Comparison of two simple methods for the measurement of detection thresholds for basic, umami and metallic tastes. *Journal of Sensory Studies, 13*(3), 299-314. https://doi.org/10.1111/j.1745-459X.1998.tb00091.x

* Harvey, L. O. (1986). Efficient estimation of sensory thresholds. *Behavior Research Methods, Instruments, & Computers, 18*, 623-632. https://doi.org/10.3758/BF03201438

* Hough, G., Methven, L., & Lawless, H. T. (2013). Survival analysis statistics applied to threshold data obtained from the ascending forced-choice method of limits. *Journal of Sensory Studies, 28*(5), 414-421. https://doi.org/10.1111/joss.12067

* ISO 13301. (2018). *Sensory analysis — Methodology — General guidance for measuring odour, flavour and taste detection thresholds by a three-alternative forced-choice (3-AFC) procedure*.

* Jaeger, S. R., de Silva, H. N., & Lawless, H. T. (2014). Detection thresholds of 10 odor-active compounds naturally occurring in food using a replicated forced-choice ascending method of limits. *Journal of Sensory Studies, 29*(1), 43-55. https://doi.org/10.1111/joss.12085

* Kim, M., Jang, H., & Lee, K.-G. (2020). Characterization of key aroma-active compounds isolated from omija fruit treated differently based on odor activity values and descriptive sensory analysis. *Foods, 9*, 638. https://doi.org/10.3390/foods9050638

* Kim, M. K., Drake, S. L., & Drake, M. A. (2011). Evaluation of key flavor compounds in reduced- and full-fat Cheddar cheeses using sensory studies on model systems. *Journal of Sensory Studies, 26*, 278-290. https://doi.org/10.1111/j.1745-459X.2011.00343.x

* Lawless, H., Harono, C., & Hernandez, S. (2010). Thresholds and supra-threshold intensity functions for capsaicin in oil and aqueous based carriers. *Journal of Sensory Studies, 15*, 437-447. https://doi.org/10.1111/j.1745-459X.2000.tb00281.x

* Lawless, H. T. (2010). A simple alternative analysis for threshold data determined by ascending forced-choice methods of limits. *Journal of Sensory Studies, 25*(3), 332-346. https://doi.org/10.1111/j.1745-459X.2009.tb00262.x

* Lawless, H. T., & Heymann, H. (1999). Measurement of sensory thresholds. In H. T. Lawless, & H. Heymann (Eds.), *Sensory evaluation of food: Principles and practices*. Boston, MA: Springer US.

* Lee, H.-S., & Van Hout, D. (2009). Quantification of sensory and food quality: The R-index analysis. *Journal of Food Science, 74*(6), R57-R64. https://doi.org/10.1111/j.1750-3841.2009.tb01204.x

* Linne, B., & Simons, C. T. (2017). Quantification of oral roughness perception and comparison with mechanism of astringency perception. *Chemical Senses, 42*(7), 525-535. https://doi.org/10.1093/chemse/bjx029

* Lundahl, D. S., Lukes, B. K., McDaniel, M. R., & Henderson, L. A. (1986). A semi-ascending paired difference method for determining the threshold of added substances to background media. *Journal of Sensory Studies, 1*, 291-306. https://doi.org/10.1111/j.1745-459X.1986.tb00179.x

* Macmillan, N. A., & Creelman, G. D. (1991). *Detection theory: A User's guide, Cambridge university press, Cambridge, UK*. Detection theory: A User's guide. Cambridge university press.

* McBride, R. L. (1987). Taste psychophysics and the Beidler equation. *Chemical Senses, 12* (2), 323-332. https://doi.org/10.1093/chemse/12.2.323

* Meilgaard, M. C. (1993). Individual differences in sensory threshold for aroma chemicals added to beer. *Food Quality and Preference, 4*(3), 153-167. https://doi.org/10.1016/0950-3293%2893%2990158-3

* Meilgaard, M. C., Carr, B. T., & Civille, G. V. (2006). *Sensory evaluation techniques (4th ed.)*, pp. 129-139). Taylor & Francis.

* Munoz-Gonzalez, C., Canon, F., Feron, G., Guichard, E., & Pozo-Bayon, M. A. (2019). Assessment wine aroma persistence by using an in vivo PTR-ToF-MS approach and its relationship with salivary parameters. *Molecules, 24*(7). https://doi.org/10.3390/molecules24071277

* Nuvoli, C., Fillion, L., Lacoste Gregorutti, C., & Labbe, D. (2023). Comparison of sensitivity to taste and astringency stimuli among vegans and omnivores. *Physiology & Behavior, 262*, Article 114092. https://doi.org/10.1016/j.physbeh.2023.114092

* Oppermann, A. K. L., de Graaf, C., Scholten, E., Stieger, M., & Piqueras-Fiszman, B. (2017). Comparison of rate-all-that-apply (RATA) and descriptive sensory analysis (DA) of model double emulsions with subtle perceptual differences. *Food Quality and Preference, 56*, 55-68. https://doi.org/10.1016/j.foodqual.2016.09.010

* Peng, M., Jaeger, S. R., & Hautus, M. J. (2012). Determining odour detection thresholds: Incorporating a method-independent definition into the implementation of ASTM E679. *Food Quality and Preference, 25*(2), 95-104. https://doi.org/10.1016/j.foodqual.2012.02.006
Piornos, J. A., Delgado, A., de La Burgade, R. C. J., Methven, L., Balagiannis, D. P., Koussissi, E., et al. (2019). Orthonasal and retronasal detection thresholds of 26 aroma compounds in a model alcohol-free beer: Effect of threshold calculation method. *Food Research International, 123*, 317–326. https://doi.org/10.1016/j.foodres.2019.04.034

Schoumacker, R., Martin, C., Thomas-Danguin, T., Guichard, E., Le Quéré, J. L., & Labouré, H. (2017). Fat perception in cottage cheese: The contribution of aroma and tasting temperature. *Food Quality and Preference, 56*, 241–246. https://doi.org/10.1016/j.foodqual.2016.04.003

Valentová, H., Škrovánková, S., Panovská, Z., & Pokorný, J. (2001). Determination of astringent taste in model solutions and in beverages. *Czech Journal of Food Sciences, 19*(5), 196–200. https://doi.org/10.17221/6607-CJFS

Valentová, H., Skrovánková, S., Panovská, Z., & Pokorný, J. (2002). Time-intensity studies of astringent taste. *Food Chemistry, 78*(1), 29–37. https://doi.org/10.1016/S0308-8146(01)00330-2

Wang, M., Septier, C., Brignot, H., Martin, C., Canon, F., & Feron, G. (2022). Astringency sensitivity to tannic acid: Effect of ageing and saliva. *Molecules, 27*(5), 1617. https://www.mdpi.com/1420-3049/27/5/1617.