an important aspect of any research is the use of appropriate methodologies either to control or to reduce the effects of potential confounding factors .
a matter of great concern that can influence the results of epidemiological studies of dental caries is the variation in disease diagnosis between two or more examiners ( interexaminer error ) and for the same examiner in two or more occasions ( intraexaminer error ) .
therefore , it is very important that data collection measures are standardized in order to minimize measurements variations1 . the calibration process including the determination of reliability , with both previous and ongoing epidemiological survey , is a basic step to understand and standardize the examination criteria , and also to evaluate the interexaminer variability in order to ensure accurate results15,17 .
the assessment of reliability is the most employed measure in dental caries surveys during the examiners ' calibration .
reliability is related to the extent to which examiners agree in their evaluations16 . the most used measures to assess reliability in epidemiological studies of dental caries are the overall percentage of agreement and the kappa statistics11 .
kappa test is a measurement of reliability that takes into consideration the agreement among raters by chance , providing better evaluation of interexaminer disagreement during calibration processes5 .
the purposes of this study were : a ) to evaluate interexaminer reliability in caries detection considering different diagnostic thresholds and b ) to indicate , by using kappa statistics , which is the best way of measuring interexaminer agreement during the calibration process in dental caries surveys .
the epidemiological examinations were initiated after approval of the study design by the research ethics committee of the dental school of piracicaba , state university of campinas , brazil ( protocol no .
the volunteers ' parents signed an informed consent form authorizing the enrollment of the children in the study .
dentists with previous experience in epidemiological surveys examined schoolchildren at baseline , 3 and 6 months after initial training , using two diagnostic thresholds on dental caries : who criteria , traditionally used in epidemiological surveys17 and who+il including the diagnostic of initial caries lesions ( il , white spot lesions ) , after being calibrated by a " gold standard " examiner , a dentist who routinely uses the who criteria for exams and had been previously trained and calibrated in il diagnosis .
eleven dentists with previous experience in epidemiological surveys of dental caries were invited to participate in this study .
schoolchildren aged 6 - 7 years from two public schools in piracicaba , sp , brazil , were selected by a dentist , according to their caries activity .
the dentists used mirror , cpi probe , air - drying for examination after the children brushed their teeth .
the exclusion criteria were : use of fixed orthodontic device , presence of severe fluorosis and/or hypoplasias , and severe systemic diseases .
for each training or calibration period of training , 10 to 13 different children were selected .
two diagnostic thresholds were used to record dental caries : 1 ) who threshold ( dmft index ) following the who codes and criteria17 , in which a tooth is considered as decayed when a cavitation is present ; 2 ) who+il threshold , in which active initial lesions were also recorded following criteria adapted from nyvad , et al.13 and fyfee , et al.6 .
an il was defined as an active carious lesion which , upon visual assessment by a calibrated examiner , presented intact surface , no clinically detectable loss of dental tissue , with a rough , whitish / yellowish colored area of increased opacity presumed to be carious ( when the cpi probe was used , its tip should be moved gently across the surface ) .
il locates close to gingival margin in smooth surfaces or extending along walls of fissure in occlusal surfaces .
each examiner was helped by a recorder during the study . a benchmaker examiner ( " gold standard " )
conducted the training processes with both theoretical and practical activities , which lasted 20 hours ; and the calibration exercises , which lasted 8 hours at each phase : baseline , 3 and 6 months after initial training .
the training and calibration processes were conducted by the gold - standard examiner using two diagnostic thresholds on dental caries , as described in previous studies2,3 . during theoretical discussions ,
the benchmaker examiner showed the examiners some photographic slides with clinical examples of each criterion that would be used in the study , in order to determine the examiners ' knowledge about epidemiological diagnosis , to instruct them on the criteria and examination method to be used , and finally , to achieve an initial standardization among them .
the clinical training consisted of 4 periods of 4 hours each , and was conducted in an outdoor setting .
each dentist examined 10 to 13 children , with distinct caries activity and prevalence , per period . during this phase , examiners discussed clinical diagnosis , study codes and criteria , recording and other errors in order to reach an acceptable level of agreement ( kappa>0.8517 ) .
the calibration exercises , in which the examiners did not discuss their findings , were carried out in 2 periods of 4 hours each , with a 1-week interval .
these were also undertaken after 3 and 6 months , after the first calibration phase ( baseline ) .
the epidemiological examinations were carried out in an outdoor setting , under conditions such as natural light , with dental mirror and ball - ended cpitn probes with a diameter of 0.5 mm ( to remove debris , assess the presence of fissure sealants and , in case of doubt , to check the surface texture of il ) .
toothbrushing with fluoridated dentifrice was performed by the children , under supervision of a dental hygienist , following the bass modified technique during approximately two minutes .
tooth air - drying ( approximately 5 seconds per tooth ) was performed by using compressed air delivered by a dental compressor ( wetzel : medical line 3.6/30  0.5 hp ) .
a program using microsoft excel has been developed by the department of community dentistry of fop - unicamp to calculate the interexaminer reliability by means of the kappa statistics7 that has been recommended by the who17 and the british association of community dentistry15 for evaluation of agreement among examiners in oral health surveys .
the code recorded for each dental unit or surface was entered for each examiner , in accordance with the different diagnosis thresholds ( who ; who+il ) used in the three calibration phases .
the objective of the statistical analysis was to assess interexaminer reliability under different caries diagnostic thresholds and different ways of analysis .
thus , interexaminer reliability was calculated by using kappa statistics , according to two diagnosis thresholds ( who ; who+il ) and considering : a ) the entire dentition ; b ) upper / lower jaws ; c ) sextants : upper / lower right / left , upper / lower anterior ; d ) each teeth individually . for each evaluation ,
codes from examination made by each examiner were compared to those of other examiners , ( example : 1x2 , 1x3  1x11 ; 2x3 . 2x11 ; 10x11 ) .
the epidemiological examinations were initiated after approval of the study design by the research ethics committee of the dental school of piracicaba , state university of campinas , brazil ( protocol no .
the volunteers ' parents signed an informed consent form authorizing the enrollment of the children in the study .
dentists with previous experience in epidemiological surveys examined schoolchildren at baseline , 3 and 6 months after initial training , using two diagnostic thresholds on dental caries : who criteria , traditionally used in epidemiological surveys17 and who+il including the diagnostic of initial caries lesions ( il , white spot lesions ) , after being calibrated by a " gold standard " examiner , a dentist who routinely uses the who criteria for exams and had been previously trained and calibrated in il diagnosis .
eleven dentists with previous experience in epidemiological surveys of dental caries were invited to participate in this study .
schoolchildren aged 6 - 7 years from two public schools in piracicaba , sp , brazil , were selected by a dentist , according to their caries activity .
the dentists used mirror , cpi probe , air - drying for examination after the children brushed their teeth .
the exclusion criteria were : use of fixed orthodontic device , presence of severe fluorosis and/or hypoplasias , and severe systemic diseases .
for each training or calibration period of training , 10 to 13 different children were selected .
two diagnostic thresholds were used to record dental caries : 1 ) who threshold ( dmft index ) following the who codes and criteria17 , in which a tooth is considered as decayed when a cavitation is present ; 2 ) who+il threshold , in which active initial lesions were also recorded following criteria adapted from nyvad , et al.13 and fyfee , et al.6 .
an il was defined as an active carious lesion which , upon visual assessment by a calibrated examiner , presented intact surface , no clinically detectable loss of dental tissue , with a rough , whitish / yellowish colored area of increased opacity presumed to be carious ( when the cpi probe was used , its tip should be moved gently across the surface ) .
il locates close to gingival margin in smooth surfaces or extending along walls of fissure in occlusal surfaces .
each examiner was helped by a recorder during the study . a benchmaker examiner ( " gold standard " ) conducted the training processes with both theoretical and practical activities , which lasted 20 hours ; and the calibration exercises , which lasted 8 hours at each phase : baseline , 3 and 6 months after initial training .
the training and calibration processes were conducted by the gold - standard examiner using two diagnostic thresholds on dental caries , as described in previous studies2,3 . during theoretical discussions ,
the benchmaker examiner showed the examiners some photographic slides with clinical examples of each criterion that would be used in the study , in order to determine the examiners ' knowledge about epidemiological diagnosis , to instruct them on the criteria and examination method to be used , and finally , to achieve an initial standardization among them .
the clinical training consisted of 4 periods of 4 hours each , and was conducted in an outdoor setting .
each dentist examined 10 to 13 children , with distinct caries activity and prevalence , per period . during this phase , examiners discussed clinical diagnosis , study codes and criteria , recording and other errors in order to reach an acceptable level of agreement ( kappa>0.8517 ) .
the calibration exercises , in which the examiners did not discuss their findings , were carried out in 2 periods of 4 hours each , with a 1-week interval .
these were also undertaken after 3 and 6 months , after the first calibration phase ( baseline ) .
the epidemiological examinations were carried out in an outdoor setting , under conditions such as natural light , with dental mirror and ball - ended cpitn probes with a diameter of 0.5 mm ( to remove debris , assess the presence of fissure sealants and , in case of doubt , to check the surface texture of il ) .
toothbrushing with fluoridated dentifrice was performed by the children , under supervision of a dental hygienist , following the bass modified technique during approximately two minutes .
tooth air - drying ( approximately 5 seconds per tooth ) was performed by using compressed air delivered by a dental compressor ( wetzel : medical line 3.6/30  0.5 hp ) .
a program using microsoft excel has been developed by the department of community dentistry of fop - unicamp to calculate the interexaminer reliability by means of the kappa statistics7 that has been recommended by the who17 and the british association of community dentistry15 for evaluation of agreement among examiners in oral health surveys .
the code recorded for each dental unit or surface was entered for each examiner , in accordance with the different diagnosis thresholds ( who ; who+il ) used in the three calibration phases .
the objective of the statistical analysis was to assess interexaminer reliability under different caries diagnostic thresholds and different ways of analysis .
thus , interexaminer reliability was calculated by using kappa statistics , according to two diagnosis thresholds ( who ; who+il ) and considering : a ) the entire dentition ; b ) upper / lower jaws ; c ) sextants : upper / lower right / left , upper / lower anterior ; d ) each teeth individually . for each evaluation ,
codes from examination made by each examiner were compared to those of other examiners , ( example : 1x2 , 1x3  1x11 ; 2x3 .
for both diagnostic thresholds , high mean values of kappa were obtained ( tables ) . moreover ,
interexaminer agreement was constant when considering the entire dentition , jaws and sextants ( tables 1 and 2 ) .
kappa values above 0.85 were obtained by analysis of sextants according to who threshold . however , when considering the who+il threshold , the values for posterior sextants decreased ( tables 1 and 2 ) .
the results of interexaminer reliability considering each tooth individually showed that the main difficulty was related to caries diagnosis in posterior teeth , especially the permanent first molars , for both thresholds with , in general , lower values for the who+il threshold ( tables 3 and 4 ) .
e1 = exercise at baseline ; e2= exercise after 3 months ; e3= exercise after 6 months .
e1=exercise at baseline ; e2= exercise after 3 months ; e3= exercise after 6 months .
the process of examiners ' calibration is an important aspect in planning and conducting oral health surveys .
brazilian surveys4,10 have shown that training and calibration of examiners have been an aspect of great concern for measuring interexaminer agreement , according to the recommendations of the who , which has indicated the use of kappa statistics17 .
kappa test provides a better evaluation of disagreement among examiners during calibration processes since it is a measurement of adjusted agreement by taking into consideration the ratio of chance agreement5 .
analysis of variance and post - hoc tests , such as scheff , have also been used to assess significant differences in caries indices among examiners9 .
the present study showed , in general , high means of interexaminer reliability for both diagnosis thresholds when considering the entire dentition , the upper / lower jaws and sextants ( tables 1 and 2 ) . on the other hand , lower kappa values
were observed for dental units ( each tooth individually ) , especially when considering the most sensitive diagnosis threshold ( who+il ) ( tables 3 and 4 ) . as a consequence , good interexaminer agreement for
the entire dentition may not be as real as if one considers separately the posterior teeth , in which cavitated and non - cavitated carious lesions are concentrated8 . moreover , considering the current epidemiological profile of dental caries , the higher number of sound teeth ( fewer errors in diagnosis ) in comparison to carious teeth ( more errors in diagnosis ) may dilute the errors attributed to carious teeth , leading to a positive vision of the results achieved in examiners ' calibration14 .
therefore , one may speculate that the analysis of kappa values considering the entire dentition rather than each tooth individually may not be the best way to evaluate interexaminer reliability , especially in areas with low caries prevalence .
it may be suggested the need for future reformulations in conducting examiners ' calibration , paying more attention to diagnosis of posterior teeth and selecting children of distinct caries activity and prevalence1 .
low kappa values under the who+il threshold can also be explained by the inherent difficulty in diagnosing il , especially in surveys1 .
although the examinations were carried out in sunny days under high luminosity conditions , the use of artificial light could generate an increase in interexaminer agreement by facilitating the view of posterior teeth .
further studies are needed to determine the relevance of using artificial light in dental caries surveys , mainly for detecting initial lesions . in general , the use of more detailed measures to determine interexaminer agreement , such as the evaluation by dental unit ( each tooth individually ) , improves the calibration process by showing which teeth are leading to great disagreements and indicating the possible need for greater efforts in training examiners14 .
however , it must be emphasized that the method of evaluating interexaminer agreement also depends on the study design and objectives , the desired degree of accuracy and the available resources . as an example , the calibration process using more rigorous statistical measures , such as the analysis by dental units , would be indicated in either clinical trials or case - control studies , in which the effect of preventive measures on the reduction of caries levels , including the detection of initial lesions , must be evaluated .
therefore the kappa statistics considering reliability values according to each code / clinical condition can be employed12 .
on the other hand , in order to know and evaluate the epidemiological profile of dental caries in an underprivileged community , for instance , these more robust measures could be dispensed .
it is important to mention that the present study , which is part of a 12-month longitudinal study on examiners ' calibration , presents some limitations , such as lack of validity results by comparing the examiners ' results to those from the gold - standard examiner , and lack of intraexaminer errors .
such measures were not taken into consideration because the main goal of this study was to evaluate interexaminer reliability by using different diagnostic thresholds for caries detection as well as to verify the behavior of kappa statistics in order to indicate the most adequate way to measure reliability during the examiners ' calibration process in dental caries surveys .
the results of this study showed that the interexaminer reliability and its maintenance for six months were possible , under both caries diagnosis thresholds .
nevertheless , great disagreement was observed for the posterior teeth , especially when the who+il criteria were used .
the analysis considering dental elements individually was the best way of detecting disagreements among examiners during the calibration sections .
 objectivesa ) to evaluate the interexaminer reliability in caries detection considering different diagnostic thresholds and b ) to indicate , by using kappa statistics , the best way of measuring interexaminer agreement during the calibration process in dental caries surveys.methodseleven dentists participated in the initial training , which was divided into theoretical discussions and practical activities , and calibration exercises , performed at baseline , 3 and 6 months after the initial training . for the examinations of 6 - 7-year - old schoolchildren , the world health organization ( who ) recommendations were followed and different diagnostic thresholds were used : who ( decayed / missing / filled teeth  dmft index ) and who + il ( initial lesion ) diagnostic thresholds . 
 the interexaminer reliability was calculated by kappa statistics , according to who and who+il thresholds considering : a ) the entire dentition ; b ) upper / lower jaws ; c ) sextants ; d ) each tooth individually.resultsinterexaminer reliability was high for both diagnostic thresholds ; nevertheless , it decreased in all calibration sections when considering teeth individually.conclusionthe interexaminer reliability was possible during the period of 6 months , under both caries diagnosis thresholds . 
 however , great disagreement was observed for posterior teeth , especially using the who+il criteria . 
 analysis considering dental elements individually was the best way of detecting interexaminer disagreement during the calibration sections . 