the national cancer institute ( nci ) of the united states of america has proposed a reporting system for thyroid fine needle aspiration ( fna ) cytology , so - called bethesda system , which became an international standard of thyroid cytology [ 1 , 2 ] . following this recommendation , the uk royal college of pathologists ( the uk system ) [ 3 , 4 ] and italian societies of endocrinology and the italian society for anatomic pathology and cytology joint with the italian division of the international academy of pathology ( the italian system ) [ 5 , 6 ] updated their diagnostic schema comparable with the bethesda system . in 2013 , the japan thyroid association ( jta ) published guidelines for clinical practice for the management of thyroid nodules , including its diagnostic system for reporting thyroid fna cytology as shown in table 1 [ 79 ] , using criteria similar to those used in the papanicolaou society recommendation , the bethesda system [ 1 , 2 ] , the uk system [ 3 , 4 ] , and the italian system [ 5 , 6 ] . the jta reporting system of thyroid cytology further recommends risk stratification of follicular neoplasms ( fn ) into favor benign ( low risk : lr ) , borderline ( moderate risk : mr ) , and favor malignant ( high risk : hr ) , which was adopted from the practice of high - volume thyroid centers in japan . this study examined observer variation in the subclassification of fn among 4 thyroid experts to validate the usefulness and limitations of this characteristic risk stratification of fn recommended in the jta guidelines . it is not our purpose to address in detail all the morphological issues of thyroid cytology . conventional smear samples of indeterminate diagnosis ( n = 389 ) were selected from files ( n = 3843 ) from the year 2005 at ito hospital , tokyo , japan , by one ( k. kameyama ) of the authors . inadequate samples or those with poor preparation and cases under clinical follow - up ( no final histologic diagnosis ) were excluded . there were 91 ( 23.4% ) surgically treated patients with thyroid nodule under indeterminate cytological diagnoses . these patients visited ito hospital in 2005 and underwent diagnostic surgery or curative surgery in the subsequent years between 2005 and 2008 . there were 48 cases of benign diagnoses , including 29 follicular adenomas ( fas ) , 19 adenomatous nodules ( an ) , and 43 malignant diagnoses , including 13 follicular carcinomas ( ftcs ) , 24 papillary carcinomas ( ptcs ) , 1 poorly differentiated carcinoma , 2 c - cell carcinomas , and 3 malignant lymphomas . twenty cases of cytological smear samples with follicular patterned lesions were randomly selected by one ( k. kameyama ) of the authors and they were further analyzed for subclassification of fn and diagnoses by 4 reviewers , as shown in table 2 . the present study is a reproducibility study undertaken by 4 reviewers ( kennichi kakudo , kaori kameyama , mitsuyoshi hirokawa , and ryohei katoh ) who have special interest in thyroid pathology , all of whom are members of the clinical guideline committee of the jta . they were requested to subclassify fn following the new jta reporting system of thyroid cytology as shown in table 1 . those 20 cases of smear samples ( one representative smear sample each stained by the papanicolaou method ) were circulated among the 4 reviewers without clinical information . the 4 reviewers examined cytological samples independently without exchanging opinions and were unaware of the original cytological diagnoses and final histologic diagnoses . because this study was conceived as an audit of cytology performance and the results are anonymized , institutional ethical committee permission was not required for its conduct . informed consent of all 4 reviewers was obtained for this study protocol . for statistical analysis of observer concordance ,  was first introduced as a measure of the level of agreement between pairs of raters and extended to multiple raters , known as composite  [ 11 , 12 ] . composite  statistical analysis was performed by using data analysis and statistical software , version 13 ( stata press , college station , texas , usa ) . values of  can be interpreted as follows : 0.000.20 , slight or very weak agreement ; 0.210.40 , weak to fair agreement ; 0.410.60 , moderate agreement ; 0.610.80 , substantial or good agreement ; and values over 0.81 , optimal or almost perfect agreement . there were 8 malignant cases ( 2 ftcs , widely invasive ; 4 ftcs , minimally invasive ; and 2 ptcs ) and 12 benign cases ( 1 an and 11 fas ) . all diagnoses made by the 4 reviewers on the 20 follicular pattern lesions are shown in table 2 . there were only 6 ( 30% ) cases with a consensus diagnosis among all 4 of the reviewers : 3 in fa and 3 in ftc . diagnoses of fn and hr were made in only 7 cases ( 35% ) , including 4 malignant and 3 benign cases . composite  statistical analysis clearly demonstrated good concordance for the hr diagnosis in malignant cases (  = 0.7714 ) ( table 3 ) , while this was not found for the other subcategories in both benign and malignant lesions (  < 0.4 ) . it is of note that there was no interobserver variation in ftcs , widely invasive ( n = 2 ) , and all 4 reviewers classified the 2 cases into the hr subcategory . it is clear that ftcs , widely invasive , have different degrees of cellular abnormality in our study . in contrast , there were significant disagreements among other types of malignancy and benign lesions , which clearly confirmed that the subclassification of fn is not a definite diagnosis but risk stratification useful for triage patients . concerning ftc , minimally invasive ( n = 4 ) , there was only one concordant case among the 4 reviewers and all reviewers made a mr diagnosis in 1 case . there were disagreements between mr and hr in 1 case and between lr and mr in 2 cases . as for ptc ( n = 2 ) , the 4 reviewers gave 2 diagnoses each , which included 3 lrs , 3 mrs , 1 hr , and 1 suspicious for ptc . however , it is of note that there was no single case in our 20 cases whose subclassification varied among all 3 subcategories . in benign final histologic diagnosis ( n = 12 ) , there were 9 split diagnoses , including 6 cases between lr and mr and 3 cases between mr and hr , but no case between lr and hr . the incidence of lr diagnoses by the 4 reviewers in the 12 benign lesions was 31.3% ( 15/48 ) and that of lr diagnoses in the 6 cases of ftc was 16.7% ( 4/24 ) ( p = 0.186 ) . the incidence of hr diagnosis in the 12 benign lesions was 10.4% ( 5/48 ) and that of hr in the 6 cases of ftc was 47.1% ( 10/24 ) ( p = 0.002 ) . the incidence of malignancy of the hr was high at 60% for reviewer a , 75% for b , 75% for c , and 75% for d ( 6075% ) . the incidence of malignancy in the mr was lower than that of hr and was 33.3% for reviewer a , 35.7% for reviewer b , 25% for c , and 30.8% for d ( 2535.7% ) . the incidence of malignancy in the lr was lower than that in the hr and was 33.3% for reviewer a , 0% for b , 37.5% for c , and 33.3% for d ( 037.5% ) . although the risks of malignancy in the subcategories of fn differed among the 4 reviewers , it is of note that hr cytological diagnosis was significantly correlated with malignant histological diagnosis , and the risk of malignancy ( 6075% ) was significantly higher than that in the other 2 subcategories ( 035.7% ) in our study ( p = 0.002 ) . although the standardization of terminology and diagnostic criteria is important for accurate communication among patients , clinical doctors , and cytopathologists [ 110 ] , there are still differences among reporting systems in thyroid cytology as to how to interpret cytological diagnoses and how to decide on the clinical management of patients . in japan , cytopathologists attempted to use an internationally accepted reporting system , but it had to be modified to fit our practice . in japan , all patients with indeterminate cytology undergo further diagnostic procedures , without immediate surgery , to search for higher risk patients who should undergo surgery [ 79 , 1315 ] . it is because the majority of thyroid carcinomas in indeterminate cytology are indolent and more conservative approaches , other than immediate diagnostic surgery , usually do not create any harm to the patient with malignancy [ 1620 ] and diagnostic surgery to all patients with indeterminate cytology results in risks of unnecessary surgery to the patients with benign nodules , more than 80% of the patients [ 79 , 1922 ] . the proportion of malignancy found at thyroidectomy from patients with indeterminate cytology in this clinical setting will increase in number , and the malignancy rate of patient with indeterminate cytology in our study was calculated as 47.3% as shown in materials and methods . takezawa et al . from japan retrospectively analyzed their 1606 cytological samples using bethesda system , although it was written in japanese with english abstract , and they identified 115 ( 7.9% ) cases of aus / flus ( atypia of undetermined significance / follicular lesions of undetermined significance ) and 61 ( 4.2% ) cases of fn / sfn ( follicular neoplasms / suspicious for follicular neoplasms ) . the resection rate of their aus / flus nodules was 30.4% ( 35/115 cases ) and its malignancy rate was 88.6% ( 31/35 cases ) , and the resection rate of fn / sfn nodules was 36.0% ( 22/61 cases ) and its malignancy rate was 72.4% ( 16/22 cases ) , which were very different from the ranges reported in most of the literatures from western countries using the bethesda system or the uk system [ 1 , 2 , 2023 ] . as it is clear , further triage of patients with indeterminate nodules reduces resection rates and increases malignancy rates in any diagnostic systems including the bethesda system . this clinical management was also true in some other countries [ 2427 ] , as crippa and dina commented in a letter to an editor that thyroid cytology is the most important but not the only important deciding factor and therefore it must be integrated with other diagnostic procedures . we propose that future thyroid cytology classification schemes should reconsider clinical managements of indeterminate categories and how to reduce unnecessary surgeries for patients with benign thyroid nodules [ 79 ] . the risk stratification of follicular pattern lesions into 3 subcategories ( cellular follicular lesion , fn favor benign , and fn favor malignant ) was suggested by the papanicolaou society in 1996 , but it did not become popular in thyroid cytology worldwide , apart from japan [ 8 , 9 ] . there have been some reports in the literature on risk stratification of fn [ 10 , 2834 ] . kelman et al . reported that 31/52 ( 60% ) nodules with nuclear atypia consistent with fn were malignant   and it was 4/9 ( 44.4% ) in fn with atypia by goldstein et al . . reported that atypical proliferation was more often malignant than follicular group ( 53% versus 19% ) in their indeterminate category ( tir3/thy3 ) . some researchers pointed out that the malignancy rate of fn without atypia is low and assessment later on could be an alternative approach [ 31 , 32 ] and patients with high - risk cytological features such as nuclear overlapping ( crowding ) should be advised to have a surgical intervention [ 3133 ] . gerhard and da cunha santos using the papanicolaou society guidelines studied reproducibility between 2 observers in 97 diagnoses . they reported a substantial level of diagnostic interobserver (  = 0.71 ) and intraobserver (  = 0.66 ) reproducibility , although interobserver disagreement in the cytological diagnosis occurred in 23 cases ( 24.7% ) and 18 ( 41.7% ) of them were for fn . in an interobserver reproducibility study using the uk system , kocjan et al . reported that the  statistic was very poor ( 0.11 ) for the thy3a category and that for thy4 was 0.17 , in contrast to moderate to good agreement for thy1 ( 0.69 ) , thy2 ( 0.55 ) , thy3f ( 0.51 ) , and thy5 ( 0.61 ) . the observer variation of fn in our study occurred more often between lr and mr ( 9 cases , 45% ) , followed by between mr and hr ( 5 cases , 25% ) , but it is remarkable to note that none occurred between lr and hr . in other words , discordance is limited to between lr and mr or mr and hr , so we may conclude that the mr subcategory has an essential role in minimizing discordance between lr and hr . another choice of subclassification of follicular neoplasms would be two categories ( low cancer risk and high cancer risk )   instead of three categories ( lr , mr , and hr ) , and this modification ( two categories ) is also described as acceptable in the reporting system recommended by the japan thyroid association [ 7 , 8 ] . the second conclusion we may draw is that hr in the jta system is a powerful cytological subcategory to be used for the triage of patients for diagnostic surgery because the risk of malignancy of hr is high ( 6075% ) equivalent to suspicious for malignancy category in the bethesda system , with good concordance among the 4 reviewers (  = 0.7714 ) . abele and levine reported their rate of indeterminate category to be 5% of 51,000 adequate fnas and they suggested that the national rate of 15% was in large part due to overdiagnosis . this significant difference in ratio of indeterminate categories may be due to experience of thyroid cytology and not patients ' background . clary et al . commented in their interobserver variability study that some pathologists make greater use of indeterminate categories such as follicular lesion , favor nonneoplastic or follicular lesion , and favor neoplastic lesion , whereas others show more definitive categorization into benign and neoplastic groups . cibas et al . also stated in their report on interobserver variability that cytopathologists with experience of thyroid cytopathology are more likely to make a definitive interpretation ( i.e. , benign or malignant ) . this tendency was seen in our present study in which reviewers b and d ( whose indeterminate diagnosis rates are about 15% in their practice ) made mr diagnosis more often ( 70% and 65% , resp . ) and lr diagnosis less frequently ( 10% and 15% , resp . ) than those of reviewers a and c ( whose indeterminate diagnosis rates are about 5% in their practice ) . the difference in the prevalence of benign and fn / sfn may explain the different rates of lr ( low risk ) and mr ( moderate risk ) among reviews in our study , because incidence of one category may expand or contract depending on the rates of other categories . some fn / sfn lesions with benign pattern would be classified as benign by different authors and the incidence of fn / sfn of bethesda system in recent 7 series varied from 1.5 to 9.7% and that of benign category was between 54 and 77.4% summarized by ohori and schoedel . rapid development of molecular analyses on thyroid cytology may lead us possibly in the near future to more accurately identify patients who should be referred to surgery . until that time comes , thyroid fna cytology remains a main stay in the management of patients with thyroid nodules integrated with other clinical tests , such as ultrasound image diagnosis . as a conclusion the hr subcategory has a high predictive value of malignancy and good agreement among the 4 reviewers which is clinically helpful to triage patients for surgery . we believe that patients with cytological diagnosis of hr subcategory of fn in the jta system should be surgically treated especially if other risk factors coexisted . on the other hand , patients with lr or mr category therefore , surgical resection rate of indeterminate category is low in japan usually less than 50% , particularly in cases with fn .
background . 
 the japan thyroid association recently published guidelines for clinical practice for the management of thyroid nodules , which include a diagnostic system for reporting thyroid fine needle aspiration cytology . 
 it is characterized by the subclassification of follicular neoplasms , which is different from other internationally accepted reporting systems . 
 materials and methods . 
 this study examined observer variability in the subclassification of follicular neoplasms among 4 reviewers using papanicolaou - stained smear samples from 20 surgically treated patients with indeterminate cytology . 
 results . 
 the favor malignant subcategory had high predictive value of malignancy ( risk of malignancy : 6075% ) and good agreement among the 4 reviewers (  = 0.7714 ) . 
 conclusion . 
 these results clearly confirmed that the risk stratification of follicular neoplasms , which was adapted from cytology practice of high - volume thyroid centers in japan , can provide clinically helpful information to estimate the risk of malignancy and to triage patients for surgery .