February 15, 2019

DISCLAIMER: This is a working document provided by the Centers for Disease Control and Prevention (CDC) technical support contractor, SC&A, for use 

in discussions with the National Institute for Occupational Safety and Health (NIOSH) and the Advisory Board on Radiation and Worker Health (ABRWH), 

including its Working Groups or Subcommittees. Documents produced by SC&A, such as memorandum, white paper, draft or working documents are not 

final NIOSH or ABRWH products or positions, unless specifically marked as such. This document prepared by SC&A represents its preliminary evaluation 

on technical issues.

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

ABRWH Dose Reconstruction Review Methods Work Group: Comment Matrix for the White Paper,

“Review of Areas of Professional Judgement in the Dose Reconstruction Process and Approaches for 

Assuring Consistency” (NIOSH 2017)

Observation 1 – Personal Dose Reconstructor Judgements

Several areas were identified where personal dose reconstructor judgements could potentially result in inconsistent assessment of a portion of 

an individual’s dose. Details of the individual professional judgements identified in the review of the Savannah River Site (SRS) and Linde 

Ceramics sites are included in the NIOSH 2017 attached SRS and Linde Ceramics reports and include: 

• Judgements regarding worker location for purposes of internal dose estimates and external dose estimates (photon, neutron, electron, 

and assumptions regarding sources of internal exposure) and assumed energy distribution 

• Judgements regarding job title and the associated potential for exposure (e.g., whether a job, not listed in ORAU-OTIB-0052—

Parameters to Consider when Processing Claims for Construction Trade Workers—should be treated as a construction trade worker 

job for purposes of estimating external dose, job title can affect the decision to assign ambient dose, coworker dose based on the 50th 

percentile of the distribution or coworker dose based on the 95th percentile of the distribution, etc.) 

• Judgements in the calculation of missed external and internal dose (using limit of detection (LOD)/2, coworker data, use of “nearby” 

data to fill gaps in dosimetry data, determination and use of minimum detectable activity (MDA) for assessing missed internal doses) 

• Consistency in reconciling discrepancies in available dosimetry data (e.g., annual external summary data versus cycle data) 

• Judgements in calculating internal dose based on in vivo and/or in vitro measurements for best estimate cases (fitting models, 

approach using measured data and values less than the MDA, estimate of doses for long periods without monitoring data, cases where 

both in-vivo and in-vitro data are available) 

• Judgements regarding calculating dose associated with incidents/events noted in the claimant interview or DOE records 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 2 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Status 

Update

Date

Commenter Comments/Response/Recommendations

11/5/2017 NIOSH

2017 

White Paper Recommendation 1: Assessments should be performed in the areas identified where personal 

professional judgements were made by individual dose reconstruction staff to determine consistency of 

judgements or assumptions. There are several means of assessment which may be useful in achieving the 

goal of determining whether there are inconsistencies in the areas of professional judgement including: 

1. Oak Ridge Associated Universities (ORAU) blind reviews – one case done by two different dose 

reconstructors (for SRS two different staff that work on SRS cases). 

2. ORAU focused reviews – select areas of one case (or multiple cases) compared to determine if 

judgements were consistent. These type of reviews could lead to the identification of areas where 

procedures or guidance is ambiguous. 

3. National Institute for Occupational Safety and Health (NIOSH) blind reviews – similar blind review 

as described in item 1 conducted by NIOSH staff rather than ORAU. 

4. NIOSH focused reviews – similar reviews as described in item 2 above. 

5. Advisory Board on Radiation and Worker Health (ABRWH, Board) blind reviews – Board review 

(with Board contractor). Could be particularly useful for Atomic Weapons Employer (AWE) sites that 

do not have a technical basis document. 

6. ABRWH focused reviews – Board review (with Board contractor). The external Board review may be 

difficult if the cases do not include detailed documentation and basis for the professional judgement. 

7. Refine current peer review conducted by NIOSH to assure a greater percentage of best estimate cases 

undergo a comprehensive review. The current procedure provides for 5% random sample of all 

completed cases undergo extensive peer review by NIOSH. NIOSH should consider biasing the 

sampling to select a greater percentage of best estimate cases for the comprehensive review. 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 3 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Status 

Update

Date

Commenter Comments/Response/Recommendations

9/13/2018 Methods 

WG

These observations/recommendations were discussed at the Methods Work Group (WG) meeting held on 

September 13, 2018. It was determined that Oak Ridge Associated Universities Team (ORAUT) does not 

have the mechanism in place to do blind reviews, and NIOSH has resource limitations. Recommendation 

bullet point 6 was discussed as a good starting point. SC&A was tasked with identifying the findings from 

past reviews where a consistency/professional judgement issue was raised. This will be used to identify the 

areas where consistency issues have arisen in the past and to establish priorities.

Observation 2 – Program Judgments

Professional judgements are necessarily a part of the development of technical basis documents (TBDs) and procedures. These decisions are 

made at a program level and undergo extensive internal and external review, including review by the ABRWH through the Subcommittee for

Procedure Reviews, the site profile reviews, and, in some instances, the reviews associated with Special Exposure Cohort evaluation reports. 

Some of these judgements are site specific, while some are judgements made regarding cross-cutting or what have sometimes been 

characterized as “global” issues. Some of these issues include: 

• The method for developing and using coworker models and the assumption that the approach is bounding for all workers has been 

reviewed and debated extensively by the ABRWH – in general terms and regarding site-specific use. A summary of the general 

approach may be useful. 

• The logic used in determining whether potential exposure from an incident (short-term acute exposure) is or is not bounded by 

assessment of dose from routine measurements over a long period of time (assuming a chronic exposure). 

• The method used for estimating uncertainty associated with external and internal doses. While it is clear this issue has been discussed 

extensively over the life of the program, it may be useful to summarize the approach to uncertainty and the reviews conducted. 

• For some sites (specific time period at a site) there is an underlying, general, assumption that if individuals should have been 

monitored they were monitored (assumption of a “robust” radiation safety program). The SRS dose reconstruction (DR) guidance 

document states that “For 1989 and later it is generally assumed all employees that needed monitoring were monitored.” The basis for 

the assumption is included within the guidance; nonetheless, it is a program assumption that could influence the approach for 

estimation of doses during the time period in question. For SRS it is also assumed, due to the inexpensive nature of tritium sampling, 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 4 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

that all workers requiring monitoring were monitored. These assumptions were extensively reviewed (internal and ABRWH review), 

and it may be useful to summarize the basis for the assumptions and the ABRWH review. 

Status 

Update 

Date

Commenter Comments/Response/Recommendations

9/13/2018 Methods 

WG

This observation was discussed at the Methods WG meeting held on September 13, 2018.

Observation 3 – Program Judgments

The external dose matrix for the Linde Ceramics site operational period (ORAUT-TKBS-0025, Table 4-24) is based on a variety of data and 

is rather difficult to recreate from underlying data. However, the approach used in deriving the values in this table (which are the basis for the 

calculations of individual external doses) has been extensively reviewed by SC&A and the ABRWH, with all issues being “closed” during the 

resolution process. Since site matrices for several other AWE sites are based on similar types of underlying data, a review and comparison for 

consistency may be useful.

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

11/5/2017 NIOSH

2017 

White Paper Recommendation 2: A summary document should be developed for several of the program 

assumptions, including but not limited to what NIOSH has defined as global issues. A document similar to 

that produced by NIOSH regarding the treatment of residual contamination seems appropriate.

[date] [name] Comment on Recommendation 2: (placeholder line left blank for future comments)

11/5/2017 NIOSH

2017 

White Paper Recommendation 3: Since site matrices for several AWE sites, in addition to Linde Ceramics,

are based on similar types of underlying data, a review and comparison for consistency in methods may be 

useful. 

[date] [name] Comment on Recommendation 3: (placeholder line left blank for future comments)
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 5 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Observation 4 – Detailing Approach Used in Dose Reconstruction

ORAU has made great improvements in including all work in case files. Most notably, due to a recommendation from the ABRWH, the 

inclusion of DR guidance within each case file became standard procedure. Additionally, ORAU includes multiple IMBA dose calculation 

runs to demonstrate that the most claimant-favorable approach regarding internal dose estimation was adopted for the final DR report. It 

appears, however, that the level of specificity for the DR Notes or DR guidance varies from site to site. This may be due to the nature of the 

specific issues with each site; however, it may be something that should be discussed by DR teams and, where appropriate, standardized. 

Status 

Update 

Date

Commenter Comments/Response/Recommendations

[date] [name] (placeholder line left blank for future comments)

Observation 5 – Detailing Approach Used in Dose Reconstruction

The DR guidelines are not controlled documents, and yet it seems they are very important “procedures” regarding site-specific DR “rules.” It 

is unclear what triggers a change in DR guidelines, whether the change in DR guidelines results in other changes (e.g., workbook 

modification), and whether cases that could be affected by such a change should be recalculated based on new guidelines. It seems that cases 

are reviewed, when warranted, after significant changes are made to a controlled document (e.g., TBD) but not necessarily when DR guidance 

is changed. This is a difficult issue to resolve because, on the one hand, the TBD review by the Board can take quite some time, but on the 

other hand, doing multiple reworks of cases is not efficient and also could bring into question the credibility of the program. It is of note that 

the SRS TBD has not been revised since 2005, and a few important technical information bulletins have also not been updated since 2003. 

The latest SRS DR guidance was developed in 2016, and there have been 16 revisions since 2009. To look at this more closely, an early 

version of the DR guidance (April 27, 2011) was compared with a recent version of the DR guidance (August 2, 2016) (NIOSH 2017, 

Attachment 3a), and a comparison was made between DR guidance (September 2, 2015) and the more recent version (August 2, 2106) 

(NIOSH 2017, Attachment 3b). It is clear that there are significant differences, even when comparing the 2015 version with the 2016 version. 

One example of a change made after 2015 was the addition of a table including MDAs to be used when assessing post-1990 lung count data. 

Again, the question is whether reassessment of any cases should be done after “significant” changes are made to DR guidance (prior to 

updates of the TBD that would trigger such a review).
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 6 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

[date] [name] (placeholder line left blank for future comments)

Observation 6 – Detailing Approach Used in Dose Reconstruction

The inclusion of what has been called a “DR timeline” within the case file seems like something that would be valuable, especially for best 

estimate cases. This would allow for a more straightforward review process and would allow a better understanding of assumptions (where 

applicable, e.g., assumed work location). ORAU mentioned that this was done for some SRS DRs but was not required. Some Hanford cases 

do include a Microsoft Excel spreadsheet titled “dose reconstruction timeline.” In the notes on the timeline preparation sheet, the first note 

states, “The supplied time line or others may be applied. However, a timeline should be used for most all but the simplest of Dose 

Reconstructions, since they help the DR assure consistent and systematic dose reconstruction, assure all information is considered, and 

provide for a final check of completion. They help the PR understand the DR’s approach, thus expediting the review process.” It seems this 

approach should be standardized and included in files for at least all best estimate cases.

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

9/13/2018 Methods 

WG

This observation was discussed at the Methods WG meeting on September 13, 2018. NIOSH will look into 

the possibility of including a timeline in cases. It is unclear the level of work required to add this detail or if it 

is necessary in all cases. NIOSH will report back with what might trigger the inclusion of a timeline.

Observation 7 – Detailing Approach Used in Dose Reconstruction

While great strides have been made over the life of this program regarding the inclusion of documentation within the DR case file, better 

documentation in the individual case files, for best estimate cases, would be very helpful in determining if the appropriate process was 

followed. This is very important for areas where professional judgement comes into play. Rather than trying to “read the mind” of why a DR 

staff person made the assumption they did, it should be documented so as to avoid any confusion. While there is certainly a limit to how 

detailed of a roadmap is needed, these kind of questions have come up over the years in the ABRWH Subcommittee for Dose Reconstruction 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 7 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Reviews. It slows the resolution process when those involved in the review are left to speculate what they believe the dose reconstructor did 

and how they got the result they did. 

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

11/5/2017 NIOSH

2017 

White Paper Recommendation 4: NIOSH/ORAU should review DR notes or guidance to consider whether 

some degree of standardization is warranted or useful. NIOSH/ORAU should (a) consider using a more 

standardized form for the DR guidelines for sites where they are necessary and (b) consider requiring the 

inclusion of a case narrative document that specifies the judgements made and the basis for the judgement. 

Such details are included within some case files (sometimes within comment fields within DR workbooks), 

but it does not appear to be done on a consistent basis. This may be useful for best estimate cases but 

probably not as important for over- and underestimating approaches. 

[date] [name] Comment on Recommendation 4: (placeholder line left blank for future comments)

11/5/2017 NIOSH 

2017 

White Paper Recommendation 5: NIOSH/ORAU should consider whether reassessment of any cases 

should be done after “significant” changes are made to DR guidance (prior to updates of the TBD that would 

trigger such a review).

[date] [name] Comment on Recommendation 5: (placeholder line left blank for future comments)

11/5/2017 NIOSH 

2017 

White Paper Recommendation 6: NIOSH/ORAU should consider including a timeline for, at a minimum, 

best estimate cases.

[date] [name] Comment on Recommendation 6: (placeholder line left blank for future comments)

Observation 8 – Quality Assurance and Quality Control

The quality assurance/quality control program has been greatly enhanced since about 2005, and even more so after about 2011. This can be 

seen by the reduction in “errors” shown on PDF p. 21 of the professional judgment white paper (NIOSH 2017). It is interesting to note that 

these recorded technical errors include “differences due to disagreement about technical approach.” A very important aspect of the continuous 

improvement of the program is to understand how these differences were resolved. 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 8 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

From September 2012 to September 2016, there were 18 SRS cases with technical comments (for a total of 35 technical comments – several 

cases had multiple comments). ORAU did not categorize any of the comments as professional judgement comments (the NIOSH 2017 

independent categorization resulted in 4 of the 35 having a professional judgement component). Earlier data from NIOSH (from November 

2003 through April 2004) included a total of 304 technical findings and 108 SRS findings. Thirty-nine of the SRS findings included a 

professional judgement component (based on the NIOSH 2017 assessment – not categorized by NIOSH). Quality assurance and quality 

control tracking are very important components of the program, and it seems clear that these elements have been improved (especially since 

2012 – a much-improved system for tracking and categorizing comments). 

It is clear that since 2012, the Division of Compensation Analysis and Support review comments (ORAUT-PROC-0077) are being tracked in 

a database. The case-specific comments (Form 35 comments – DR Comments) are also included in the individual case file. It is less clear, 

however, how these findings/comments are considered in aggregate and whether they result in changes in the DR process. 

It appears that since about 2005, a formal internal review process was in place (ORAUT-PROC-0059). The case-specific comments (Form 41 

comments) do not appear to be included within the case file – these are internal ORAU documents. It is clear that these comments are 

considered in aggregate, and improvements in the tracking system and feedback process have been made. It seems that the database tracking 

this information is internal to ORAU. It is clear, based on a November 2012 presentation to the ABRWH Subcommittee for Dose 

Reconstruction Reviews (ABRWH 2012), that this information is considered and used to make improvements in the DR program. 

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

11/5/2017 NIOSH 

2017 

White Paper Recommendation 7: It is recommended that a tracking mechanism should be developed, to the 

extent possible, to consider findings/comments from all reviews (peer review, NIOSH review, ABRWH 

review, other?), in an aggregate fashion, for purposes of improving dose reconstruction methods and in 

assuring consistency in areas of professional judgement.

[date] [name] Comment on Recommendation 7: (placeholder line left blank for future comments)
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 9 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

11/5/2017 NIOSH 

2017 

White Paper Recommendation 8: For cases where a “significant” amount of professional judgements were 

necessary, it may be useful to have additional level of review prior to finalizing the case. ORAU has 

indicated that since 2012, they have required a second peer review for all cases with a probability of 

causation between 40% and 52%. Based on ORAUT-PROC-0077, NIOSH conducts an extensive review on 

5% of all cases (randomly selected from NOCTS). NIOSH may want to consider a biased sampling approach 

to select a greater percentage of the best estimate type cases.

[date] [name] Comment on Recommendation 8: (placeholder line left blank for future comments)

Observation 9 – Use of CATI Information

The current approach requires the dose reconstructor to consider all incidents or accidents mentioned by the claimant in completing the DR. 

In the course of the NIOSH 2017 review, ORAU commented that “the presence of enough technical information to address an incident in 

detail is unusual” and that “the ORAU Team uses the best information available from the claim files and the interviews, but most mentions of 

incidents are usually generic in nature and cannot be specifically assessed without further technical information” (PDF p. 22). While this is 

understandable, it is worth noting that in the course of the NIOSH 2017 assessment, a DR was identified in which an individual described a 

period of neutron exposure, and although the claimant had no recorded neutron dose for that time period, the initial DR report included an 

estimate of missed neutron dose from this activity that was much greater than the missed dose that would have been assigned. It may be 

useful to know whether others were involved in this job during this specific time period. This information, while often lacking the desired 

detail, may be useful if it is possible to extract such information without too great of a burden and consider it in aggregate. Further, in 

querying specific job types, NIOSH 2017 identified a great deal of computer-assisted telephone interview (CATI) reports with detailed 

accounts of events/incidents (often with specific buildings and time periods). This likely is not the case for many of the CATI reports, but 

consideration of this information in aggregate may provide valuable and useful information. 
DR Professional Judgement Issues Matrix Date: 2/15/2019 Page 10 of 10

NOTICE: This document has been reviewed to identify and redact any information that is protected by the Privacy Act 5 USC §552a and has been cleared 

for distribution. 

Status 

Update 

Date

Commenter Comments/Response/ Recommendations

11/5/2017 NIOSH 2017 White Paper Recommendation 9: NIOSH and ORAU may want to consider the use of CATI interview 

information and outreach information (information obtained from outreach meetings and employee/expert 

interviews conducted by SC&A and NIOSH) in aggregate form. Perhaps a pilot test (perhaps based on 

looking at data provided by a certain subgroup of claimants from a site) can be done on one site to 

consider the feasibility of extracting such data and their utility in the overall dose reconstruction program.

References

ABRWH 2012. Transcript: Meeting of the Advisory Board on Radiation and Worker Health, Subcommittee on Dose Reconstruction Review, 

Hebron, KY. November 27, 2012. 

NIOSH 2005a. Dose Reconstruction Error Tracking and Reporting, ORAUT-PROC-0077, Revision 00, National Institute for Occupational 

Safety and Health, Cincinnati, OH. March 28, 2005. 

NIOSH 2005b. Peer Review of Dose Reconstructions, ORAUT-PROC-0059, Revision 00, National Institute for Occupational Safety and 

Health, Cincinnati, OH. September 6, 2005. 

NIOSH 2014. Parameters to Consider when Processing Claims for Construction Trade Workers, ORAU-OTIB-0052, Revision 02, National 

Institute for Occupational Safety and Health, Cincinnati, OH. July 24, 2014. 

NIOSH 2015. An Exposure Matrix for Linde Ceramics Plant (Including Tonawanda Laboratory), ORAUT-TKBS-0025, Revision 04, 

National Institute for Occupational Safety and Health, Cincinnati, OH. May 8, 2015. 

NIOSH 2017. Review of Areas of Professional Judgement in the Dose Reconstruction Process and Approaches for Ensuring Consistency

[White Paper], National Institute for Occupational Safety and Health, Cincinnati, OH. November 5, 2017.