Centers for Disease Control and Prevention 

National Institute for Occupational Safety and Health 

Ninth Meeting of the 

Advisory Board on Radiation and Worker Health
 

December 12, 2002
 

Conference Call Meeting 
National Institute for Occupational Safety and Health (NIOSH) 

Advisory Board on Radiation Worker Health (ABRWH) 

Record of the Teleconference held December 12, 2002 

The Advisory Board on Radiation Worker Health (ABRWH) held a teleconference on December 

12, 2002. The purpose of this meeting was to discuss the scope of work and technical evaluation 

plan for the procurement of technical consultation for the ABRWH for its review of dose 

reconstructions conducted in fulfillment of the Energy Employees Occupational Injury 

Compensation Program Act (EEOICPA). This procurement will aid the ABRWH in meeting its 

responsibility to review the scientific validity and quality of dose reconstructions completed by 

NIOSH and to assist the Board in advising the Secretary of Health and Human Services in 

making determinations for or against additions to the Special Exposure Cohort. Public comment 

was solicited during this call, but it was explained that questions regarding the scope of work and 

technical evaluation may not be answered due to specifications of the procurement process. 

When the final scope of work is published as a Request for Proposals (RFP), questions may be 

raised as specified in the RFP. 

ABRWH Chair Dr. Paul Ziemer convened the teleconference at 1:15 p.m. Committee members 

present were: 

Paul L. Ziemer 

Larry J. Elliott, Executive Secretary 

Henry A. Anderson 

Roy L. DeHart (attended until 3:00 p.m.) 

Richard L. Espinosa 

Mike Gibson 

Mark A. Griffon 

James M. Melius 

Wanda I. Munn 

Leon Owens 

Robert W. Presley 

Genevieve S. Roessler 

Member Antonio Andrade, Ph.D., did not attend. 

Attendees Included: 

Department of Health and Human Services (DHHS): 

Centers for Disease Control and Prevention (CDC), National Institute for Occupational Safety 

and Health (NIOSH): Larry Elliott, Cori Homer, James Neton, David Sundin, Elizabeth HomokiTitus, David Naimon, Ted Katz, Twila Saitow 

Department of Energy (DOE), Rocky Flats: Bob Eisline 

Department of Labor (DOL): Rose Toufexis, Jeff Kotsch 

Members of the public: 

Jeff Clemm, SAIC, Inc. 

Ray Green, Court Reporter 

Sylvia Keating, Pace International Union 

Arjun Makhijani, Institute for Energy and Environmental Research (IEER) 

Richard Miller, Government Accountability Project (GAP) 

2
 
 

John Morrow, Sanford Cohen and Associates 

Marie Murray, Recorder 

Herman Potter, Pace International Union 

Bob Tabor, Fernald Atomic Trades and Labor Council, Harrison, OH 

Opening Comments: 

Dr. Ziemer reported that former member Sally Gadola had resigned to avoid a possible 

perception of conflict of interest. Ms. Gadola is employed by the Oak Ridge Associated 

Universities (ORAU), which had been awarded the contract to conduct the dose reconstructions 

associated with EEOICPA claims. The Board agreed that Ms. Godola’s departure was a great 

loss for the Board. 

Dr. Ziemer thanked Mr. Griffon and the Dose Reconstruction Workgroup for their development 

of several working documents with which to procure technical consultative support to the Board. 

The documents: Attachment C: Statement of Work; Attachments D and E: Example Tasks 1 and 

2 (Basic and Advanced Reviews respectively); and, Attachment A: Technical Evaluation Criteria 

will be the main parts of the Request for Proposal (RFP) and incorporated into the subsequent 

contract. Attachment D and E provide basic and advanced, respectively, examples of the tasks 

the bidders will be asked to quote upon. Consensus on these documents is necessary to begin the 

procurement process. Thus, they were the focus of this teleconference meeting. For the process 

to advance in a timely manner, only minor changes to the SOW and the technical evaluation 

criteria would still be possible at the next (January 2003) meeting. 

After an overview of Attachments A and C was provided by Mr. Griffon, the Board discussed 

each section. This review and discussion was not editorial in character (e.g., for typographical or 

grammatical errors, etc.), but on the substance of the documents. Editorial comments and minor 

corrections were not discussed on this call and were to be sent in to NIOSH/OCAS. The 

following summary highlights discussion on specific passages from the documents. 

ATTACHMENT C: STATEMENT OF WORK 

This is delineated by criteria for basic and advanced review, as well as a “blind” dose 

reconstruction review. (Note: ¶ indicates “paragraph”) 

C.1: Purpose of contract: No comment 

C.2: Background and Need: 

¶3, Munn: Sentence 2: It is unclear to what “‘as least as likely as not’” relates; the 

employment, or the dose. The Act addresses employment. Edit to ensure this parallels 

the Act. 

¶3 Elliott: This paragraph contains language taken from another contract; NIOSH will adjust 

the date and ensure that “proposed” is dropped from reference to the rule. 

Attachment C.3: Contract Tasks lists three primary tasks: A) individual dose reconstruction 

review, B) NIOSH OCAS “Site Profile” and “Worker Profile Review”, and C) Review of 

Special Exposure Cohort (SEC) Petitions. The text of the latter was intentionally broad, since 

the SEC regulation has not yet been finalized. 

3
 
 

 

 

C.3.A Individual dose reconstruction reviews 

¶1 Elliott: Change line 2 to “as needed to adjudicate the claim”. 

¶1 Ziemer: Add for quality control, whether case determination includes if NIOSH followed 

their own guidelines in doing the dose reconstruction. Agreed to change this to “was 

done in a manner consistent with NIOSH dose reconstruction regulations with CFR...” or, 

after “other cases”, to add “...,and whether NIOSH followed its own guidelines in the 

dose reconstruction”. 

¶1 Change “contractor shall determine” to “contractor shall evaluate” 

C.3.A.1 Basic Review 

B-2: Naimon: Data for the dose estimate comes from many places, not just from the interview. 

Agreed: add “and if inconsistent, to evaluate if there is reasonable justification why the 

inconsistency is so” (e.g., the inconsistency could be due to a misinformed survivor’s 

comments). 

C.1 Munn: How could it be determined to what extent the benefit of the doubt was resolved 

in the claimant’s favor? Are they being asked to quantify it? Agreed: delete “and to 

what extent” 

C.3.A.2 Advanced Review 

B.2 Review work history interview and documentation provided by claimant. 

Ziemer: If the claim is successful, the customer satisfaction is probably a moot point; but it is 

unclear how satisfaction can be rated with an interview. Is that an appropriate role for an audit, 

or should it just be ascertained from the record that the interview was properly conducted? 

Comments included: 

!	 The question is what is to be validated? Is it the tool’s effectiveness in gathering all 

necessary information or the interview itself? If the latter, an independent auditor should 

ask the same questions. Changing them may garner different information but will not 

validate the tool. How can all the potentially needed questions be known from the start? 

Reopening interviews could open Pandora’s box. 

!	 Griffon: The idea was to both audit the method itself (does the questionnaire get the right 

result?) and the specific process (did the interview capture the relevant information from 

the interviewee and report it accurately?). 

!	 Issues impeding the re-interview relate to protecting confidentiality of the individuals 

(Privacy Act); the inference of an obligation/burden put back on the population; the 

requirement of an Office of Management and Budget (OMB) review and clearance to 

interact in this manner; and, assuring all legal requirements are met under such an 

approach. 

!	 Melius/Gibson: This does place the burden back on the claimants or their survivors, many 

of whom do not understand the process or what is/is not important information to convey. 

A tool is needed to check if the appropriate information was followed up on, which 

requires a re-interview. 

!	 The re-interview is not meant to re-adjudicate claims, but to ensure that the system in 

place works. Doing the audit of the process can be accomplished with the records at 

4
 
hand for a given claim. It is preferable to do so without going back to the claimants or 

persons interviewed. 

!	 The NIOSH process is straightforward. Standardized questionnaires are used, the 

claimant reviews both the draft interview report and the draft dose reconstruction, and 

signs the OCAS-1 form to signify agreement that all the information they had available 

was captured and addressed. All this information is in the administrative record and 

should be the initial basis for the audit of the process. There are many points along the 

process where a claimant can raise concerns about the development of their claim. 

!	 The review is a critically important part of the process and it is not envisioned many such 

follow-up interviews would be necessary. People are frequently less comfortable in an 

oral interview setting, which may produce inaccuracies. 

!	 Possible solutions: 

–	 Munn: Have the last question be whether the interviewee believes everything 

necessary was covered, or if they would like the interview to be conducted in 

some other way. 

–	 Anderson: To avoid irritating the claimants, pre-select cases and get their 

agreement at the first interview to doing a follow-up, or to tape the interviews for 

later use by the reviewer. 

!	 Mr. Elliott stated that the policy is not to record or tape interviews. He advised deleting 

B.1's text from the point of transcript on, and the parenthetical statements in B.1 and B.2. 

Related sensitivities are that this imposes another burden on the claimants (not all 

claimants would be comfortable with recording their interview); the potential causing 

further anguish of survivors by finding on follow-up that the claimant had passed away; 

the need for an OMB clearance (which will take an undetermined period of time to 

receive); and, assuring that all legal requirements are met under such an approach. If the 

details can be worked out later, that can be effected through a separate Task Order, but 

waiting to develop such details would prevent issuing a RFP with a proposed requirement 

now. 

–	 Responses: 

- Dr. DeHart would be satisfied with that since time is of the essence. 

- Dr. Melius was not willing to advance the SOW without a clear Board 

method to review the appropriateness of the process, such as this interview 

audit. He would prefer waiting for a DHHS response to the present 

proposed language. 

- Mr. Owen: The Paducah union often finds that the worker is deceased, but 

the widow generally appreciates the follow-up. And, the longer the 

program is delayed, the more claimants will pass on and the more the 

program’s credibility is eroded. 

- A literature review could explore if there is any gold standard for such 

interview processes to elicit the desired response. NIOSH can provide 

references to scientific literature on how survey instruments are used. 

!	 Elliott: The technical consultant contractor could attend and review interviews while they 

are in progress, and provide an evaluation of those cases after adjudication. 

– Problem: This would not be representative of the normal interview, and there 

could be legal problems if the claimant becomes involved in litigation. 

5
 
 

–	 If the claim is denied, the follow-up will surely entail bias. However, while this 

can be accounted for (e.g., if 20 awarded cases are happy and 20 denied claims 

are not), to make it fair, this would have to be done before the adjudication. 

!	 Mr. Griffon stated that the workgroup had originally had a task to review the method and 

procedures, but then rolled those in the individual cases. Had that not been done, the 

comparison with the literature would be possible. 

Conclusion: Reexamine this issue in January. To properly address the concerns of both the 

agency and the Board, it was agreed to table this until the January meeting. More information 

and direction from the Department should be available by that meeting. 

2.A.3	 Munn: If these data are not readily available, it would be a huge job to identify all 

relevant data sources, especially those outside the official record. Agreed: change 

“determine” to “evaluate if a reasonable effort was made to identify relevant sources...” 

There is a similar question in B.3. A rule of thumb for these eventualities should be 

generated at some point in future. 

C.3.B NIOSH OCAS Site and Worker Profiles and Review. The site profile incorporates all 

dosimetry data taken from all DOE sites in NIOSH’s database, and the worker profile includes 

all data and analysis on co-workers’ exposures, that could be potentially useful to claimants 

without data. These will be reviewed by the independent contractor. Both C.3.B and C.3.C are 

broad, as noted earlier. This is a ‘marker’ to alert the contractor that they might have to do tasks 

such as these. 

¶2 Line 1: Change “The contractor shall investigate...” which infers onsite study, to 

“review”, as in the next sentence. This is not intended to be comprehensive, as a site 

profile will be. This is the same issue as in 2.A.3 above. 

¶2 Delete “ / Contractor”; this refers only to NIOSH. 

¶2 The intent of the last sentence is that these interviews could be conducted at a hotel, not 

on a DOE site. 

¶2 Munn: as with 2.A.3, above, the term “site ‘experts’” will have to have some criteria 

assigned to it. The intent is to indicate people familiar with the site, but a minimum 

benchmark will be needed. 

¶4 All ten sites are expected to involve visits for interviews, but that is not realistic in the 

first year. Ten comprises 33% of the major DOE sites, all of which are huge. 

-- Neton: Change the number of sites to 5. Just because the site profile is not 

complete does not automatically mean a dose reconstruction cannot be done. 

–	 Elliott: Use phrases like “as needed, as deemed appropriate,” etc. Or, strike these 

two sentences completely, since Attachments D&E address that. 

–	 Agreement: leave this for a decision at the January meeting. 

C. Review of SEC petitions. This was left broad since there is no established policy as yet, and 

inserted to allow for technical assistance to the Board. This can be in the SOW despite the 

absence of the rule, but in view of the latter, NIOSH will strike the text in #2 to simply say the 

contractor “will review SEC petitions.” Again, this was inserted to be a marker for the 

contractor’s bid estimation. 

6
 
 

C.4 Work Assignments: No comments. This text and that of C.5 is boilerplate from past NIOSH 

work, slightly modified for this application. 

C.5 Report Preparation 

5.1 Ziemer: at the end of C.5.1, have the report sent simultaneously to the NIOSH Project 

Officer and the Chair of the Board. Agreed. 

Closing Comments on Attachment C:. 

The issue of reviewing the interview process will be a problem, preventing the SOW from being 

entered into the procurement process. However, NIOSH can work with Mr. Griffon to make the 

edits/changes discussed and to identify the issues remaining. Mr. Elliott will seek guidance from 

DHHS and the Office of General Counsel on how to proceed with regard to the interest to reinterview and record interviews. 

ATTACHMENT A 

A Personnel: No comment 

B Management Approach: No comment 

C Technical Approach: No comment 

D Past Performance. This Section D and Section E are designed to help evaluate the 

contractor’s expertise and approach to this work, so as to provide a level playing field for 

all the proposers. Dr. Neton reported that this was boilerplate NIOSH language. Some 

changes in department procedures will require some minor (not substantive) changes that 

will be in the text provided at the January meeting. Agreed; re-review in January. 

E Conflict of Interest: Will discuss at future meeting. 

¶¶2&3 The evaluation criteria involved areas of disagreement in the workgroup. Scrutiny will 

be focused on work with DOE, and atomic weapons employer (AWE), or contractors of 

either, in the past 5 years. This criterion includes key personnel who the contractor may 

engage to supplement their staff expertise. It also includes persons who had served as 

expert witnesses at any time in the past, on behalf of DOE, AWE, or contractors of either. 

The 5-year term was set arbitrarily, to address any perceptions of conflict of interest. 

Aspects discussed on this issue were: 

!	 Roessler: The 5-year term may eliminate the most technically up to date contractors for 

no valid reason. This should be either dropped or shortened. Previous statements 

developed by the workgroup left it up to the contractor to provide a conflict of interest 

form that could be evaluated. 

!	 Since NIOSH is engaging the services of both the contractor and the reviewing panel, 

being as prescriptive as possible seemed advisable to avoid any perception of conflict of 

interest. 

7
 
! The general feeling of the Board was that five years is too long; most contracts cite 1 

year. Two years would more than enough and probably still may eliminate most of those 

best qualified to do the work. Consultants may have multiple contracts, some with a 

minor DOE component. Those with DOE as the major source of funding are the ones 

bearing scrutiny, not 1-2 short projects. This should be listed as an evaluation 

consideration; then, the Board would want to discuss rationale of why or why not that 

person was chosen. 

! Gibson: People justifiably do not trust DOE so it is important to have someone 

completely divorced from DOE to do the evaluation. 

! If a perception of mistrust is the consideration, this should go much farther back than 5 

years, to work done when the exposures occurred, in addition to the key personnel and 

contractors involved in the epidemiologic dose estimate studies. 

! Mr. Elliott pointed out that NIOSH, not ORAU, will be evaluated, and the evaluation 

panel will have one Board member to participate in the selection process. The technical 

review panel works with confidential information and the names are not made public. 

But the Board representative can reassure the ABRWH members that this issue is 

addressed. 

! Dr. Ziemer suggested a minimum of 2 years without DOE work and looking back at work 

done over the last five years. 

! Melius: The perception of conflict could relate to the type and extent of the contractor’s 

past work for DOE (e.g., doing lab or quality control work versus work in radiation 

protection or dose reconstruction). As a compromise, he suggested “a minimum of 2 

years not working with DOE, and evaluation of a history of longer than 5 years, 

balancing the conflict of interest with expertise and other criteria”. 

! Although there was some feeling that two years may be more restrictive than necessary, 

there was agreement on the compromise language for this criterion of “at a minimum not 

working with DOE for the past 2 years, and evaluated on the degree/extent of work 

performed for DOE or a DOE contractor. If necessary, the bidder should include 

justification for key personnel if there is conflict of interest.” 

¶4 Issue of those involved in litigation. If a case was adjudicated for DOE, the contractor 

should not be involved in the panel at all; if they did so for an individual, they would just 

not address that particular case if it arose. 

Aspects discussed on this issue were: 

!	 Dr. DeHart felt there should be parity between the two situations, but had had to leave the 

teleconference before the SEC text review began. 

!	 Munn: There is no way of knowing how many people this would affect, and she disliked 

automatically eliminating anyone credentialed who testified on the science. 

!	 ORAU allegedly has 90 people on staff that meet this criterion, and ORAU accepted 

these criteria. The contractor should be at least as stringent. 

!	 Due to the deep distrust of DOE in the field, even the perception of any bias relative to 

the claimants must be removed. This criterion mostly pertains to perception. Most 

scientists are honest, but testifying for DOE clouds the issue. 

!	 There was agreement to retain the current language, but Dr. DeHart may raise it again in 

January. 

8
 
 

Attachment E: Example Task 2 

Footnote 1: Since information has been considered but not included as relevant since more data 

already supported the claim, add “that should have been considered.” 

Item B. When the interview issue is resolved, adjust this text as well. 

The agenda for the next meeting (January 7-8) will include: 

January 7/Day one: 

•	 A critique of the interview process 

•	 Vote on the document 

•	 Program status report 

•	 Update on dose reconstruction 

• Review of the latest version of these documents with wording resulting from this call. 

January 8/Day two: 

•	 Administrative Housekeeping and Board Work Schedule 

•	 Close session for development, review, and discussion of the proposed independent 

Government cost estimate for a contract. 

9
 
!	 If closure is reached on the scope of work on day 1 (Tuesday), then tentatively on 

Wednesday morning, a review may be done with NIOSH staff of the dose 

reconstructions completed by NIOSH and adjudicated to date (there are currently 

DOL decisions on ~7 of the 13 completed). At the start of the meeting, what 

can/cannot be publicly discussed from the administrative record will be presented. 

In small workgroups, the information used for the finalized claims will be 

reviewed with a staff member present to assist as needed. Since this will be a 

public meeting, the entire administrative record for each case will need to be 

redacted. NIOSH OCAS will determine if and how such a review might be 

accomplished. 

! On January 8, there will be a closed session to develop the independent 

government cost estimates for the work discussed on this call. 

! NIOSH was requested to: 

–	 Have the Office of General Counsel present the legal issues of the 

evaluation contract pertaining to the interviews. 

–	 Review NIOSH’s current plans for internal evaluation of the interview 

process, by NIOSH and by the contractor, to inform the Board’s 

recommendation on this issue. 

Since the redrafted SEC rule, which includes the comments from the Board, public input 

and the town hall meeting comments, has been substantially changed from the rule 

proposed last summer, it will not be ready for review and discussion in the January 

meeting. Another Notice of Proposed Rule Making will have to be issued. 

Public Comment: 

John Mauro, of Sanford Cohen & Associates, asked if the RFP announcement was 

scheduled and would it include anticipated work hours. Mr. Elliott replied that the RFP 

will specify a “not to exceed” amount of work hours and those interested should call 

OCAS or get on the list to receive the announcement of the RFP at the NIOSH/OCAS 

Website. 

Arjun Makhijani, Institute for Energy and Environmental Research, offered several 

comments. 

! He found the Attachment C.3.A criterion of “performed fairly ... consistent with 

other cases” to be ambiguous about the standard for judging performance, since 

some DOE work has been sloppy. A better thought-through statement is needed. 

!	 Regarding the workers whose applications are denied, he felt that historically, 

workers have generally been proven more right than the establishment in stating 

effects. Their technical credibility should be given greater weight than that given 

to establishment views or measurements, particularly since some DOE data are 

fraudulent. Their testimony should not be dismissed nor should they be 

considered hysterical or afraid of radiation. The distrust of DOE was well earned 

by its actions. This approach should be part of the criteria to select the 

independent reviewer. 

!	 The dose reconstruction should include assessment of whether the DOE data are 

fraudulent. Without that, neither the dose reconstruction process nor its audit will 

10
 
be very credible. A more straightforward process of putting people in the SEC

should consider that as well.

Richard Miller, of the Government Accountability Project, noted that the specifications

for the technical review personnel do not include previous experience in conducting dose

reconstructions. If more than one contractor is not selected, a blind review is not

possible.

Dr. Neton defined Mr. Miller's method as a double blind review (i.e., done by two

reviewers). This process of blind review starts from scratch and re-conducts the dose

reconstruction from the data used without any knowledge of the adjudication. NIOSH

could not provide any interpretation of this language until the SOW is fully developed,

but suggestions were welcomed.

Mr. Miller offered two forDls of advice: I) specify two auditors for a double blind review,

given the huge scope of an audit and the desirability of making it as "bullet proof' as

possible. And 2) the auditor should be someone with actual dose reconstruction

experience, and experience in dealing with contradictory and fraud records as well as

with uncertainty analysis and bounding techniques. These are all central to what needs to

be looked at in this process. He had circulated an e-mail with suggested text.

JeffClemm, ofSAIC, Inc., asked if the Section E text on conflict of interest refers to the

prime contractor pr the affiliation/team member. Again, there could be no response until

the RFP is released. And, given the Rule 83 as proposed, a dose reconstruction that

cannot be done should be reviewed by the Board and its contractors.

With no further comment, the conference adjourned at 4:07 p.m.

...

I hereby confirm that these minutes are accurate, to the best of my knowledge

. ~~~;~~~ A""'-:

Paul L. Zi , D., Chair

~--£ ? 

Date

-

-~

~

11 I