Department  of Health  and  Human  Services 
OFFICE  OF 
INSPECTOR  GENERAL 
THE  CANCER  INFORMATION  SERVICE 
Expanding  Telephone Access 
JUNE GIBBS BROWN 
Inspector  General 
MAY  1998 
OEI-09-97-0360  OFFICE  OF INSPECTOR  GENERAL 
The mission of the Office of Inspector General(OIG), asmandatedby Public Law 95-452, is to 
protect the integrity of the Department of Health and Human Servicesprograms aswell asthe 
health and welfare of beneficiaries servedby them. This statutory mission is carried out through 
a nationwide program of audits, investigations, inspections,sanctions,and fraud alerts. The 
Inspector Generalinforms the Secretaryof program and managementproblems and recommends 
legislative, regulatory, and operational approachesto correct them. 
Office  of Evaluation  and Inspections 
The Office of Evaluation and Inspections (OEI) is one of severalcomponentsof the Office of 
Inspector General. It conductsshort-term managementand program evaluations (called 
inspections)that focus on issuesof concernto the Department, the Congress,and the public.  The 
inspection reports provide findings and recommendationson the efficiency, vulnerability, and 
effectivenessof departmentalprograms. 
OEIs SanFranciscoRegional Office  preparedthis report under the direction of Kaye Kidwell, 
Regional Inspector General. Principal OEI staff included: 
REGION  HEADOUARTERS 
Paul Gottlober, Project Leader 
 Alan Levine, Program Specialist 

Carrie Lozano, Lead Analyst 
 BarbaraTedesco,Mathematical Statistician 

StevenZerebecki, Lead Analyst 

Jennifer Miller,  Consultant 

Kathy Dezotte, Program Assistant 

To obtaincopiesof this report,pleasecall the SanFranciscoRegionalOffice at 41S-437-7900. 
Reportsare also available on the World Wide Web at our home page address: 
http://www.dhhs.gov/progorg/oei  EXECUTIVE  SUMMARY 
PURPOSE 
To identify and evaluatethe causesfor the high busy signal rates, abandonmentrates and wait 
times on the Cancer Information Service (CIS) toll-free telephone service and to determine how 
to reduce or eliminate them. 
BACKGROUND 
The Cancer Information Services (CIS) mission is to be the voice of the National Cancer 
Institute (NCI) by providing the most current NC1 information to cancer patients, their 
families and friends, health professionals and the public at large.  The CIS disseminatescancer 
information directly to consumersthrough its toll-free telephone service (l-800-4CANCER) 
and indirectly through its outreach program. 
Since 1976, CIS has provided cancer information to the public via its network of contractors, 
currently numbering 19.  Callers are automatically routed to the contractor responsible for 
their geographic area. At eachcontractor, information specialistsanswer calls about the 
causes,prevention, diagnosis and treatment of cancer. They also refer callers to local 
community organizations.  Contractors are also required to provide assistanceto community 
organizations through an outreach program.  Estimated contractor funding in 1997 was 
approximately $15.8 million.  
The CIS Branch, located in NCIs Office of Cancer Information,  Communication and 
Education, is responsible for contractor and program oversight. 
Accessto the service can be measuredin total calls taken, busy signal rates, abandonmentrates 
(the percentageof callers waiting in queuewho disconnectbefore reaching staff) and wait 
times (the averageamount of time a caller is on hold before speaking with staff).  In 
1997, almost 1 out of every 3 of the attempted calls (or 240,860 calls out of 834,185) to CIS 
failed to reach an information specialist. Busy signal rates for the individual contractors 
ranged from 10 to 52 percent during the sameperiod.  Network-wide  averagebusy signal rates 
for 1995 and 1996 were similarly  high.  According to a National Performance Review Study, 
the top ranked private sector telephone servicesexpect busy signal rates of less than 2 percent. 
Our methodology consistedof mail surveys, telephone and in-person interviews, focus groups 
and a review of CIS-related statistics and information. 
1  Source: CIS Branch 
i  FINDINGS 
The CIS is a valuable public  service enhanced  by its dedicated  staff and strong  training  and 
quality  assurance programs 
The CIS is a unique service that the public relies on for the most up-to-date, scientific 
information on cancer. Despite stressand frustration, staff remain dedicated to the mission 
and goals of CIS.  A comprehensivetraining program and practical evaluation tool strengthen 
the service and serve as models for others to emulate. 
The CIS does not take full  advantage  of current  telephone  technology  to improve  access and 
customer  service 
The CIS does not use its phone technology to route callers to the first available information 
specialist.  Furthermore, it does not provide callers with important recorded messagesthat 
could preclude them from reaching a busy signal or waiting on hold. 
Data gathering  requirements  adversely affect call efficiency 
Although random sampling is a widely-accepted practice in both the private and public sectors, 
CIS requires contractors to document 100 percent of calls and collect demographic data on half 
of them.  The software used to record the data is poorly designedand is accompaniedby 
unnecessarilyrestrictive policies, such as requiring information specialiststo document the 
sameinformation in both code and narrative form.  The CIS staff regard the software and 
related policies as critical factors that inhibit their efficiency. 
Information  specialists  could  be more efficient  if their  resources  were more readily  accessible 
and user-friendly 
Staff identified severalproblems with their main resource, the Physician Data Query.  They 
also reported that non-computerized resources,such as Fact Sheets, are difficult  to maintain 
and update and that their community service information is frequently inaccurate and 
incomplete. 
The regional  structure  does not contribute  to a consistent  and efficient  phone  service 
Demographic differences, the number of phone lines and the distribution of resourcesare some 
of the variables that have led to inconsistent contractor performance.  Contractor performance 
is also affected by the level of support received from parent institutions as well as local 
managementpractices. While contractor staff continually and consistently volunteered that the 
current regional structure enhancesthe personalized service that callers receive, we found 
little  supporting evidence. 
ii  The USs  customer  service standards  do not address access to the toll-free  service 
The CIS Branch has not set minimum accessstandardsfor CIS contractors.  For example, CIS 
contractors are not expectedto keep their busy signal rates under a specific threshold.  Similar 
health information organizations and other telephone service centersthroughout the country 
have implemented minimum  standards. 
RECOMMENDATIONS 
The Cancer Information Service, through its telephone service, provides the public with an 
invaluable resource for information about cancerprevention, causesand treatment.  To expand 
accessto the toll-free service, the Office of Cancer Information,  Communication and Education 
should: 
. 	 Complete and implement plans to upgrade CIS telephone technology to enhance 
contractors ability to respondto calls and provide information to all callers. 
.  Establish technical requirements and performance standardsfor contractors. 
.  Modernize and correct deficiencies in the Physician Data Query. 
. 	 Improve efficiency by reducing data gathering requirements and computerizing 
more reference materials, such as Fact Sheets. 
. 	 Discontinue collecting and disseminating community service information; 
instead, partner with national organizations to provide this information. 
.  Re-evaluatethe regional structure. 
.  Encourage contractors to further enhanceCIS staff training. 
AGENCY  COMMENTS  AND  OIG  RESPONSE 
We received written comments on the draft report from NC1 (seeappendix B). We are 
encouragedthat NC1 already has taken action on some of our recommendations. However, we 
remain concernedabout inefficiencies in vital areasof the program and would like to clarify 
and reassertthe following  recommendations. 
We continue to believe that (1) to improve efficiency, NC1 should quickly adopt a statistically 
valid sampling methodology for all CIS data recording and collection efforts, (2) the CIS 
Branch and its contractors should ceaseduplicating private sector efforts by collecting, 
maintaining and disseminating information on community resourcesand (3) NC1 should 
. . . 
111  reconsider the regional structure of the phone service. The relationships between the phone 
service, the outreach program and researchactivities do not justify  the current inefficient 
regional structure. 
iv 
 PAGE 

EXECUTIVE  SUMMARY 

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

FINDINGS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..f............................  6 

Valuable public service,dedicatedstaff, strongtraining and quality assurance ........  6 

Currenttelephonetechnology  ..............................................  . 

Burdensomedata gathering ...............................................  .9 

Accessibility of resources ................................................  11 

Questionableregional structure ............................................  12 

Customer servicestandards ...............................................  15 

RECOMMENDATIONS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 

APPENDICES 
A:  CIS Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  A-l 
B: NCIs comments on the draft report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-l  PURPOSE 
To identify and evaluatethe causesfor the high busy signal rates, abandonmentrates and wait 
times on the Cancer Information  Service (CIS) toll-free telephone service and to determine how 
to reduce or eliminate them. 
BACKGROUND 
The Cancer Information  Services (CIS) mission is to be the voice of the National Cancer 
Institute (NCI) by providing the most up-to-date information from NC1 to cancer patients, 
their families and friends, health professionals and the public at large.  The CIS disseminates 
cancer information directly to consumersthrough its toll-free telephone service 
(l-800-4CANCER)  and indirectly through its outreachprogram. 
The NC1 is the federal governments principal agency dedicatedto understanding the nature of 
cancer. The NC1 is a scientific institution with the majority  of its resources(more than 
$2 billion  in 1997) being spent on researchinto the causes,treatment and prevention of cancer. 
In 1971, President Nixon signed the National Cancer Act, authorizing additional funds for 
cancer research, prevention and education through the initialization  of the National Cancer 
Program.  In 1974, an amendmentto the Act made the CIS possible by mandating that the NC1 
Director: 
. ..provide a contract for a program to disseminateand interpret, on a current basis, for practioners 
and other health professionals, scientists, and the general public scientific and other information 
respecting the cause,prevention, diagnosis, and treatment of cancer. 
In the following  2 years, NCIs Office of Cancer Communications and Division of Cancer 
Prevention and Control jointly  conceived a national toll-free telephone information service. 
The NC1 offered contracts to its ComprehensiveCancer Centers (i.e., universities, hospitals 
and researchcenters) (1) to maintain regional information services, using information on both 
researchand treatment provided by NC1 and (2) to develop and maintain regional databaseson 
clinics and organizations serving cancerpatients.  In 1988, the Office of Cancer 
Communications assumedfull oversight of the contractsthrough the CIS Branch.  The CIS 
Branch and the International Cancer Information Center (the publisher of the Physician Data 
Query) are now under a new division called the Office of Cancer Information,  Communication 
and Education. 
In late 1976, the CIS telephone service becameoperational.  By that time, NC1 had contracted 
with  17 ComprehensiveCancer Centersthat respondedto calls within  separate,specific 
geographic regions using individual toll-free telephone numbers.  Available telephone 
technology in 1976 did not allow call routing to different regions via a single, national toll-free 
number.  The new CIS contractors did not service the entire nation.  Calls from areasnot 
1  covered by CIS contractors were answeredby an alternate centrally-located contractor.  In 

1983, CIS implemented a single, national toll-free number (l-800-4CANCER).  Calls are 

routed automatically from the single number to the CIS contractors, basedon the callers 

locations. 

The current number of contracts is 19.  Each contract respondsto calls within  a specific 

geographic region.  The sizesof the regions vary from covering a portion of a single stateto 

up to six full states. Together, the 19 contractsprovide service for the entire nation and Puerto 

Rico in English, Spanishand teletypewriter (TTY)  for the hearing impaired.  In addition to the 

19 contracts, CIS contracts for a Publications Ordering Service in Maryland.  The publication 

service fulfills  NC1 publication requestsfor the entire nation and also is reachedvia 1-800-

4CANCER. 

In addition to a small Federal staff in the CIS Branch, the CIS is staffed by a network of 

information specialists, telephone service managers,project directors, outreach managersand 

coordinators and support staff who are employeesof the 19 CIS contractors.  Information 

specialistsare the voice of NCI;  they answer incoming calls and provide information requested 

by callers.  They use NC1 publications and NCIs on-line databaseof cancer treatment, 

prevention and clinical trial information called the Physician Data Query (PDQ) as first-line 

resourcesto respond to callers.  Information specialistsalso use community service databases, 

individually  developed and maintained by the 19 contractors, to respond to callers requestsfor 

community information.  Information  specialistsdocument eachcall on-line using the 

Electronic Call Record Form.  The form is used to record the reason for and responseto the 

call, the resourcesused during the call and the callers demographic information. 

The CIS Branch regularly evaluatesthe phone service by performing anonymous test calls to 

all contractors.  The tool used for this evaluation is the CIS Telephone Evaluation and 

Reporting System (CISTERS).  In short, CISTERS evaluates: 

t  the correctnessof the information provided; 

b  adherenceto policies and procedures; 

b  the completenessof the call; 

b  the appropriate use of resources; 

t  customer service (making sure the caller understood and received the information he or 

she desired); 
b  the information specialists demeanor, communication skills and credibility  and 
t  overall customer service. 
On the basis of CISTERS test call results, CIS identifies areasthat require improvement.  The 
CIS Branch regularly provides contractors with reports of CISTERS test calls. 
Telephone call statistics are maintained on a daily basis by the contractors and a monthly basis 
by the CIS Branch.  Daily  statistics are aggregatedinto monthly reports.  The Basic Call 
Management System, AT&Ts  Federal Telecommunications Service (FTS2000) Management 
System and the Electronic Call Record Form are CISs instruments for measuring telephone 
statistics. Each contractor collects daily telephone statistics using the Basic Call Management 
2  System, a computer system linked to the information specialists phone extensions. The 

AT&T  provides separatedata from the FTS2000 Management System. Both the Basic Call 

Management System and FTS2000 record the number of calls taken, busy signal rates, 

abandonmentrates and the time connected. The Electronic Call Record Form records talk time 

for eachcall, a count of calls answeredduring a given period of time and information about the 

content of the calls. 

The number of calls reported by the Basic Call Management System, FTS2000 and the 

Electronic Call Record Form do not match.  As a result, accuratefigures on the actual number 

of calls CIS is processing do not exist.  The CIS Branch recently contracted with an 

engineering firm to explain the data discrepanciesand recommend changes. The firm also is 

analyzing telephone staffing and regional managementconfigurations. 

In addition to answering the toll-free line, CIS contractors are required to operate an outreach 

program and conduct limited research. Each contractors outreach coordinator servesas a 

liaison between CIS and other organizations interested in disseminating NC1 information and 

promoting NC1 messages. Outreach efforts also addressspecific cancer issues, such as breast 

and cervical cancer and clinical trials recruitment, with a particular focus on underserved 

populations and those individuals who have difficulty  accessinghealth information. 

Total funding for the 19 CIS contractors in 1997 was $15,745,983, a decreaseof 9.4% from 

the 1996 funding level due to an NC1 mandate. In previous years, funding was: 

1993  1994  1995  1996 
$7,640,094  $17,272,420  $18,047,134  $16,786,158 
Phone  Service Access 
Since its inception, CIS has answeredan increasing number of calls eachyear, from a low of 
approximately 47,000 in 1976 to a high of almost 600,000 in 1997.3 The phone service 
operatesfrom 9:00 AM  to 4:30 PM, local time, Monday through Friday.  Accessto the 
service can be measuredin total calls taken, busy signal rates, abandonmentrates (the 
percentageof callers waiting in queuewho disconnectbefore reaching staff) and wait times 
(the averageamount of time a caller is on hold before speaking with staff). 
In 1997, almost one out of every three calls to l-800-4CANCER  during operating hours failed 
to reach an information specialist.  Almost 29 percent of the 834,185 attempted calls (i.e., 
240,860 calls) made to l-800-4CANCER  resulted in a busy signal.4 Additionally,  there are 
2  Source: CIS Branch 
3  Source: AT&T. 
4 	 Using CISs presentAT&T technology, it cannot be determinedhow representativethe busy 
signal rate is of the actual number of people who are attempting to reach the service and failing. 
3  wide ranges in the accessstatistics for the 19 individual contractors.  For example, in 1997 the 
averagebusy signal rate per contractor varied from 7 percent to 53 percent.  If the Spanish 
telephone lines are included, the margin widens from 4 percent to 67 percent.  The CIS Branch 
reports similarly  high busy rates for 1995 and 1996. 
Contract funding for 19975indicates contractor spendingper completed call varied from 
$29 to $74 for 18 of the 19 contractors.  The averagecost per call for the 19thcontractor was 
approximately $150.  According to CIS, this unusually high cost per call can be attributed to 
the contractors reliance on outreach and the low volume of calls it receives. The averagecos 
per call (total contract spending divided by total number of completed calls) for the entire CIS 
network was $41.  The variance among contractors may be explained, in part, by contractor 
spending on their outreach programs and the fact that contractors are required to employ 
managementstaff in equal numbers, regardlessof the number of lines they operate. 
CIS  Contractor  Variances  (1997) 
Average  Average  Average  number  of 
Lines  Contractors 
busy  signal  rates  cost  per call  completed  calls  per  line 
3  I  1  15%  $1 506  480 
4  7  7-30%  $29- 74  2921  - 4977 
5  I  5  22-32%  $23-41  3933-6301 
6  6  9-50%  $29-55  3799-4744 
Telephone  Service Benchmarks 
On September 11, 1993, President Clinton signed Executive Order 12862 on Setting 

Customer Service Standardsin government.  The directive required all executive agenciesthat 

provide direct public servicesto set benchmarks for customer service performance. 

In February 1995, Vice President Al Gores National Performance Review included a Federal 

Consortium Benchmark Study Report.  The report focused on best practices in telephone 

service and found that the top ranked private sector organizations used the following  measures 

to predict customer satisfaction: 

t  averagespeedof answer:  less than 15 seconds 

b  abandonedcall rate:  less than 2 percent 

b  busy signal rate:  less than 1 percent 

t  service level (total calls minus busy signals and abandonedcalls):  98 percent 

t  first call resolution (one agent/no transfers):  85 percent resolution 

5 	 This is basedon the most recent budget figures received from the CIS Branch. During the course 
of the inspection, we received severaldifferent setsof contract funding figures. 
6 	 This region was closed for 3 months in 1997. Its budget was adjusted;therefore, the averagecost 
per call reflects 9 months of phone service. 
4  Other attributes of the top ranked private sector call centers include: 
a recognition that state-of-the-arttelephone and computer hardware and software 
b 
technologies are top businessimperatives, 
t  peak demand times are managedby offering callers the choice of live or automated 
service while scheduling staff to meet call demand and 
b  substantial site consolidation. 
METHODOLOGY 
In addition to a mail survey to all information specialists, we conducted on-site focus groups 
with information specialistsat six contractor sites. Eighty-four percent (163 out of 195) of the 
information specialistsrespondedto our mail survey.  We selectedthe six contractor sites 
basedon their call statistics, the populations they serve and their proximity  to other national 
cancerorganizations.  Using informal discussion guides, we also interviewed (in-person or by 
telephone) the project directors, telephone service managersand outreach staff at all 
19 contractors as well as telephone service supervisorsat most of the contractors. 
We chosea purposive sample of national cancer organizations to reflect the various types of 
organizations that could benefit from or make use of CIS information.  The sourcesused for 
the sample included CancerCaresA Helping Hand--The ResourceGuidefor  People With 
Cancer, organizations identified by the CIS Branch and the Internet.  Inclusion was basedon: 
1) the information or topic the organization addresses,2) their constituency and 3) the nature 
of servicesthe organization provides.  Prior to conducting interviews, we verified that each 
organization serveda national constituency. We conductedinterviews with principal staff, 
which included directors and managers, from 56 organizations regarding their scopeof 
services, their information resourcesand their experienceswith CIS and the PDQ. 
We visited 10 privately- and publicly-funded health information and cancer organizations. 
Staffs were interviewed regarding their experienceswith CIS.  We observedthe operations of 
health information servicessimilar to CIS and reviewed their information  systems, call 
statistics and relevant policies and procedures. 
We reviewed and analyzed policy and training manuals, reports, call statistics, contracts and 
plans that CIS has developed over the last severalyears to improve the telephone service. We 
also interviewed staff from the outreach program, CIS Branch, CISTERS, the International 
Cancer Information Center and the contractor that is responsible for the Publications Ordering 
Service, portions of the PDQ and programming the Electronic Call Record Form. 
The findings in this report reflect the experiences,opinions and observations of contractor, 
Branch, cancer and health organizations and OIG staff.  A companion report entitled The 
Cancer Information Service--A Resourcefor National Cancer Organizations 
(OEI-09-97-00361) contains additional information regarding how national cancer 
organizations use and value the Cancer Information Service and its resources. We conducted 
our review in accordancewith the Quality Standardsfor  Inspections issuedby the Presidents 
Council on Integrity and Efficiency. 
5  THE  CIS  IS A VALUABLE  PUBLIC  SERVICE  ENHANCED  BY  ITS  DEDICATED 
STAFF  AND  STRONG  TRAINING  AND  QUALITY  ASSURANCE  PROGRAMS 
The CISprovides  a unique  and valuable public  service 
The CIS is a unique health information service. It is the only cancer information  service that 
addressesall cancersand usesonly NC1 developed, reviewed and/or approved literature and 
data.  Callers are the general public, cancerpatients, their friends and families and health 
professionals. Callers receive information from CIS that helps them to understand cancerand 
make better informed decisions about treatment or prevention.  Through CIS, the public learns 
first-hand what advanceshave been made as a result of this nations investment in cancer 
research. The public relies on CIS for the most comprehensiveand reliable clinical trials 
listing available through a single toll-free phone service. Congressspecifically requires NC1 to 
inform the public of its findings, and CIS is one of NCIs principal modes of communication. 
Other cancer organizations confirm that CIS is the most comprehensiveand credible sourcefor 
cancer information.  One organization praised CIS saying, You get more information from 
CIS than anyone else, and the information is scientific.  Nationally,  they are the most 
credible.   The sentiment was echoedby many other organizations.  For additional information 
on other cancer organizations perceptions of CIS, seethe companion report entitled 
The Cancer Information  Service--A Resourcefor National Cancer Organizations 
(OEI-09-97-0036 1). 
Despite  stress and frustration,  staff remain  dedicated  to the mission  and goals  of CIS 
The staff of CIS, information specialistsin particular, confront the reality of cancer on a daily 
basis. Cancer patients and their friends and family,  often desperatefor answers, turn to CIS. 
To respond, CIS staff must master volumes of complicated information.  They must be 
sensitive to emotionally charged callers and to callers relationships with their physicians. 
They must also follow  many confusing and arbitrary policies and procedures. For example, 
they: 
must obtain their supervisors permission before sending certain types of publicly-
b 
available information (e.g., information for health professionals from Cancernet); 
b  cannot send information to a third party even when the caller statesshe is inquiring on 
someoneelses behalf; 
t  must follow  an information hierarchy, even if the information is not relevant; 
b  must ask if the caller has spoken with her physician about support groups before 
providing support group referrals and 
t  can quote, but cannot send, certain NC1 and other publicly-available materials 
(e.g., Medline information). 
6 
 The CISs goal is to provide callers with accurate, up-to-date information on very complex 
diseasesthat afflict 40 percent of the nations population.  Staff are pressuredon two distinct 
fronts:  (1) they must give callers all of the information relevant to their condition (and not 
rush the caller in the process)and (2) they must be able to answer and respond to all the callers 
who are trying to get through.  Information  specialistsare genuinely concernedabout the busy 
signal rate.  In at least two contractor offices, information specialistshave resorted to covering 
up the queue indicators on their phonesbecausethey get %tressedout when they seethem 
while on a call.  Accessproblems are a sourceof stressin their lives, and their self-evaluations 
of job performance are clearly affected by the services inability  to deal with public demand. 
At most CIS contractors, information specialistsare afforded little  opportunity for career 
advancement. Salary ranges, which are determined by eachcontractors parent institution, 
typically  fall below $30,000 per year.  Still,  information specialistsremain dedicatedto CIS; 
more than 65 percent of them have been with the service longer than 3 years, and more than 
23 percent have been with the service longer than 5 years. 
Many information specialistsvolunteered that their ability to help people provides them with 
personal andjob satisfaction.  At one CIS contractor, a retired physician decided to work for 
CIS becausehis son had been misinformed about cancertreatment options by his doctors.  He 
felt that he could make a difference in the lives of cancerpatients while working at CIS.  At 
another contractor, information specialistswere volunteering their own time for additional 
training that the contract was not funding. 
The CIS invests in an ambitious  and comprehensive  trainingprogram 
Information specialist training is comprehensiveand includes a continuing education 
component. All  information specialistsare required to complete an initial  8-week training and 
certification process. Currently, applicants must have college degreesor must be registered 
nursesto qualify as information specialists. In the absenceof thesequalifications, they must 
have serveda minimum of 2 years as a CIS information specialist at another time.  Initial 
training covers CIS policies and procedures, a general overview of cancer topics such as 
treatment, detection and staging and expandedtraining on breast, prostate and lung cancer. In 
order to maintain certification,  information specialistsare required to complete a minimum of 
12 hours of continuing education per quarter, must answer the phones a minimum of 12 hours 
per week and must maintain quality per CISTERS standards. Contractors choosea variety of 
methods for information specialiststo complete continuing education, such as participation in 
hospital grand rounds, attendanceat lectures or assignedreadings.  Finally,  CIS contractors 
are required to conduct special training programs at the discretion of the CIS Branch. 
Eighty-six percent of information specialistsranked CIS training as either excellent or very 
good.  While information specialistshighly regard their cancer training, they believe that 
managementtraining, software training, training on technical information and Internet training 
would be useful. 
7 
 Cancer organizations are impressedwith CIS training.  Two cancer organizations indicated that 
they would like to participate in CIS training when askedhow they would like to work more 
closely with CIS.  A third commented that CIS training is a real highlight  of a generally 
wonderful service. 
The CISTERSprogram  is an excellent  evaluation  tool, although  it is misunderstood  by some 
con tractor  staff 
The CIS Telephone Evaluation and Reporting System (CISTERS) program evaluatescontractor 
performance consistently and without bias, with an eye toward developing staffs ability to 
handle calls.  With the participation of contractors, the staff at CISTERS developstest call 
scenarios that accurately representreal calls.  They call all contractors and document the 
outcome of the calls on a custom computer software application.  More than half of the 
contractors believe that CISTERS improves call efficiency.  Several mentioned that it helps 
organize calls, although its suggestionsmay increasecall length.  According to staff, Test 
calls are valuable; they hit on important issuesand CISTERS improved call efficiency 
tremendously.  Call guidelines are helpful; they bring information specialistsback to key 
issues.  In a customer satisfaction survey that was conductedby the CIS Branch, most 
contractors said CISTERS was helpful and reported different and creative ways they use 
CISTERS reports to improve service. 
Staff from the CIS Branch suspectthat some contractors totally disregard CISTERS 
recommendations, while others employ varying degreesof action.  Some contractor staff are 
concernedabout not following  CISTERS criteria and view the reports as punitive rather than 
instructive.  Comments from those staff members indicate that not everyone understandsthe 
purpose of CISTERS.  We visited severalother health information telephone lines.  None of 
them has an evaluation tool that comparesin usefulness. 
The program is not without its limitations,  but the deficiencies we detectedwere minor relative 
to its merits.  For example, CISTERS does not develop scenariosthat involve highly emotional 
callers, a scenariothat information specialistsappearto be struggling with.  One information 
specialist asserted,They (CIS Branch) dont want us to think of ourselvesor call ourselves 
counselors.  But we are, we really are.  We observedanother information specialist who 
appearedawkward on a highly emotional call, reading scripted customer service questionsat 
its conclusion.  The program actually performs relatively few test calls, less than 10 per 
contractor per quarter.  Some contractors have complained that such a small sample cannot be 
used to gaugetheir performance accurately.  Sampleswill  be increasedin 1998, however, 
according to CISTERS staff. 
8 
 THE  CIS  DOES  NOT  TAKE  FULL  ADVANTAGE  OF ITS  CURRENT  TELEPHONE 
TECHNOLOGY  TO  IMPROVE  ACCESS  AND  CUSTOMER  SERVICE 
Callers  are not routed  to the first  available  information  specialist 
The CISs single toll-free phone number automatically routes calls basedon the callers 
geographic location.  The contractors ability to promptly answer the call is not taken into 
consideration.  Basedon CIS call statistics and our on-site visits, we discovered marked 
variancesin demand and accessamong the different contractors. We observedat least one 
contractor with phone lines open for most of the day, while other offices had persistent queues 
of waiting callers.  The CISs policy of routing callers to geographic contractors is not the 
most efficient way to ensurethe highest level of accessand customer service.  Although 
current CIS telephone technology allows callers to be transferred or routed among contractors, 
this technology is used only occasionally to offset temporary, unplanned closings. 
The CIS lacks important  recorded  messages 
The CIS does not provide callers with the option of obtaining recorded information that could 
preclude them from reaching a busy signal or waiting on hold.  For example, recorded 
information is not offered about CANCERNET  (a website for NC1 Fact Sheets,the PDQ and 
other cancer information),  CANCERFAX  (an automatedfax service to obtain NC1 printed 
materials), the PDQ SearchService for Health Professionals, frequently addressedcancer 
topics and other sourcesof cancer information,  even though such information may help callers 
when the phone lines are full. 
Standardinformation,  such as medical disclaimers (explaining that information  specialistsare 
not physicians) and source citations (generally NC1 materials), is repeatedby information 
specialiststo every caller instead of through an introductory recorded message. Information 
specialistsmentioned that the requirement to repeat the disclaimers and citations is excessive, 
that the wording is stiff and scripted and that disclaimers often interrupt the flow of the call 
and damagetheir credibility.  In addition, when CIS is inundated with calls about a particular 
topic arising from intense media coverage, CIS does not take full advantageof telephone 
technology by giving callers the option to listen to explanatory messages. 
DATA  GATHERING  REQUIREMENTS  ADVERSELY  AFFECT  CALL  EFFICIENCY 
The Electronic  Call Record  Form  software  is problematic 
The Electronic Call Record Form is marred by technical problems and restrictive policies on 
its use. At best, the form is only a slight improvement over paper forms.  At worst, it 
significantly impedes information specialistsfrom quickly and efficiently  taking calls. 
Managers, supervisors and information specialistsagreethat the Electronic Call Record Form 
software is a source of inefficiency and a weaknessin the telephone service.  Information 
specialists, in particular, believe that the software is in dire need of improvement.  Examples 
of software problems identified by information specialistsinclude: 
9  the inability  to move to a new Electronic Call Record Form without completing the 
b 	
current form on the screen. This meansthat information specialistsmust complete the 
form even if there is a queue; 
t 	 the need to manually insert codes. For example, if a caller refusesto answer the 
demographic questions, the software does not automatically input refusal codesto all 
questions. Instead, information specialistsmust input a refusal code for eachquestion; 
t 	 the need to type a narrative for the call as well as enter codes for the sameinformation 
provided in the narrative and 
computer crashesand freezescausedby the software under certain circumstances. 
b 
One telephone service supervisor, who also answerscalls, capsulized the frustrations of many 
respondents, 
The form isjust storage. It doesntreally takeadvantageof computerabilities. If 
theresa problemwith it, everyoneneedsto log-outandstart-up. This totally disrupts 
calls. NC1doesntunderstand.If you leavea field blank, an error comesup, andit is 
difficult to correct. Centralsupportrespondedto this by saying,fill in all blanks. 
The Electronic Call Record Form software causesthe computers to go down two to three times 
per week, according to a manager at another contractor. 
100 percent  data gathering  is excessive 
To capture detailed information about the nature of the calls, the demographics of callers and 
trends in eachregion, CIS requires that information specialistscomplete an Electronic Call 
Record Form for each call.  Becauseof this, information specialists cannot move quickly from 
call to call.  According to information specialists, completing demographic questionsadds 
severalminutes to every call.  Documenting everything else, from the reason for the call to the 
responseto the inquiry,  adds even more time.  Nevertheless,CIS has mandatedthat 
information specialistsdocument 100 percent of calls and gather demographic data on half of 
them, despite the negative impact on phone service and current accessproblems. 
Randomly sampling populations for multi-variate analysis (including the type CIS does with its 
call data) is a widely acceptedpractice in government and the private sector.  Service 
providers, including those providing health information,  regularly use random samplesto 
monitor their customer service, quality improvement and quality control.  A government health 
information service similar to CIS is sampling 1 in 30 callers for all  data gathering.  This 
includes the reasonsfor the call, responsesby staff and customer satisfaction. 
10 
 INFORMATION  SPECIALISTS  COULD  BE MORE  EFFICIENT  IF  THEIR 
RESOURCES  WERE  MORE  READILY  ACCESSIBLE  AND  USER-FRIENDLY 
The Physician  Data  Query  inhibits  call efficiency 
The Physician Data Query (PDQ) is technologically archaic and time consuming.  While 
contractors believe that PDQ information is typically  excellent, nearly all of them identified 
problems with the program.  Although most of the problems relate to the PDQs 
user-friendliness, staff reported myriad other problems such as incomplete, incorrect and 
out-dated information.  One staff member described the amount of problems as ungodly, and 
another statedthat the PDQ needsto be completely overhauled, becauseit isnt easyto use 
when on a call.  
BecauseCIS usesa DOS-basedcomputer version of the PDQ, information  specialistscannot 
conduct keyword searchesor easily move within  or between documents. As a result, they 
cannot quickly scroll through information,  and they may not be able to locate information on 
specific topics such as hot flashes or nausea. While there is a clinical trial searchtool, staff 
mentioned that the tool is not consistentand doesnot always return appropriate trials.  To 
avoid navigational and searchproblems throughout the PDQ, several regions take time to 
download it into WordPerfect files or Folio View,  and at least one region usesa hard-copy 
PDQ rather than the computerized version. 
While CIS contractors report most PDQ problems to NCI,  staff from several contractors 
believe that NC1 is not responsive. According to NCI,  the programming requirements of the 
PDQ do not allow problems to be addressedeasily. 
Community  service databases are inaccessible,  incomplete  and inaccurate 
More than half of all information specialistsreported that their community service databases 
are inaccessible,mostly becausethey are not computerized.  Information  specialists in several 
regions must thumb through a Rolodex or binder to identify community service referrals 
during calls.  They claim that calls would be shorter if the community serviceswere indexed 
on their computers. While some regions have computerized their community service 
databases,not all regions have the technology, time or expertise to do this.  Further, 
computerization does not mean that the databasesare accurateor up-to-date. 
The lack of a clear policy and direction for gathering and maintaining community service 
information has resulted in inaccurate and incomplete databasesat some contractors.  While a 
majority  of contractors reported that community service referrals are important, there was 
inconsistency in the staffs understanding of (1) NC1 policy and direction with regard to 
community referrals, (2) what standardsshould be imposed for inclusion of community 
organizations and (3) when and how to update their databases. For example, some contractors 
have opted to use databasesprovided by other organizations.  One contractor is mainly using 
Fact Sheetson how to find and choosea community organization, while other contractors are 
providing referrals to national organizations only.  Information  specialists complained that 
their community service databasesare the least complete and accurateresourcethey use. 
11  Managers corroborated with comments like,  Referral servicesare very problematic, especially 
for the staff.  We only have a few resources, and Having current resourcesis a sourceof 
great trouble.  We need to be up-to-date.  For additional information on community service 
referrals, seethe companion report entitled, The Cancer Information  Service--A Resourcefor 
National Cancer Organizations (OEI-09-97-00361). 
Contractors  cannot  easily maintain  and update NCI  %act Sheets  
Staff at CIS contractors complain that the processof updating Fact Sheetsis wasteful.  Fact 
Sheetsare one of the primary resourcesinformation specialistsuse during calls, becausethey 
cover a range of frequently addressedtopics in a concise and easy-to-understandmanner. 
Becausemost regions maintain Fact Sheetsin binders at eachdesk, the current system for 
updating them (which includes downloading, photocopying, distributing and making sure the 
old ones are discarded) is inefficient.  Updating is complicated by the fact that one Fact Sheet 
may be revised severaltimes within  a short time period.  According to staff, computerization 
would easethe updating processand allow easier organization and access. One contractor is 
using a scannerto computerize the sheets;however, most contractors do not have this 
capability. 
THE  REGIONAL  STRUCTURE  DOES  NOT  CONTRIBUTE  TO  A CONSISTENT  AND 
EFFICIENT  PHONE  SERVICE 
Demographic  differences  among  the regions  may affect demand  on CIS 
There are measurabledifferences in the contractors regional populations, and it doesnot 

appear that CIS resourcesare equitably allocated to addressthe most basic of thesedifferences. 

For example, total resident population and percentageof population over age 

65 (most cancersoccur in those over age 65) probably influence contractor demand. Regions 

consist of populations that range from approximately: 

t  1.2 million  to 22.4 million7 in total residents, 

b  10.5 percent to 16.7 percent in residentsover age 65, 

b  395,000 to 3.7 million  in population per available CIS phone line in the region and 

b  51,000 to 508,000 in residentsover age 65 per available CIS phone line in the region. 

In addition to the variances in demographics among the regions, there are variancesin 

demographics among contractors who have the samenumber of phone lines.  For example, a 

contractor with only 4 phone lines is serving a region with 3.9 million  residents, while another 

contractor with 6 phone lines is serving only 1.4 million  residents: 

7 Source: 1997projections--US CensusBureau and the CIA World Factbook (for Puerto Rico only) 
12  Number  of Telephone  Lines  Compared  to Regional  Demographics 
Regionat  population  Resident  population  over 
Lines  Contractors 
per  line  age  65 per  line 
3  1  395,000  51,000 
4  7  2.6 - 3.9 million  312,000  -482,000 
5  5  1.4 - 3.7 million  152,000  - 505,000 
6  6  1.4 - 3.7 million  185,000  - 508,000 
It is not clear  that the regional  structure  positively  contributes  to customer  service 
Despite CISs assertion that the regional structure enhancescustomer service, we were unable 
to identify any factual basis to support the claim.  Furthermore, we interviewed staff from 
health information servicesproviding telephone servicessimilar to CIS and found that 
disseminating technical health information and dealing with callers in crisis from a central 
location can be done efficiently  and achievea high level of customer satisfaction.  One cancer 
organization statedthat they have not had any objections to their centralized service (that was 
recently consolidated from many regional locations), becauseif you give [callers] what they 
want, they dont care where you are. 
We found no evidencethat callers are better servedby information specialists located within a 
geographic region that the caller is likely  to identify with.  In fact, the opposite is true.  It is 
unlikely,  given CISs current regional structure, that 1) a large majority  of callers identify with 
the location of their CIS contractor office and 2) CIS contractor staff are necessarily more 
aware of all of the areasthey serve simply becauseof their location (seethe appendix for the 
current CIS regional structure). 
The CIS also claims that the regional configuration is important for the outreach program. 
While it is true that outreach targets communities within the region, the relationship between 
outreach and the phone service is superficial at best. Although the telephone service and the 
outreach coordinators are housedin the samecontractor offices, most contractors did not 
identify a strong relationship or interdependencebetweenthe two programs.  In most regions, 
the information specialistsdo not participate in outreach and the only relationship between the 
two programs is that outreach promotes the telephone service, the telephone service fields calls 
for outreach and information specialistssometimesprovide the outreach coordinator with 
technical information.  In fact, severalproject directors indicated that they are struggling to 
find a relationship between the two programs becausethey have different missions, reach 
different groups and are two separatepiecesthat do not always function together. 
Con tractors  receive different  levels of support from  their parent  institutions 
Most project directors believe that they have good relationships with their parent institutions, 
but somebenefit more from theserelationships than others.  For example, some contractors 
receive money towards salaries, free spaceor renovated space. They also receive equipment 
13  (including computers), furniture,  technical support, administrative support, promotion and a 
general sensethat the parent institution believes in the CIS mission and goals.  Some 
contractors have enough office spaceto provide information specialists with their own work 
area away from the phone room.  Among contractors that receive considerable support from 
their parent institutions, project directors told us that the center fills gaps that the contract 
doesnt cover and [Our parent institution] understandsthat we are part of NCI, but they still 
provide us with whatever we need. 
Unlike thesemore fortunate contractors, other contractors receive only benefits that are 
available to all employees of the institution such as discounts on tuition and accessto training 
and speakers. They are often housed in less desirable spacewhich is not on, or even adjacent 
to, the main campus. Their computer equipment is outdated and technical support is catch as 
catch can.  Information  specialistsmust sharedesksand have no private study space. They do 
not even have dividers between their workstations.  Similarly,  the more fortunate contractors 
have the capacity to computerize information to make it more accessible,while thesedo not. 
Personnelclassifications vary by contractor, as well.  Some classifications do not seem 
appropriate for information specialists.  For example, information specialistsat one contractor 
had been classified as phone operators while at another they had been classified as 
coordinators.  In contrast, the parent institution of another contractor worked closely with the 
project director to create a (medical information specialist classification to acknowledge the 
special skills that the job entails. 
Contractor  policies,  procedures  and management  practices  affect  telephone  access and 
employee morale 
Although the CISTERS program and other quality assurancemeasuresinsure that CIS 
contractors provide consistent information,  there is little consistencyin the way contractors 
managetheir workload.  These include the ways they conduct clinical trial searches,assemble 
mailings and retrieve information.  Though most contractors have assignedstaff who are 
scheduledto conduct clinical trial searches,other contractors allow information specialiststo 
complete clinical trial searchesat any time.  Similarly,  at some contractors, mailings are 
handled by administrative or assignedstaff or during scheduledoff-phone time, while at others 
information specialistsassembletheir mailings between eachcall, even if there is a queueof 
waiting callers.  Further, some contractors have insured that assembling mailings is as efficient 
as possible by having commonly used items pre-copied and commonly sent packagesprepared, 
but others photocopy information and assemblepackagesfor eachmailing.  Some contractors 
have said that information specialists, being highly trained professionals, should not spend 
their time filling  and stamping envelopes. The CIS Branch has plans to institute an electronic 
publication ordering service, but this can only be done after contractor computer hardware is 
upgraded. Furthermore, only some contractors have the capacity, support and interest to 
computerize as much information as possible, thus making the information easier to accessand 
organize. 
s Note: the OIG did not collect personnelclassifications from all 19 contractors. 
14  Variations in how managersinteract and allocate work frustrate some staff and may affect 
accessto the phone service.  During focus groups, information specialists expressedfrustration 
that managersare not involved enough in the daily operations of the telephone room.  They felt 
that this made it difficult  when managersinstituted change, becausemanagerswere not always 
aware of how changeswould affect telephone room operations. While a majority  of telephone 
service managersanswer the phone lines, information specialists at a few contractors expressed 
frustration that their managersdo not take calls.  At one contractor, staff complained that they 
are not receiving the breaks they are entitled to becauseof the demandsof the telephone line; 
however, their manager does not take calls to help alleviate this problem.  Staff also 
complained that their manager interrupts them during calls to give them written reminders 
and make sure they give callers complete information.  Information  specialists find this 
practice frustrating and demoralizing.  Additionally,  severalproject directors expressed 
concern about the inadequatemanagementand supervisory skills in their offices. 
There also are variances in the opportunities for professional growth that contractors offer 
information specialists. Severaltelephone service managersexpressedfrustration about their 
ability to provide information specialistswith opportunities for professional growth.  However, 
some contractors have tried to addressthis by expanding the role of the information specialist 
to include rotating supervisory responsibilities or other assignments,such as database 
development and maintenance. 
THE  CIS  CUSTOMER  SERVICE  STANDARDS  DO  NOT  ADDRESS  ACCESS  TO  THE 
TOLL-FREE  SERVICE 
Benchmarks  or standards  aimed at reducing  access barriers  have not been developed 
Contractors are not required to keep their busy signal rates, wait times and abandonmentrates 
under any specific threshold.  Standardshave not been developed, such as minimum expected 
number of calls, maximum allowable cost per call or maximum allowable busy signal rate. 
Contractors do not know what the CIS Branch expectsin relation to accessindicators, if 
anything.  Some contractors are uncertain about how to balanceproviding quality, personalized 
and detailed attention to callers with their need to reduce busy signals and wait times.  One 
telephone service manager commented that the number of standardsin the contract should be 
investigated and suggestedthat standardsbe developed separatelyfor each contractor.  In the 
absenceof standardsprovided by the CIS Branch, this manager has instituted a standardfor the 
number of calls that her staff should complete per day. 
Similar  national  health  information  telephone  services have benchmarks  and standards 
One nonprofit telephone health information service is applying private sector standardsand 
techniquesthat are similar to those mentioned in the National Performance Review.  Located in 
a single national call center, the service recruited and hired a manager who had manageda 
corporate call center.  The service has goals for wait times, abandonmentrates and service 
levels.  Technology permits them to virtually  eliminate their busy signal rate.  The service 
currently experiencesa 15-secondaveragewait time with a goal of 12 secondsand has a 
15  3.5 percent abandonmentrate with a goal of less than 3 percent.  The percentageof calls 
answeredin 30 secondsor less is 88 percent, with a goal of 90 percent. 
Another health information service, operating under a federal government contract, has call 
volume standardsset by the contracting agency. The service is under contractual obligation to 
complete a minimum number of calls per year. 
16 
 The Cancer Information  Service, through its telephone service, provides the public with an 
invaluable resource for information about cancerprevention, causesand treatment.  To expand 
accessto the toll-free service, the Office of Cancer Information,  Communication and Education 
should: 
1. 	 Complete  and  implement  its plans  to upgrade  CIS  telephone  technology  to enhance 
contractors  ability  to provide  information  to all callers. 
Among the improvements that should be consideredare: 
the ability to route callers nationally to the first available information specialist; 
b 
t 	 the use of recorded messagesthat (1) notify callers of their estimated hold time 
or place in queue, (2) prompt callers to call back if all lines are full,  (3) prompt 
callers to chooseeither the first available information specialist nationally or an 
information specialist in their region, (4) provide basic cancer information and 
(5) include the medical disclaimer and a citation that only NC1 resourcesare 
used unless otherwise noted by an information specialist and 
t 	 the option to listen to recorded messagesthat provide telephone numbers for 
CANCERFAX  and the PDQ SearchService for Health Professionals, 
information about CANCERNET  and up-to-date NC1 responsesto current 
media eventson cancer (e.g., green tea and cancer, the nuclear fallout study). 
2. 	 Establish  minimal  technical  requirements  and  performance  standards  for 
contractors. 
This could be accomplishedby: 
t 	 implementing minimum requirements regarding CIS contractors computer 
hardware and software, staff computer aptitude and technical support, 
creating a single LAN  for CIS contractors to (1) eliminate many of the problems 
b 	
with maintaining and updating databasesand software and (2) facilitate the 
computerization of more information and 
using benchmarks similar to those in the National Performance Review. 
b 
3.  Modernize  and  correct  deficiencies  in the Physician  Data  Query. 
t 	 Continue efforts to redesign the PDQ to enhancesearchand navigational 
capabilities, completenessand accuracy. 
17 
 Insure that CIS staff, including information specialists, continue to be involved 
b 	
in PDQ improvement efforts. 
4. 	 Improve  the  efficiency  of information  specialists  by reducing  data  gathering 
requirements  and  computerizing  more  reference  materials,  such as Fact  Sheets. 
t 	 Improve the Electronic Call Record Form by eliminating technical problems and 
restrictive policies, such as the needto manually insert codesand enter codesas 
well as narrative. 
Require documentation of calls and collection of demographic data by utilizing 
b 	
statistically valid random sampling. 
Allow  contractors to use disclaimers and citations at their discretion, given that 
b 	
callers will  hear a recorded message. 
w 	 Move forward with plans to institute a system whereby contractors would not be 
responsible for the actual mailing of publications to callers. 
5. 	 Discontinue  collecting  and  disseminating  information  on community  services; 
instead,  partner  with  national  cancer  organizations  who  would  provide  this 
information. 
The CIS has been unable to routinely collect, update and disseminate information on 
community resources. Two national cancer organizations have committed considerable 
resourcesto this effort and would like to partner with CIS to provide referrals.  Seethe 
companion report entitled The Cancer Information Service--A Resourcefor National 
Cancer Organizations (OEI-09-97-00361) for more information on this subject. 
6.  Re-evaluate  the regional  structure  of contractors. 
Basethe structure on objective criteria, analysis and performance indicators. 
b 
t 	 Consider reducing the number of contractors and increasing the number of lines 
to eliminate inconsistenciesand allow contractors to answer more calls. 
b 	 Reallocate resourcesbasedon historic and expecteddemographic predictors of 
call volume (e.g., resident population, cancer incidence). 
7.  Encourage  contractors  to further  enhance  training. 
Regularly evaluatethe training needsof CIS staff with the goal of not only 
b 	
increasing cancer knowledge but also career advancementskills. 
Clarify  and publicize the instructive purpose of CISTERS and contractors 
b 	
obligations to addressCISTERS recommendations. 
18  AGENCY  COMMENTS  AND  OIG  RESPONSE 
We received written comments on the draft report from NC1 (seeappendix B). We are 
encouragedthat NC1 already has taken action on some of our recommendations. However, we 
remain concernedabout inefficiencies in vital areasof the program and would like to clarify 
and reassertthe following  recommendations. 
1. DATA  GATHERING  PLAN 
Sampling is a scientifkally  valid methodfor  collecting caller information 
To improve the efficiency of CIS, NC1 should quickly adopt a statistically valid sampling 
methodology that would capture caller data and information.  For example, our statisticians 
have suggestedthat an annual sample of no more than 6 to 10 thousand callers could provide 
adequateand reliable data to project to the universe of callers.  In order to addressCISs 
concernsabout collecting adequatedata specifically from small groups of under-servedcallers, 
our statisticians suggestthat sample sizescould be modified using a screening question. 
2. COMMUNITY  SERVICE INFORMATION 
Private sector organizations are in better positions to provide the public  with information about 
community resources,and CIS should no longer attempt to duplicate their efsorts 
Since considerableprivate sector resourcesare being spentto develop, maintain and 
disseminateinformation for the public on community resources,we must emphasizeour 
recommendation that CIS discontinue duplicating private sector efforts.  Given their finite 
resourcesand inability  to implement efficient systemsfor developing and maintaining a 
community referral system, we recommend that CIS contractors no longer operate such 
systems. Instead, CIS should rely on national organizations who also have toll-free 
information servicesto provide this service. 
Using one national database,CIS contractors could refer callers to organizations that maintain 
up-to-date information on local resourcesfor cancerpatients, their families and friends.  By 
doing so, contractor resourcescould be committed to where they are neededmost--providing 
NC1 information about the prevention, causesand treatment of cancer to the public in a timely 
and easy-to-understandmanner. 
3. CIS REGIONAL  STRUCTURE 
Thephone services current regional structure is ineficient.  Furthermore, it cannot be 
justified  by the phone services relationships with the outreach program and research activities. 
Regardlessof CISs outreach program and researchactivities, the current regional structure 
does not strengthenthe telephone service. Specifically, there is no correlation between the 
19  regions sizes, budgets, number of attempted and completed calls, number of phone lines and 
number of staff. 
While the CIS telephone service and outreachprograms are collocated, they are not 
interdependent. The programs have different missions, different audiencesand, with few 
exceptions, different staff members. 
During the course of the inspection, there was little mention of the researchactivities or their 
effect on the phone service.  Contractors who did mention researchparticipation explained that 
such activities may consist of asking callers additional questionsor providing them with 
additional information.  Contractors participation in the researchprojects varies.  Some 
contractors did not believe that such projects interfered with the day-to-day work of the phone 
service, while others did not believe that it is appropriate for CIS staff to conduct research. 
20 
 APPENDIX  A 

THE  CIS  REGIONAL  CONFIGURATION 

A-l  THE  CIS  REGIONAL  CONFIGURATION 

Region  I Area  Served  I 

1  Connecticut, Maine, MassachusettsN, ew Hampshire, Rhode Island, and Vermont 
2  New York City, Long Island, and WestchesterCounty, NY 
3  I New York State(excluding NYC and WestchesterCounty), and WesternPennsylvania  I 

I I 

4  Delaware,New Jersey,and EasternPennsylvania 
I I 

District of Columbia, Maryland, andNorthern Virginia 
I I 

6  Georgia, North Carolina, and South Carolina 
7  Florida and Puerto Rico  I 

8  Alabama, Louisiana, and Mississippi  I 

I 

9  Arkansas,Kentucky, and Tennessee 
I 

10  Ohio, WestVirginia, and SouthernVirginia 
I 

11  Iowa, North Dakota, Minnesota, SouthDakota, and Wisconsin 
I I 

12  Indiana and Michigan 
I I 

13  Illinois,  Kansas,Missouri andNebraska 
Oklahoma and Texas 
I
15  Alaska, Montana, Oregon, Washington StateandNorthern Idaho 
I
16  Arizona, Colorado, New Mexico, Utah, Wyoming, and SouthernIdaho 
I
Nevada andNorthern California 
I I 

SouthernCalifornia 
I 

A-2 
 APPENDIX  B 

NCIS  COMMENTS  ON THE  DRAFT  REPORT 

B-l  DEPARTMENT  OF HEALTH  & HUMAN  SERVICES  Public  Health  Service 
National  Institutes  of  Health 
APR 24  1998  Bethesda,  Maryland  20892 
TO: 	 June Gibbs Brown 
Inspector General, OS 
FROM:  Deputy Director for Management 
SUBJECT: 	 Office of Inspector General (OIG) Draft Reports, 77~ Cancer Information 
Service: Expanding TelephoneAccess,OEI-09-97-00360, and The Cancer 
Information Service: A Resourcefor National Cancer Organizations, 
OEI-09-97-00361 
Attached are the National Institutes of Health comments,preparedby the National Cancer 
Institute (NCI), on the subject draft reports. These reports provide confirmation of previous NC1 
findings aswell as further insight into the issuesaffecting accessto this valuable resource for the 
American public. 
We agreegenerally with the OIG recommendationsor with the objectives of the 
recommendations. However, in somecaseswe believe that theseobjectives can be met through 
actions or initiatives  other than those recommendedin the reports. Our commentsdescribe the 
stepsthat NC1 has taken or plans to take to addresseachrecommendation. 
Should your staff have any questions, pleasecall William Gillen, Office of Management 
Assessment,at 301-496-2462. 
Attachments 
MAY.7i 
B-2 
 DEPARTMENT  OF HEALTH  & HUMAN  SERVICES  Public  Health  Service 
National  Institutes  of  Health 
National  Cancer  Institute 
Bethesda,  Maryland  20892 
TO: 	 June Gibbs Brown 
Inspector General 
APR2 3  1998 
Through:  Director, NIH 
FROM:  Director, National CancerInstitute, NIH 
SUBJECT: 	 Offke  of Inspector General Draft Report:  Cancer Information  Service: 
Expanding  Telephone Access(OEI-09-97-00360) 
Office of Inspector General Draft Report: The Cancer Information  Service: 
A Resource for National  Cancer Organizations  (OEI-09-97-00361) 
We have reviewed the above-referenceddraft inspection reports that assessaccessto the National 
Cancer Institutes Cancer Information Service. The inspections were conducted in cooperation 
with NC1 program staff and the contractor staff at the regional CIS offices.  The reports provide 
confirmation of previous NC1 findings aswell as further insight into the issues affecting accessto 
this valuable resourcefor the American public. 
Our detailed comments on both reports are attached. Thank you for the opportunity to review 
and comment. 
-% 
Richard D. Klausner, M.D. 
Attachments 
B-3 
 National CancerInstitute Commentson 

Oftlice of Inspector General Draft Report: 

Cancer Information Service: Expanding Telephone Access 

OEI-09-97-00360 

I 
NC1 GENERAL COMMENTS 

The Office of Inspector General study provides confirmation of NC1 findings and tbrther insight 

into the issuesaffecting telephone accessto the Cancer Information Service. It is useful to have 

this additional, independent assessmenatsthe National Cancer Institute proceedswith its plans to 

upgrade the CIS and improve accessto this important source of accurate, up-to-date cancer 

information for the public, patients, and health professionals. 

OIG RECOMMENDATION 

Completeand implement plans to upgrade US telephone technolog), to enhance contractors 

ability  to respond to calls andprovide  information to all callers. 

NC1 RESPONSE 

The Institute agrees. Plans to upgrade the CIS telephone technology have been in 

progress since 1996, and the NCI-commissioned report of an independent engineering 

assessmenotf the system,routing, and equipment hasbeen completed. On the basisof 

this information, implementation of upgradesto telephone equipment is under way and 

will be completed in FY 1998. 

OIG RECOMMENDATION 
Ektablish minimum technical requirements and performance stanakr& for  contractors. 
NC1 RESPONSE 

The Institute agreesthat standardsare necessary. With the completion of the NCI

commissionedengineering assessmenotf the CIS telephone service, the program has 

baselinemeasurementsthat allow for implementing technical, operational, and 

performance standards. This implementation is under way. 

OIG RECOh4lvfENDATION 
Modernize and correct deficiencies in the PDQ database. 
NC1 RESPONSE 
B-4 
 The Institute is undertaking a major redesignof its entire clinical trial information system, 
including PDQ. The new clinical trial information systemwill increasefUnctionaMy, 
integrate all NCI-produced information products, tailor information to meet the needsof 
diverse users,and makeinformation available in a variety of mechanismsand formats, 
including the World Wide Web. 
OIG RECOMMENDATION 
Improve the efjkiency of information specialists by reducing data-gathering requirements and 
computerizing more reference material. 
NC1 RESPONSE 

The Institute agreesthat efficiency can be improved. The Institute is working with the 

CIS Evaluation Task Force, statisticians, and Institute staff to analyzethe data gathering 

requirements, sampling plan, and the requirement for a narrative that documentsthe call in 

addition to coding. However, the Institute believes that data collection is critical if the 

program is to 1) continue to respondto information requestedby the public, Congress,the 

press,or individuals and organizations interested in what the public wants to know about 

cancer; 2) allow the CIS to participate in cancercontrol researchprojects; and 3) conduct 

quality assurance. 

The CIS is preparing to install a new document managementsystemto facilitate the 

development, updating, and accessof CIS referencematerials. When implemented,the 

systemwill allow the assemblyof all resourcesinto a searchable,computerized collection 

accessibleby subject and keyword. 

OIG RECOMMENDATION 
Discontinue collecting anddisseminating information on community services; instead, partner 
with national cancer organizations who would provide this information. 
NC1 RESPONSE 

The Institute agreesthat a more efficient and effective community servicesreferral 

program is needed,but we continue to think that this type of information is helpfil  to 

many people. Redesign of a centralized listing of national organizations providing 

community resourcesand referrals is under way. The Institute is actively seeking 

partnerships with other national organizations to shareand reciprocate in this task. 

OIG RECOMMENDATION 
Re-evaluate the regional structure. 
NC1 RESPONSE 

The CIS program includes not only the telephone service but also an outreach program 

and a researchcomponent that are strengthenedby a regional structure. 

The CIS outreach program develops partnershipswith nonprofit, private, and government 

B-5 
 agenciesat the national, regional, and state levels. The regional CIS offtces reach partners 
that have an establishedpresencein the region, are trusted within their communities, and 
are dedicatedto serving minority and underservedpopulations. Through collaborations 
with cancercenters and universities, regional CIS ofices participate in investigator-
initiated cancercontrol and health communications research. All three program 
componentsmust be consideredwhen evaluating the regional structure of the program. 
AI1 options for structuring the program will be consideredwhen preparing for the renewal 
of the CIS contracts. 
OIG RECOMMENDATION 

Encourage contractors tojirther  enhance CIAstaff training. 

NC1 RESPONSE 

The Institute agreesthat staff development is important. The CIS program begana 

managementinitiative in March 1998 to emphasizethe importance of professional 

development and skills building in regional CIS offices. Additional training programsto 

enhancecareerdevelopment for CIS staff are appropriate for contractors and indicate 

institutional commitment to the CIS program. 

B-6