Department of Health and Human Services 

OFFICE 

INSPECTOR GENERAL


DISSEMINATION OF RESULTS OF AOA'


DISCRETIONARY FUND PROJECTS


stRVICEs. -.rRichard P. Kusserow l. INSPECTOR GENERAL 

JUNE 1991 1y 


d3a 

;. 

Deparment of Health and Human Services 

OFFCE OF


INSPECTOR GENERAL


DISSEMINATION OF RESULTS OF AOA'


DISCRETIONARY FUND PROJECTS


tRVICES. 

Richard P. Kusserow 

INSPECTOR GENERAL '0 

1y~1t d3a 

OEI-04-91-0011O 
EXECUTIVE SUMMARY


PURPOSE 

This inspection was conducted to evaluate the dissemiation practices for discretionar grant 

projects supported by the Admnistrtion on Agig (AoA). 

BACKGROUND 

Title IV of the Older Americans Act authorizes a progr of discretionar funds to support trg, education, researh, and demonstration projects. All grant applicants ar required to 

submit a plan describing how they wil disseminate their project results. Although AoA no 

longer has a unit devoted to dissemiation, its sta contiue a number of dissemination activities. 

FINDINGS 

Accordig to the Older Americans Act, an integral par of the Title IV program is dissemination 

of information gaied from the grant projects. AoA's dissemiation efforts do not assure that this 

information reaches other organizations who can use it. 

AoA relies priarly on grantees, whose capabilties var widely, for dissemination of 

project results. 

In evaluatig applications, the value assigned to dissemination activities has declined. 

AoA' s expanded guidace to applicants has had little impact on actUal dissemination 

activities. 

AoA does not adequately assess project outcomes to determine the utility of the 

information to others. 

AoA pursues a broad dissemination strategy with too limited resources. 

RECOMMENDATION 

AoA should assur the establishment and adequate funding for a permanent function, responsible 

for dissemination of results of discretionar fund projects. 
COMMENTS 

The drt report was circulated for comment to the Commssioner on Aging, the Assistat 

Secrtar for Legislation (ASL) and the Assistat Secreta for Planning and Evaluation (ASPE). 

The ASL had no comments; ASPE concured with the fmdings and recommendation. AoA 

found our report to be accurate and the recommendation relevant. In her response, the 

Commssioner on Aging also summarzed the agency s curent and planned dissemination 

activities. 
.................... ........ ........ ............ .......... ................................................. .. ........ ................ ...... .................. ........ .......................................... .......................................................... .... .................. ............................................................... ...... ............ ........ .

TABLE OF CONTENTS


EXECUTIV SUMMARY 

INRODUCTION.................................................................................................... .. 

Purose ...... 

Background.............. .................. ............ .................................... ..........................


Scope.... .... .... .... .......... 


Methodology ................ ......................................................................................... 3


FIN IN GS .................................................................................................................. 4


AoA Relies Prmarly on Grantees, Whose Capabilties Var Widely, for 

Dissemination of Project Results ...................................... ............ ...................... 

In Evaluating Applications, the Value Assigned to Dissemination Activities 

has Declined.. .......... .................. ........................................................................... 

AoA' s Expanded Guidace to Applicants Has Had Litte Impact on Actual 

Dissemination Activities...... ...................... ............ .............................................. 

AoA Does Not Adequately Assess Prject Outcomes to Determne


the Utiity of the Information to Others ..............................................................


AoA Puues a Broad Dissemination Strategy with


Too Limited Resources .............................................. .......... ................................


RECO MMEND A TIO N 

AGENCY CO MMENTS 

APPENDIX A: AoA Comments............................................................................ 

... 

, 

INTRODUCTION


PURPOSE 

Ths inspection was conducted to evaluate the dissemiation practices for discretionar fund 

projects supported by the Admistrtion on Agig (AoA). 

BACKGROUND 

The Admistrtion on Aging was established in 1965 by the Older Americans Act (OAA). 

Through formula grants, AoA gudes and assists states and communities to develop and 

implement services for elderly persons. The agency also admnisters a discrtionar funds 

progr. 

AoA Discretionar Funds Program

Title IV of the Older Americans Act authorizes a progr of discretionar funds to support 

trning, education, research and demonstrations, as well as dissemination of information. For 

FY 1991 , $25. mion has been alocated to the Title IV program. This is two percent more 

than the 1990 budget, when a tota of 140 new grts were awarded. 

discrtionar funds progr, as stated in the Older Americans Act, is 

to, 

expand the Nation s knowledge and understading of aging and the aging process, 

to design and test innovative ideas in progrs and services for older individuals, 

and to help meet the needs for trned personnel in the field of aging though 

The purose of the Title 


(1) education and trning 

(2) research. .. 

(3) demonstration projects ..- and 

(4) dissemination of information...acquired though such programs... 

Including dissemiation as an integral par of the program establishes that the information gained 

from the projects should extend well beyond the initial grtees- Effective dissemination 

promotes utiization by others in the field, and avoids subsequent researchers reinventing the 

. wheel." The authorizing legislation recognizes the need to disseminate project results in order to 

gai full benefit from the resoures expended on each project. 

In prepartion for the reauthorization of the OAA in 1991, the House Subcommittee on Human 

Resources of the Select Committee on Agig is holdig a series of hearngs. One hearng, in 

September 1990, focused on the Title IV progr, parcularly the effectiveness and usefulness 

of dissemination efforts. The General Accounting Offce (GAO) presented testimony on a 

survey of all state agencies on aging to determne their use of Title IV results. GAO said While 
Title IV dissemination is having some positive impact, it is not achieving maximum results ... 

AoA does not have a comprehensive dissemiation strategy. 

Disseminaton Practies in Other HHS Agencies 

Dissemination of project results is important to many HHS agencies which support research and 

demonstrtion projects. The Deparent has no standad process to assure effective 

dissemination of the results of grant projects. Dissemiation is handled differently by the 

varous operatig divisions of the Deparent. 

For example, each of the National InstitUtes of Health maintai public informtion offices. 

These offces, to varing degres, disseminate the results of projects funded by their agencies. 

However, accordig to the National InstitUte on Aging s (NI) Public Information Offce, NIA 

grantes, lie those funded by other branches of Nl, generaly do not rely on the Institute to 

dissemiate the results of their projects. Rather, grantees seek to publicize their own results, which commonly focus on medical and biological issues, in professional jourals and forums. 

Under the Public Health Service, the Maternal and Child Health Bureau has a cooperative 

agreement with the National Center for Education in Maternal and Child Health at Georgetown 

University to handle dissemination of the grants funded by the agency. 

In the Health Care Financing Admnistrtion (HCFA), grantees can, with approval from HCFA dissemiate the results of their own projects, but ar not required to do so. HCFA routinely 

sends all approved fmal project reports to an information clearnghouse. 

In another example, the Offce of Human Development Services (OHDS) instrcts applicants 

that dissemiation is an essential priciple of its grt progr. However, in the 1990 Federal Register announcement, dissemiation activity was drpped as an evaluation criterion for rating 

grant applications. 

SCOPE OF THIS INSPECTION 

This stUdy evaluated dissemination of the results of AoA' s discrtionar fund projects. It 

examned both the grtees ' and the agency s dissemination activities. All AoA discretionar 

grants admistered by the AoA central offce, and completed between April 1 , 1989 and March 

, 1990 were included in the scope of the study. Grants that were completed within this time 

period had responded to the announcements published in the Federal Register 
 for fiscal years 

1985 though 1988. 

For informtion about dissemiation activities of the agency, OEI staf interviewed professionals 

within and outside AoA who ar familar with the discretionar funds program. AoA staff 

included regional and centr offce administrators and project offcers from the Title IV 

progr. Outside organizations included: the Gerontological Society of America, the 

Association for Gerontology and Higher Education, the National Institute on Aging, the 

American Association of Retired Persons, and all information clearnghouses used by AoA. 
METHODOLOGY 

A list of the grts completed between April 1, 1989 and March 31 , 1990 was obtaned in August 

1990 from OHDS' s Grants Management Information System (GMIS). Those 88 grnts were 

classifed by tye (researh, traiing, and demonstrtion). A sample of 30 was selected for study. 

Since there were only 5 research grants, all were included in the sample. The remaider of the 

sample was selected using a random number generator. 

OEI staf examned the project fIes for each grt to determne what dissemination 

commtments were made when the grant application was accepted for fundig, and what 

dissemiation activities were subsequently reported in the quarerly progress report and the final 

report. Since the offcial fies, which are maintaned by OHDS, ar sent to archives when the 

project closes, the project offcers' fies were used for this stUdy. The project offcers usually 

maita a copy of al items that ar in the offcial file, but they ar not required to have copies

of al offcial papers including the final reports. In fIes where the final report was missing, the 

project offcers provided inormation on the projects' dissemiation activities. In some cases, the 

grantees were interviewed by phone to confi or supplement the recollections of the project 

offcers. 
FINDINGS


Accordig to the Older Americans Act, an integral par of the Title IV program is dissemination 

of information acquired from the grant projects. AoA's dissemination efforts do not assure that 

this inormation reaches other organizations who can use it. 

AOA RELIES PRIARILY ON GRANTEES, WHOSE CAPABILITIES VARY 

WIDELY, FOR DISSEMIATION OF PROJECT RESULTS 

AoA has relied on its grtees to tae the lead responsibilty for dissemination of project results, 

accordig to AoA offcials. One of the distinguishing charcteristics of AoA's discretionar 

funds progr is the wide varety of grantees, raging frm smal nonprofit community 

organzations to major national gerontological associations and universities. The capabilities of 

these organzations to effectively dissemiate their project results var. Despite the range of 

abilties, the legislation requires al research and demonstration grts to "include provisions for 

the appropriate dissemiation of project results." Furer, the Federal Register announcement 

indicates that al applicants wil be evaluated in par on how they plan to disseminate their 

products. The result is most grtees submit minimal dissemination plans while some plan more 

extensive activities. 

Experts generaly attrbute the dierence in dissemiation activities to the resources and 

reputation of the grtee. Some grtees, with a wide network of contacts, sufficient resources, 

and experienced sta, dissemiate their project results extensively and effectively. For example, 

one grtee, a national business organization provided the funds to publish the project s report 

on retiment planning and send it to the human resource executives of leading corporations in 

the U. S. Ths organization also had the connections to anange presentations at forums such as 

breakast meetigs of business executives acrss the countr. However, most grtees do not 

demonstrte such capabilties in their dissemination activities. 

IN EVALUATING APPLICATIONS, THE VALUE ASSIGNED TO DISSEMINATION 

ACTIVITIES HAS DECLINED 

The criteria for evaluating applications ar published in the Federal Register
 announcement of 

the availabilty of discretionar funds. A panel of expert from outside the Federal Government 

comments on and scores the applications. The criteria and the value assigned to each one have 

changed over the year, but generaly include: 

need for, or objectives, of the project 

methodology or approach 

results or expected outcomes/benefits 

dissemination! utilzation plan 

level of effort or staff background and resources 
AoA requies projects to include a dissemination plan. However, the relative importance of the 

dissemination crterion in the proposal-ratig process has declined. In 1987 each of the criteria 

was wort the same amount - 20 points. Over the year, the point values assigned to the five 

crteria have shited. The dissemination crterion has been reduced from 20 points in 1983 

though 1987, to 15 points in 1988, and only 10 points since 1989. 

AOA' S EXPANDED GUIDANCE TO APPLICANTS HAS HAD LITTLE IMPACT ON 

ACTUAL DISSEMIATION ACTIVIES 

At the same tie that AoA has been reducing the value assigned to the dissemination criterion 

the agency has been giving the applicants more substative guidace on dissemination. 

Pror to 1988, the guidelines in the Federal Register for the dissemination crterion instrcted 

applicants to describe: 

the methods for sharing their findings, 

the steps to promote utilization of products, and 

the specifc audiences to be adressed. 

Starng in 1988, the instrctions became more deliberate and precise. In 1988, applicants were 

asked to add: 

a description of why the proposed steps are expected to be successful in


disseminating the products and findings;


reasons why specifc audiences wil benefit; and


detailed steps to get the products adpted by the audiences.


The 1989 and 1990 announcements contan a separate section on dissemination. Applicants are 

notied that they ar expected to be awar of projects in the same area as their proposal. To help 

locate related projects, the announcement gives a list of sources for trcking CUITent and previous 

AoA-sponsored projects. Furher, these announcements provide the following principles of 

dissemination, and advise applicants to consider them in developing their applications: 

The most widely utilzed projects make dissemination and utilization a central 

aspect of the project, not a peripheral one. 

Dissemination starts at the beginning of a project. 

Potential users should be involved in planning. 

Products should be prepared with the needs of potential users in mind. 

Dissemination is a networking process. 

At a minimum, dissemination includes getting your final products into the hands 

of appropriate users and maing presentations at conferences. 

Coordination with other related projects may increase the chances of your 

products being used.

Review of the dissemiation plans in grant applications did not reveal a qualtative difference 

between plans submitted prior to 1988 and those responding to the 1988 announcement, when 

the instrctions became more detaed. Both before and after 1988, plans list a few stadard 

dissemiation activities. The most frequently mentioned activities ar: submitting to 

professional jourals for publication, applying to present results at professional conferences, 

includig project descrption in state and local newsletters, and makng presentations at local! 

professional meetigs. 

Most projects included in this study responded to the 1987 and 1988 announcements. This stUdy 

did not cover projects initiated in 1989 or 1990. However, project offcers said they have not 

noticed a signifcant difference in the quality of the plans submitted in the past two years, 

compard to previously. 

This review found that 90 percent of grants completed between April I , 1989 and March 31, 

1990 implemented the dissemination plans that were proposed at the outset of the projects. 

However, the grtees' dissemination plans do not go far enough to assure that project results 

reach those who are most likely to use them. Accomplishing this requirs the kid of efforts 

AoA has descrbed in its gudelines for dissemiation since 1988. Even though the grantees 

plans fail to adhere fully to the dissemination guidelines, the projects continue to be approved for 

fundig since the dissemiation criterion accounts for such a small porton of the overall 

evaluation score. 

Accordig to project offcers and experts outside AoA, the miimal dissemination activities of 

grantees can be trced to the lack of emphasis from AoA, as well as insufficient time and funds, 

and lack of capabilty of smal grantees. 

AOA DOES NOT ADEQUATELY ASSESS PROJECT OUTCOMES TO DETERMINE 

THE UTILITY OF THE INFORMATION TO OTHERS 

AoA' s procedurs place heavy emphasis on evaluation of grant applications for feasibility and 

utity of results. However, once the project is completed, the final results 
 are not adequately

assessed for their potential utility to others in the field. 

The purpose of AoA' s grant projects is to test new and innovative ideas. Because they explore 

heretofore untested approaches, some projects wil not meet expectations. Since AoA does not 

adequately review all completed projects, those that should be replicated ar not distinguished 

from unsuccessful projects and projects whose results ar less useful to others. 

The same project close-out proedure is followed for all projects. All final reports are sent to the

information clearnghouses. Here the good, new ideas may get lost in the volume of final reports 

released each year. 

Respondents most often noted the lack of resources, including staff, as the main reason AoA 

does not follow a thorough, systematic process for reviewing completed projects. Additionally, 
some feel that AoA' s offcially judgig the relative utility of the projects would politicize the 

process. Despite these diffculties, a number of outside expert feel assessing project results and 

targetig them to specific users would greatly incrase the application of project results by 

others. In fact, severa experts with outside organizations said that projects must be evaluated 

and summarzed fist, before effective dissemiation can occur. One major organization said its 

policy is to review al of its own projects to determne which ones to include in their 

database/librar and to select the ones they wil disseminate more extensively. An offcial with a 

major organzation in the field of aging commented that, "trly responsible dissemination goes 

way beyond rote dissemination plans...this organization spends half of its tie on dissemination 

activities. " 

AOA PURSUES A BROAD DISSEMINATION STRATEGY WITH TOO LIMITED 

RESOURCES 

Respondents noted a number of problems with AoA' s dissemiation efforts. Congress withdrw 

authorization for the agency s internal information clearghouse in 1981. AoA has not had a 

sta devoted to dissemiation since then. In addtion, fundig for the TItle IV progr has been 

reduced drasticaly from a high of $54 milion in 1980 to $26 millon this year. In spite of these 

reductions, the agency s staf has tred to maintai a wide rage of dissemiation activities, along 

with their other duties. 

AoA' s dissemiation activities include: 

sendig reports to inormation clearnghouses 

holdig meetings of grantees workig on similar projects 

convenig regional dissemination conferences 

publishig booklets, a magazne, and information memoranda


supportng 11 resource centers.


Clearinghouses 

Using clearnghouses as a method of dissemination is essentially passive. Technically, the 

reports ar avaiable, but an organization or individual must take initiative to obtain them. 

Gettg a repon out of these archives requirs famliarty with database research and access to 

the computerized soures. Additionally, users of the clearnghouses do not have a means of 

distiguishing excellent projects from those of less utiity without looking at each report. 

AoA sends all final reports and products to the following information clearnghouses: 

National Technical Informtion Service (NTIS) 

AgeLine Database 

S. Governmenr Prinring Offce


Project SHARE (closed August 31, 1990)

The clearnghouses fIe the projects ' fial reports in a computerized information system. Only 

NTS had information on the number of requests for AoA reports. Over the past ten years, NTIS 

fied an average of the requests per month. 

AgeLine is a commercialy avaiable database admistered by the American Association of 

Retid Persons (AARP). AARP selects some of AoA's reports for inclusion in its database. The 

most frquent users of AgeLine are faculty and stUdents. 

The Governent Prnting Offce sends microfiche copies of projects ' fmal report to the 

depository librares that have requested in advance to receive the general category of 

publications that includes AoA reports. 

Cluster Meetings and Networking 

One of the priciples of dissemination is linkng people and organizations and vigorously 

encouragig netWorkig activities. AoA's formal and informal efforts in this ara are hampered 

by lack of funds and lack of procedures for apprising staf of signifcant projects. 

AoA has implemented the priciple of netWorkig though cluster meetings. AoA funds several 

grants in a given area, for example elder abuse, for the same project period, and convenes 

meetigs of the grantees to discuss the projects. Project offcers are enthusiastic about this 

approach and feel the grtees lear from each other. Factors that hamper "clustering" include 

lack of funds to bring grtees together more frquently, and the very common practice of 

approvig grtees' requests for time extensions. When extensions ar granted, "clustered" 

projects are no longer on the same schedule, and therefore the grntees cannot share their full 

fmal results at the fmal meetig. 

Some project offcers have linked new and former grantees with similar interests informally 

when there is no formally recognized cluster. However, among project offcers, there is 

considerable varation in involvement with grantees. According to AoA offcials, there is no 

provision to keep AoA staf apprised of the Title IV projects outside of each offcer s immediate 

responsibility. This general lack of an information netWork within AoA inhibits the distrbution 

of information by staf to grantees. In addition, AoA does not routinely cover the costs for staff 

to attend national conferences on aging. These conferences provide opportunities for 

professionals to form netWorks which facilitate exchange of information. 

Federal Regional Offices' Disseminaton Activities 

A number of AoA central and regional offce staf feel a major weakness in the use of existing 

resources is failing to include regional offces in more dissemination activities. Regional offces 

are geographically closer to the recipients of AoA funded services and have the potential to 

contrbute substantially within existing resoures. Accordng to one aging specialist, the regional 

offces could, "function as the eyes and ears of the Commissioner and be a conduit of 

information. "

Severa respondents noted that the curnt Commissioner has formally convened a number of 

priority areas. They feel that an informal network through the grssroots roundtables to identiy 


regional offces could build on the Commssioner s current efforts to get local input. One 

regional admnistrtor of AoA said that dissemination effort could be improved by staring at 

the begiing of the funding process. He felt regional offces could help identify research needs 

and thereby also identify the potential users of project results. 

Since 1983, the Offce of Prgr Policy and Legislation in the Office of Human Development 

Services has sponsored regional meetings to disseminate information on discrtionar grants, includig AoA's, and to encourage formation of networks among interested pares. The 

meetigs are convened by the federa regional offces. OHDS considers the meetings to be 

successfu. However, only nie meetings ar scheduled for 1991. 

AoA Publicatons 

AoA prouces a number of publications that are distrbuted nationally. Several respondents 

noted that some of these publications would be better utilized if they were targeted to specific 

audiences. For example, a project offcer suggested that one of the recent publications, 

Dissemination By Design, should be given routiely to all grtees. This booklet describes 

effective dissemination priciples and practical techniques for disseminating information. A 

Many respondents suggested 

that the compendium would be more useful if done as a cumulative volume which describes all 

completed, rather than ongoing, projects arged by subject area. 

frequent suggestion concerned the Compendium of Active Grants. 


National Aging Resource Centers 

A number of AoA sta say the resource centers ar AoA's most effective means for "getting the 

word out. AoA funded 11 resource centers acrss the countr for 3 years to provide training, 

technical assistance, short term research, and infonnation dissemination for state and area 

agencies on aging and others serving elderly people. Most of the centers ar located 

universities. Each one focuses on a specific area of aging, such as long term care, elder abuse 

health promotion, miority populations, and rual elderly. The future of the centers is uncertain 

since the funding termates in 1991. 
. .. 

. . .

RECOMMENDATION


The Admistration on Agig, with limited resources for dissemination, has engaged in a broad sttengthen and better 

range of activities to dissemiate results of its Title IV projects. To 


coordiate these efforts, the Inspector Genera makes the following recommendation.


AoA should assure the establishment and adequate funding for a permanent function, discretionar fund projects. This could be


responsible for dissemination of results of would include: 

done internally or by contrct. Responsibilti 

various dissemination activities; providing a focus for AoA' 


in the grant application review process, reconsidering the dissemination 

criterion and the point value assigned to it; 

utility to


establishing and maintaining a process to assess project outcomes for 

others; 

developing methods to assist grantees who lack the resources and expertise to 

target replicable results to potential users; 

evaluating the role of clearinghouses in an overall dissemination strategy and 

using clearinghouses accordingly; and 

regional offces on aging in establishing and implementing a role for federal 

disseminating project results. 

10. 
AGENCY COMMENTS


The Commssioner on Aging, in commenting on the report summarzed AoA's ongoing 

dissemination activities. The Commssioner furher described AoA's plan to fund a 3 year 

cooperative agreement to evaluate and disseminate Title IV grt products. Appendix A contains 

the full text of the Commissioner s comments. 

The comments from the Assistat Secreta for Planning and Evaluation concur with the report. 

The Assistant Secrta for Legislation had no comments. 
APPENDIX A


AoA Comments

: ;.

VI 

,,,,,(

I A' '-3 - '3 1 1"1 0 t- 15:1':. H H S , 0 I G - H E A D 1;1 U ART E R	 €!2 

DEPARTME T Of HE. LTH &. HU.ItAN SEiVICES	 Of of 118 s.et 

20' Administration on Aging Washngton. D. 

M ::

A \00\	 r1' m 

- m_
 :=;z-­

TO:	 Richard P. Kusserow


Inspector General


FROM:	 u. s. Commissioner on Aging 

I s 

SUBJECT:	 OIG Draft Report: "Dissemination of Results of AoA 

Discretionary Fund proj ects, " OEI-04-91-00110 

We have revie ed your report on the dissemination efforts of 

the Ad inist=ation on Aging (AoA). We find that the report is 

acc	 e and
 e recommendations for improving our

disse ination system are relevant. 

At the same time, we feel that the draft report does not pay
 has been
 su::icient attention to the considerable effort that 


made in this area. For a n er of years, AoA has awarded ation. grants to projects essentially to disseminate info


Exa ples of sucb projects include the eleven National Aging 

Resource Centers with major responsibilities for the provision 

of technical training and assistance and information ination dissemination. The usefulness of the information Qisse


o the States by these Centers was acknowledged in a recent


General Accounting Office (GAO) study presented to the House


Select Committee on Aging on 9/11/90. Other projects funded


have disseminated in!o ation on health promotion and senior 

centers. 

Taking into consideration both your report and the GAO study, 

we have begu to address the issues raised by both agencies. 

Our Announcement of the FY 1991 Oiscretionary Grant Program 

will include the award of a three year cooperative agreement, 

for up to $300, 000 per year, to actively and systematically 

evaluate and disseminate Title IV grant products and to examine


gaps in research, demonstration, training and practice in 


field of aging. I believe that this prospective cooperative 

ag=eeme t will assist us in making major progress in this area.