Healthy Marriage and Responsible Fatherhood Initiative: Further Progress Is Needed in Developing a Risk-Based Monitoring Approach to Help HHS Improve Program Oversight
Published: 2008-09-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Strengthening marriages and relationships in low-income families has emerged as a national strategy for enhancing the well-being of children. The Deficit Reduction Act of 2005 (DRA) appropriated $150 million in discretionary grants each year from 2006 through 2010 to implement the Healthy Marriage and Responsible Fatherhood Initiative. To provide insight into how these programs are being implemented and monitored, GAO is reporting on (1) how the Department of Health and Human Services (HHS) awarded grants and the types of organizations that received funding; (2) what activities and services grantees are providing, including those for domestic violence victims; (3) how HHS monitors and assesses program implementation and use of funds; and (4) how program impact is measured. GAO surveyed grantees, interviewed HHS staff, reviewed HHS records and policy, and visited several programs.
section_title: What GAO Found
paragraphs: Operating under a deadline that allowed HHS 7 months to award grants, HHS shortened its existing process to award Healthy Marriage and Responsible Fatherhood grants to public and private organizations. During this process, HHS did not fully examine grantees' programs as described in their applications, including the activities they planned to offer, and this created challenges and setbacks for grantees later as they implemented their programs. For example, some grantees told us that they were informed that certain activities were not permitted months into program implementation even though HHS had approved these same activities described in their grant applications. The Healthy Marriage and Responsible Fatherhood programs provide similar activities, but their focus and target populations differ. Healthy Marriage programs are more likely to provide marriage and relationship activities, while Responsible Fatherhood programs are more likely to provide parenting skills. Additionally, both programs serve low-income and minority groups, but Healthy Marriage grantees are more likely to target teenaged youth, and Responsible Fatherhood grantees are more likely to target incarcerated parents. Both programs' grantees reported that they refer domestic violence victims to specialists in their communities. HHS uses methods that include site visits and progress reports to monitor grantees, but it lacks mechanisms to identify and target grantees that are not in compliance with grant requirements or are not meeting performance goals, and it also lacks clear and consistent guidance for performing site monitoring visits. Moreover, HHS's ability to readily identify which grantees are not in compliance or not meeting goals is hindered because it currently lacks uniform performance indicators and a computerized management information system that would enable HHS to more efficiently track key information on individual grantees. HHS told us that it is in the process of developing a management information system and has submitted uniform performance indicators for review. HHS has established a rigorous research agenda to gauge the long-term impact of healthy marriage and responsible fatherhood activities on diverse, low-income populations. HHS is sponsoring three multiyear impact evaluations of the Healthy Marriage program and one of the Responsible Fatherhood program.

section_title: Letter
paragraphs: Strengthening marriages and relationships in low-income families has  emerged as a national strategy for enhancing the well-being of children.  With the passage of the Deficit Reduction Act of 2005 (DRA), Congress  appropriated $150 million in discretionary grants each year from 2006  through 2010 to implement the Healthy Marriage and Responsible  Fatherhood Initiative (Initiative). The Initiative represents an  unprecedented financial commitment by the federal government to  support marriage and fatherhood programs. The focus of the Healthy  Marriage program is to encourage the formation and maintenance of two- parent households through healthy marriage promotion activities, while  the focus of the Responsible Fatherhood program is to strengthen the role  of the father in a child’s life. The Initiative supports two goals under  Temporary Assistance for Needy Families (TANF), the federally funded  block grant that funds programs designed to help needy families achieve  self-sufficiency. The goals are to prevent and reduce the incidence of out- of-wedlock pregnancies and to encourage the formation and maintenance  of two-parent families. To implement the Initiative, the Department of  Health and Human Services (HHS) competitively awarded grants to  various organizations to support a broad range of activities to promote  healthy marriage and responsible fatherhood. To address domestic  violence concerns, DRA required all grantees to consult with a domestic  violence expert and to include information on how they will address  domestic violence issues in their grant applications to HHS. It also  required that participation by individuals in the program be voluntary.
To gain insight into how these programs are being implemented, you  asked that we determine (1) how HHS awarded grants and the types of  organizations that received funding; (2) what activities and services  grantees are providing, including those for domestic violence victims; (3)  how HHS monitors and assesses program implementation and use of  funds, and (4) how program impact is measured.
To respond to these questions, we conducted a web-based survey of all  122 Healthy Marriage and 94 Responsible Fatherhood grantees that  provide direct services to program participants, asking them to provide  information about various aspects of their programs including the  characteristics of their organization, services they offered, curricula used,  and their process and procedures for identifying domestic violence. Of the  216 grantees to whom we sent our survey, 211 responded for a response  rate of 98 percent. Throughout this report survey results are based on the  number of grantees responding to a particular question. Additionally, to  obtain more in-depth information about services marriage and fatherhood  grantees are providing, we visited 14 grantees in Washington, Oklahoma,  New Mexico, Indiana, Oregon, and the District of Columbia. On 2 of these  visits, we accompanied HHS staff responsible for monitoring grantees. We  selected grantees to achieve variation in geographic location, type of grant  awarded, award amount, services, organization type, and the programs’  target populations. In addition, we conducted telephone interviews with  organizations that were awarded grants to provide technical assistance to  grantees, and help organizations develop fatherhood programs. Moreover,  to understand the criteria HHS used to award grants and the manner in  which HHS monitors and assesses program implementation, we randomly  selected 40 Healthy Marriage and Responsible Fatherhood grantee case  files to review. In this review, we examined several documents, including  applications, semiannual progress and financial reports, grantee selection  panel score sheets, and correspondences between grantees and agency  officials. To determine how program impact is measured, we interviewed  organizations that have received contracts to conduct impact evaluations  of Healthy Marriage and Responsible Fatherhood interventions and  assessed their methodological approach to measuring impact. We also  interviewed HHS officials about the uniform, program-wide performance  indicators under development and surveyed grantees about how they  measure program performance. We conducted this performance audit  from July 2007 to September 2008, in accordance with generally accepted  government auditing standards. Those standards require that we plan and  perform the audit to obtain sufficient and appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives. For  additional information on our scope and methodology, see appendix I.
subsections: section_title: Background
paragraphs: Welfare reform in 1996 made sweeping changes to the national welfare  policy, including a new emphasis on marriage as an area of societal and  governmental concern. With the passage of the Personal Responsibility  and Work Opportunity Reconciliation Act of 1996, which established the  Temporary Assistance for Needy Families (TANF) program, Congress  wrote into law that marriage is the foundation of a successful society and  promotes the interests of children. Congress was, in part, prompted to  address this issue because of what it deemed a “crisis in our Nation” in the  rate of pregnancies and births to unmarried women. In the legislation,  Congress cited the negative consequences to children that result from  these pregnancies and births, including greater risk for child abuse and  neglect, higher rates of poverty, and lower educational aspirations.
TANF was reauthorized under the Deficit Reduction Act of 2005 (DRA),  and signed into law in February 2006. DRA appropriated $150 million a  year for 5 years in discretionary grants for the Healthy Marriage and  Responsible Fatherhood Initiative (Initiative). While the Initiative was  established as part of TANF, the nation’s welfare program, it does not  impose income limits for program participants. However, HHS designated  a few priority groups for funding under the Initiative, including  incarcerated fathers and low-income, unwed, expectant or new parents. In  structuring the Initiative, HHS created two distinct grant programs—one  relating to Healthy Marriage and one to Responsible Fatherhood—but with  common aims. The Healthy Marriage program is aimed at encouraging the  formation and maintenance of two-parent households to improve child  well-being through healthy marriage promotion, and the Responsible  Fatherhood program is designed to strengthen the role of the father as a  means of promoting child well-being, specifically within the context of  marriage. HHS has stressed that the overarching Initiative is not designed  to encourage couples to stay in unhealthy marriages.
In the legislation, Congress prescribed the “allowable” activities for the  Initiative (see table 1). Given the broadness of these allowable activities,  HHS developed examples of services grantees could provide, such as  providing after-school programs for high school students and marriage  education courses that incorporate information on financial literacy.  Although providing services to victims of domestic violence is not an  allowable activity (see table 1), organizations were required by DRA to  describe in their grant application how their programs or activities would  “address” issues of domestic violence, and commit in their application to  consult with experts in domestic violence in developing their programs  and activities. The DRA also required that organizations describe in their  application what they would do to ensure and how they would inform  individuals that participation in programs is voluntary.
In fiscal year 2007 most of the funding, approximately $113 million, was  used to support Healthy Marriage and Responsible Fatherhood  demonstration grants, while the remaining funds were used for research,  technical assistance, administrative costs, and other TANF-related  activities (see fig. 1).
As part of the agency’s overall research agenda, HHS has sponsored  several impact evaluations of its programs. These evaluations are  considered to be the best method of determining the extent to which the  program, rather than other factors, is causing specific participant  outcomes. Impact evaluations, which are awarded through a competitive  bid process to experienced research firms, often are complex, multiyear  studies that can be difficult and costly to undertake and require particular  attention to both study planning and execution. Moreover, maintaining  proper incentives to obtain and sustain the participation of populations  that do not have financial and familial stability can be challenging. In  previous work, we found that HHS has established a rigorous research  agenda that regularly evaluates how well its programs are working. In  particular, HHS has a diverse research agenda focused on TANF that  includes research on strategies to help low-income individuals gain self- sufficiency.
section_title: HHS Awarded Grants to a Range of Public and Private Organizations, but the Awards Process Contributed to Challenges for Some Grantees
paragraphs: HHS awarded grants to a range of public and private organizations, but its  awards process later contributed to challenges for these grantees. HHS  shortened its awards process to meet a deadline specified in legislation  that allowed 7 months to award grants. HHS awarded grants to a diverse  set of grantees that provided direct services to program participants in 47  states, the District of Columbia, and American Samoa. However, as part of  its awards process, HHS did not fully examine grantees’ programs as  described in grantee applications, including the activities they planned to  offer, contributing to challenges for some grantees as they were  implementing their programs.
subsections: section_title: HHS Shortened Its Existing Awards Process to Meet DRA Deadline for Awarding Grants
paragraphs: HHS shortened its process to award grants by the end of the fiscal year  (September 30). Under DRA, which became law in February 2006, HHS  had to award grants in 7 months. Within this time frame, HHS had to  perform several tasks related to the awards process. Specifically, HHS  staff said they developed the grant announcements and the criteria for  selecting grantees under tight time constraints and limited the amount of  time organizations could apply for grants to fewer than the 60 days  recommended in HHS’s policy manual. HHS officials, who told us they had  not expected that more than 1,650 organizations would apply for funding,  hired The Dixon Group, a management consulting firm, to receive  applications, locate grant application reviewers, and assist with reviewer  training. At the same time The Dixon Group was receiving applications,  they also were selecting peer reviewers. Approximately 600 peer reviewers  served on 40 to 50 review panels for 4 weeks during July and August.  While the grant announcements stated that grant application reviewers  should be experts, HHS allowed peer review of the applications and The  Dixon Group and HHS characterized graduate students, professors, and  practitioners as peer reviewers. Further, because individuals who were  experts in the field of marriage and fatherhood applied for the grants, it  limited the pool of available expert reviewers. We reviewed several of the  resumes of the peer reviewers and found that while most had experience  as federal reviewers, their professional and volunteer experiences were  not always directly relevant to marriage and fatherhood services. For  example, one peer reviewer had experience in nursing and another listed  experience as a social studies teacher.
To determine which organizations would receive funding, HHS developed  guidance that outlined a five-part criteria for most grants, with each  criterion worth a specific amount of points. Reviewers scored  organizations’ applications using the guidance provided by HHS and by  judging how well the applicant responded to each criterion. For example,  a major criterion was the applicant’s “approach,” worth 40 points. For this  criterion, applicants were asked to describe their approach to recruiting  and retaining participants, their proposed activities, and time frames for  accomplishing specific milestones. Applicants also were required to  demonstrate that their proposed activities were consistent with the needs  of their target population and that the rationale for the approach was  based on the demonstrated effectiveness of similar activities. Finally,  under their approach section, applicants also had to describe how they  planned to address issues of domestic violence and ensure voluntary  participation. For the “organizational profile” criterion, worth 20 points,  organizations had to provide information that demonstrated their  qualifications to serve participants, including organizational charts,  financial statements, resumes, letters of support, and the qualifications of  partnering organizations. As part of other criteria, applicants were asked  to provide a budget and budget justification, and information on how they  proposed to measure the outcomes of their programs. Applicants could  receive up to 5 bonus points if they demonstrated prior experience in  developing, implementing, or managing skills-based marriage or  fatherhood education programs. See appendix II for a table of the criteria  used for each type of grant.
The peer reviewers used these criteria to score applicants, and HHS  ranked the applications based on the scores. With some exceptions,  applications that received the highest scores were awarded grants. HHS  made exceptions to ensure, among other things, that grants were  geographically distributed and reflected a diversity of target populations  and communities served.
In September 2006, HHS began notifying grantees of their awards, but  experienced a setback when they had to reconvene review panels to  rescore 31 applications. When scoring some applications, some reviewers  incorrectly gave applicants zero points for the “approach” section.  According to the grant announcements, if applicants failed to discuss how  they would inform individuals that program participation was voluntary,  as well as discuss specific issues relating to domestic violence issues, they  would receive no points for the “approach” criterion. HHS discovered that  reviewers had incorrectly interpreted whether applicants satisfied this  portion of the “approach” criterion, and after clarifying the criteria,  required that they rereview the applications.
section_title: HHS Awarded Grants to a Diverse Set of Grantees
paragraphs: HHS awarded grants to a diverse set of grantees that included 216 different  organizations—122 were Healthy Marriage and 94 were Responsible  Fatherhood demonstration grants that provided direct services to program  participants in 47 states, the District of Columbia, and American Samoa  (see app. III). In responding to our survey, grantees selected multiple  categories to describe their organizational type. The majority—89 percent  of the grantees—classified themselves as nonprofits. However, faith- based, for-profit, and private organizations also received funding. Awards  for Healthy Marriage demonstration grants ranged from $225,000 to $2.4  million, and awards for Responsible Fatherhood demonstration grants  were for smaller amounts, ranging from $188,000 to $1 million.
Over two-thirds of our survey respondents indicated that their  organization had prior experience related to healthy marriage or  responsible fatherhood activities. This experience included providing  workshops for couples and singles, parenting classes, and relationship  workshops for high school students. Some of these organizations also  provided a broader array of other services to the community, such as  mental health services and counseling services, and substance abuse  treatment. Also, at least a dozen of the grantees had provided abstinence  services and some Healthy Marriage grantees were previous recipients of  grants from HHS for related purposes, including healthy marriage  curriculum development and fostering healthy marriage within  underserved communities.
section_title: HHS’s Grant Awards Process Contributed to Challenges Grantees Had Implementing Programs
paragraphs: HHS’s grant awards process contributed to challenges grantees later faced  implementing their programs. HHS was able to announce grant awards by  September 30; however HHS did not fully examine grantees’ programs as  described in grantee applications. Specifically, we found during 5 of our 14  site visits that grantees, whose program activities had initially been  approved by HHS, were later told that those same activities were not  allowed under the conditions of their award. For example, during a site  visit, one grantee reported that it proposed providing services to  unmarried couples in its application and was doing so until HHS informed  them that these services were not allowed under the conditions of their  award. Another grantee told us that it was providing General Educational  Development (GED) education as part of its Healthy Marriage program,  but was later notified that the activity was not allowed. These grantees  were well into program implementation when they were told to  discontinue certain activities. One grantee we visited said it engaged in  activities that were not allowable under the grant for a full year before  being informed by HHS that the activities were not permissible. The  grantee told us that it would have benefited from more timely review and  feedback from HHS. In another case, HHS told a grantee that it would have  to extend the length of its workshops for participants from 60 minutes to  90 minutes to 8 hours, even though the grant application noted that short,  workshops would be provided. To implement this change, the grantee  said it would likely incur additional expenses, such as paying facilitators  for extra time and spending more for rental space.
HHS told us that it received more applications than expected and this was  the first time it awarded these grants. HHS also said it had learned from  this experience.
section_title: Programs Offer a Range of Similar Activities, but Their Focus and Target Populations Differ
paragraphs: While the range of activities offered and populations served by Healthy  Marriage and Responsible Fatherhood programs’ grantees are similar,  their focus and target populations differ. Both programs offer a range of  similar activities, but a greater percentage of marriage programs provided  activities related to marriage and relationship skills and a larger  percentage of fatherhood programs provided parenting skills. Grantees for  both programs reported that they refer domestic violence victims to  specialists when appropriate. Additionally, while both programs target  such groups as minority and low-income populations, Healthy Marriage  grantees are more likely to target high school or teenaged youths, and  Responsible Fatherhood grantees are more likely to target incarcerated  parents.
subsections: section_title: Both Programs Offer a Range of Similar Activities and Refer Domestic Violence Victims to Specialists When Appropriate, but They Differ in Which Activities They Offer Most Frequently
paragraphs: Both programs offer a range of similar activities, and grantees from both  programs said they refer victims of domestic violence to specialists in  their communities when appropriate (see fig. 2). However, according to  our survey, while both programs offer many similar activities, Healthy  Marriage programs focus more on those related to marriage and  relationship services, whereas Responsible Fatherhood programs are  more likely to focus on providing services teaching parenting skills.  Specifically, 94 percent of Healthy Marriage grantees, compared to 55  percent of Responsible Fatherhood grantees, reported offering marriage  and relationship activities. During our visits to several Healthy Marriage  grantees, we often observed activities related to marriage and  relationships. For example, we observed a Healthy Marriage workshop  where couples took quizzes to determine how well they knew one another  and then participated in a discussion about commitment, chemistry, and  compatibility. Conversely, 92 percent of Responsible Fatherhood grantees,  compared to 47 percent of Healthy Marriage grantees, reported in our  survey that they provide services related to teaching parenting skills. For  example, a Responsible Fatherhood grantee program we visited included  in its curriculum parenting skills training, such as lessons on a child’s  developmental needs and how to communicate with children of different  ages. In addition, Responsible Fatherhood grantees were more likely than  Healthy Marriage grantees to report that they focused on providing  programs with specific services to help participants achieve economic  stability, including assistance with finding a job. Healthy Marriage grantees  also reported that they focus on economic stability activities, but to a  lesser extent than Responsible Fatherhood programs. According to HHS,  Healthy Marriage grantees can provide these services only within the  context of allowed activities (see table 1). For example, Healthy Marriage  grantees might discuss financial issues as part of marriage and relationship  skills. Depending on the conditions of the award, grantees might provide  more than one of the services or activities listed in figure 2.
Both Healthy Marriage and Responsible Fatherhood grantee programs  offer services for varying lengths of time and in various settings. Some  programs have one intensive session in a lecture setting, while others offer  classroom settings that are more interactive and may be offered for 1 or 2  hours 1 night a week for up to 17 weeks. One grantee program we visited  offered marriage workshops to participants at weekend retreats with paid  lodging, and two Responsible Fatherhood programs we visited included  optional home visits by staff. In addition, some grantees run  advertisements or sponsor advertising campaigns that discuss the  importance of healthy marriage and responsible fatherhood. For example,  one advertising campaign designed a billboard that read “a diamond isn’t  the only thing that should last forever.”
According to our survey, the majority of grantees—98 percent—deliver  their services through classroom instruction using a curriculum (see fig.  3). Many survey respondents said they developed and used their own  curriculum (41 percent of Healthy Marriage and 47 percent of Responsible  Fatherhood respondents). For example, one grantee we visited said it  developed its own Spanish-language curriculum because the few existing  Spanish-language curricula for Responsible Fatherhood programs did not  meet the specific needs of the Latino population the grantee served. Other  grantees adapt commercially available curricula to meet the needs of  participants. The most-commonly-used, commercially available curriculum  was the Prevention and Relationship Enhancement Program. This  curriculum focuses on identifying strengths and weaknesses of a marriage,  improving communication skills, and increasing the connection between  the partners. Technical assistance providers make information about  curricula available to grantees on their Web site. A list of curricula used by  multiple grantees is in appendix IV.
Most grantees—about 93 percent—reported in our survey that they  include information on domestic violence in their programs. For example,  several grantees modified their curriculum to include a discussion of  domestic violence with participants. One survey respondent noted that it  leads a discussion on domestic violence issues that helps participants self- identify and understand domestic violence. During our site visits, some  Healthy Marriage grantees told us that they focus on the characteristics of  a healthy relationship. In addition to discussing topics related to  relationship health and domestic violence awareness, grantees also  distribute informational materials about domestic violence (see fig. 4). For  example, during a site visit to a Healthy Marriage grantee, we observed  classroom instructors distributing pamphlets on recognizing signs of  domestic violence. Handouts include state Directories of Domestic  Violence Support Services; handbooks for domestic violence victims, and  victims’ rights; and pamphlets on topics ranging from “recipes for safety”  to the characteristics of an abusive relationship.
Additionally, most grantees reported in our survey that they have  protocols for how staff should handle instances where program  participants may be victims of domestic violence, and many grantees train  their staff on identifying signs of domestic violence, as well as on teaching  program participants the signs of unhealthy relationships. Moreover, most  grantees reported that they consult with domestic violence organizations  and refer potential domestic violence victims to them. For example, one  grantee we visited told us that it consulted with two different domestic  violence organizations when designing its Responsible Fatherhood  program. The domestic violence organizations helped the grantee develop  part of a workshop related to domestic violence and also presented  information to program participants. During our site visits, grantees also  told us they refer program participants to domestic violence specialists  when appropriate. For example, one of the grantees we visited said that  when it encountered a potential domestic violence situation, it held a joint  meeting with a caseworker, domestic violence expert, and a family  services coordinator. Collectively they determined the appropriate referral  for the person. The DRA does not include domestic violence services as an  allowed activity, but does require that programs have in place mechanisms  for addressing domestic violence.
section_title: Programs Focus Services on Different Target Populations
paragraphs: Healthy Marriage and Responsible Fatherhood grantee programs focus on  providing services to different populations, but they both target low- income and minority populations. According to our survey, 58 percent of  Healthy Marriage and 52 percent of Responsible Fatherhood grantees  target low-income individuals, and 39 percent of Healthy Marriage and 36  percent of Responsible Fatherhood grantees target minorities (see fig. 5).  Healthy Marriage grantee programs target high school or teenaged youths  at higher rates than Responsible Fatherhood grantee programs, in part,  because education in high schools is one of the Healthy Marriage  program’s allowed activities. On the other hand, Responsible Fatherhood  programs target incarcerated parents, typically fathers, because HHS  designated a portion of the program’s funding for this population. Both  grantee programs allow men and women to participate in their  programs—even though the Responsible Fatherhood programs were  created specifically to target men, they are both open to men and women.  An administrative complaint was filed by a legal advocacy organization  centering on whether women have equal access to the program and  subsequently HHS reminded grantees that the Responsible Fatherhood  programs are open to eligible men and women.
Grantees use a variety of methods to attract participants to the program.  According to our survey, grantees rely heavily on word of mouth, but they  also attract participants through educational handouts and brochures,  referrals, and advertisements such as promotion campaigns (see fig. 6).  For example, one grantee we visited, which targets Latinos, indicated that  while it advertises through a variety of methods including community- based advertising, radio, and door-to-door recruiting, it had difficulty  attracting participants. Some grantees told us they devised numerous  incentives to better retain participants. For example, one grantee we  visited told us it provides food and child care at each session,  transportation subsidies, and Wal-Mart and Babies R Us gift cards once  participants completed the program.
Participation in the Healthy Marriage and Responsible Fatherhood  programs must be voluntary as required by DRA, and according to our  survey, grantees used a variety of methods to inform participants that  participation was voluntary. Specifically, 95 percent of survey  respondents indicated they provide verbal notification that  participation is voluntary, while 89 percent indicated that they provide  written notification (see fig. 7).
section_title: HHS Has a Program Monitoring System, but Lacks Mechanisms to Identify and Target Grantees Not in Compliance with Grant Requirements or Not Meeting Performance Goals
paragraphs: HHS has a program monitoring system, but it lacks the mechanisms to  identify and target grantees not in compliance with grant requirements or  not meeting performance goals. HHS uses multiple tools to monitor  grantee programs, such as site visits and reviews of reports submitted by  grantees. However, HHS lacks specific guidance for conducting  monitoring site visits. Moreover, HHS’s ability to target grantees in need of  assistance is hindered by the lack of an effective Management Information  System.
subsections: section_title: HHS Uses Multiple Tools to Monitor Grantees
paragraphs: To monitor Healthy Marriage and Responsible Fatherhood grantee  performance, HHS uses multiple tools including a combination of phone  calls, e-mails, grantee progress reports, and site visits. HHS also reviews  grantee Single Audit Act reports. HHS is responsible for monitoring the  216 Healthy Marriage and Responsible Fatherhood grantees and according  to our survey; almost all grantees reported some contact from HHS staff.  According to the grantees we visited, HHS staff contact them at least once  a month. Grantees said that HHS staff typically contact them to notify  them of opportunities for technical assistance, address errors or issues  that arise during review of required programmatic and financial progress  reports, and to notify them of upcoming events. In addition, some grantees  also initiate communication with HHS to ask questions regarding policy, to  request approval for certain activities, or to request budget modifications.
Semiannually, HHS requires grantees to submit both programmatic and  financial progress reports, which, among other things, provide HHS with  updates on grantees’ progress toward meeting performance goals that  grantees established for themselves in their applications, as well as  provide information on grantees’ compliance with domestic violence and  other HHS policies. For example, some grantees report to HHS on the  number of participants they expect to serve. Some grantees also may  report on the types of activities and participant satisfaction with programs  or services as well as changes in participant behavior before and after  programs. They also may report on any problems they may be  experiencing, including recruiting challenges. Because grantees can set  their own program goals and establish their own measures for these goals,  there is considerable variation among the information being collected.  Financial progress reports contain information, such as financial  statements, that allow HHS to track the use of grant funds. HHS also  monitors grantees’ use of funds by tracking grantees’ draw down of funds.  Specifically, HHS also is able to compare financial progress reports  submitted by grantees with reports from the HHS electronic grant payment  management system to monitor grantees’ withdrawal of funds. For  example, if HHS observes that a grantee has not withdrawn funds  according to its schedule, they will contact the grantee to determine the  reason the grantee has not been withdrawing funds. For grantees that  received federal funds in excess of $500,000, HHS monitors and reviews  audit reports in accordance with the Single Audit Act. According to HHS,  its review of grantee Single Audit Act reports covers compliance with  audit standards, completeness, timeliness, and other audit considerations.
As part of HHS’s on-site monitoring, at least one HHS staff member will  interview grantee staff, review program documents, and in some instances  observe programs in operation. For example, when we accompanied HHS  during two grantee site visits in March of this year, HHS and one of the  grantees discussed challenges the grantee was experiencing with  recruiting participants. HHS discovered that the grantee, whose target  population included a rural district, was struggling to meet its goal for the  number of participants it initially believed it would serve. The HHS official  referred the grantee for technical assistance in order to help it improve  participant recruitment and retention. HHS officials told us that  monitoring site visits was a priority for them and their goal was to visit all  grantees within the first 3 years of the award period. As of August 2008,  HHS told us that approximately 84 percent of grantees had received a site  visit from HHS since September of 2006, when the programs were first  funded. Our survey results confirmed that HHS had visited most of the  grantees in the first 2 years.
section_title: HHS Lacks Guidance for Conducting Site Visits and Other Monitoring Activities
paragraphs: HHS staff lack specific guidance for conducting site visits and other  monitoring activities, according to our interviews with HHS staff, visits  and interviews with grantees, and file reviews. As a result, the length and  types of issues reviewed and documentation examined by HHS during site  visits varied depending on who conducted the visit. HHS officials told us  that staff responsible for monitoring are to use the legislation, grant  announcements, and site visit protocol as guidance to monitor grantee  performance. Although legislation and grant announcements provide some  general guidance, they do not specifically define what is permitted under  each allowed activity. For example, the grant announcement lists marriage  education as an allowed activity for some grantees, but does not  specifically describe what marriage education activities are permitted  under the grant. We also found the site visit protocol provided by HHS was  limited to a checklist of topics for HHS to cover during grantee site visits.  The checklist did not detail the process, the criteria for conducting  monitoring site visits, or the key items to be examined, leaving each  monitoring staff member the discretion to determine what information to  gather and how best to gather it. Moreover, we found other  inconsistencies in how HHS conducts monitoring visits. For example,  during some monitoring site visits, HHS staff observed grantees providing  services while in other instances they did not. According to HHS officials,  HHS staff are required only to observe services if the timing of the visit  coincides with services, but they are not required to schedule monitoring  site visits to coincide with sessions. Because some HHS officials do not  observe grantees providing services, they cannot confirm that the services  are in fact being provided or that the funding is being spent as intended.
The lack of sufficient guidance from HHS may have led HHS staff to  inconsistently apply HHS policy among some grantees. For example, through  our interviews and file review, we found that some monitoring staff members  allowed several Healthy Marriage and Responsible Fatherhood grantees to  use incentives to retain program participants, while others were told they  were not permitted to use similar incentives. From our review of grantee files,  we found instances where HHS staff worked with grantees to adjust or lower  the goals they developed for themselves to meet second-year targets. Other  grantees who did not meet their year-1 performance goals were not permitted  to adjust their performance targets. In another example, HHS officials told us  that abstinence education was not allowable under the Healthy Marriage  program, but we observed during our site visits and review of grantee data  several Healthy Marriage grantees operating programs that focused on  abstinence education.
section_title: HHS’s Ability to Target Grantees Not in Compliance with Grant Requirements or Not Meeting Performance Goals Is Hindered by the Lack of an Effective Management Information System
paragraphs: The lack of an effective management information system that captures key  information on individual grantees hinders HHS’s ability to appropriately  identify which grantees are not in compliance with grant requirements or are  not meeting performance goals. Although it maintains paper files on each  grantee, the breadth and detail in these files vary considerably. For example,  some HHS staff keep very detailed logs on grantees, while others maintain  minimal records. Moreover, the information in these files is not always used  to target grantees in need of assistance or to identify how grantees are using  their funds. For example, one grantee used grant funds to provide marriage  education services not allowed under its grant to participants. Although  information such as how grantees are using their funds should be contained  in the files, the grantee in this instance was notified months after initiating  services that the program was not allowed, causing the grantee to use  alternative sources of funding to provide services. Moreover, through our  case studies, we found instances where grantees did not receive timely  feedback on progress reports, documents that are part of the files HHS  maintains on individual grantees. These files provide an early alert to  problems grantees may be experiencing and could potentially identify  grantees at risk of not meeting performance goals. Despite HHS having this  information, some grantees told us that they did not receive timely feedback  from HHS, causing them setbacks in implementing program activities.
Without an effective management information system, HHS has not been able  to take a strategic approach to conducting grantee site visits and other  monitoring activities. Although HHS told us that grantees experiencing  challenges should receive priority for site visits, our review of a random  sample of grantee files showed that several grantees were having difficulty  recruiting participants, yet HHS did not always give them priority for on-site  review. Moreover, during our site visits, some grantees told us they were  experiencing difficulty meeting participation goals or recruiting the number of  participants they indicated to HHS they would serve through their program.  These grantees also were not targeted specifically for on-site monitoring.  Specifically, the decision of which grantees to visit and in what order was left  to the discretion of HHS staff, according to HHS officials. Because grantees  that were experiencing challenges did not always receive priority for  monitoring site visits and these site visits were scheduled based on HHS staff  scheduling preferences, we found that monitoring was not always based on  grantee risk or need.
HHS told us it is in the process of developing a database that will help it  standardize and combine grantee communications and performance  information. According to HHS, the first phase of the web-based  management information system has been completed. The system is  designed to replace the paper files and, according to HHS, will  considerably reduce or eliminate inconsistencies in HHS’s recordkeeping.  The management information system will capture performance indicators  developed by the grantee and submitted semiannually in grantee  programmatic progress reports, such as grantees’ progress toward meeting  participant recruitment goals and changes in participant behavior. The  new system should allow HHS to better manage and search for grantee  information, upload grantee communications, and track data from grantee  programmatic progress reports. It is not clear, however, when HHS will be  able to include uniform performance indicators that it plans to collect  from individual grantees. HHS officials told us that performance indicators  have been developed, but are pending implementation while they are  currently under review by the Office of Management and Budget. HHS said  it anticipates having grantees begin collecting data in autumn or early  winter of 2008, the start of the third year of funding for the 5-year  initiative. According to HHS, the uniform performance indicators will  eventually be part of its planned management information system.
section_title: HHS Has Long-term Research Underway Intended to Assess Program Impact
paragraphs: HHS has four multiyear studies of marriage and fatherhood programs  underway that are intended to assess the impact of the programs on  various populations and understudied groups, the final results of which  are expected between 2011 and 2013. Funded partially by the DRA, HHS  awarded contracts to three organizations—RTI International; Mathematica  Policy Research; and MDRC—that competitively bid to conduct the  evaluations, which run over several years and across several marriage or  fatherhood programs. Two of the impact studies will exclusively follow  grantees funded under the Healthy Marriage and Responsible Fatherhood  Initiative, while the other two studies will follow a mix of grantees and  healthy marriage programs not funded under the Initiative. In all cases, the  programs being studied primarily offer participants skills-based marriage  or fatherhood education. The primary focus of HHS’s research is to  determine the impact, if any, marriage and fatherhood programs have on  couples, families, and fathers as a result of participation in the programs.  Impact evaluations are the strongest method for assessing the efficacy of a  program because they allow for a comparison between similar groups that  differ only with respect to whether they received a service or “treatment.”  However, they often are difficult and expensive to conduct because they  take years to complete and it often is difficult to retain enough participants  to produce meaningful results. Prior research has focused on the impact of  marriage services on middle-income families and couples. A review of the  literature, sponsored by HHS, on the overall impact of marriage and  relationship programs found that, on average, middle-income couples  receiving services showed increased relationship satisfaction and  improved communication skills. HHS’s research agenda represents the  first major federal effort to study the impact of healthy marriage and  responsible fatherhood programs on low-income populations and is part of  a wider body of research being developed by HHS.
Two of the three healthy marriage studies—the Building Strong Families  (BSF) and the Supporting Healthy Marriage (SHM) evaluations—focus on  low-income couples who are expecting or have recently had a child. The BSF  is following 5,103 low-income unmarried couples across seven marriage  programs around the time of the birth of a child using data collected at three  stages of participants’ lives. The SHM study is examining the effects of  healthy marriage programs on 6,860 married couples across eight marriage  programs. The third healthy marriage study—the Community Healthy  Marriage Initiative—expands its focus beyond specific target populations to  entire communities: the initiative is comparing couples in three different  geographic communities with federally funded healthy marriage programs— Milwaukee, Wisconsin; Dallas, Texas; and St. Louis, Missouri—with three  demographically similar communities—Cleveland, Ohio; Ft. Worth, Texas;  and Kansas City, Missouri—where there are no federally funded healthy  marriage programs. The study, which involves 4,200 participants, will explore  whether the presence of intensive healthy marriage programs promotes  changes in attitudes and behavior toward marriage in the communities being  studied. In addition to the three healthy marriage evaluations, HHS also is  funding an impact evaluation of Responsible Fatherhood programs. The  National Evaluation of the Responsible Fatherhood, Marriage and Family  Strengthening Grants for Incarcerated and Re-entering Fathers and Their  Partners (MFS-IP) began in 2006, when the first year of Responsible  Fatherhood funds became available, and is currently enrolling participants.  The MFS-IP, much like the three marriage studies, will explore changes in  couple quality and changes in attitudes toward marriage. In addition, the  MFS-IP will assess changes in outcomes for employment and economic  stability, in line with the parameters of activities allowed under the legislation  for Responsible Fatherhood grantees (see fig. 8).
For all four studies, evaluators will collect outcome data for the couples  participating in programs at various stages of the study and then compare  the results against groups of couples who did not participate in the  programs. Because the two groups are, by nature of the study design,  similar in every major respect, any differences between the two groups  can be attributed to the program. The evaluators for the four studies  differed on the methods they used to create these two groups. Two of the  four studies, the BSF and the SHM, randomly assign couples to either a  group that receives services (the experimental group) or group that does  not (the control group). The other two studies are quasi-experimental.  This type of study uses methods other than random assignment to create a  comparison group, such as selecting a set of individuals who have similar  characteristics to the group receiving the program services under study.
To compare these groups in the four studies over time, the evaluators are  conducting surveys and interviews, generally 1 year and 3 years after  participating in a program, in order to gauge couples’ and families’  outcomes. The surveys ask questions about how couples are  communicating after participating in a program; whether they are using  the skills they learned in the program; and how they would rate, overall,  the quality of their relationship since participating in the program. The  evaluators also will administer the same surveys to the couples not  participating in Healthy Marriage or Responsible Fatherhood programs in  order to make comparisons between the two groups. For example, the  BSF study will examine a range of outcomes, including whether marriage  services improved marital relationships, reduced marital instability, and  improved child well-being.
In general, we found the evaluations to be well-designed and rigorous,  however, there are inherent difficulties presented by the Community  Healthy Marriage Initiative, which assesses the impact of healthy marriage  programs on entire geographic areas. Specifically, it may be difficult to find  and study true comparison communities. One positive feature of the study is  the collection of baseline data for each of the participating communities;  however, it is difficult to determine if the contractors have captured and  controlled for the important variables needed to match the communities. In  addition, it will be difficult to determine if changes in the community stem  from Healthy Marriage program services or some other factors.
section_title: Conclusions
paragraphs: Marriage and fatherhood programs have emerged as a national strategy for  improving the well-being of children. The federal government has committed  $150 million annually for 5 years for these programs and provided for an  evaluation the Healthy Marriage and Responsible Fatherhood Initiative to  determine how well the Initiative is working for low-income populations.  While HHS has made an effort to visit nearly all of the programs in their first 2  years of operations, absent mechanisms for detecting grantee compliance and  performance issues, some grantees did not receive monitoring and technical  assistance soon enough and had to make modifications to their program well  into implementation. Moreover, effective monitoring was hampered by a lack  of an effective management information system that captures key  information, including uniform performance indicators for grantees, and the  lack of consistent and clear monitoring guidance. Without an effective  monitoring system or clear and consistent monitoring guidance, grantees may  continue to be at risk of noncompliance with HHS policy or of not meeting  performance requirements.
section_title: Recommendations for Executive Action
paragraphs: In order to improve monitoring and oversight of Healthy Marriage and  Responsible Fatherhood grantees, we are recommending that the  Secretary of HHS:    employ a risk-based approach to monitoring grantees and conducting  grantee site visits, using its planned management information system  and information from both progress reports and uniform performance  indicators to help identify those grantees at risk of not meeting  performance goals or not in compliance with grant requirements; and    create clear, consistent guidance and policy for monitoring Healthy  Marriage and Responsible Fatherhood grantees.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to HHS for its comments; these appear in  appendix V. In its comments, HHS concurred with our recommendation that  it employ a risk-based approach to monitoring using its planned management  information system and performance indicators to help identify grantees for  monitoring, saying these tools would further enhance oversight and  monitoring efforts currently underway. In its comments, HHS states that it  has already developed and implemented this portion of the recommendation,  including developing a customized approach to prioritizing site visits and  technical assistance. However, HHS caveats that only the first phase of its  web-based management information system has been completed and that  performance indicators that would help them identify those grantees at risk,  are still awaiting approval by OMB. A fully implemented management  information system with performance indicators in place will further enhance  HHS’s ability to monitor grantees based on risk.
HHS disagreed with the portion of our recommendation that HHS lacks  specific guidance for conducting monitoring site visits. In its comments, HHS  stated that it developed a clear, comprehensive, and thorough protocol and  trained project officers on the critical and essential items that must be  covered during grantee site visits. As we stated in our report, this protocol  was limited to a checklist of topics to be covered during the site visit and did  not describe the process to be followed or criteria to be used to monitor  grantees. Moreover, the lack of clarity in this protocol may have contributed  to the inconsistencies in how site visits were administered by HHS staff, as  noted in our report.
HHS also stated in its response that fiscal oversight or monitoring a  grantee’s fiscal compliance can be used as an alternative mechanism to  confirm whether grantees are providing services or spending funds as the  grant intended. While we agree that monitoring grantee’s fiscal compliance  is essential, HHS’s comments do not change our view that observing  activities is critical to confirming that grantees are actually providing  services as intended by the grant.
Finally, HHS commented on our finding that some grantees were operating  programs focused on abstinence education. HHS stated that it is  impermissible to use Deficit Reduction Act (DRA) funding for abstinence  education, however, grantees may use funding from other sources to  provide abstinence education through programs separate from the Healthy  Marriage and Fatherhood programs. We visited one such program whose  staff told us that they used DRA funding to support their abstinence  education program and that abstinence education was not provided as a  single lesson, but was the focus of the entire curriculum.
HHS also provided technical changes to a draft of the report, which we  incorporated into the report as appropriate.
As agreed with your office, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after its  issue date. At that time, we will send copies of this report to the Honorable  Michael O. Leavitt, Secretary of Health and Human Services, relevant  congressional committees, and other interested parties. In addition, the  report will be available at no charge on GAO’s Web site at  http://www.gao.gov. Please contact me on (202) 512-7215 if you or your  staff have any questions about this report. Contact points for our offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are listed in appendix VI.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: To gain insight into how Healthy Marriage and Responsible Fatherhood  programs are being implemented, we were asked to report on (1) how the  Department of Health and Human Services (HHS) awarded grants and the  types of organizations that received funding; (2) the activities and services  grantees are providing, including those for domestic violence victims; (3)  the manner in which HHS monitors and assesses program implementation  and use of funds; and (4) how program impact is measured.
To address the objectives, we conducted a Web-based survey of 122  Healthy Marriage and 94 Responsible Fatherhood grantees asking them to  provide information about various aspects of their programs. We received  a response rate of 98 percent. We also visited 14 grantees in Washington,  Oklahoma, New Mexico, Indiana, Oregon, and the District of Columbia. In  addition, we conducted telephone interviews with organizations that  provide technical assistance to grantees and help other organizations  develop fatherhood programs. To further understand the criteria HHS  used to award grants and the manner in which HHS monitors and assesses  program implementation, we reviewed 50 grantee case files, 40 randomly  and 10 deliberately selected, examining documents such as applications,  semiannual progress and financial reports, grantee selection panel score  sheets, and correspondences between the grantees and agency officials.  To determine how program impact is measured, we interviewed  organizations that received contracts to conduct impact evaluations of  Healthy Marriage and Responsible Fatherhood interventions and assessed  their methodological approach to measuring impact.
subsections: section_title: Survey of Marriage and Fatherhood Programs
paragraphs: To address all of our objectives, we conducted a Web-based survey of all  216 demonstration grantees that provided direct services to participants,  122 Healthy Marriage and 94 Responsible Fatherhood grantees. We asked  grantees about various aspects of their programs, including the  characteristics of their organization, services they offered, experience  providing similar services, curricula used, their process and procedures  for identifying domestic violence, staff training; and any evaluations the  grantees were conducting on their own. In order to identify respondents  for our survey, we obtained lists of grantees and contact information from  HHS’s Administration for Children and Families and their Office of Grants  Management. We compared the two lists to compile the most accurate list  of grant recipients and contact information. In some cases, we contacted  the organization directly to determine the appropriate contact person and  obtain updated information. Of the 216 grantees contacted, 211 provided  information, for a response rate of 98 percent. The survey data was  collected from February 2008 to April 2008.
Because this was not a sample survey, it has no sampling errors. However,  the practical difficulties of conducting any survey may introduce errors,  commonly referred to as nonsampling errors. For example, difficulties in  interpreting a particular question, sources of information available to  respondents, or entering data into a database or analyzing them can  introduce unwanted variability into the survey results. We took steps in  developing the questionnaire, collecting the data, and analyzing them to  minimize such nonsampling error. For example, prior to launching our  survey, we worked with social science survey specialists to develop the  questionnaire and minimize error. We tested the content and format of the  questionnaire with multiple grantees prior to administering the survey to  address issues such as differences in question interpretation, and  differences in data tracking. We conducted 10 survey pretests. As a result  of our pretests, we changed survey questions as appropriate and tested  those changes with grantees that participated in our original pretests.  Further, the final pretests were performed using the Web-based survey  tool, which checked for accuracy and usability. To ensure grantees  responded to the survey, we sent e-mail reminders and conducted follow- up telephone calls with nonrespondents. Since this was a Web-based  survey, respondents entered their answers directly into the electronic  questionnaire, eliminating the need to key data into a database, minimizing  error. We used content coding, computer edits, and independent analysts  to assess the reliability of the information collected.
section_title: Site Visits to 14 Grantees
paragraphs: To gather information to respond to all of these questions, we visited 14  grantees—9 Healthy Marriage grantees and 5 Responsible Fatherhood  grantees—in Washington, Oklahoma, New Mexico, Indiana, Oregon, and  the District of Columbia. We selected grantees to achieve variation in  geographic location, type of grant awarded, award amount, services,  organization type, program curriculum, and the programs’ target  populations. During each site visit we asked the grantees about the grant  application process and their programs, including accessibility of funds,  services provided, guidance and communication with HHS, and challenges  the grantees experienced. During seven of these site visits, we observed  the implementation of marriage and fatherhood services. Further, we also  observed HHS staff in the process of conducting two grantee site visits. In  analyzing our site visit interviews we arrayed and analyzed narrative  responses thematically. The site visits were conducted from December  2007 through April 2008.
section_title: File Review
paragraphs: Further, to learn about the criteria used to award grants and HHS’s  monitoring activities, we conducted a review of 50 grantee case files out of  the total 229 grants awarded in September 2006. We conducted a simple  random sample of 40 Healthy Marriage and Responsible Fatherhood  grantee case files—28 Healthy Marriage grantees and 12 Responsible  Fatherhood grantees. We also deliberately selected and reviewed an  additional 10 grantee case files; the team deliberately reviewed case files  for 1 technical assistance grantee, 6 grantees that assist other  organizations with developing fatherhood programs, and 3 grantees we  visited. During the case file review, we examined documents contained in  the grantee’s case file including, the grantee’s original and continuation  application, semiannual progress and financial reports, grantee selection  panel score summary sheets, correspondences between the grantee and  agency officials, and site visit reports. We reviewed the documents to  assess HHS’s compliance with its grants policy manual and to understand  how HHS monitors use of funds. We also reviewed Single Audit Reports  for the selected sample of grantees. To facilitate the case file review, we  developed a data collection instrument to record specific information for  each case file reviewed. We used content coding to analyze the qualitative  information from our data collection instrument. We conducted our  review on-site at HHS’s Administration for Children and Families.
section_title: Review of Internal HHS Documents and Interviews with HHS Officials
paragraphs: We also reviewed the HHS grant selection criteria included in the grant  announcements and HHS’s internal guidance on grant selection processes  which we compared to the selection of Healthy Marriage and Responsible  Fatherhood grant recipients. In addition to these reviews, we interviewed  HHS and the contractor responsible for hiring reviewers and organizing  the review panels.
To determine how HHS measures program impact, we collected survey  instruments, design papers, and program guidelines for each of the four  impact evaluations underway in order to assess their methodological  soundness. In addition, we interviewed HHS staff responsible for  overseeing the contractors responsible for the impact evaluations. To  gauge how HHS is monitoring the progress of grantees, we interviewed  HHS staff regarding its process for monitoring grantees, including  guidance used and staff training provided to determine how HHS monitors  and assesses program implementation and use of funds.
section_title: Interviews with Experts
paragraphs: To identify critical components that should be included in services  provided by grantees, we interviewed multiple experts in the areas of  marriage, fatherhood, and domestic violence. We also interviewed  grantees and contractors that were not direct providers of healthy  marriage and responsible fatherhood services but received funding under  the Healthy Marriage and Responsible Fatherhood Initiative to provide  technical assistance to demonstration grantees, conduct research, and  help other organizations develop fatherhood programs.
We conducted this performance audit from July 2007 to September 2008,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Grantee Selection Criteria
paragraphs: Healthy  Marriage  Resource  Center (1 grant  awarded) awarded) grants awarded) grants awarded)
section_title: Appendix III: States and Territories with Grantees That Provide Direct Services to Participants as of February 2008
section_title: Appendix IV: Curricula Being Used by Healthy Marriage and Responsible Fatherhood Grantees and Frequency of Use
paragraphs: Prevention and Relationship Enhancement Program (PREP)
section_title: Appendix V: Comments from the Department of Health and Human Services
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: Sherri Doughty (Assistant Director) and Ramona L. Burton (Analyst-in- Charge) managed all aspects of the assignment. Michelle Bracy, Melissa  Jaynes, and Chhandasi Pandya made significant contributions to this  report, in all aspects of the work. In addition, Cathy Hurley, Kevin  Jackson, Stuart Kaufman, and Luann Moy provided technical support in  design and methodology, survey research, and statistical analysis; Daniel  Schwimer provided legal support; and Jessica Orr assisted in the message  and report development.