As a federal-state partnership, the framework for the UI program is established by federal law, which sets broad requirements that state programs must follow, including categories of workers that must be covered by the program. While states design and administer their own programs, DOL’s Employment and Training Administration (ETA) is responsible for ensuring that state UI laws conform to, and state program operations comply with, federal law. For example, as required by the Social Security Act, ETA is to ensure state laws include provisions that allow for full payment of UI benefits when they are due. ETA also sets overall program policy, monitors state performance, and provides states with technical assistance. For example, ETA sets acceptable levels of performance and monitors states on measures related to benefits, program integrity, appeals, tax, and reemployment (see appendix II). ETA oversees state UI programs through its Office of Unemployment Insurance (OUI) and its six regional offices. The regional offices are the states’ main points of contact with DOL and serve as a link between the department and the states for providing technical assistance and clarifying program policies, objectives, and priorities. While federal law sets forth broad provisions for the categories of workers that must be covered by the program, some benefit provisions, and certain administrative requirements, the specifics of regular UI benefits are determined by each state and the District of Columbia. This results in essentially 51 different programs. States administer their own programs and have considerable flexibility to set benefit amounts and their duration, and establish eligibility requirements and other program details. States are also responsible for customer service in their UI programs. For a general overview of the process for filing UI claims, see fig. 1. Typically, eligible unemployed workers can receive UI benefits for up to 26 weeks in most states. In addition to state UI benefits, the Federal- State Extended Benefit Program provides additional weeks of benefits during high and rising unemployment. UI programs are generally funded by federal and state payroll taxes levied on employers. State benefits for unemployed workers are primarily financed by state employer payroll taxes and are placed in a trust fund that the federal government maintains on behalf of states. Ideally, states build reserves in their trust fund accounts through revenue from employer taxes during periods of economic expansion in order to pay UI benefits during economic downturns. Federal taxes paid by employers are used to fund the costs of administering UI programs in all states, among other purposes. As part of the President’s Budget, DOL uses a combination of national claims-related workload projections and other factors to develop the request for UI administrative funding for states. After Congress appropriates funds, DOL uses a formula to allocate the funding to states and considers state workloads estimates, as well as other information provided by states, such as cost accounting information. Since funding is calculated in part based on claims-related workloads, the federal funding available to states is generally sensitive to changes in total claims, with more funding available when claims increase and less when they decrease. States may also receive additional federal administrative funding above base levels on a quarterly basis when claims-related workloads exceed base funding levels. ETA may also award supplemental funds to states for special UI projects, such as supporting state efforts to reduce improper payments. In addition, states may provide additional state funding for the administration of their UI programs. State UI programs rely extensively on Information Technology (IT) systems to carry out their UI program functions. Specifically, IT systems are used to administer the programs and to support related administrative needs. For example, these systems are used to support benefit eligibility determinations, record claimant filing information, and calculate benefit amounts. The majority of the states’ existing systems for UI operations were, however, developed in the 1970s and 1980s, and we previously reported that states face challenges modernizing these legacy systems to operate more efficiently. These challenges included limited staff with the technical and project management expertise to manage system modernization efforts, limited funding for modernization efforts, and limited resources for operating legacy systems while implementing modernized systems. In recognition of the importance of service delivery in effective governance, in April 2011 the White House issued Executive Order 13571—Streamlining Service Delivery and Improving Customer Service— which required federal agencies to, among other things, work with the Office of Management and Budget (OMB) to develop a customer service plan for federally-administered programs, establish mechanisms to solicit customer feedback, and improve customer service by adopting best practices. Agencies’ plans were required to address approximately three to five of their highest volume services. In response, DOL developed a customer service plan which includes workers in federal contracting, worker safety and health, and worker rights; however, it does not cover the UI program because it is administered by states. According to DOL officials, while agencies were not required to update their plans and DOL has not done so, a committee report accompanying the Consolidated Appropriations Act, 2016, directed OMB to report in March 2016 on agencies’ progress in developing customer service standards and incorporating them into their performance plans. In a 2010 report on federal agency customer service standards, we identified several examples of quality customer service, including understanding customers’ needs and organizing services around those needs, offering self-service options, and providing citizens with the information necessary to hold government accountable for customer service performance. In 2014, we reviewed customer service standards for federally-administered programs at five federal agencies and found that none of the agencies’ standards included all the key elements of customer service standards, such as performance targets or goals and performance measures. Claimant Quote “It’s pretty much been the same, the same problems as in 2009 when I first applied, specifically, the wait times, trying to get a hold of someone taking so long. I usually am on the phone for 20, 30 minutes, get frustrated, hang up, call back another day.” Claimant Quote “So then I call them on Monday, but I can’t get through until Wednesday or Thursday or Friday or sometimes the whole week.” sometimes over days or weeks—before reaching a representative, or said it took several days for a representative to call them back or respond to their email messages. For example, claimants told us that when they missed a call from a representative, it was difficult to reach them to return the call. One claimant reported making 15 attempts to leave a message because a representative’s voice mailbox was full. Claimant Quote “So you’ve already got three prompts before you even get to one that will allow you to ask questions. And then that prompt has two or three prompts. So you probably go through six before you get to a line that says we have a heavy volume and we can’t service you now and click.” In all six focus groups, claimants told us they had difficulty using automated phone systems, and in five groups, claimants said they were unable to reach program representatives by phone. Claimants said they had difficulty using automated phone systems because phone menus were long and complex, involving numerous steps, or because it was easy to make mistakes and difficult to fix them. For example, claimants told us that if they selected the wrong option from the menu, they were forced to start over. One claimant said it took 40 minutes to file a claim after accidentally pressing the wrong button. In other examples, claimants told us they had to hang up—or the system ended the call—after they reached a “dead end” and were unable to return to the previous menu. In addition, in five groups, claimants told us they were unable to reach program representatives by phone. For example, claimants said that when they called to file a claim, they received a message indicating that the phone system was at full capacity and could not accept additional calls. Claimant Quote “I haven’t actually had anyone that has been mean. I have spoken to people that were stressed and overworked, but for the most part they were as helpful and kind as they could be under the circumstances that they had to work with. Especially when I was falling apart.” Claimant Quote “I would say maybe 70 percent of the time it’s somebody that is new and I feel like I am educating them.” one claimant said a representative gave helpful advice about the best day of the week to file a claim. However, other claimants said representatives seemed to lack knowledge or training, or that different representatives provided conflicting information. Also, in four of six groups, some claimants said representatives were courteous, while others said they were not. Claimant Quote “I felt like it was easy. I just found the website and put in pretty basic stuff and it pulled up all my employers that I have had and the earnings and everything was already in there- -I didn’t have to enter any of that. They already knew. And then I got the stuff in the mail a few weeks later, pretty quick.” In contrast, in all six focus groups, claimants who filed their claims online said it was generally easy to do so, though they occasionally experienced system disruptions. For example, claimants described the Internet filing process as “quick,” “user-friendly,” and “convenient,” and said they found certain features helpful, such as the ability to view their past and future benefit payments and print out information about their claims. In addition, claimants said they preferred filing online to filing by phone because it was faster, or because they were less likely to make mistakes. Less frequently, claimants in five groups told us that they experienced challenges filing online. For example, claimants said that some state UI program websites only accepted claims during business hours or were temporarily unavailable due to system outages. Claimant Quote “The online works well. The amount of information, if you actually spend the time to look through all of the information, it answers many more questions than they would have time for on the phone.” In all six focus groups, claimants had mixed opinions about the sufficiency of the information provided by their state UI programs. For example, in all six groups, some claimants reported that the state UI program website or written program materials provided answers to their questions—such as next steps in the application process, or the amount of their benefit payment and when they could expect to receive it. However, others reported that the UI program website or written materials did not answer their questions—such as how to calculate their quarterly wages, or what activities qualify as “work.” In five of six groups, claimants reported that these sources did not provide answers to questions about their unique or complex situations, such as how medical leave or severance pay affected their claims, or how to report income from multiple jobs. In addition to the challenges reported by recent UI claimants, advocacy groups told us that claimants with limited English proficiency have experienced challenges accessing translated materials and program representatives who speak their language when filing claims. For example, in one state, advocates told us that program materials are available in a limited number of languages, despite the many languages spoken in the state. In addition, advocates in two states said that program representatives speak a limited variety of languages, which can make it difficult for claimants to provide additional information after they file claims. Advocates noted that these challenges can be particularly difficult for claimants who speak languages that are less common in their states. For example, in one state, advocates said that an Arabic-speaking claimant did not receive benefits for over 3 months because of delays he experienced in receiving translated application materials and receiving an interpreter for his appeal hearing. In addition, advocacy groups in two states told us that the quality of translated materials—as well as the quality of interpretation—is sometimes poor, which can contribute to benefit delays or result in erroneous determinations that claimants are not eligible for benefits. For example, advocates in one state said that program representatives have sometimes incorrectly translated the reason for a claimant’s separation from employment, which has resulted in claimants being deemed ineligible for benefits. Furthermore, advocates in two states noted that when claimants with limited English proficiency appeal these eligibility determinations, the interpretation provided at the hearings is sometimes of poor quality, and that it is sometimes difficult to even find an interpreter. In addition to language challenges, advocacy groups in two states told us that special populations of claimants, such as individuals with limited English proficiency and individuals with disabilities, also face challenges filing claims over the Internet or by phone because it can be difficult for them to communicate without face-to-face contact. Given these challenges, advocates said it is important for state UI programs to ensure that these special populations and individuals without access to computers or phones are able to file claims and ask questions in person. In our survey, most states reported that they have taken various steps to make their UI programs accessible to special populations, for example, by offering assistance in languages other than English and by making their websites accessible for individuals with disabilities. Most states collect some data on the extent to which UI claimants experience customer service challenges when filing claims by phone and over the Internet. In our survey of state UI programs, 39 of the 43 states that allow unemployed workers to file for UI benefits by phone (91 percent) reported that they collect some related customer service information, such as whether claimants are able to reach program representatives by phone in order to file claims or ask questions. For example, 38 states (88 percent)—including the three we visited—reported collecting data on the total number of calls answered by program representatives and average call wait times. In addition, 26 states (60 percent)—including two of the states we visited—reported collecting data on the number or percentage of calls dropped by automated phone systems (see table 1). As noted above, long call wait times, difficulties reaching program representatives by phone, and difficulties using automated phone systems were cited as challenges by claimants in most of our focus groups. Furthermore, of the 48 states that allow unemployed workers to file claims over the Internet, 39 (81 percent) collect customer service data for these claims. Over two-fifths of states (21 states, or 44 percent) reported monitoring how often claimants are unable to complete filings due to system disruptions, such as website timeouts or crashes, and over half (28 states, or 58 percent) conduct usability testing for their UI program websites. While some states reported using this data to make changes to customer service, officials in two states we visited said that funding constraints may make it challenging to implement these changes. In our survey, 28 of the 43 states that allow claims to be filed by phone (65 percent) reported that they have used data on these claims to make changes to customer service for UI claimants. For example, officials in one state we visited said that they use the data to adjust the number of program representatives available to answer calls and to forecast future claim volumes. In addition, 32 of the 48 states that responded to our survey (67 percent) reported collecting feedback directly from claimants. The two most commonly reported methods for soliciting feedback, according to the states we surveyed, was through surveys and the UI program website. For example, program officials in one state we visited told us they conduct annual and ad hoc surveys of claimants to assess their experiences in filing claims. Officials we met with in another state told us that while they do not collect systematic feedback from claimants, they do solicit informal feedback via social media and their website. States responding to our survey reported facing various customer service challenges—such as staffing, outdated IT systems, funding constraints — which may help explain some of the challenges reported by claimants in our focus groups. While these issues were reported by many states, they were generally reported by fewer states during the last 12 months than during the recent recession. Although 43 states reported that claims can be filed by phone, 4 of these states did not respond to questions about phone-related challenges in our survey. 48) cited staff turnover as a challenge to at least a moderate extent during the last 12 months—compared to the 27 states (56 percent) that reported facing similar challenges during the recession. Staff retention may reflect the complexity of the job, which was noted by UI officials in two states we visited. Also, 12 of 48 states (25 percent) reported staff training and expertise as a challenge to at least a moderate extent in the last 12 months, as compared to 28 states (58 percent) during the recession. While fewer states reported that staff training was a challenge recently than during the recession, it remains challenging due to the complexity of the job and the length of time required to adequately train new hires, according to officials in two states. Outdated IT systems are another challenge for many states, according to our survey. Specifically, 29 of 48 states (60 percent) reported that their current IT systems have significant limitations. These limitations can include legacy mainframe technology from the 1970s and 1980s or outdated programming languages that have implications for state programs’ ability to efficiently process claims and serve claimants. Moreover, 26 states (54 percent) reported that their IT systems were a challenge to a large or moderate extent during the last 12 months, compared to the 34 states (71 percent) that reported these systems were a challenge to at least a moderate extent during the recession (see fig. 3). In the aftermath of the recession, according to state UI program officials, frequent reprogramming of their IT systems was required, in part, to reflect changes in a temporary federal UI benefit program. Additionally, officials in one state we visited explained that their outdated system continues to present challenges because UI program staff must check multiple systems for information on claims, which can lead to errors in processing claims. Additionally, while several states that responded to our survey reported that they were planning to modernize their IT systems, officials we spoke to in all three of our site visit states explained that they faced challenges in fully modernizing their systems. In our survey, 19 of 48 states (40 percent) reported that they have implemented a modernized IT system. Of the remaining 29 that had not yet done so, 16 indicated that their state plans to implement a modernized IT system by 2019. However, officials in three states we visited identified challenges in implementing fully modernized systems—primarily, federal administrative funding constraints. In these states, in the absence of full modernization, officials told us they responded by making modifications to selected capabilities within their existing systems or that they were considering adopting practices from other states that have upgraded their existing systems. A larger number of states that described their current IT systems as having significant limitations reported other challenges than did states with modernized systems. For example, a larger number of states reporting IT system limitations cited call handling times as a challenge to at least a moderate extent during the recession than did states with modernized systems. Officials in one state without a modernized system told us that because claimants cannot check status updates and other information online, they must phone the call center, which consumes significant staff resources for routine requests that a modernized system could handle easily. Federal Legislative Changes to Temporary Benefits Program During the recession, federal legislative changes included relatively frequent changes to a temporary federal benefits program. In GAO’s survey, 39 states (81 percent) cited federal legislative changes as challenges to a large or moderate extent during the recession. Comments from State Unemployment Insurance Program Officials Related to Federal Legislative Changes: State A: “…the excessive programming changes…caused program issues in often unrelated areas of the system which were difficult to diagnose (result of a very old and fragile system). This often exacerbated the difficulty in meeting service demand and also created additional frustration (for) both staff and the public.” State B: “(the temporary federal UI benefit program) was constructed… and administered…in a manner certain to create challenges for applicants and therefore for state programs. We never really had problems with the volume of actual applicants, but rather with the unnecessary volume of phone calls and administrative transactions…” State C: “(the temporary federal UI benefit program) was difficult to administer due to the large number of law changes and their retroactive effective dates.” Federal funding constraints was one area cited as a challenge to a large or moderate extent by more states during the last 12 months (28 of 48 states, or 58 percent) than during the recession (24 states, or 50 percent), as shown in fig. 4. All states rely on federal funding for the administration of their UI programs, and the total federal funding available to states has declined to pre-recession levels. Officials in one state we visited explained that after the recession ended, claim volumes decreased and, consistent with the funding model DOL uses to allocate funding to the states, federal administrative funding provided to the state was reduced. However, according to officials in this state, other aspects of UI program administration have remained constant or increased, such as efforts to identify and collect benefit overpayments, which also rely on federal funding. Moreover, in our survey, half of states said their UI programs receive no additional state administrative funds (24 states, or 50 percent), and nearly one quarter reported receiving less than 5 percent of their total administrative funds from their states (11 states, or 23 percent). While officials in all 3 states we visited cited limited federal administrative funding as a challenge, only one of these states requested and received additional state administrative funds. Representatives from an advocacy group we spoke with in that state noted that the additional funds, which the state used to hire more call center staff, resulted in a significant reduction in blocked calls. Some states reported experiencing challenges related to claims filed by phone to a large or moderate extent, both during the recession and during the last 12 months (see fig. 5). According to a DOL regional office official and officials in one state we visited, challenges related to claims filed by phone are likely due to staffing issues. For example, if claimants are unable to reach program representatives by phone within a reasonable amount of time, it may indicate that the UI program has an insufficient number of call center staff. Also, 18 of the 39 states (46 percent) who responded in our survey about claims filed by phone reported that call handling times remain a challenge to at least a moderate extent. Officials in one state we visited explained that as the state has made it easier for claimants to manage their own claims, calls to the state UI program, including claimant calls directed from American Job Centers, are becoming more complex and may take longer to address. Officials in that state told us that call handling times remain challenging because program representatives have to balance the competing goals of completing calls quickly and meeting customer needs. In addition, 13 of 39 states (33 percent) reported continuing challenges related to calls abandoned by callers, and 8 of 39 states (21 percent) reported continuing challenges with calls dropped by the automated phone system, to at least a moderate extent. In addition, some states described challenges related to claims processing delays or backlogs. For example, 16 of 48 states (33 percent) cited delays or backlogs in claims processing during the last 12 months as a challenge to a large or moderate extent, while these issues were challenges for 34 states (71 percent) during the recession. Although claim volumes have generally declined, since federal administrative funding is tied, in part, to the volume of claims, some states may face challenges in this area. In one state we visited, for example, officials said they feel understaffed despite having a lower workload, particularly when there are unexpected spikes in claim volumes due to seasonal unemployment fluctuations or changes in economic conditions, for example. Many states reported that they took or are planning to take some actions to improve customer service, such as providing self-service options on state UI program websites and implementing automated telephone systems (see fig. 6). For example, almost all of the 48 states responding to our survey reported having provided self-service options on the state UI program website (94 percent) or implementing an automated telephone system (88 percent), both of which are designed to allow claimants to file their own claims with little or no staff assistance. Other commonly-reported practices included posting “Frequently Asked Questions” on state UI program websites (94 percent) and providing customer service training to call center staff (77 percent). Other practices, though potentially helpful, have not been implemented as frequently. For example, 22 states reported having implemented virtual hold or courtesy call back on their phone lines, which allows claimants to opt out of holding and instead receive a call back, and an additional 9 states said they planned to do so. Several states have also undertaken efforts to streamline program processes to improve efficiency. For example, officials in California cross- trained program staff so that all staff could answer incoming calls, and reduced call center hours from a full business day to a half day. Prior to this change, some call center staff answered calls while others worked on eligibility or payment issues throughout the day. As more staff are cross- trained in all functions, their competencies to answer all types of calls are enhanced, according to officials. Officials told us that these changes allowed them to answer more calls than with their previous staffing model in which staff specialized in different tasks. Officials described the changes as helping to create additional capacity in an environment of constrained resources. Additionally, in our survey, several states described efforts to streamline or modify their processes to improve services. DOL’s ETA monitors states on a range of UI performance measures, some of which directly address certain aspects of customer service. Under the federal-state partnership, ETA sets overall program policy and monitors state performance, and states provide customer service to claimants. ETA sets acceptable levels of performance and monitors states on a total of 15 core measures within the following categories: 1) benefits, 2) program integrity, 3) appeals, 4) tax, and 5) reemployment. (For a comprehensive list of core performance measures, see appendix II.) According to ETA officials, while most of these measures indirectly address customer service issues, four of them directly address certain aspects of customer service. Specifically, ETA measures the timeliness of first payments to eligible claimants and of appeals decisions and assesses the accuracy of two types of non-monetary eligibility determinations. Officials told us they consider these measures to be important indicators of customer service (see table 2). ETA provides states with technical assistance for the UI program overall, and provides technical assistance on customer service under certain circumstances. According to ETA officials, ETA’s six regional offices monitor state performance on a quarterly basis and provide technical assistance—which may address customer service—if states fail to meet acceptable levels of performance over a prolonged period of time. For example, if a state fails to pay benefits to claimants in a timely manner, ETA officials said they may review the state’s call center operations and provide related assistance. Officials told us they have also provided technical assistance when states have faced major customer service challenges that are not addressed in the performance measures, such as significant delays in answering calls. In addition, ETA provides ongoing technical assistance to states on program administration, which has occasionally addressed customer service issues. For example, officials said they have developed training, hosted webinars, provided supplemental funding, and maintained an online community of practice through which states can share information. According to officials, ETA is currently piloting an effort to ensure states are routinely assessing program operations and processes, complying with applicable laws, and effectively administering their UI programs. As part of this effort, officials expect that beginning in fiscal year 2017, states will conduct annual self- assessments of their benefit processes—including reviews of their call center operations and supporting IT infrastructure—and ETA will provide related assistance, such as sharing best practices for areas in which states are experiencing challenges. ETA has also provided states with technical assistance or guidance on IT modernization, staffing issues, and program access, which were reported as customer service challenges by states and advocacy groups. ETA provides states with technical assistance on IT modernization by funding and overseeing the Information Technology Support Center (ITSC), which is operated by the National Association of State Workforce Agencies (NASWA). For example, in March 2015, ETA and ITSC issued a checklist to help states prepare to launch modernized UI IT systems. ITSC has also developed a comprehensive guide for UI IT modernization projects, among other resources. In addition, ETA has provided states with supplemental funding to establish consortia in which states work together to develop and share a common, modernized IT system. With respect to staffing issues, ETA officials said they have provided technical assistance under certain circumstances, such as when states have been unable to adequately staff their call centers due to high volumes of claims during the recent recession or the implementation of modernized IT systems. Regarding program access, in October 2015, ETA issued guidance directing states to ensure that all individuals—including individuals with limited English proficiency, individuals with disabilities, and older individuals—can effectively access the UI program. Officials in all three states we visited told us that they have consulted with other states to learn from their experiences administering the UI program. For example, officials said they have consulted with other states about process improvements and online chat capabilities, among other issues. In two states, officials told us that they have contacted other states directly because they have sometimes been unable to obtain timely responses to questions posted on ETA’s online community of practice. In one of these states, officials explained that it can be challenging for staff to monitor the online community of practice on a regular basis and respond to other states’ requests for information in a timely manner. In a third state, officials said that they have contacted other states directly because officials are more willing to openly share challenges and lessons learned in private discussions. In addition, officials told us that NASWA has helped them gather information about other states’ UI programs. For example, officials in one state said NASWA helped them survey all states about their claims workloads and staffing, among other issues, which helped them determine that they have more claims per staff member than other states. While states are sharing some information, officials in all three states said their UI programs could benefit from more information about other states’ successful customer service practices. For example, officials in all three states said it would be helpful to have more information—beyond ETA’s recent guidance—on successful practices for serving special populations, such as limited English proficiency claimants and those with disabilities. In particular, officials in all three states said that their programs could benefit from more information about other states’ successful practices for addressing ongoing challenges, such as insufficient staffing and IT limitations for states that have been unable to implement fully modernized systems. ETA has taken some steps to help states share successful customer service practices, and plans to continue to help states do so—including those that the states we visited said would be helpful. According to ETA officials, one of ITSC’s core activities is to collect lessons learned from state IT modernization efforts and disseminate them to states, including successful practices for partial modernization when states are unable to fully modernize their systems. In addition, ETA regional offices have helped states share successful customer service practices through ongoing technical assistance, including periodic conference calls and conferences, and by connecting states facing challenges with other states that have successfully addressed similar challenges. Furthermore, ETA is currently in the process of conducting a national study of UI call center operations which officials expect will identify best practices on a range of issues, including staffing. Officials also expect that ETA’s efforts to comprehensively assess state UI program operations and processes will identify best practices in a range of areas related to customer service, such as call center operations, IT infrastructure, and program access. When these efforts are completed, ETA plans to share the best practices it identifies with all states it its online community of practice. We provided a draft of this report to the Department of Labor for review and comment. The agency provided technical comments, which we incorporated where appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Labor, appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. Please contact me on (202) 512-7215 or at brownbarnesc@gao.govif you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found of the last page of this report. Key contributors to this report are listed in appendix III. We examined (1) the customer service challenges, if any, recent Unemployment Insurance (UI) claimants have faced, and the extent to which states collect information on claimants’ challenges, (2) any challenges states have faced in providing customer service to UI claimants, and any improvements they have made, and (3) the extent to which Department of Labor (DOL) monitors states’ customer service efforts and provides assistance to help them make improvements. To address our objectives, we reviewed relevant federal laws, regulations, and guidance; conducted a survey of state unemployment insurance (UI) programs; interviewed DOL and state UI programs officials; conducted site visits to 3 states; and held 6 focus groups with recent UI claimants in all 3 site visit states. We also interviewed various stakeholders, including representatives of national associations and advocacy groups that represent UI claimants. We conducted this performance audit from November 2014 to May 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. To address all three objectives, we conducted a web-based survey of UI programs in all 50 states and the District of Columbia. The survey included questions about available methods for filing UI claims in each state, the status of information technology (IT) systems modernization in each state, the data states collect related to customer service, the challenges that states have experienced with respect to UI claims during the recent recession and during the previous 12 months, the practices that states have implemented to improve customer service, and the views of state UI officials about the assistance that DOL has provided related to customer service. We received responses from 48 states, for a response rate of 94 percent. To minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same, we conducted pretests with five state UI programs. To ensure that we obtained a variety of perspectives on our survey, we selected states with diversity on the following criteria: 1) Employment and Training Administration (ETA) region, 2) average unemployment rate, 3) UI claims volume, and 4) the status of IT modernization. In addition, we considered whether the state had been identified as employing certain practices related to customer service, such as conducting surveys of claimants. Based on feedback from these pretests, we revised the questionnaire in order to improve the clarity of the questions. An independent survey specialist within GAO also reviewed a draft of the questionnaire prior to its administration. After completing the pretests, we administered the survey. On June 25, 2015, we sent an e-mail announcement of the questionnaire to state UI directors, notifying them that our online questionnaire was available for them to complete, and provided them with unique passwords and user names. To encourage state UI programs to respond, we followed up with non-respondents by phone and email through October 6, 2015. We also followed up by email with state UI directors to clarify their survey responses. We collected responses through February 2016. We used standard descriptive statistics to analyze responses to the questionnaire. Because we surveyed all states, the survey did not involve sampling errors. To minimize non-sampling errors, and to enhance data quality, we employed recognized survey design practices in the development of the questionnaire and in the collection, processing, and analysis of the survey data. For instance, as previously mentioned, we pretested the questionnaire with state UI programs to minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same. We further reviewed the survey to ensure the ordering of survey sections was appropriate and that the questions within each section were clearly stated and easy to comprehend. To reduce nonresponse, another source of non-sampling error, we followed up by email with states that had not responded to the survey to encourage them to complete it. In reviewing the survey data, we performed automated checks to identify inappropriate answers. We further reviewed the data for missing or ambiguous responses and followed up with states when necessary to clarify their responses. On the basis of our application of recognized survey design practices and follow-up procedures, we determined that the data were of sufficient quality for our purposes. Concurrently with our survey, we conducted site visits to three states: California, New York, and Texas. We selected these states because they had the largest number of new UI claims in their respective regions in calendar year 2014, which was the last year for which data were available. We also selected these states because they are located in geographically diverse ETA regions. In each state, we interviewed state UI program officials, as well as officials at two ETA regional offices and organizations that advocate for UI claimants. The results from our interviews with state UI programs and ETA regional offices are not generalizable. In all three states, we conducted focus groups with recent UI claimants (see additional discussion below). We also identified and interviewed representatives of advocacy groups that represent UI claimants in our site visit locations. In part, we selected these groups to provide perspectives on the challenges faced by claimants with limited English proficiency and claimants with disabilities, who were not represented in our focus groups. The opinions expressed by these groups represent their points of view, and may not represent the views of all advocacy groups or the customer service experiences of all claimants with limited English proficiency and claimants with disabilities. In our interviews with state UI program officials, we asked about the data collected by the program, such as call handling times, and the ways the data are used; the extent to which and how the state UI program collected feedback from claimants; customer service challenges experienced by the state UI program; and practices to improve customer service. In our interviews with ETA regional offices, we asked about federal monitoring and technical assistance efforts and the extent to which they relate to customer service. In our interviews with advocacy organizations, we asked about the customer service challenges faced by state UI programs and by claimants, communications with claimants, access for special populations of claimants, ways in which the state UI program is currently working well, and state practices to improve customer service. To learn about recent UI claimants’ challenges related to customer service, we conducted 6 focus group sessions with a total of 58 claimants at 3 locations, using a contractor to recruit and screen participants and record and transcribe the sessions. In order to recruit focus group participants, we provided participant selection criteria to the contractor. Specifically, we stipulated that potential participants be 21 years of age or older, speak English, have personally applied for UI benefits within the 12 months preceding the time the contract was awarded, or from July 2014 to July 2015, and be able to provide written verification that they applied for UI benefits in their states. The contractor then contacted and screened potential participants from its database, and over-recruited a total of 15- 20 individuals for each session as necessary to ensure that 8-10 eligible individuals participated. We conducted these focus groups in September 2015. These sessions involved structured small-group discussions designed to gain more in-depth information about specific issues that could not easily be obtained from another method, such as a survey or individual interviews. Consistent with typical focus group methodologies, our design included multiple groups with varying characteristics but some similarity on one or two homogeneous characteristics. In all focus groups, the participants had filed UI claims within the last 12 months in the state where the group was held. Most participants said they had filed their claims online or by phone, although other filing methods were represented. Our overall objective in using a focus group approach was to obtain views, insights, and feelings of UI claimants who had filed claims within the last 12 months. Specifically, we wanted to learn about challenges they faced in filing claims, including their experiences with state UI program websites and phone lines, as well as their views about the courtesy and responsiveness of UI program staff, their thoughts about ways in which the state UI program is currently working well, their views about the timeliness of agency actions, their experience with information provided by the state UI program and opportunities to provide feedback, and their recommendations for improvement. By including UI claimants who had filed using different methods, and claimants who varied according to age, gender, ethnicity, and self-reported education level and income, we intended to gather a range of perspectives regarding state UI programs’ customer service efforts. All of the participants selected for the focus groups were fluent English speakers. We selected three cities as focus group locations. We selected these locations because they corresponded to the states we selected for site visits. We conducted two sessions in each of the three cities—Albany, New York; Austin, Texas; and Sacramento, California. Discussions were structured, guided by a moderator who used a standardized list of questions to encourage participants to share their thoughts and experiences. We conducted one pretest focus group session in Rockville, Maryland prior to beginning our travel for the sessions. Each of the 6 focus groups was recorded and transcripts were created, which served as the record for each group. Those transcripts were then evaluated using content analysis to develop our findings. The analysis was conducted in two steps. In the first step, three analysts jointly developed a set of codes to track the incidence of various responses and themes during focus group sessions. In the second step, each transcript was coded by an analyst and then those codes were verified by two other analysts. Any coding discrepancies were resolved by all three analysts agreeing on what the codes should be. Methodologically, focus groups are not designed to (1) demonstrate the extent of a problem or to generalize results to a larger population, (2) develop a consensus to arrive at an agreed-upon plan or make decisions about what actions to take, or (3) provide statistically representative samples or reliable quantitative estimates. Instead, they are intended to generate in-depth information about the reasons for the focus group participants’ attitudes on specific topics and to offer insights into their concerns about and support for an issue. The projectability of the information produced by our focus groups is limited for several reasons. First, the information includes only the responses from recent UI claimants from the six selected groups. Second, while the composition of the groups was designed to ensure a range of age, gender, and ethnicity, the groups were not randomly sampled. Third, participants were asked questions about their experiences or expectations, and other UI claimants not in the focus groups may have had other experiences or expectations. Because of these limitations, we did not rely entirely on focus groups, but rather used several different methods to corroborate and support our conclusions. Appendix II: Unemployment Insurance (UI) Performance Measures and Acceptable Levels of Performance Performance Measure Benefits Measures First Payment Promptness: % of all 1st payments made within 14/21 days after the week ending date of the first compensable week in the benefit year (excludes Workshare, episodic claims such as Disaster Unemployment Assistance, and retroactive payments for a compensable waiting period). Nonmonetary Determination Time Lapse: % of Nonmonetary Determinations (Separations and Nonseparations) made within 21 days of the date of detection of any nonmonetary issue that had the potential to affect the claimant’s benefit rights. Nonmonetary Determination Quality- Nonseparations: % of Nonseparation Determinations with Quality Scores equal to or greater than 95 points, based on the evaluation results of quarterly samples selected from the universe of nonseparation determinations. Nonmonetary Determination Quality- Separations: % of Separation Determinations with Quality Scores equal to or greater than 95 points, based on the evaluation results of quarterly samples selected from the universe of separation determinations. Program Integrity Measures Detection of Overpayments: % of detectable, recoverable overpayments estimated by the Benefit Accuracy Measurement survey that were established for recovery. Benefit Year Earnings (BYE) Measure: % of the amount overpaid due to BYE issues divided by the total amount of UI benefits paid. Improper Payments Measure: % of UI benefits overpaid plus UI benefits underpaid minus overpayments recovered divided by the total amount of UI benefits paid. UI Overpayment Recovery Measure: % of Amount of overpayments recovered divided by (Amount of overpayments established minus overpayments waived) (example Improper Payments Information Act (IPIA) Reporting Year 2013 = July 1, 2012 – June 30, 2013) Appeals Measures Average Age of Pending Lower Authority Appeals: The sum of the ages, in days from filing, of all pending Lower Authority Appeals divided by the number of Lower Authority Appeals. Average Age of Pending Higher Authority Appeals: The sum of the ages, in days from filing, of all pending Higher Authority Appeals divided by the number of Higher Authority Appeals. Lower Authority Appeals Quality: % of Lower Authority Appeals with Quality Scores equal to or greater than 85% of potential points, based on the evaluation results of quarterly samples selected from the universe of lower authority benefit appeal hearings. Tax Measures New Employer Status Determinations Time Lapse: % of New Employer Status Determinations made within 90 days of the last day in the quarter in which the business became liable. Performance Measure Tax Quality: Tax Performance System (TPS) assessment of the accuracy and completeness of the tax program determined by scoring, on a pass/fail basis, samples of the 13 tax functions. Effective Audit Measure: Evaluates whether a state’s employer audit program meets or exceeds minimum levels of achievement in the following four factors: Factor 1 - % of Contributory Employers Audited Annually, Factor 2 - % of Total Wages Changed from Audits, Factor 3 - % of Total Wages Audited, Factor 4 - Average Number of Misclassifications Detected per Audit, and meets or exceeds a minimum overall score of the four factors. Reemployment Measure Facilitate Reemployment: % of UI claimants who are reemployed within the quarter following the quarter in which they received their first UI payment. Cindy S. Brown Barnes, (202) 512-7215, brownbarnesc@gao.gov. In addition to the contact named above, Mary Crenshaw, Assistant Director; Divya Bali, Analyst-in-Charge; Daniel Berg, Caitlin Croake, Christopher Morehouse, and Betty Ward-Zuckerman made significant contributions to this report. Also contributing to this report were Jessica Botsford, David Chrisinger, Carol Henn, Jill Lacey, Kathy Leslie, Mimi Nguyen, Lisa Pearson, Carl Ramirez, Jerome Sandau, Almeta Spencer, Valerie Melvin, Margaret Weber, Charles Willson, and Charles Youman. Unemployment Insurance: States’ Reductions in Maximum Benefit Durations Have Implications for Federal Costs. GAO-15-281. Washington, D.C.: April 22, 2015. Managing for Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service. GAO-15-84. Washington, D.C.: October 24, 2014. Information Technology: Department of Labor Could Further Facilitate Modernization of States’ Unemployment Insurance Systems. GAO-12-957. Washington, D.C.: September 26, 2012. Unemployment Insurance: Economic Circumstances of Individuals Who Exhausted Benefits. GAO-12-408. Washington, D.C.: February 17, 2012. Managing for Results: Opportunities to Strengthen Agencies’ Customer Service Efforts. GAO-11-44. Washington, D.C.: October 27, 2010.

Summary: UI benefits are a critical source of income for millions of unemployed Americans. Overseen federally by DOL and administered by states, the UI program requested $32.5 billion in benefits in fiscal year 2015 for approximately 7 million UI claims. During the 2007-2009 recession, states faced challenges processing record numbers of claims, and questions were raised about the quality of customer service. GAO was asked to review customer service issues in state UI programs. GAO examined (1) customer service challenges, if any, recent UI claimants have faced and the extent to which states collect information on claimants' challenges, (2) any challenges states have faced providing customer service to claimants, and any improvements they have made, and (3) the extent to which DOL monitors states' customer service efforts and provides assistance to help them make improvements. GAO surveyed state UI programs in 50 states and the District of Columbia (with 48 responding); interviewed officials from state UI programs and advocacy groups in California, New York, and Texas (selected for geographical diversity and large numbers of UI claims); and conducted six focus groups with recent UI claimants in these three states. Focus group results are not generalizable, but provide important insights into the experiences of some UI claimants. GAO also reviewed relevant federal laws and DOL guidance. In GAO focus groups of recent claimants filing for unemployment insurance (UI) benefits, those who filed by phone reported experiencing various challenges, such as long call wait times; however, those filing for benefits online reported that it was generally easy to do. In all six of the focus groups GAO conducted in the three states it visited, individuals who had claimed benefits by phone between July 2014 and July 2015 reported experiencing challenges with the state UI program's customer service. GAO defined customer service as including ease of program access, courtesy, timeliness, and accuracy, as well as responsiveness to customer needs and expectations. Some participants in all six focus groups reported experiencing long call wait times and difficulties using automated phone systems. In addition, in two of the states visited, representatives of advocacy groups reported that special populations, including individuals with limited English proficiency, had difficulty accessing the UI program. In GAO's survey of state UI programs, most states—including those GAO visited— reported collecting some data on customer service challenges for claimants. For example, 38 states reported collecting data on average call wait times. Many states reported challenges in providing customer service, including staffing issues, and most have taken some steps to improve customer service, such as increasing self-service options. Specifically, more than half the states GAO surveyed reported insufficient staffing, outdated Information Technology (IT) systems, and funding constraints, all of which could play a role in claimant challenges. Many states also reported that they have taken or are planning to take some actions to improve customer service. For example, 45 states reported taking action to provide self-service options, and 42 states reported taking action to provide customer service training to program representatives. The Department of Labor's (DOL) Employment and Training Administration (ETA) provides states with monitoring and assistance on some aspects of customer service. ETA monitors and measures state performance on the timeliness of benefit payments and appeals decisions. ETA also monitors and measures the accuracy of states' non-monetary eligibility determinations, such as whether states accurately assess reasons for claimants' separation from employment. In addition, ETA provides states with various technical assistance. ETA has provided states with assistance on IT modernization, staffing issues, and program access for special populations. UI program officials in the three states GAO visited said they could benefit from more information on other states' successful customer service practices, including practices for addressing continuing staffing and IT challenges. ETA plans to share these practices on an ongoing basis, officials said. For example, officials said ETA plans to share successful practices—including those related to staffing—obtained from its national study of call center operations through its online community of practice. ETA also plans to continue to collect lessons learned from state IT system modernization efforts and disseminate them to states. GAO is not making recommendations in this report.