Veteran-Owned Small Businesses: Planning and Data System for VA's Verification Program Need Improvement
Released: 2013-01-14T00:00:00

Letter

Background

The 2006 Act requires VA to establish annual contracting goals for SDVOSBs and other nonservice-disabled VOSBs; the goal for SDVOSBs must at least match the government-wide SDVOSB contracting goal of 3 percent of federal contract dollars. VA set its goals at 3 percent for SDVOSBs and 7 percent for all VOSBs (SDVOSBs and other VOSBs) for fiscal years 2006 and 2007, and subsequently has set and achieved increasing contracting goals for VOSBs and SDVOSBs, as shown in figure 1. VA’s total contracting awards to VOSBs increased from $616 million (including $356 million to SDVOSBs) in fiscal year 2006 to $3.6 billion (including $3.2 billion to SDVOSBs) in fiscal year 2011.

VA’s OSDBU has overall responsibility for the SDVOSB/VOSB verification program. Within OSDBU, the Center for Veterans Enterprise (CVE), maintains a database of eligible SDVOSBs and VOSBs and is responsible for verification operations, such as processing applications. To implement the requirements of the 2006 Act, VA began verifying businesses in May 2008 under interim final rules, which the agency did not finalize until February 2010.the verification program, see fig. 2.) To be eligible for verification under VA’s rules (For a timeline of major events affecting the small business concern must be unconditionally owned and controlled by one or more eligible parties (veterans, service-disabled veterans, or surviving spouses); the owners of the small business must have good character (any small business owner or concern that has been debarred or suspended is ineligible); the applicant cannot knowingly make false statements in the application process; the firm and its eligible owners must not have significant financial obligations owed to the federal government; and the firm must not have been found ineligible due to an SBA protest decision.

VA launched its verification process under the 2006 Act in 2008 and shifted to a more robust process in 2010. VA’s verification process under the 2006 Act (2006 process) initially consisted of (1) checking VA databases to confirm veteran status and, if applicable, service-disability status; and (2) reviewing publicly available, primarily self-reported information about control and ownership for all businesses that applied for verification. Beginning in September 2008, VA also adopted a risk-based approach to conducting site visits or other means, such as additional document reviews and telephone interviews, to further investigate selected high-risk businesses. VA adopted a more thorough verification process in 2010 (2010 process), which included reviewing and analyzing a standardized set of documents that each applicant is required to submit. VA refined the 2010 process over time so that, as of October 2012, the verification process consisted of four phases—initiation, examination, evaluation, and determination. Denied applicant firms are able to request a reconsideration of the denial decision.

Initiation: CVE employees are to confirm that applicants meet minimum requirements for the program by, among other things, verifying the owners’ veteran and service-disability status and determining that they have submitted all of the required documents or adequate explanations for missing documents.to check the Excluded Parties List System to ensure that the applicant business and all owners are not on the list.

Examination: Contractors are to review completed applications to determine whether firms meet the eligibility requirements and make an initial recommendation for approval, denial, or additional review (i.e., a site visit).

Evaluation: Contractors and staff are to review the initial recommendations to ensure that the screening has met quality standards and that firms have received an appropriate recommendation. They may decide as well that a site visit is necessary. Contractors are to conduct site visits if they are recommended, and CVE employees are to recommend approval or denial.

Determination: CVE supervisors are to review staff recommendations and issue eligibility decisions. A determination letter is to be emailed to the applicant, and approved companies appear as verified in the Vendor Information Pages (VIP) database.

Request for Reconsideration: Through an optional Request for Reconsideration process, denied applicants can remedy the issue(s) that caused their applications to be initially denied. Based on a review by staff from VA’s Office of General Counsel, VA may approve the application, deny it on the same grounds as the original decision, or deny it on other grounds. If VA denies a request for reconsideration solely on issues not raised in the initial denial, the applicant may ask for reconsideration as if it were an initial denial. Denied applicants can also request a legal review if they believe their application was denied in error.

VA’s database of SDVOSBs and VOSBs previously listed unverified and verified firms, but currently is required to list only verified firms, as a result of the Veterans Small Business Verification Act (2010 Act), part of the Veterans’ Benefits Act of 2010.2008, VA modified its VIP database of self-certified SDVOSBs and After the verification program began in VOSBs to receive verification applications and publicly display names of verified firms. Once VA approved a business, the business name appeared with a verified logo in VIP, but the database continued to display self-certified firms as well. The 2010 Act requires that no new applicant appear in the VIP database unless it has been verified by VA as owned and controlled by a veteran or service-disabled veteran. The 2010 Act also required VA, within 60 days of enactment, to notify all unverified (self-certified) firms in VIP about the verification requirement and also required firms to apply for verification within 90 days or be removed from the database. VA officials reported that by September 2011, the agency had removed from the VIP database all firms that had self-certified so that the database would include only verified firms. As of October 2012, the database included both firms verified under the 2006 Act process and the 2010 process.

VA Has Made Changes to Improve Its Verification Program, but Continues to Face Challenges in Its Strategic Planning Efforts and Information Technology Infrastructure

VA has made significant changes to its verification processes in an effort to improve its operations and address program weaknesses, but continues to face challenges in establishing a stable and efficient program to verify firms on a timely and consistent basis. Since December 2011, VA has instituted a number of significant operational changes, including revising standard operating procedures and enhancing quality assurance protocols. However, it has not had a comprehensive, long-term strategic plan for the verification program and has consistently prioritized addressing immediate operational challenges, contributing to programmatic inefficiencies. In response to our observations, VA’s OSDBU initiated action in late October 2012 to compile a strategic planning document that encompasses the verification program. OSDBU appears to have at least partially applied key leading strategic planning practices in its initial planning effort. But the plan lacked performance measures to assess whether the desired outcomes are being achieved and had a shorter-term focus than typically associated with a strategic plan. Furthermore, VA had not shared the plan with key stakeholders, such as veteran support organizations and business associations or congressional staff. In addition, the verification program’s information technology (IT) system has shortcomings that have hindered VA’s ability to operate, oversee, and monitor the program. VA is planning to modify or replace the system, but has not directly tied this effort into its long-term strategic planning efforts to ensure that the new system meets the verification program’s long-term information needs.

VA Has Adopted Changes to Improve Verification Operations and Address Some Identified Weaknesses and Concerns

As of September 30, 2012, the VIP database listed 6,257 firms that had been verified as VOSBs or SDVOSBs. Of these, VA reported that 1,733 were verified under the initial 2006 Act process and 4,524 under the more rigorous 2010 process. VA’s database also listed a substantial number of pending cases at that time: 691 new applications for verification, 131 firms seeking reverification to remain in VIP, and 165 requests for reconsideration from firms that were denied verification. See appendix II for additional data on the verification program as of September 30, 2012.

GAO-12-697. process that VA previously used represented a continuing vulnerability. By September 30, 2012, VA had reverified or removed from the VIP database 622 of the firms that were originally verified under the 2006 Act process, but had yet to reverify the remaining 1,733 firms, according to VA’s inventory of verified firms. The inventory indicated that the 2-year verification period for 1,159 of these remaining firms expired on or before September 30, 2012, so they were not eligible to receive VA contracts after that date and were due to be removed from the database. VA officials said that firms whose 2-year verification period had not yet expired would be removed from the database upon expiration if they had not been verified under the 2010 process. According to VA, fewer than 120 companies that were verified under the 2006 process remained in VIP as of December 1, 2012.

In interviews with us between April 2012 and June 2012, veterans’ organizations cited applicant concerns about other aspects of the verification program, such as the rationale for determinations and the time it took VA to make determinations, as the following examples illustrate.

We observed several outreach sessions that VA conducted in May and June 2012 with veterans’ organizations and an association of organizations that provided technical assistance with procurement. In these sessions and in our follow-up interviews with participants, the organizations stated that VA’s guidance for applicants did not always adequately explain how VA interpreted some of the subjective eligibility standards in its regulations, such as the requirements that owners have good character. They also said that they and applicants sometimes found the rationale for denial to be unclear or inconsistent.

Representatives from two of the veteran service organizations that we interviewed also raised concerns about the length of time it could take to process an application. According to VA officials, it took on average more than 130 days after receiving a complete application to make a determination in July 2011. As of October 2012, it took approximately 85 days.

With the hiring of a new CVE director in December 2011, CVE conducted a review of the verification process to identify ways to increase its efficiency and began adopting changes to the verification process to improve its operations and address program weaknesses and applicants’ concerns. For example, as a result of this review, CVE did the following:

CVE revised its Standard Operating Procedures (SOP) to reflect current practices and help ensure greater consistency in its verification processes. These procedures describe the purpose, scope, statutory references, staff roles, and implementation steps.

CVE instituted a more robust quality assurance process to ensure that staff adhered to the approved procedures. For example, CVE employees and contractors are now subject to both scheduled and spot audits and must resolve any major deficiencies within 10 days.

CVE hired its first training officer and revised its training program with the goals of ensuring that CVE staff were properly trained and qualified to perform their duties, achieved high performance, and were responsive to changing business requirements, among other things. The training officer is responsible for coordinating training for staff (including contractors), including weekly training on the verification program and customer service, as well as monthly fraud education.

CVE added specific methods of communicating with applicant firms, with the goal of ensuring that applicants receive an email from VA at least every 30 days with an update on the status of their application.

CVE began using the initiation phase, rather than the examination phase, to determine whether an application was complete, so that any missing documentation or inadequate explanations can be addressed before the examination process begins.

CVE began tracking staff productivity levels and more closely monitoring the quality of their work. For example, beginning in the spring of 2012, CVE started setting targets for the numbers of cases that individual staff members should review each day. Also, in September and October 2012, CVE’s evaluation team reviewed the results of contractors’ examinations to identify cases that had not been properly completed or in which the recommended finding should be overturned.recommended for additional training.

Contractor staff with the most rejected cases were VA has also revised the organizational structure for the verification program, but VA officials said that human capital challenges remained. In general, CVE is structured so that a federal employee oversees a team of contractors. VA uses a mix of federal employees and contractors to complete verifications because contractors have greater flexibility to adjust staffing levels in response to variations in the number of applications submitted, according to VA officials. Between December 2011 and October 2012, VA adopted changes to the verification program’s organizational structure to make it more efficient, increase oversight of federal staff and contractors, and strengthen functions that did not previously have dedicated staff, such as training. VA officials (1) reorganized and increased the number of employees and contractors assigned to the verification process and (2) created several new teams including quality assurance, training, records management, and customer service. During this period, VA added about 3 full-time equivalent staff and 64 contractors to the verification program.organization charts as of December 2011 and October 2012.) However, VA officials said that the verification program faces ongoing human capital challenges. For example, 5 of the verification program’s 27 full-time federal positions were vacant as of November 2012. As of early November 2012, CVE was developing a business case to justify the staff organization necessary to support verification operations, including revised federal employee labor categories and modifications to contractor support. (See app. III for the verification program’s VA has also sought to improve outreach to applicants through additional online resources and a new Verification Counseling program. In November 2011, VA began posting on its website verification assistance briefs intended to clarify aspects of the program’s rules. These briefs cover topics that VA officials have determined cause the majority of denials, such as full-time control and transfer restrictions. In addition, VA launched a self-assessment tool in June 2012 to help applicants understand the rules, regulations, eligibility criteria, and review process for verification. Recognizing that some applicants needed additional support, VA launched a Verification Counseling program in June 2012. According to VA, this program integrates the Verification Assistance Partner counselors (initially, selected veterans’ support organizations and business associations and Procurement Technical Assistance Centers) into the regular training provided to CVE examination and evaluation staff. These counselors in turn provide counseling to firms interested in becoming verified. The program is intended to increase understanding of the verification program’s eligibility requirements so that ineligible firms would be less likely to apply and eligible firms would be more likely to submit the materials necessary for them to succeed in their initial applications.

To mitigate an anticipated increase in its workflow over time, VA initiated two efforts in early 2012 to modify its approach to reverifying firms’ eligibility. VA’s verification regulations issued in May 2008 limited the term of the verification status of a firm to a 1-year time period. However, as growing numbers of firms verified under the 2010 process began to require reverification in early 2012, VA recognized that it would face a mounting workload over time if it reverified firms annually using its full examination procedures. As a result, VA began to develop procedures for a simplified reverification process, which it introduced in early June 2012. VA also began the process of modifying the verification program regulations to extend the verification period from 1 year to 2 years and published an interim final rule to this effect in late June 2012. As a result of the rule change, additional firms eligible for simplified reverification will not begin reaching the expiration of their verification period until February 2013. In late October 2012, VA determined that a firm would be eligible for simplified reverification one time before again requiring full examination (i.e., once every 4 years).

Despite the steps that VA had taken since December 2011, the Secretary of Veterans Affairs acknowledged ongoing concerns about the program and announced the creation of a senior executive task force to review the verification program and determine whether it had sufficient resources and support. The task force, created in June 2012, was initially charged with reporting back within 60 days with suggested changes that would help streamline the verification process. In August 2012, the task force adopted a charter stating that its purpose was to review all aspects of the verification program, including processes, operating policies, management information systems, staffing, and resources. The task force presented its preliminary findings to the VA Chief of Staff in early November 2012. The review results and recommendations of the task force were expected to be provided to the Office of the Secretary for final approval during the second quarter of fiscal year 2013.

OSDBU’s Initial Strategic Planning Effort Applied Some Best Practices, but Stakeholder Involvement, Performance Metrics, and Long-term Focus Could Be Improved

During the period covered by our review, VA had not created a formal strategic plan for the verification program. However, in response to our inquiries, OSDBU compiled a strategic planning document in late October 2012 that covered the verification program. This plan was based on a series of planning documents that were initially developed between June and December 2011 for internal discussions and conversations with congressional staff. This initial strategic planning effort appears to have at least partially followed key leading federal strategic planning practices, but additional progress is needed to improve the usefulness of the plan. We have previously reported that agency-wide strategic planning practices required under the Government Performance and Results Act of 1993 (GPRA)—which was amended by the GPRA Modernization Act of 2010 (GPRAMA)—can also serve as leading practices for planning at lower levels within federal agencies, such as individual programs or initiatives. We have also previously identified six leading practices in federal strategic planning that are most relevant to initial strategic planning efforts: (1) defining the mission and goal; (2) defining strategies that address management challenges and identifying resources needed to achieve goals; (3) ensuring leadership involvement and accountability; (4) involving stakeholders; (5) coordinating with other federal agencies; and (6) developing and using performance measures.of each of these practices, see appendix I.

According to OSDBU and CVE officials, VA did not develop a formal strategic plan when it was initially developing the verification program because the primary concern at the time was to develop and implement initial verification procedures and program regulations. Once the program was launched in 2008, CVE continued to make its immediate operational challenges a higher priority than long-range strategic planning. Although VA’s focus on getting the verification program running and reacting to legislative change may have seemed reasonable at the time, its failure to develop a comprehensive strategic plan contributed to programmatic inefficiencies. For example, as discussed in greater detail later, VA developed the data system for the verification program without fully considering its long-term information needs. Resulting shortcomings of the system have required CVE to develop inefficient workarounds to operate and oversee the program.

After the new OSDBU executive director started in April 2011, OSDBU began developing planning documents for 2011 through 2012 that covered OSDBU and its three mission areas—the verification program, strategic outreach, and acquisition support. After we asked VA about the lack of a strategic plan for the verification program, OSDBU officials compiled the separate OSDBU planning documents into a single document and updated them to include some milestones, tasks, and metrics for 2013. VA officials said that they considered this document to be the strategic plan for OSDBU and the foundation of its efforts for fiscal year 2013, and found that the compiled document could serve as a more comprehensive basis for future planning.

Based on our review of the strategic plan and the six documents OSDBU drew upon to compile it, as well as OSDBU officials’ description of the process they undertook to develop these documents, OSDBU appears to have at least partially applied the six leading federal strategic planning practices that we previously identified, as described below.

Defining the mission and goals. The plan provides OSDBU’s primary mission and alludes to the components of the mission for the verification program (verifying eligible firms and preventing ineligible firms from being verified), but does not explicitly describe the verification program’s mission. The plan identifies broad, long-term goals for OSDBU, which according to OSDBU officials were initially intended to be achieved by 2012. These goals include achieving a sustainable organizational structure to support its mission and ensuring compliance with all statutory requirements. Long-term objectives for the verification program include, among other things, meeting all regulatory requirements, providing quality customer experience, certifying CVE’s processes and staff, and preventing ineligible firms from being verified through rigorous quality control. As we have previously reported, goals in strategic plans should ideally explain what results are expected and when to expect those results. Thus, such goals are an outgrowth of the mission and are often results-oriented. However, based on the broad wording of some of the goals and objectives for the verification program, assessing whether they have been accomplished and the results achieved would be difficult.

Defining strategies that address management challenges and identify resources needed to achieve goals. The planning documents identify management challenges that affect the verification program, such as human capital and technology. For example, the planning documents note that verification staff need training on the verification requirements. While the compiled strategic plan does not identify the specific resources necessary to overcome these challenges, it lays out strategies with more specific tasks to address them, such as developing and conducting staff training for verification.

Ensuring leadership involvement and accountability. According to OSDBU officials, more senior VA officials were aware of OSDBU’s long-term goals, and OSDBU regularly briefed VA’s Chief of Staff and other senior VA officials on its plans and progress. However, while OSDBU compiled the strategic plan itself in late October 2012, it had not yet been reviewed or approved outside of OSDBU as of early November 2012, and we could not assess whether or how senior VA leaders would be involved in monitoring its implementation. To help hold managers accountable for elements in the plan, OSDBU officials said that the Executive Director met regularly with staff to discuss their plans and performance. For example, the officials said that OSDBU and CVE officials hold weekly meetings to discuss the status of the verification program and applications reviewed.

Involving stakeholders. OSDBU officials said that they had briefed stakeholders, including congressional staff and committees, while developing the initial planning documents for 2011 and 2012 that formed the basis for OSDBU’s strategic plan. The officials said that OSDBU’s planning was informed by extensive feedback on the verification program from the VA acquisition community, congressional staff and committees, veteran support organizations and business associations, and veteran-oriented media, as well as through direct contact with applicants. However, since the strategic plan was only recently compiled in response to our review, VA had not shared the plan with key stakeholders, thus missing an opportunity to promote transparency of the verification program’s plans and priorities and to facilitate continued stakeholder involvement.

Coordinating with other federal agencies. OSDBU officials said that they met with officials from other agencies’ OSDBUs prior to the development of the planning documents to discuss the verification program, in particular the program’s potential government-wide expansion. An official said that they did not coordinate with SBA— which administers the government-wide SDVOSB contracting program and certifies the eligibility of firms for other government-wide contracting programs—when they were developing the planning documents.

Developing and using performance measures. The strategic plan that OSDBU compiled contained “metrics” related to the verification program that consisted of a combination of output, efficiency, and customer service measures but lacked quality and outcome measures aligned with long-term goals. Over 80 percent of the metrics in the plan (31 of 38 items) related to the implementation of a specific task rather than whether the desired outcomes are being achieved. For example, the verification-related metrics for 2013 include “provid improved training of CVE staff” and “review fraud training program with OIG,” in support of a strategy to improve the capability of CVE staff to perform accurate and timely evaluation of applications and detect misrepresentation and fraud. But the plan does not identify measures that could be used to assess the impact of the identified long-term goals and strategies, such as a reduction in the number of examinations that are not properly completed. As previously discussed, CVE has begun tracking staff productivity levels and more closely monitoring the quality of their work, but these measures are not included in the strategic plan. Recognizing some of the challenges with its existing measures, VA has undertaken a recent initiative with a university to improve OSDBU’s performance measures. OSDBU officials expected to incorporate these measures into future planning efforts.

Lastly, OSDBU’s initial strategic planning effort was more short-range than long-range in focus. GPRAMA requires that agency-level strategic plans cover at least a 4-year period.OSDBU developed in 2011 only covered 2011 through 2012 because they expected the program to have achieved its initial long-term goals within that time, according to OSDBU officials. In compiling the strategic plan to respond to our enquiries, OSDBU officials told us that they recognized the value of expanding the coverage of the plan to include strategies and metrics for activities to be completed in 2013. But the plan did not include strategies and metrics beyond 2013. The longer-term focus of a strategic plan is one of the key distinctions from a performance plan that focuses on annual goals and measures. Without a longer-term perspective, the current strategic plan serves as more of a short-term management plan rather than as a longer-term guide to help frame the needs and direction of the verification program.

Information Technology Limitations Have Hampered VA’s Ability to Manage the Verification Program

The verification program’s current data system lacks certain data fields and reporting and workflow management capabilities needed to provide key information for program management. We have previously reported that an agency must have relevant, reliable information to run and control its operations. More specifically, we have noted that pertinent information should be identified, captured, and distributed to the right people in sufficient detail, in the right form, and at the appropriate time to enable them to carry out their duties and responsibilities efficiently and effectively.

Since the verification program began in 2008, VA has relied on data systems that it developed on an incremental, ad hoc basis in response to immediate needs, without an overarching plan or vision, and without centralized oversight by VA’s Office of Information and Technology (OI&T). As stated earlier, VA initially did not develop a strategic plan that might have provided a framework for envisioning the verification program’s information needs from the outset. Rather, VA initially modified its existing VIP database to address only its immediate need to accept firms’ application forms and identify verified firms. VA staff also created a separate database to track the results of checks that it used to verify that firms met its basic eligibility requirements, such as veteran and service- disability status. When VA began requiring firms to submit a standardized set of documents under the 2010 process, these documents were collected in a variety of formats, and paper copies had to be manually uploaded to CVE secure servers, according to VA officials.According to VA, in some cases documentation was shredded to protect confidential information without first being uploaded to the server. In response to these problems, VA hired a contractor to develop the Verification Case Management System (VCMS), which went online in 2011. VCMS was integrated with VIP to enable VA to better track and retrieve documents and manage the verification process. According to VA officials, the project was managed by the program office because VIP and VCMS were funded by VA’s Supply Fund, and not through appropriated information technology funds, which are overseen by OI&T.

The resulting VIP/VCMS system aids in performing some tasks. For example, VIP/VCMS allows applicants to upload documents directly into the web-based system, and applicants and VA to track applicants’ broad phase of review (i.e., initiation, examination, or evaluation). VA staff and contractors can also use VIP/VCMS to send and maintain a record of emails to the applicant to, for example, request additional documentation, provide status updates, or send the determination letter. The system also allows VA officials to run some reports, such as the number of initial applications and requests for reconsideration that have been approved, denied, withdrawn or completed by year as well as the open applications that have been in the system for more than 90 days.

However, VIP/VCMS has significant shortcomings that could have been avoided with better planning for and oversight of the system’s development. Specific areas with remaining shortcomings include the following:

Data fields. VA officials said that, because of the need to get a system in place quickly, the responsible staff at the time did not consider all of the data elements that would be useful for monitoring program trends and staff performance and did not plan for future phases that would add more data fields. For example, VCMS did not include data fields to track the reasons for denial (i.e., specific eligibility, ownership, or control issues); the basis for requests for reconsideration and their outcomes; and the incidence and reasons for applications being returned to a lower level of the process for rework or for a reversal of a contractor or staff member’s recommendation to approve or deny an application. VCMS also lacks fields to facilitate monitoring the reasons for and results of customer service inquiries.

Reporting. VCMS currently also allows only limited reporting, and users cannot always customize their search criteria to obtain data in the form they need to monitor the program. We noted in our August 2012 report that VCMS’s limited reporting capabilities, and the lack of certain data within the system, resulted in inconsistent aggregate reporting and made tracking the inventory of firms difficult for VA. In response, VA conducted a laborious process to develop a manual inventory of firms that have been verified under the 2006 and 2010 Act processes and the dates that those firms were verified, which VA staff could not obtain directly from VCMS.

Workflow management. VCMS has the capability to track which broad phase of the verification process an application is in and to record which staff completed certain actions in the system, but it does not meet VA staff or contractors’ needs for assigning and monitoring the progress of applications. As a result, the contractor that initially examines applications relies on a workflow management system outside of VCMS to assign and track applications as they move through the steps of the examination phase. Similarly, each of the team supervisors that we talked to has created spreadsheets to track the status of applications as their team reviews them. The reliance on these other systems is inefficient and increases the risk that data will not be completely or accurately recorded across systems.

In addition, VCMS experienced periodic outages following its initial launch in 2011 and after its most recent modification. According to VA officials, VCMS crashed almost immediately after its launch in May 2011 because it could not handle the volume of data that VA began receiving. The system was also off-line for a month in September 2011 following another modification. During these outages, firms that had a contract pending and needed to be verified in order to receive the contract could be manually processed on a case-by-case basis. More recently, VCMS was unavailable from May 9 to June 6, 2012, during which time applicants could not submit new applications, and staff could not request additional documentation through the system. This outage was caused by a security problem that was identified in routine testing by OI&T as VA was preparing to launch a modification to the system.

VA is in the process of planning to either modify or replace the current version of VCMS to address the identified shortcomings, but this planning effort has not been tied to broader long-term strategic planning for the verification program. VA officials have identified elements that the next iteration of VCMS should include. For example, the officials would like the program to automate some aspects of the background company research and generally make the verification process less burdensome on veterans. Following the program outage in May 2012, verification program officials began reaching out to OI&T for assistance in overseeing both the current system and a potential modification or replacement. These discussions received further emphasis through the previously discussed senior executive task force, which included representatives of OI&T. As a result of an expected recommendation by the task force, OI&T assigned staff in July 2012 to begin formally planning for either a modification or a replacement system, a process that OI&T will manage. VA is considering short- and long-term information needs as it defines the business requirements for the system. But, as we have seen, the initial strategic plan that OSDBU developed in late October 2012 does not specify longer-term goals for the verification program or define program strategies and activities beyond 2013. Without tying the effort to modify or replace the verification program’s data system to more comprehensive, long-term strategic planning, the resulting system risks again failing to meet the verification program’s long-term needs and goals.

Expanding Its Verification Program Government-wide Would Require VA to Improve the Program and Address Policy Issues

Expanding VA’s verification program to support the government-wide SDVOSB contracting program would require VA to increase the scale of its program to verify potentially thousands of additional firms. VA has faced ongoing challenges implementing its verification program, and it would need to continue to stabilize and improve its verification operations by addressing remaining vulnerabilities to fraud and abuse, demonstrating whether recent operational changes have resulted in improved performance and whether new methods for educating applicants are effective, and addressing data system limitations. Also, as VA revises its verification program regulation, it is considering policy issues that would impact a government-wide verification program.

VA has not formally projected how many firms it might need to verify under a government-wide SDVOSB verification program, and a number of factors make such a projection difficult. For example, the scale of a program would depend on whether firms would be required to obtain verification to bid on contracts or only to receive contract awards, how likely firms that are already self-certified as SDVOSBs would be to seek verification if it were required, and how many new or existing SDVOSBs that have not yet self-certified might seek verification in the future.

The estimate of 12,800 firms is based on the number of self-certified SDVOSBs (i.e., registered in CCR but not yet verified by VA as of Sept. 30, 2012) that did not receive contract obligations in fiscal years 2010 or 2011 (the last full fiscal year available), according to FPDS-NG. future. As a result, predicting how many would actually be motivated to seek verification if it were required is difficult.

Beyond firms that have already registered as prospective federal contractors, thousands of existing or new SDVOSBs could eventually register and seek verification if it were required. We did not identify a current estimate of the number of SDVOSBs in the United States. However, an SBA analysis of data from the Census Bureau’s 2007 Survey of Business Owners found that there were around 200,000 service-disabled veteran-owned businesses (of any size) at that time.

Considering the additional operational challenges that VA would face in preparing to verify potentially thousands of additional firms, VA would need to continue to address existing program weaknesses to stabilize and improve its verification program. Our prior and current work indicates that several aspects of VA’s current verification program, specified below, would have to be addressed before the program could be effectively implemented government-wide.

U.S. Small Business Administration, Office of Advocacy, Veteran-owned Businesses and their Owners—Data from the Census Bureau’s Survey of Business Owners (Washington, D.C.: March 2012). SBA reported that there were more than 2.1 million nonservice-disabled veteran-owned businesses (of any size) in 2007. debarments and prosecutions of firms found to be misrepresenting their SDVOSB status, had not been implemented. As of January 4, 2013, we were reviewing documentation that VA had recently provided to determine whether VA’s actions are sufficient to consider some of the recommendations implemented.

Operations. A major expansion of the verification program would have a greater chance at success if its priorities and operations were more stable and if the recent changes that VA adopted were shown to have improved the program’s performance. For example, the steps VA has taken to standardize its procedures and make them more efficient, improve its quality assurance process, and enhance training of CVE employees and contractors are promising. However, it is too soon for us to test the effectiveness of these evolving procedures, and it is not clear whether VA will adopt further significant changes as a result of the recommendations of the senior executive task force reviewing the verification program.

Applicant Education. Because a government-wide program would potentially affect thousands of additional firms, VA would need to have in place effective methods for educating business owners about the program and for obtaining and responding to their feedback. VA officials suggested that the agency’s recent efforts to clarify online guidance for applicants and to partner with organizations to better educate applicants about the verification requirements were intended to help firms understand the rationale for the required documentation and explain how VA interprets the documents submitted in making its determinations. The officials described plans for collecting the data they would need to evaluate these efforts, which included assessing whether denial rates differed for firms that used the online guidance or received assistance from a partner organization and those that did not.

Information technology. As we noted earlier, the limitations of the verification program’s information system (VIP/VCMS) have hampered VA’s ability to effectively manage, monitor, and report on the program’s operations and results. Addressing these limitations by, for example, ensuring that the information system collects the data needed to monitor the consistency and accuracy of VA’s determinations, allows customized reporting to meet managers’ needs, and supports efficient workflow management would also help position VA to manage an expanded government-wide program. Furthermore, CVE and OI&T officials said that, in planning to modify or replace VIP/VCMS, they were factoring in the potential need for the system to have the capacity and flexibility to expand to a government- wide scale and to be adapted for automated interagency information sharing. For example, the officials said they were planning to consider how to enable contracting officers from other agencies to determine whether an SDVOSB was verified without having to manually search for the firm in VIP.

In addition, VA has begun a process to revise the verification program’s regulations, which would likely serve as the starting point if VA were charged with implementing a government-wide verification program. VA officials said that they were planning to revise the regulations partly in response to applicants’ and veterans’ organizations concerns about VA’s eligibility standards. For example, two veterans’ organizations questioned VA’s regulatory requirement that veteran owners be able to transfer their ownership interest without restriction by nonveteran owners, effectively suggesting that VA’s standard for establishing control of a firm is too strict. The organizations stated that because nonveteran owners might reasonably expect to have a say in such transfers, the requirement limited the ability of SDVOSBs and VOSBs with nonveteran minority owners to participate in the Veterans First program. VA officials said that they would weigh this and other concerns as they developed proposed revisions to the regulation, a process that they expected to result in a final rule by mid-2014.

Any changes to VA’s verification requirements could create or widen differences between the various government-wide small business contracting programs’ requirements and VA’s, a consideration that would likely be of even greater importance if VA’s verification program were expanded. In addition to the government-wide SDVOSB program, federal contracting preference programs give federal agencies the authority to set aside contracts for small business concerns and specific types of small businesses: women-owned small businesses, businesses located in historically underutilized business zones (HUBZone), and socially and economically disadvantaged small businesses participating in SBA’s 8(a) program. While the SDVOSB and women-owned small business programs allow firms to self-certify their eligibility, SBA reviews supporting documentation to certify HUBZone and 8(a) firms, with the 8(a) program requiring more extensive documentation similar to what is required under VA’s verification program. (See app. IV for a description of these programs and their verification requirements.) Some veterans’ organizations and others with whom we spoke have cited perceived differences between VA’s eligibility standards and SBA’s standards for the government-wide SDVOSB program and the 8(a) program, whose certification process is most similar to VA’s verification program.

However, VA and SBA officials worked together to compare the three programs’ regulations and VA’s and the 8(a) program’s documentation requirements. Initially, VA and SBA officials told us that they did not find major differences in the programs’ regulatory eligibility requirements, the agencies’ interpretation of them, or the documentation requirements for verification. In commenting on a draft of this report, SBA subsequently stated that, while the wording of the regulations pertaining to eligibility requirements was comparable, there was a distinction regarding ownership by spouses of disabled veterans. SBA also stated in its comment letter that there were some key differences in how the agencies interpreted the regulations and that the agencies were consulting with one another to determine whether those differences could or should be resolved. Going forward, if VA adopts unilateral changes to its verification policies and procedures, these changes could have the effect of making it more difficult to align the programs. VA officials told us that the tension between competing calls for VA to ease its requirements and to be consistent with the government-wide SDVOSB and 8(a) programs would be a major consideration as VA considered changes to its regulations— particularly considering the potential for a government-wide SDVOSB verification program. Accordingly, the officials said that they were consulting with SBA as they began to develop proposed changes to VA’s verification program regulation.

Conclusions

The opportunity to receive set-aside or sole-source contract awards under the Veterans First program is a significant benefit that provides billions of dollars in contracts annually to SDVOSBs and VOSBs. As a result, the program warrants strong internal controls to provide reasonable assurance that the contracts VA enters into are awarded to eligible firms. At the same time, an inherent tension exists between the need for effective internal controls and the Veterans First program’s goal of increasing contracting opportunities for SDVOSBs. If VA fails to correctly verify eligible firms, or if firms’ concerns about the verification process deter them from applying, VA’s ability to sustain its high levels of contracting with SDVOSBs and VOSBs could ultimately be at risk. VA has made progress toward reducing its vulnerability to fraud and abuse, and CVE’s new management team has initiated a variety of operational changes in an effort to improve the program. VA has also initiated efforts to develop a comprehensive strategic plan for the verification program. This initial strategic planning effort represents a positive step that appears to have at least partially applied key leading federal strategic planning practices. However, the initial plan includes only goals intended to be met within 2 years, and many of the performance measures focus on the implementation rather than the outcomes of activities. Additionally, VA has not shared the plan with key stakeholders. As it continues to develop and refine its strategic plan, VA could strengthen its effort by ensuring that the plan articulates results-oriented, long-term goals and objectives for the verification program, that the metrics are focused on outcome measurements that can be used to monitor the verification program’s performance and demonstrate results, and that key stakeholders are involved in evaluating the plan.

The initial lack of a comprehensive strategic plan for the verification program has also contributed to the development of a data system that has proven to be inadequate. The system does not collect data for monitoring program trends and staff performance, has limited reporting and workflow management capabilities, and has been unable to accept applications for extended periods, hindering VA’s ability to operate and monitor the verification program. VA has started taking steps to address the shortcomings in the data system by shifting responsibility for developing plans to enhance or replace VCMS from CVE to OI&T. But without tying that effort to long-term strategic planning, VA risks failing to meet the program’s information needs going forward.

As VA revises its verification program regulations and considers the relationship between its policies and those of other federal small business contracting preference programs, the agency faces a tension between competing calls to reduce the burden on applicants and to be vigilant in preventing and detecting fraud. This tension would underlie a government-wide SDVOSB verification program as well. Addressing these policy issues for its own program—or ultimately for a government- wide verification program—will require VA to weigh certain tradeoffs. These include deciding how to reduce the administrative burden that the verification process places on eligible firms and maintain sufficient fraud prevention and detection controls to provide reasonable assurance that the billions of VA contract dollars set aside for SDVOSBs and VOSBs reach their intended beneficiaries.

Recommendations for Executive Action

To improve the management and oversight of VA’s SDVOSB and VOSB verification program, we recommend that the Secretary of Veterans Affairs take the following two actions:

Direct OSDBU to continue to develop, refine, and implement a formal strategic plan to provide a comprehensive framework to guide, integrate, and monitor the verification program’s activities over time. As OSDBU refines the strategic plan, it should incorporate longer- term goals and objectives for the verification program. The plan should also incorporate outcome measures that OSDBU can use to better monitor the verification program’s progress and demonstrate its results. OSDBU should also share the plan with key stakeholders.

Direct OSDBU and OI&T, as they modify or replace the verification program’s data system, to integrate their efforts with OSDBU’s broader strategic planning effort for the verification program to ensure that the new system not only addresses the short-term needs of the program but also can be readily adapted to meet longer-term needs.

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of Veterans Affairs and the Small Business Administration for comment. In its written comments, VA generally agreed with GAO’s conclusions and concurred with the two recommendations. VA stated that it had actions under way that would address each recommendation. VA indicated that it anticipated submitting a strategic plan to the Office of the Secretary in fiscal year 2013 and would develop a schedule to brief VA senior leaders and other key stakeholders once the plan is approved. VA also provided additional information about its efforts to replace the verification program’s data system. VA noted that it had begun the process of replacing the existing system and had developed a work statement for the replacement system. VA also provided technical comments that we incorporated as appropriate into the report. In its technical comments, VA disagreed with the status of some of the prior GAO recommendations that we noted had not been fully implemented, including the provision of regular fraud awareness training and unannounced random and risk-based audits of verified firms to ensure compliance with the program rules. We have revised the report to indicate that, as of January 4, 2013, we were reviewing documentation provided by VA in December 2012 to determine if VA’s actions taken to address some of our prior recommendations are sufficient to consider them implemented. We also noted that we will continue to review documentation provided by VA in the future to assess whether the remaining recommendations have been implemented.

In its written comments, SBA provided additional information on its views on eligibility requirements for VA’s Veterans First Contracting Program, the government-wide SDVOSB contracting program, and the 8(a) program. In particular, SBA stated that a statement in our draft report was not accurate—specifically, our comment that VA and SBA did not find major differences in the programs’ eligibility requirements, the agencies’ interpretation of the requirements, or the documentation required for verification. SBA noted that, statutorily, surviving spouses of disabled veterans might be eligible for VA verification but that they were not eligible under SBA’s regulations for the government-wide SDVOSB program. SBA also noted that it provided an avenue of appeal through its SDVOSB status protest and 8(a) eligibility processes but that VA did not have a similar appellate procedure. Finally, SBA stated that the wording of the regulations pertaining to VA’s and SBA’s eligibility requirements was similar but that there were some key differences in interpretation that the two agencies were reviewing. We have revised our discussion of VA’s and SBA’s effort to compare the programs’ eligibility and documentation requirements, citing the difference noted by SBA with respect to the eligibility of surviving spouses and noting that the agencies were consulting with each other to determine whether differences of interpretation could or needed to be resolved. We also added clarifying language in appendix I describing how we obtained information on VA and SBA efforts to compare program regulations. In addition, we clarified the differences between SBA’s and VA’s status protest mechanisms in appendix IV. VA’s and SBA’s comments are reprinted in appendixes V and VI.

We are sending copies of this report to the appropriate congressional committees, the Administrator of SBA, the Secretary of Veterans Affairs, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII.

Appendix I: Scope and Methodology

Our objectives were to (1) describe and assess the progress that the Department of Veterans Affairs (VA) has made in establishing a program to verify the eligibility of service-disabled veteran-owned small businesses (SDVOSB) and veteran-owned small businesses (VOSB) on a timely and consistent basis, and (2) describe the key operational and policy issues that VA would need to address should its verification program be implemented government-wide. Because VA was introducing significant changes to its procedures and operations at the time of our study, we determined that evaluating VA’s compliance with its past procedures would be of limited value and that testing the effectiveness of verification procedures that were still evolving would be premature. We focused instead on issues related to planning for and designing the verification program and on changes in its management and operations that the Center for Veterans Enterprise (CVE) instituted since December 2011.

75 Fed. Reg. 6098 (Feb. 8, 2010); 77 Fed. Reg. 38181 (June 27, 2012).

November 1, 2011, and September 30, 2012, and VA’s Verification Master Inventory List, a manually maintained inventory of all verified firms that VA uses to supplement VCMS. We assessed the data by interviewing knowledgeable VA officials, reviewing related documentation, and checking the data for illogical values or obvious errors and found them to be sufficiently reliable for the purpose of illustrating general characteristics of the verification program. We also interviewed officials from three of the contractors who perform aspects of the verification process—GCC Technologies, LLC; HeiTech Services, Inc.; and Addx Corporation—to understand their roles in the verification program, and representatives from three veteran service organizations and a technical assistance association that were participating in the Verification Counseling program–VETForce, American Legion, National Veteran Small Business Coalition, and Association of Procurement Technical Assistance Centers—to discuss their views on the verification program and their expectations of the Verification Counseling program.

Pub. L. No. 103-62 (August 3, 1993); Pub. L. No. 111-352 (Jan. 4, 2011). GPRAMA provides federal agencies with an approach to focusing on results and improving government performance by, among other things, developing strategic plans. Examples of GPRAMA plan components include a mission statement; general goals and objectives, including outcome-oriented goals; and a description of how the goals and objectives are to be achieved, including the processes and resources required. leading practices that we had previously identified as being relevant to agencies’ initial strategic planning efforts. We reviewed a strategic planning document that OSDBU compiled in 2012 in response to our study, and six planning documents prepared between June 2011 and December 2011 that OSDBU officials said provided the basis for the strategic plan. We compared these documents, and the planning activities associated with them, to the six leading practices, as shown in table 1. Because VA prepared the initial strategic planning document as we were completing our draft report, we did not conduct a comprehensive review of the strategic plan, the supporting documents that VA provided, or the process that VA undertook to develop these documents.

We also assessed the extent to which the verification program’s data system provided the information needed to run and control the verification program’s operations, a key standard for effective internal controls. In particular, we focused on the timely availability of pertinent information sufficient to enable people to carry out their duties efficiently and effectively, a factor that we previously identified as important in assessing this standard. We reviewed data system documentation and reports that the system produces and interviewed officials from VA and the contractors that perform aspects of the verification process to determine how the data system was developed and how VA uses it, and to identify the capabilities and limitations of the data system.

For verified SDVOSBs that did not appear as self-certified in CCR as of March 2012, we cross-referenced the Small Business Administration’s (SBA) Dynamic Small Business Search, which includes supplemental information on registered firms that meet SBA’s size standard for the firms’ industries. under VA’s existing program regardless of whether a government-wide program was adopted. In addition, we could not determine how many of these firms have been actively seeking contracts or how likely they would be to do so in the future, making it difficult to predict how many would actually be motivated to seek verification if it were required. We assessed these data by interviewing VA officials knowledgeable about the VA data, reviewing documentation related to all of the data systems, and checking the data for illogical values or obvious errors and found them to be sufficiently reliable for the purpose of illustrating the potential scale of a government-wide verification program. We also reviewed our prior work on the verification program and that of the VA Office of Inspector General, as well as our assessment of the current status of the program, to identify issues that VA would need to address in implementing a government-wide program. Because of the Small Business Administration’s (SBA) role administering the government-wide SDVOSB program, we also interviewed VA and SBA staff about how the statutory and regulatory provisions implemented by the two agencies compare. In addition, we reviewed SBA documents and interviewed SBA staff for their views on a potential government-wide verification program. However, the SBA staff said that it would be inappropriate for them to comment on VA’s or SBA’s potential roles or other considerations in implementing a potential program.

For both objectives we interviewed officials in VA’s Office of Small and Disadvantaged Business Utilization (OSDBU), CVE, Office of the General Counsel, and the Office of Information and Technology to understand their historical, current, and expected roles in the verification program. We also reviewed prior GAO reports and a VA Office of Inspector General report on the verification program and testimonies from congressional hearings on the government-wide SDVOSB program and VA’s verification program.

We conducted this performance audit from February 2012 to January 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Verification Program Data as of September 30, 2012

We reviewed VA’s database known as the Verification Case Management System (VCMS) to obtain data on the status of initial applications, requests for reconsideration, and applications for reverification submitted to VA between November 1, 2011, and September 30, 2012. We chose this period because fiscal year 2012 roughly coincided with the December 2011 to November 2012 period that was the focus of our work. Because we were primarily interested in the progress that VA had made processing applications that were submitted during the period that was the focus of our work, we excluded from our analysis applications VA processed during the period but that were submitted prior to November 1, 2011. We used these data to determine the volume of applications that VA received between November 1, 2011, and September 30, 2012, and their status (pending, withdrawn, approved, or denied) as of September 30, 2012. Because our analysis included applications that had been submitted less than 90 days ago, we expected a significant number of the cases to be pending as of September 30, 2012.

Based on our analysis of VCMS data, VA received approximately 4,900 initial applications between November 2011 and September 2012. The monthly volume of initial applications fluctuated during this period, with VA receiving an average of about 450 initial applications per month. As shown in figure 3, approximately 14 percent of the 4,900 initial applications submitted during the period were pending a determination as of September 30, 2012, and another 43 percent had been withdrawn.

The remaining 43 percent of applications had received a determination and, of these, 61 percent were approved and 39 percent were denied.

Applicants can withdraw their applications at any time, or VA can withdraw an application if the applicant does not respond to requests to provide missing or additional requested documentation within 30 days.

Appendix III: Verification Program Organizational Structure, December 2011 and October 2012

Between December 2011 and October 2012, VA revised the organizational structure for the verification program. As shown in figures 6 and 7, VA officials (1) reorganized and increased the number of employees and contractors assigned to the verification process and (2) created several new teams including quality assurance, training, records management, and customer service. As of October 2012, the verification program had about 28 full-time equivalent federal employees and 174 contractors, an increase of about 3 full-time equivalent staff and 64 contractors to the verification program since December 2011.

Appendix IV: Verification or Certification Requirements Associated with Federal Small Business Contracting Preference Programs

All federal agencies have the authority to set aside contracts for small business concerns and for several specific types of small businesses: SDVOSBs, women-owned small businesses, businesses located in historically underutilized business zones (HUBZone), and socially and economically disadvantaged small businesses participating in SBA’s 8(a) program (table 2). Some programs are also authorized to make sole- source awards to these groups. For the government-wide SDVOSB program, business owners are required only to certify their eligibility online in the System for Award Management (SAM) and do not need to submit any supporting documentation. SBA does not verify the eligibility of these firms. Women-owned small businesses may obtain certification by an entity approved by SBA or self-certify their eligibility online in SAM; in either case, the firms must upload supporting documents to SBA’s online Women-Owned Small Business Program Repository for potential review by contracting officers or SBA. In contrast with these self- certification programs, SBA must certify firms’ eligibility to receive contracts under the HUBZone and 8(a) programs. SBA reviews supporting documentation to certify HUBZone and 8(a) firms, with the 8(a) program requiring more extensive documentation that is similar to that required by CVE for its verification program. For each of the government-wide small business contracting preference programs except for the 8(a) program, SBA provides a “status protest” mechanism for interested parties to a contract award to protest if they feel a firm misrepresented its eligibility in its bid submission. SBA’s status protest mechanism for the SDVOSB and women-owned small business programs and its certification process for the 8(a) program also provide interested parties with an avenue of appeal to SBA’s Office of Hearings and Appeals. However, VA’s OSDBU decides any SDVOSB or VOSB status protests arising from a VA solicitation.appellate procedure for such decisions.

Appendix V: Comments from the Department of Veterans Affairs

Appendix VI: Comments from the Small Business Administration

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Harry Medina (Assistant Director), Emily Chalmers, Pamela Davidson, Julianne Dieterich, Julia Kennon, Cory Marzullo, John McGrail, Daniel Newman, Jena Sinkfield, James Sweetman, and William Woods made key contributions to this report.