Veteran-Owned Small Businesses: Planning and Data System for VA's Verification Program Need Improvement
Released: 2013-01-14T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: VA is required to give contracting preference to service-disabled and other veteran-owned small businesses. It must also verify the ownership and control of these firms to confirm eligibility. Prior reports by GAO and VA's Office of Inspector General identified weaknesses in VA's processes and controls that allowed ineligible firms to be verified. GAO was asked to review the verification program. For this report, GAO assessed (1) VA's progress in establishing a program for verifying firms' eligibility on a timely and consistent basis and (2) key operational and policy issues that VA would have to address should its verification program be implemented government-wide. GAO reviewed VA's policies and procedures; compared its initial strategic planning effort with previously identified leading strategic planning practices; interviewed VA officials and veterans' organizations; and analyzed government-wide contracting databases.

section_title: What GAO Found
paragraphs: The Department of Veterans Affairs (VA) has made significant changes to its verification processes for service-disabled and other veteran-owned small businesses to improve operations and address program weaknesses, but continues to face challenges in establishing a stable and efficient program to verify firms on a timely and consistent basis. Since December 2011, VA has instituted a number of significant operational changes, including revising standard operating procedures and enhancing quality assurance protocols for its verification program. However, GAO found that VA did not have a comprehensive, long-term strategic plan for the program and had prioritized addressing immediate operational challenges, contributing to programmatic inefficiencies. In response to this observation, VA's Office of Small and Disadvantaged Business Utilization (OSDBU) initiated action in late October 2012 to compile a strategic planning document that encompassed the verification program. VA's OSDBU appears to have partially applied key leading strategic planning practices in its initial planning effort. But the plan lacks performance measures to assess whether the desired outcomes are being achieved and has a short-term focus that is not typically associated with a strategic plan. VA also has not shared the plan with key stakeholders, including congressional staff. Further, the verification program's data system has shortcomings that have hindered VA's ability to operate, oversee, and monitor the program. Among other things, the system does not collect important data and has limited reporting and workflow management capabilities. VA plans to modify or replace the system, but has not directly tied this effort into its long-term strategic planning efforts to ensure that the new system meets the verification program's long-term information needs.

Expanding VA's verification program to support the government-wide contracting program for service-disabled, veteran-owned small businesses would require VA to improve its verification process and address a number of operational and policy issues. GAO estimated that between about 3,600 and 16,400 currently self-certified firms could seek verification under an expanded program, but VA has experienced ongoing challenges verifying the volume of firms currently participating in the program. GAO's prior and current work indicates that VA would need to further reduce its program's vulnerability to fraud and abuse, demonstrate whether recent operational changes have improved performance, have in place effective methods for educating applicants, and address the limitations of the program's data system in order to expand successfully. Also, VA has begun a process to revise the verification program's regulations, partly in response to concerns about VA's eligibility standards being too stringent. However, any changes to VA's verification requirements could create or widen differences between the various government-wide small business contracting programs' requirements and VA's, a consideration that would likely be of even greater importance if VA's verification program were expanded. Addressing these issues for its own program--or ultimately for a government-wide program-- requires weighing tradeoffs between reducing the burden of verification on eligible firms and providing reasonable assurance that contracting preferences reach their intended beneficiaries.

section_title: What GAO Recommends
paragraphs: To improve the long-term effectiveness of the program, VA should (1) refine and implement a strategic plan with outcome-oriented long-term goals and performance measures, and (2) integrate its efforts to modify or replace the program's data system with a broader strategic planning effort to ensure that the system addresses the program's short- and long-term needs. VA concurred with both recommendations.

section_title: Letter
paragraphs: The Department of Veterans Affairs (VA) has dramatically increased its  contracting with small businesses owned by veterans, including service- disabled veterans, since 2006—from $616 million in obligations in fiscal  year 2006 to $3.6 billion in fiscal year 2011. VA’s success in contracting  with veteran-owned small businesses (VOSB), including service-disabled  veteran-owned small businesses (SDVOSB), stems from its Veterans  First Contracting program, established in response to the Veterans  Benefits, Health Care, and Information Technology Act of 2006 (2006  Act). The 2006 Act requires VA to give preference in its small business  contracting to SDVOSBs and VOSBs. It also gives the agency unique  authority to make noncompetitive (sole-source) awards to these firms and  to restrict competition for awards to them (set-asides). Along with  establishing VA’s contracting preferences, the 2006 Act makes VA  responsible for maintaining a database of SDVOSBs and other VOSBs.  The act requires VA to verify that all firms entered in the database are  actually owned and controlled by one or more veterans and to confirm the  status of any owner who indicates a service-connected disability.

Sustaining its high levels of contracting with SDVOSBs and VOSBs and  ensuring the integrity of the Veterans First program requires VA to verify a  pool of firms eligible to receive contracts and to correctly disqualify  ineligible firms. Yet since VA launched its verification program in 2008, we  and VA’s Office of Inspector General (OIG) have reported on  weaknesses, such as in VA’s efforts to define an organizational structure  to manage the program, establish and monitor verification operations, and  institute effective controls to prevent and detect fraud by ineligible firms.At the same time, some veteran small business owners and their  advocates have complained that the verification process has been slow  and burdensome and have questioned VA’s rationale for denying some  firms’ verification applications.

The government-wide SDVOSB contracting program, which is  administered by the Small Business Administration (SBA), authorizes  other federal agencies to award set-aside or sole-source contracts to  firms that self-certify as SDVOSBs. As such, the program risks awarding  contracts to firms that are ineligible or deliberately misrepresenting their  SDVOSB status. We and the Department of Defense Inspector General  have identified several instances in which agencies awarded contracts  under the government-wide SDVOSB program to ineligible firms.

Accordingly, we have previously suggested that a government-wide  SDVOSB verification program should be considered.

We were asked to review VA’s verification program and assess the steps  that would be necessary to implement it government-wide. Because VA  was introducing significant changes to its procedures and operations at the  time of our study, we determined that evaluating VA’s compliance with its  past procedures would be of limited value and that testing the effectiveness  of verification procedures that were still evolving would be premature. We  focused instead on issues related to planning for and designing the  verification program and on changes in its management and operations.  Accordingly, in this report, we (1) describe and assess the progress that VA  has made in establishing a program to verify the eligibility of SDVOSBs and  VOSBs on a timely and consistent basis, and (2) describe the key  operational and policy issues that VA would have to address should its  verification program be implemented government-wide.

To describe and assess the progress VA has made in establishing a  program to allow it to verify the eligibility of SDVOSBs and VOSBs on a  timely and consistent basis, we reviewed relevant statutes, regulations,  and procedures for the verification program, as well as planning and  organizational documents. We compared the strategic planning document  that the Office of Small and Disadvantaged Business Utilization (OSDBU)  compiled in 2012 in response to our study to six leading practices that we  had previously identified as being relevant to agencies’ initial strategic  planning efforts. We also interviewed VA officials and representatives  from three veteran service organizations and a technical assistance  association that were participating in an outreach program VA had  launched to assist applicant firms. To describe the key operational and  policy issues that VA would need to address should its verification  program be implemented government-wide, we developed rough order of  magnitude estimates of how many more SDVOSBs—beyond those that  VA had already verified or was in the process of verifying as of  September 30, 2012—might seek verification if it were required  government-wide. Specifically, our estimation method relied on the  number of SDVOSBs listed in the Central Contractor Registration (CCR)  database that had not been verified by VA and were not in the process of  being verified. Next, we determined whether or not these self-certified  SDVOSBs received contracts from agencies other than VA in fiscal years  2010 or 2011 using the Federal Procurement Data System-Next  Generation (FPDS-NG). These estimates do not take into account the  number of firms that would seek verification under VA’s existing program  regardless of whether a government-wide program were adopted nor the  number that actually would be motivated to seek verification if it were  adopted. We assessed these data by interviewing VA officials  knowledgeable about the VA data, reviewing documentation related to all  of the data systems, and checking the data for illogical values or obvious  errors and found them to be sufficiently reliable for the purpose of  illustrating the potential scale of a government-wide verification program.  We also reviewed our prior work on the verification program and that of  the VA OIG. Further, because of SBA’s role administering the  government-wide SDVOSB program, we reviewed SBA documents and  interviewed SBA staff for their views on a potential government-wide  verification program. However, SBA staff said that it would be  inappropriate for them to comment on VA’s or SBA’s potential roles or  other considerations in implementing a potential program. Appendix I  offers a fuller description of our objectives, scope, and methodology.

We conducted this performance audit from February 2012 to January  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The 2006 Act requires VA to establish annual contracting goals for  SDVOSBs and other nonservice-disabled VOSBs; the goal for SDVOSBs  must at least match the government-wide SDVOSB contracting goal of 3  percent of federal contract dollars. VA set its goals at 3 percent for  SDVOSBs and 7 percent for all VOSBs (SDVOSBs and other VOSBs) for  fiscal years 2006 and 2007, and subsequently has set and achieved  increasing contracting goals for VOSBs and SDVOSBs, as shown in  figure 1. VA’s total contracting awards to VOSBs increased from $616  million (including $356 million to SDVOSBs) in fiscal year 2006 to $3.6  billion (including $3.2 billion to SDVOSBs) in fiscal year 2011.

VA’s OSDBU has overall responsibility for the SDVOSB/VOSB  verification program. Within OSDBU, the Center for Veterans Enterprise  (CVE), maintains a database of eligible SDVOSBs and VOSBs and is  responsible for verification operations, such as processing applications.  To implement the requirements of the 2006 Act, VA began verifying  businesses in May 2008 under interim final rules, which the agency did  not finalize until February 2010.the verification program, see fig. 2.) To be eligible for verification under  VA’s rules   (For a timeline of major events affecting  the small business concern must be unconditionally owned and  controlled by one or more eligible parties (veterans, service-disabled  veterans, or surviving spouses);  the owners of the small business must have good character (any  small business owner or concern that has been debarred or  suspended is ineligible);  the applicant cannot knowingly make false statements in the  application process;  the firm and its eligible owners must not have significant financial  obligations owed to the federal government; and  the firm must not have been found ineligible due to an SBA protest  decision.

VA launched its verification process under the 2006 Act in 2008 and  shifted to a more robust process in 2010. VA’s verification process under  the 2006 Act (2006 process) initially consisted of (1) checking VA  databases to confirm veteran status and, if applicable, service-disability  status; and (2) reviewing publicly available, primarily self-reported  information about control and ownership for all businesses that applied for  verification. Beginning in September 2008, VA also adopted a risk-based  approach to conducting site visits or other means, such as additional  document reviews and telephone interviews, to further investigate  selected high-risk businesses. VA adopted a more thorough verification  process in 2010 (2010 process), which included reviewing and analyzing  a standardized set of documents that each applicant is required to submit.  VA refined the 2010 process over time so that, as of October 2012, the  verification process consisted of four phases—initiation, examination,  evaluation, and determination. Denied applicant firms are able to request  a reconsideration of the denial decision.

Initiation: CVE employees are to confirm that applicants meet  minimum requirements for the program by, among other things,  verifying the owners’ veteran and service-disability status and  determining that they have submitted all of the required documents or  adequate explanations for missing documents.to check the Excluded Parties List System to ensure that the applicant  business and all owners are not on the list.

Examination: Contractors are to review completed applications to  determine whether firms meet the eligibility requirements and make  an initial recommendation for approval, denial, or additional review  (i.e., a site visit).

Evaluation: Contractors and staff are to review the initial  recommendations to ensure that the screening has met quality  standards and that firms have received an appropriate  recommendation. They may decide as well that a site visit is necessary.  Contractors are to conduct site visits if they are recommended, and  CVE employees are to recommend approval or denial.

Determination: CVE supervisors are to review staff recommendations  and issue eligibility decisions. A determination letter is to be emailed  to the applicant, and approved companies appear as verified in the  Vendor Information Pages (VIP) database.

Request for Reconsideration: Through an optional Request for  Reconsideration process, denied applicants can remedy the issue(s)  that caused their applications to be initially denied. Based on a review  by staff from VA’s Office of General Counsel, VA may approve the  application, deny it on the same grounds as the original decision, or  deny it on other grounds. If VA denies a request for reconsideration  solely on issues not raised in the initial denial, the applicant may ask  for reconsideration as if it were an initial denial. Denied applicants can  also request a legal review if they believe their application was denied  in error.

VA’s database of SDVOSBs and VOSBs previously listed unverified and  verified firms, but currently is required to list only verified firms, as a result  of the Veterans Small Business Verification Act (2010 Act), part of the  Veterans’ Benefits Act of 2010.2008, VA modified its VIP database of self-certified SDVOSBs and   After the verification program began in  VOSBs to receive verification applications and publicly display names of  verified firms. Once VA approved a business, the business name  appeared with a verified logo in VIP, but the database continued to  display self-certified firms as well. The 2010 Act requires that no new  applicant appear in the VIP database unless it has been verified by VA as  owned and controlled by a veteran or service-disabled veteran. The 2010  Act also required VA, within 60 days of enactment, to notify all unverified  (self-certified) firms in VIP about the verification requirement and also  required firms to apply for verification within 90 days or be removed from  the database. VA officials reported that by September 2011, the agency  had removed from the VIP database all firms that had self-certified so that  the database would include only verified firms. As of October 2012, the  database included both firms verified under the 2006 Act process and the  2010 process.

section_title: VA Has Made Changes to Improve Its Verification Program, but Continues to Face Challenges in Its Strategic Planning Efforts and Information Technology Infrastructure
paragraphs: VA has made significant changes to its verification processes in an effort  to improve its operations and address program weaknesses, but  continues to face challenges in establishing a stable and efficient program  to verify firms on a timely and consistent basis. Since December 2011,  VA has instituted a number of significant operational changes, including  revising standard operating procedures and enhancing quality assurance  protocols. However, it has not had a comprehensive, long-term strategic  plan for the verification program and has consistently prioritized  addressing immediate operational challenges, contributing to  programmatic inefficiencies. In response to our observations, VA’s  OSDBU initiated action in late October 2012 to compile a strategic  planning document that encompasses the verification program. OSDBU  appears to have at least partially applied key leading strategic planning  practices in its initial planning effort. But the plan lacked performance  measures to assess whether the desired outcomes are being achieved  and had a shorter-term focus than typically associated with a strategic  plan. Furthermore, VA had not shared the plan with key stakeholders,  such as veteran support organizations and business associations or  congressional staff. In addition, the verification program’s information  technology (IT) system has shortcomings that have hindered VA’s ability  to operate, oversee, and monitor the program. VA is planning to modify or  replace the system, but has not directly tied this effort into its long-term  strategic planning efforts to ensure that the new system meets the  verification program’s long-term information needs.
subsections: section_title: VA Has Adopted Changes to Improve Verification Operations and Address Some Identified Weaknesses and Concerns
paragraphs: As of September 30, 2012, the VIP database listed 6,257 firms that had  been verified as VOSBs or SDVOSBs. Of these, VA reported that 1,733  were verified under the initial 2006 Act process and 4,524 under the more  rigorous 2010 process. VA’s database also listed a substantial number of  pending cases at that time: 691 new applications for verification, 131  firms seeking reverification to remain in VIP, and 165 requests for  reconsideration from firms that were denied verification. See appendix II  for additional data on the verification program as of September 30, 2012.

GAO-12-697. process that VA previously used represented a continuing vulnerability.  By September 30, 2012, VA had reverified or removed from the VIP  database 622 of the firms that were originally verified under the 2006 Act  process, but had yet to reverify the remaining 1,733 firms, according to  VA’s inventory of verified firms. The inventory indicated that the 2-year  verification period for 1,159 of these remaining firms expired on or before  September 30, 2012, so they were not eligible to receive VA contracts  after that date and were due to be removed from the database. VA  officials said that firms whose 2-year verification period had not yet  expired would be removed from the database upon expiration if they had  not been verified under the 2010 process. According to VA, fewer than  120 companies that were verified under the 2006 process remained in  VIP as of December 1, 2012.

In interviews with us between April 2012 and June 2012, veterans’  organizations cited applicant concerns about other aspects of the  verification program, such as the rationale for determinations and the time  it took VA to make determinations, as the following examples illustrate.

We observed several outreach sessions that VA conducted in May  and June 2012 with veterans’ organizations and an association of  organizations that provided technical assistance with procurement. In  these sessions and in our follow-up interviews with participants, the  organizations stated that VA’s guidance for applicants did not always  adequately explain how VA interpreted some of the subjective  eligibility standards in its regulations, such as the requirements that  owners have good character. They also said that they and applicants  sometimes found the rationale for denial to be unclear or  inconsistent.

Representatives from two of the veteran service organizations that we  interviewed also raised concerns about the length of time it could take  to process an application. According to VA officials, it took on average  more than 130 days after receiving a complete application to make a  determination in July 2011. As of October 2012, it took approximately  85 days.

With the hiring of a new CVE director in December 2011, CVE conducted  a review of the verification process to identify ways to increase its  efficiency and began adopting changes to the verification process to  improve its operations and address program weaknesses and applicants’  concerns.  For example, as a result of this review, CVE did the following:

CVE revised its Standard Operating Procedures (SOP) to reflect  current practices and help ensure greater consistency in its  verification processes. These procedures describe the purpose,  scope, statutory references, staff roles, and implementation steps.

CVE instituted a more robust quality assurance process to ensure that  staff adhered to the approved procedures. For example, CVE  employees and contractors are now subject to both scheduled and  spot audits and must resolve any major deficiencies within 10 days.

CVE hired its first training officer and revised its training program with  the goals of ensuring that CVE staff were properly trained and  qualified to perform their duties, achieved high performance, and were  responsive to changing business requirements, among other things.  The training officer is responsible for coordinating training for staff  (including contractors), including weekly training on the verification  program and customer service, as well as monthly fraud education.

CVE added specific methods of communicating with applicant firms,  with the goal of ensuring that applicants receive an email from VA at  least every 30 days with an update on the status of their application.

CVE began using the initiation phase, rather than the examination  phase, to determine whether an application was complete, so that any  missing documentation or inadequate explanations can be addressed  before the examination process begins.

CVE began tracking staff productivity levels and more closely  monitoring the quality of their work. For example, beginning in the  spring of 2012, CVE started setting targets for the numbers of cases  that individual staff members should review each day. Also, in  September and October 2012, CVE’s evaluation team reviewed the  results of contractors’ examinations to identify cases that had not  been properly completed or in which the recommended finding should  be overturned.recommended for additional training.

Contractor staff with the most rejected cases were  VA has also revised the organizational structure for the verification  program, but VA officials said that human capital challenges remained. In  general, CVE is structured so that a federal employee oversees a team of  contractors. VA uses a mix of federal employees and contractors to  complete verifications because contractors have greater flexibility to adjust  staffing levels in response to variations in the number of applications  submitted, according to VA officials. Between December 2011 and October  2012, VA adopted changes to the verification program’s organizational  structure to make it more efficient, increase oversight of federal staff and  contractors, and strengthen functions that did not previously have  dedicated staff, such as training. VA officials (1) reorganized and increased  the number of employees and contractors assigned to the verification  process and (2) created several new teams including quality assurance,  training, records management, and customer service. During this period,  VA added about 3 full-time equivalent staff and 64 contractors to the  verification program.organization charts as of December 2011 and October 2012.) However, VA  officials said that the verification program faces ongoing human capital  challenges. For example, 5 of the verification program’s 27 full-time federal  positions were vacant as of November 2012. As of early November 2012,  CVE was developing a business case to justify the staff organization  necessary to support verification operations, including revised federal  employee labor categories and modifications to contractor support.   (See app. III for the verification program’s  VA has also sought to improve outreach to applicants through additional  online resources and a new Verification Counseling program. In  November 2011, VA began posting on its website verification assistance  briefs intended to clarify aspects of the program’s rules. These briefs  cover topics that VA officials have determined cause the majority of  denials, such as full-time control and transfer restrictions. In addition, VA launched a self-assessment tool in June 2012 to help applicants  understand the rules, regulations, eligibility criteria, and review process  for verification. Recognizing that some applicants needed additional  support, VA launched a Verification Counseling program in June 2012.  According to VA, this program integrates the Verification Assistance  Partner counselors (initially, selected veterans’ support organizations and  business associations and Procurement Technical Assistance Centers)  into the regular training provided to CVE examination and evaluation staff.  These counselors in turn provide counseling to firms interested in  becoming verified. The program is intended to increase understanding  of the verification program’s eligibility requirements so that ineligible firms  would be less likely to apply and eligible firms would be more likely to  submit the materials necessary for them to succeed in their initial  applications.

To mitigate an anticipated increase in its workflow over time, VA initiated  two efforts in early 2012 to modify its approach to reverifying firms’  eligibility. VA’s verification regulations issued in May 2008 limited the term  of the verification status of a firm to a 1-year time period. However, as  growing numbers of firms verified under the 2010 process began to  require reverification in early 2012, VA recognized that it would face a  mounting workload over time if it reverified firms annually using its full  examination procedures. As a result, VA began to develop procedures for  a simplified reverification process, which it introduced in early June  2012. VA also began the process of modifying the verification program  regulations to extend the verification period from 1 year to 2 years and  published an interim final rule to this effect in late June 2012. As a result  of the rule change, additional firms eligible for simplified reverification will  not begin reaching the expiration of their verification period until February  2013. In late October 2012, VA determined that a firm would be eligible  for simplified reverification one time before again requiring full  examination (i.e., once every 4 years).

Despite the steps that VA had taken since December 2011, the Secretary  of Veterans Affairs acknowledged ongoing concerns about the program  and announced the creation of a senior executive task force to review the  verification program and determine whether it had sufficient resources  and support. The task force, created in June 2012, was initially charged  with reporting back within 60 days with suggested changes that would  help streamline the verification process. In August 2012, the task force  adopted a charter stating that its purpose was to review all aspects of the  verification program, including processes, operating policies,  management information systems, staffing, and resources. The task force  presented its preliminary findings to the VA Chief of Staff in early  November 2012. The review results and recommendations of the task  force were expected to be provided to the Office of the Secretary for final  approval during the second quarter of fiscal year 2013.

section_title: OSDBU’s Initial Strategic Planning Effort Applied Some Best Practices, but Stakeholder Involvement, Performance Metrics, and Long-term Focus Could Be Improved
paragraphs: During the period covered by our review, VA had not created a formal  strategic plan for the verification program. However, in response to our  inquiries, OSDBU compiled a strategic planning document in late October  2012 that covered the verification program. This plan was based on a  series of planning documents that were initially developed between June  and December 2011 for internal discussions and conversations with  congressional staff. This initial strategic planning effort appears to have at  least partially followed key leading federal strategic planning practices,  but additional progress is needed to improve the usefulness of the plan.  We have previously reported that agency-wide strategic planning  practices required under the Government Performance and Results Act of  1993 (GPRA)—which was amended by the GPRA Modernization Act of  2010 (GPRAMA)—can also serve as leading practices for planning at  lower levels within federal agencies, such as individual programs or  initiatives. We have also previously identified six leading practices in  federal strategic planning that are most relevant to initial strategic  planning efforts: (1) defining the mission and goal; (2) defining strategies  that address management challenges and identifying resources needed  to achieve goals; (3) ensuring leadership involvement and accountability;  (4) involving stakeholders; (5) coordinating with other federal agencies;  and (6) developing and using performance measures.of each of these practices, see appendix I.

According to OSDBU and CVE officials, VA did not develop a formal  strategic plan when it was initially developing the verification program  because the primary concern at the time was to develop and implement  initial verification procedures and program regulations. Once the program  was launched in 2008, CVE continued to make its immediate operational  challenges a higher priority than long-range strategic planning. Although  VA’s focus on getting the verification program running and reacting to  legislative change may have seemed reasonable at the time, its failure to  develop a comprehensive strategic plan contributed to programmatic  inefficiencies. For example, as discussed in greater detail later, VA  developed the data system for the verification program without fully  considering its long-term information needs. Resulting shortcomings of  the system have required CVE to develop inefficient workarounds to  operate and oversee the program.

After the new OSDBU executive director started in April 2011, OSDBU  began developing planning documents for 2011 through 2012 that  covered OSDBU and its three mission areas—the verification program,  strategic outreach, and acquisition support. After we asked VA about  the lack of a strategic plan for the verification program, OSDBU officials  compiled the separate OSDBU planning documents into a single  document and updated them to include some milestones, tasks, and  metrics for 2013. VA officials said that they considered this document to  be the strategic plan for OSDBU and the foundation of its efforts for fiscal  year 2013, and found that the compiled document could serve as a more  comprehensive basis for future planning.

Based on our review of the strategic plan and the six documents OSDBU  drew upon to compile it, as well as OSDBU officials’ description of the  process they undertook to develop these documents, OSDBU appears to  have at least partially applied the six leading federal strategic planning  practices that we previously identified, as described below.

Defining the mission and goals. The plan provides OSDBU’s primary  mission and alludes to the components of the mission for the  verification program (verifying eligible firms and preventing ineligible  firms from being verified), but does not explicitly describe the  verification program’s mission. The plan identifies broad, long-term  goals for OSDBU, which according to OSDBU officials were initially  intended to be achieved by 2012. These goals include achieving a  sustainable organizational structure to support its mission and ensuring  compliance with all statutory requirements. Long-term objectives for the  verification program include, among other things, meeting all regulatory  requirements, providing quality customer experience, certifying CVE’s  processes and staff, and preventing ineligible firms from being verified  through rigorous quality control. As we have previously reported, goals  in strategic plans should ideally explain what results are expected and  when to expect those results. Thus, such goals are an outgrowth of the  mission and are often results-oriented. However, based on the broad  wording of some of the goals and objectives for the verification  program, assessing whether they have been accomplished and the  results achieved would be difficult.

Defining strategies that address management challenges and identify  resources needed to achieve goals. The planning documents identify  management challenges that affect the verification program, such as  human capital and technology. For example, the planning documents  note that verification staff need training on the verification  requirements. While the compiled strategic plan does not identify the  specific resources necessary to overcome these challenges, it lays  out strategies with more specific tasks to address them, such as  developing and conducting staff training for verification.

Ensuring leadership involvement and accountability. According to  OSDBU officials, more senior VA officials were aware of OSDBU’s  long-term goals, and OSDBU regularly briefed VA’s Chief of Staff and  other senior VA officials on its plans and progress. However, while  OSDBU compiled the strategic plan itself in late October 2012, it had  not yet been reviewed or approved outside of OSDBU as of early  November 2012, and we could not assess whether or how senior VA  leaders would be involved in monitoring its implementation. To help  hold managers accountable for elements in the plan, OSDBU officials  said that the Executive Director met regularly with staff to discuss their  plans and performance. For example, the officials said that OSDBU  and CVE officials hold weekly meetings to discuss the status of the  verification program and applications reviewed.

Involving stakeholders. OSDBU officials said that they had briefed  stakeholders, including congressional staff and committees, while  developing the initial planning documents for 2011 and 2012 that  formed the basis for OSDBU’s strategic plan. The officials said that  OSDBU’s planning was informed by extensive feedback on the  verification program from the VA acquisition community,  congressional staff and committees, veteran support organizations  and business associations, and veteran-oriented media, as well as  through direct contact with applicants. However, since the strategic  plan was only recently compiled in response to our review, VA had not  shared the plan with key stakeholders, thus missing an opportunity to  promote transparency of the verification program’s plans and priorities  and to facilitate continued stakeholder involvement.

Coordinating with other federal agencies. OSDBU officials said that  they met with officials from other agencies’ OSDBUs prior to the  development of the planning documents to discuss the verification  program, in particular the program’s potential government-wide  expansion. An official said that they did not coordinate with SBA— which administers the government-wide SDVOSB contracting  program and certifies the eligibility of firms for other government-wide  contracting programs—when they were developing the planning  documents.

Developing and using performance measures. The strategic plan that  OSDBU compiled contained “metrics” related to the verification  program that consisted of a combination of output, efficiency, and  customer service measures but lacked quality and outcome measures  aligned with long-term goals. Over 80 percent of the metrics in the  plan (31 of 38 items) related to the implementation of a specific task  rather than whether the desired outcomes are being achieved. For  example, the verification-related metrics for 2013 include “provid  improved training of CVE staff” and “review fraud training program  with OIG,” in support of a strategy to improve the capability of CVE  staff to perform accurate and timely evaluation of applications and  detect misrepresentation and fraud. But the plan does not identify  measures that could be used to assess the impact of the identified  long-term goals and strategies, such as a reduction in the number of  examinations that are not properly completed. As previously  discussed, CVE has begun tracking staff productivity levels and more  closely monitoring the quality of their work, but these measures are  not included in the strategic plan. Recognizing some of the  challenges with its existing measures, VA has undertaken a recent  initiative with a university to improve OSDBU’s performance  measures. OSDBU officials expected to incorporate these measures  into future planning efforts.

Lastly, OSDBU’s initial strategic planning effort was more short-range  than long-range in focus. GPRAMA requires that agency-level strategic  plans cover at least a 4-year period.OSDBU developed in 2011 only covered 2011 through 2012 because  they expected the program to have achieved its initial long-term goals  within that time, according to OSDBU officials. In compiling the strategic  plan to respond to our enquiries, OSDBU officials told us that they  recognized the value of expanding the coverage of the plan to include  strategies and metrics for activities to be completed in 2013. But the plan  did not include strategies and metrics beyond 2013. The longer-term  focus of a strategic plan is one of the key distinctions from a performance  plan that focuses on annual goals and measures. Without a longer-term  perspective, the current strategic plan serves as more of a short-term  management plan rather than as a longer-term guide to help frame the  needs and direction of the verification program.

section_title: Information Technology Limitations Have Hampered VA’s Ability to Manage the Verification Program
paragraphs: The verification program’s current data system lacks certain data fields  and reporting and workflow management capabilities needed to provide  key information for program management. We have previously reported  that an agency must have relevant, reliable information to run and control  its operations. More specifically, we have noted that pertinent  information should be identified, captured, and distributed to the right  people in sufficient detail, in the right form, and at the appropriate time to  enable them to carry out their duties and responsibilities efficiently and  effectively.

Since the verification program began in 2008, VA has relied on data  systems that it developed on an incremental, ad hoc basis in response to  immediate needs, without an overarching plan or vision, and without  centralized oversight by VA’s Office of Information and Technology  (OI&T). As stated earlier, VA initially did not develop a strategic plan that  might have provided a framework for envisioning the verification  program’s information needs from the outset. Rather, VA initially modified  its existing VIP database to address only its immediate need to accept  firms’ application forms and identify verified firms. VA staff also created a  separate database to track the results of checks that it used to verify that  firms met its basic eligibility requirements, such as veteran and service- disability status. When VA began requiring firms to submit a  standardized set of documents under the 2010 process, these documents  were collected in a variety of formats, and paper copies had to be  manually uploaded to CVE secure servers, according to VA officials.According to VA, in some cases documentation was shredded to protect  confidential information without first being uploaded to the server. In  response to these problems, VA hired a contractor to develop the  Verification Case Management System (VCMS), which went online in  2011. VCMS was integrated with VIP to enable VA to better track and  retrieve documents and manage the verification process. According to VA  officials, the project was managed by the program office because VIP and  VCMS were funded by VA’s Supply Fund, and not through appropriated  information technology funds, which are overseen by OI&T.

The resulting VIP/VCMS system aids in performing some tasks. For  example, VIP/VCMS allows applicants to upload documents directly into  the web-based system, and applicants and VA to track applicants’ broad  phase of review (i.e., initiation, examination, or evaluation). VA staff and  contractors can also use VIP/VCMS to send and maintain a record of  emails to the applicant to, for example, request additional documentation,  provide status updates, or send the determination letter. The system also  allows VA officials to run some reports, such as the number of initial  applications and requests for reconsideration that have been approved,  denied, withdrawn or completed by year as well as the open applications  that have been in the system for more than 90 days.

However, VIP/VCMS has significant shortcomings that could have been  avoided with better planning for and oversight of the system’s  development. Specific areas with remaining shortcomings include the  following:

Data fields. VA officials said that, because of the need to get a system  in place quickly, the responsible staff at the time did not consider all of  the data elements that would be useful for monitoring program trends  and staff performance and did not plan for future phases that would  add more data fields. For example, VCMS did not include data fields  to track the reasons for denial (i.e., specific eligibility, ownership, or  control issues); the basis for requests for reconsideration and their  outcomes; and the incidence and reasons for applications being  returned to a lower level of the process for rework or for a reversal of  a contractor or staff member’s recommendation to approve or deny an  application. VCMS also lacks fields to facilitate monitoring the reasons  for and results of customer service inquiries.

Reporting. VCMS currently also allows only limited reporting, and  users cannot always customize their search criteria to obtain data in  the form they need to monitor the program. We noted in our August  2012 report that VCMS’s limited reporting capabilities, and the lack of  certain data within the system, resulted in inconsistent aggregate  reporting and made tracking the inventory of firms difficult for VA. In  response, VA conducted a laborious process to develop a manual  inventory of firms that have been verified under the 2006 and 2010  Act processes and the dates that those firms were verified, which VA  staff could not obtain directly from VCMS.

Workflow management. VCMS has the capability to track which broad  phase of the verification process an application is in and to record  which staff completed certain actions in the system, but it does not  meet VA staff or contractors’ needs for assigning and monitoring the  progress of applications. As a result, the contractor that initially  examines applications relies on a workflow management system  outside of VCMS to assign and track applications as they move  through the steps of the examination phase. Similarly, each of the  team supervisors that we talked to has created spreadsheets to track  the status of applications as their team reviews them. The reliance on  these other systems is inefficient and increases the risk that data will  not be completely or accurately recorded across systems.

In addition, VCMS experienced periodic outages following its initial launch  in 2011 and after its most recent modification. According to VA officials,  VCMS crashed almost immediately after its launch in May 2011 because  it could not handle the volume of data that VA began receiving. The  system was also off-line for a month in September 2011 following another  modification. During these outages, firms that had a contract pending and  needed to be verified in order to receive the contract could be manually  processed on a case-by-case basis. More recently, VCMS was  unavailable from May 9 to June 6, 2012, during which time applicants  could not submit new applications, and staff could not request additional  documentation through the system. This outage was caused by a security  problem that was identified in routine testing by OI&T as VA was  preparing to launch a modification to the system.

VA is in the process of planning to either modify or replace the current  version of VCMS to address the identified shortcomings, but this planning  effort has not been tied to broader long-term strategic planning for the  verification program. VA officials have identified elements that the next  iteration of VCMS should include. For example, the officials would like the  program to automate some aspects of the background company research  and generally make the verification process less burdensome on  veterans. Following the program outage in May 2012, verification program  officials began reaching out to OI&T for assistance in overseeing both the  current system and a potential modification or replacement. These  discussions received further emphasis through the previously discussed  senior executive task force, which included representatives of OI&T. As a  result of an expected recommendation by the task force, OI&T assigned  staff in July 2012 to begin formally planning for either a modification or a  replacement system, a process that OI&T will manage. VA is considering  short- and long-term information needs as it defines the business  requirements for the system. But, as we have seen, the initial strategic  plan that OSDBU developed in late October 2012 does not specify  longer-term goals for the verification program or define program  strategies and activities beyond 2013. Without tying the effort to modify or  replace the verification program’s data system to more comprehensive,  long-term strategic planning, the resulting system risks again failing to  meet the verification program’s long-term needs and goals.

section_title: Expanding Its Verification Program Government-wide Would Require VA to Improve the Program and Address Policy Issues
paragraphs: Expanding VA’s verification program to support the government-wide  SDVOSB contracting program would require VA to increase the scale of  its program to verify potentially thousands of additional firms. VA has  faced ongoing challenges implementing its verification program, and it  would need to continue to stabilize and improve its verification operations  by addressing remaining vulnerabilities to fraud and abuse,  demonstrating whether recent operational changes have resulted in  improved performance and whether new methods for educating  applicants are effective, and addressing data system limitations. Also,  as VA revises its verification program regulation, it is considering policy  issues that would impact a government-wide verification program.

VA has not formally projected how many firms it might need to verify  under a government-wide SDVOSB verification program, and a number of  factors make such a projection difficult. For example, the scale of a  program would depend on whether firms would be required to obtain  verification to bid on contracts or only to receive contract awards, how  likely firms that are already self-certified as SDVOSBs would be to seek  verification if it were required, and how many new or existing SDVOSBs  that have not yet self-certified might seek verification in the future.

The estimate of 12,800 firms is based on the number of self-certified SDVOSBs (i.e.,  registered in CCR but not yet verified by VA as of Sept. 30, 2012) that did not receive  contract obligations in fiscal years 2010 or 2011 (the last full fiscal year available),  according to FPDS-NG. future. As a result, predicting how many would actually be motivated  to seek verification if it were required is difficult.

Beyond firms that have already registered as prospective federal  contractors, thousands of existing or new SDVOSBs could eventually  register and seek verification if it were required. We did not identify a  current estimate of the number of SDVOSBs in the United States.  However, an SBA analysis of data from the Census Bureau’s 2007  Survey of Business Owners found that there were around 200,000  service-disabled veteran-owned businesses (of any size) at that time.

Considering the additional operational challenges that VA would face in  preparing to verify potentially thousands of additional firms, VA would  need to continue to address existing program weaknesses to stabilize  and improve its verification program. Our prior and current work indicates  that several aspects of VA’s current verification program, specified below,  would have to be addressed before the program could be effectively  implemented government-wide.

U.S. Small Business Administration, Office of Advocacy, Veteran-owned Businesses  and their Owners—Data from the Census Bureau’s Survey of Business Owners  (Washington, D.C.: March 2012). SBA reported that there were more than 2.1 million  nonservice-disabled veteran-owned businesses (of any size) in 2007. debarments and prosecutions of firms found to be misrepresenting  their SDVOSB status, had not been implemented. As of January 4,  2013, we were reviewing documentation that VA had recently  provided to determine whether VA’s actions are sufficient to consider  some of the recommendations implemented.

Operations. A major expansion of the verification program would have  a greater chance at success if its priorities and operations were more  stable and if the recent changes that VA adopted were shown to have  improved the program’s performance. For example, the steps VA has  taken to standardize its procedures and make them more efficient,  improve its quality assurance process, and enhance training of CVE  employees and contractors are promising. However, it is too soon for  us to test the effectiveness of these evolving procedures, and it is not  clear whether VA will adopt further significant changes as a result of  the recommendations of the senior executive task force reviewing the  verification program.

Applicant Education. Because a government-wide program would  potentially affect thousands of additional firms, VA would need to have  in place effective methods for educating business owners about the  program and for obtaining and responding to their feedback. VA  officials suggested that the agency’s recent efforts to clarify online  guidance for applicants and to partner with organizations to better  educate applicants about the verification requirements were intended to  help firms understand the rationale for the required documentation and  explain how VA interprets the documents submitted in making its  determinations. The officials described plans for collecting the data they  would need to evaluate these efforts, which included assessing  whether denial rates differed for firms that used the online guidance or  received assistance from a partner organization and those that did not.

Information technology. As we noted earlier, the limitations of the  verification program’s information system (VIP/VCMS) have  hampered VA’s ability to effectively manage, monitor, and report on  the program’s operations and results. Addressing these limitations by,  for example, ensuring that the information system collects the data  needed to monitor the consistency and accuracy of VA’s  determinations, allows customized reporting to meet managers’  needs, and supports efficient workflow management would also help  position VA to manage an expanded government-wide program.  Furthermore, CVE and OI&T officials said that, in planning to modify  or replace VIP/VCMS, they were factoring in the potential need for the  system to have the capacity and flexibility to expand to a government- wide scale and to be adapted for automated interagency information  sharing. For example, the officials said they were planning to consider  how to enable contracting officers from other agencies to determine  whether an SDVOSB was verified without having to manually search  for the firm in VIP.

In addition, VA has begun a process to revise the verification program’s  regulations, which would likely serve as the starting point if VA were  charged with implementing a government-wide verification program. VA  officials said that they were planning to revise the regulations partly in  response to applicants’ and veterans’ organizations concerns about VA’s  eligibility standards. For example, two veterans’ organizations questioned  VA’s regulatory requirement that veteran owners be able to transfer their  ownership interest without restriction by nonveteran owners, effectively  suggesting that VA’s standard for establishing control of a firm is too  strict. The organizations stated that because nonveteran owners might  reasonably expect to have a say in such transfers, the requirement limited  the ability of SDVOSBs and VOSBs with nonveteran minority owners to  participate in the Veterans First program. VA officials said that they would  weigh this and other concerns as they developed proposed revisions to  the regulation, a process that they expected to result in a final rule by  mid-2014.

Any changes to VA’s verification requirements could create or widen  differences between the various government-wide small business  contracting programs’ requirements and VA’s, a consideration that would  likely be of even greater importance if VA’s verification program were  expanded. In addition to the government-wide SDVOSB program, federal  contracting preference programs give federal agencies the authority to set  aside contracts for small business concerns and specific types of small  businesses: women-owned small businesses, businesses located in  historically underutilized business zones (HUBZone), and socially and  economically disadvantaged small businesses participating in SBA’s 8(a)  program. While the SDVOSB and women-owned small business  programs allow firms to self-certify their eligibility, SBA reviews supporting  documentation to certify HUBZone and 8(a) firms, with the 8(a) program  requiring more extensive documentation similar to what is required under  VA’s verification program. (See app. IV for a description of these  programs and their verification requirements.) Some veterans’  organizations and others with whom we spoke have cited perceived  differences between VA’s eligibility standards and SBA’s standards for  the government-wide SDVOSB program and the 8(a) program, whose  certification process is most similar to VA’s verification program.

However, VA and SBA officials worked together to compare the three  programs’ regulations and VA’s and the 8(a) program’s documentation  requirements. Initially, VA and SBA officials told us that they did not find  major differences in the programs’ regulatory eligibility requirements, the  agencies’ interpretation of them, or the documentation requirements for  verification. In commenting on a draft of this report, SBA subsequently  stated that, while the wording of the regulations pertaining to eligibility  requirements was comparable, there was a distinction regarding  ownership by spouses of disabled veterans. SBA also stated in its  comment letter that there were some key differences in how the agencies  interpreted the regulations and that the agencies were consulting with one  another to determine whether those differences could or should be  resolved. Going forward, if VA adopts unilateral changes to its verification  policies and procedures, these changes could have the effect of making it  more difficult to align the programs. VA officials told us that the tension  between competing calls for VA to ease its requirements and to be  consistent with the government-wide SDVOSB and 8(a) programs would  be a major consideration as VA considered changes to its regulations— particularly considering the potential for a government-wide SDVOSB  verification program. Accordingly, the officials said that they were  consulting with SBA as they began to develop proposed changes to VA’s  verification program regulation.

section_title: Conclusions
paragraphs: The opportunity to receive set-aside or sole-source contract awards under  the Veterans First program is a significant benefit that provides billions of  dollars in contracts annually to SDVOSBs and VOSBs. As a result, the  program warrants strong internal controls to provide reasonable  assurance that the contracts VA enters into are awarded to eligible firms.  At the same time, an inherent tension exists between the need for  effective internal controls and the Veterans First program’s goal of  increasing contracting opportunities for SDVOSBs. If VA fails to correctly  verify eligible firms, or if firms’ concerns about the verification process  deter them from applying, VA’s ability to sustain its high levels of  contracting with SDVOSBs and VOSBs could ultimately be at risk. VA has  made progress toward reducing its vulnerability to fraud and abuse, and  CVE’s new management team has initiated a variety of operational  changes in an effort to improve the program. VA has also initiated efforts  to develop a comprehensive strategic plan for the verification program.  This initial strategic planning effort represents a positive step that appears  to have at least partially applied key leading federal strategic planning  practices. However, the initial plan includes only goals intended to be met  within 2 years, and many of the performance measures focus on the  implementation rather than the outcomes of activities. Additionally, VA  has not shared the plan with key stakeholders. As it continues to develop  and refine its strategic plan, VA could strengthen its effort by ensuring  that the plan articulates results-oriented, long-term goals and objectives  for the verification program, that the metrics are focused on outcome  measurements that can be used to monitor the verification program’s  performance and demonstrate results, and that key stakeholders are  involved in evaluating the plan.

The initial lack of a comprehensive strategic plan for the verification  program has also contributed to the development of a data system that  has proven to be inadequate. The system does not collect data for  monitoring program trends and staff performance, has limited reporting  and workflow management capabilities, and has been unable to accept  applications for extended periods, hindering VA’s ability to operate and  monitor the verification program. VA has started taking steps to address  the shortcomings in the data system by shifting responsibility for  developing plans to enhance or replace VCMS from CVE to OI&T. But  without tying that effort to long-term strategic planning, VA risks failing to  meet the program’s information needs going forward.

As VA revises its verification program regulations and considers the  relationship between its policies and those of other federal small business  contracting preference programs, the agency faces a tension between  competing calls to reduce the burden on applicants and to be vigilant in  preventing and detecting fraud. This tension would underlie a  government-wide SDVOSB verification program as well. Addressing  these policy issues for its own program—or ultimately for a government- wide verification program—will require VA to weigh certain tradeoffs.  These include deciding how to reduce the administrative burden that the  verification process places on eligible firms and maintain sufficient fraud  prevention and detection controls to provide reasonable assurance that  the billions of VA contract dollars set aside for SDVOSBs and VOSBs  reach their intended beneficiaries.

section_title: Recommendations for Executive Action
paragraphs: To improve the management and oversight of VA’s SDVOSB and VOSB  verification program, we recommend that the Secretary of Veterans  Affairs take the following two actions:

Direct OSDBU to continue to develop, refine, and implement a formal  strategic plan to provide a comprehensive framework to guide,  integrate, and monitor the verification program’s activities over time.  As OSDBU refines the strategic plan, it should incorporate longer- term goals and objectives for the verification program. The plan  should also incorporate outcome measures that OSDBU can use to  better monitor the verification program’s progress and demonstrate its  results. OSDBU should also share the plan with key stakeholders.

Direct OSDBU and OI&T, as they modify or replace the verification  program’s data system, to integrate their efforts with OSDBU’s  broader strategic planning effort for the verification program to ensure  that the new system not only addresses the short-term needs of the  program but also can be readily adapted to meet longer-term needs.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of Veterans Affairs  and the Small Business Administration for comment. In its written  comments, VA generally agreed with GAO’s conclusions and concurred  with the two recommendations. VA stated that it had actions under way  that would address each recommendation. VA indicated that it anticipated  submitting a strategic plan to the Office of the Secretary in fiscal year  2013 and would develop a schedule to brief VA senior leaders and other  key stakeholders once the plan is approved. VA also provided additional  information about its efforts to replace the verification program’s data  system. VA noted that it had begun the process of replacing the existing  system and had developed a work statement for the replacement system.  VA also provided technical comments that we incorporated as appropriate  into the report. In its technical comments, VA disagreed with the status of  some of the prior GAO recommendations that we noted had not been fully  implemented, including the provision of regular fraud awareness training  and unannounced random and risk-based audits of verified firms to  ensure compliance with the program rules. We have revised the report to  indicate that, as of January 4, 2013, we were reviewing documentation  provided by VA in December 2012 to determine if VA’s actions taken to  address some of our prior recommendations are sufficient to consider  them implemented. We also noted that we will continue to review  documentation provided by VA in the future to assess whether the  remaining recommendations have been implemented.

In its written comments, SBA provided additional information on its views  on eligibility requirements for VA’s Veterans First Contracting Program, the  government-wide SDVOSB contracting program, and the 8(a) program. In  particular, SBA stated that a statement in our draft report was not  accurate—specifically, our comment that VA and SBA did not find major  differences in the programs’ eligibility requirements, the agencies’  interpretation of the requirements, or the documentation required for  verification. SBA noted that, statutorily, surviving spouses of disabled  veterans might be eligible for VA verification but that they were not eligible  under SBA’s regulations for the government-wide SDVOSB program. SBA  also noted that it provided an avenue of appeal through its SDVOSB status  protest and 8(a) eligibility processes but that VA did not have a similar  appellate procedure. Finally, SBA stated that the wording of the regulations  pertaining to VA’s and SBA’s eligibility requirements was similar but that  there were some key differences in interpretation that the two agencies  were reviewing. We have revised our discussion of VA’s and SBA’s effort  to compare the programs’ eligibility and documentation requirements, citing  the difference noted by SBA with respect to the eligibility of surviving  spouses and noting that the agencies were consulting with each other to  determine whether differences of interpretation could or needed to be  resolved. We also added clarifying language in appendix I describing how  we obtained information on VA and SBA efforts to compare program  regulations. In addition, we clarified the differences between SBA’s and  VA’s status protest mechanisms in appendix IV. VA’s and SBA’s comments  are reprinted in appendixes V and VI.

We are sending copies of this report to the appropriate congressional  committees, the Administrator of SBA, the Secretary of Veterans Affairs,  and other interested parties. In addition, the report will be available at no  charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix VII.

section_title: Appendix I: Scope and Methodology
paragraphs: Our objectives were to (1) describe and assess the progress that the  Department of Veterans Affairs (VA) has made in establishing a program  to verify the eligibility of service-disabled veteran-owned small businesses  (SDVOSB) and veteran-owned small businesses (VOSB) on a timely and  consistent basis, and (2) describe the key operational and policy issues  that VA would need to address should its verification program be  implemented government-wide. Because VA was introducing significant  changes to its procedures and operations at the time of our study, we  determined that evaluating VA’s compliance with its past procedures  would be of limited value and that testing the effectiveness of verification  procedures that were still evolving would be premature. We focused  instead on issues related to planning for and designing the verification  program and on changes in its management and operations that the  Center for Veterans Enterprise (CVE) instituted since December 2011.

75 Fed. Reg. 6098 (Feb. 8, 2010); 77 Fed. Reg. 38181 (June 27, 2012).

November 1, 2011, and September 30, 2012, and VA’s Verification  Master Inventory List, a manually maintained inventory of all verified firms  that VA uses to supplement VCMS. We assessed the data by  interviewing knowledgeable VA officials, reviewing related documentation,  and checking the data for illogical values or obvious errors and found  them to be sufficiently reliable for the purpose of illustrating general  characteristics of the verification program. We also interviewed officials  from three of the contractors who perform aspects of the verification  process—GCC Technologies, LLC; HeiTech Services, Inc.; and Addx  Corporation—to understand their roles in the verification program, and  representatives from three veteran service organizations and a technical  assistance association that were participating in the Verification  Counseling program–VETForce, American Legion, National Veteran  Small Business Coalition, and Association of Procurement Technical  Assistance Centers—to discuss their views on the verification program  and their expectations of the Verification Counseling program.

Pub. L. No. 103-62 (August 3, 1993); Pub. L. No. 111-352 (Jan. 4, 2011). GPRAMA  provides federal agencies with an approach to focusing on results and improving  government performance by, among other things, developing strategic plans. Examples of  GPRAMA plan components include a mission statement; general goals and objectives,  including outcome-oriented goals; and a description of how the goals and objectives are to  be achieved, including the processes and resources required. leading practices that we had previously identified as being relevant to  agencies’ initial strategic planning efforts. We reviewed a strategic  planning document that OSDBU compiled in 2012 in response to our  study, and six planning documents prepared between June 2011 and  December 2011 that OSDBU officials said provided the basis for the  strategic plan. We compared these documents, and the planning activities  associated with them, to the six leading practices, as shown in table 1.  Because VA prepared the initial strategic planning document as we were  completing our draft report, we did not conduct a comprehensive review  of the strategic plan, the supporting documents that VA provided, or the  process that VA undertook to develop these documents.

We also assessed the extent to which the verification program’s data  system provided the information needed to run and control the verification  program’s operations, a key standard for effective internal controls. In  particular, we focused on the timely availability of pertinent information  sufficient to enable people to carry out their duties efficiently and  effectively, a factor that we previously identified as important in assessing  this standard. We reviewed data system documentation and reports that  the system produces and interviewed officials from VA and the  contractors that perform aspects of the verification process to determine  how the data system was developed and how VA uses it, and to identify  the capabilities and limitations of the data system.

For verified SDVOSBs that did not appear as self-certified in CCR as of March 2012, we  cross-referenced the Small Business Administration’s (SBA) Dynamic Small Business  Search, which includes supplemental information on registered firms that meet SBA’s size  standard for the firms’ industries. under VA’s existing program regardless of whether a government-wide  program was adopted. In addition, we could not determine how many of  these firms have been actively seeking contracts or how likely they would  be to do so in the future, making it difficult to predict how many would  actually be motivated to seek verification if it were required. We assessed  these data by interviewing VA officials knowledgeable about the VA data,  reviewing documentation related to all of the data systems, and checking  the data for illogical values or obvious errors and found them to be  sufficiently reliable for the purpose of illustrating the potential scale of a  government-wide verification program. We also reviewed our prior work on  the verification program and that of the VA Office of Inspector General, as  well as our assessment of the current status of the program, to identify  issues that VA would need to address in implementing a government-wide  program. Because of the Small Business Administration’s (SBA) role  administering the government-wide SDVOSB program, we also interviewed  VA and SBA staff about how the statutory and regulatory provisions  implemented by the two agencies compare. In addition, we reviewed SBA  documents and interviewed SBA staff for their views on a potential  government-wide verification program. However, the SBA staff said that it  would be inappropriate for them to comment on VA’s or SBA’s potential  roles or other considerations in implementing a potential program.

For both objectives we interviewed officials in VA’s Office of Small and  Disadvantaged Business Utilization (OSDBU), CVE, Office of the General  Counsel, and the Office of Information and Technology to understand their  historical, current, and expected roles in the verification program. We also  reviewed prior GAO reports and a VA Office of Inspector General report on  the verification program and testimonies from congressional hearings on  the government-wide SDVOSB program and VA’s verification program.

We conducted this performance audit from February 2012 to January  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: Verification Program Data as of September 30, 2012
paragraphs: We reviewed VA’s database known as the Verification Case Management  System (VCMS) to obtain data on the status of initial applications,  requests for reconsideration, and applications for reverification submitted  to VA between November 1, 2011, and September 30, 2012. We chose  this period because fiscal year 2012 roughly coincided with the December  2011 to November 2012 period that was the focus of our work. Because  we were primarily interested in the progress that VA had made  processing applications that were submitted during the period that was  the focus of our work, we excluded from our analysis applications VA  processed during the period but that were submitted prior to November 1,  2011. We used these data to determine the volume of applications that  VA received between November 1, 2011, and September 30, 2012, and  their status (pending, withdrawn, approved, or denied) as of September  30, 2012. Because our analysis included applications that had been  submitted less than 90 days ago, we expected a significant number of the  cases to be pending as of September 30, 2012.

Based on our analysis of VCMS data, VA received approximately 4,900  initial applications between November 2011 and September 2012. The  monthly volume of initial applications fluctuated during this period, with  VA receiving an average of about 450 initial applications per month. As  shown in figure 3, approximately 14 percent of the 4,900 initial  applications submitted during the period were pending a determination as  of September 30, 2012, and another 43 percent had been withdrawn.

The remaining 43 percent of applications had received a determination  and, of these, 61 percent were approved and 39 percent were denied.

Applicants can withdraw their applications at any time, or VA can withdraw an application  if the applicant does not respond to requests to provide missing or additional requested  documentation within 30 days.

section_title: Appendix III: Verification Program Organizational Structure, December 2011 and October 2012
paragraphs: Between December 2011 and October 2012, VA revised the  organizational structure for the verification program. As shown in figures 6  and 7, VA officials (1) reorganized and increased the number of  employees and contractors assigned to the verification process and (2)  created several new teams including quality assurance, training, records  management, and customer service. As of October 2012, the verification  program had about 28 full-time equivalent federal employees and 174  contractors, an increase of about 3 full-time equivalent staff and 64  contractors to the verification program since December 2011.

section_title: Appendix IV: Verification or Certification Requirements Associated with Federal Small Business Contracting Preference Programs
paragraphs: All federal agencies have the authority to set aside contracts for small  business concerns and for several specific types of small businesses:  SDVOSBs, women-owned small businesses, businesses located in  historically underutilized business zones (HUBZone), and socially and  economically disadvantaged small businesses participating in SBA’s 8(a)  program (table 2). Some programs are also authorized to make sole- source awards to these groups. For the government-wide SDVOSB  program, business owners are required only to certify their eligibility  online in the System for Award Management (SAM) and do not need to  submit any supporting documentation. SBA does not verify the eligibility  of these firms. Women-owned small businesses may obtain certification  by an entity approved by SBA or self-certify their eligibility online in SAM;  in either case, the firms must upload supporting documents to SBA’s  online Women-Owned Small Business Program Repository for potential  review by contracting officers or SBA. In contrast with these self- certification programs, SBA must certify firms’ eligibility to receive  contracts under the HUBZone and 8(a) programs. SBA reviews  supporting documentation to certify HUBZone and 8(a) firms, with the  8(a) program requiring more extensive documentation that is similar to  that required by CVE for its verification program. For each of the  government-wide small business contracting preference programs except  for the 8(a) program, SBA provides a “status protest” mechanism for  interested parties to a contract award to protest if they feel a firm  misrepresented its eligibility in its bid submission. SBA’s status protest  mechanism for the SDVOSB and women-owned small business  programs and its certification process for the 8(a) program also provide  interested parties with an avenue of appeal to SBA’s Office of Hearings  and Appeals. However, VA’s OSDBU decides any SDVOSB or VOSB  status protests arising from a VA solicitation.appellate procedure for such decisions.

section_title: Appendix V: Comments from the Department of Veterans Affairs

section_title: Appendix VI: Comments from the Small Business Administration

section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Harry Medina (Assistant  Director), Emily Chalmers, Pamela Davidson, Julianne Dieterich, Julia  Kennon, Cory Marzullo, John McGrail, Daniel Newman, Jena Sinkfield,  James Sweetman, and William Woods made key contributions to this  report.


Question-summary annotations: questions: question: What is the current status of VA's verification processes?
answer: The Department of Veterans Affairs (VA) has made significant changes to its verification processes for service-disabled and other veteran-owned small businesses to improve operations and address program weaknesses, but continues to face challenges in establishing a stable and efficient program to verify firms on a timely and consistent basis.
child_questions: question: In response, what operational changes has the VA made?
answer: Since December 2011, VA has instituted a number of significant operational changes, including revising standard operating procedures and enhancing quality assurance protocols for its verification program.
child_questions: question: What problems exist in these changes?
answer: However, GAO found that VA did not have a comprehensive, long-term strategic plan for the program and had prioritized addressing immediate operational challenges, contributing to programmatic inefficiencies.

question: How is OSDBU working to improve the verification program?
answer: In response to this observation, VA's Office of Small and Disadvantaged Business Utilization (OSDBU) initiated action in late October 2012 to compile a strategic planning document that encompassed the verification program.
child_questions: question: To what extent has OSDBU's work been successful?
answer: VA's OSDBU appears to have partially applied key leading strategic planning practices in its initial planning effort. But the plan lacks performance measures to assess whether the desired outcomes are being achieved and has a short-term focus that is not typically associated with a strategic plan.

question: What other problems exist with the plan?
answer: VA also has not shared the plan with key stakeholders, including congressional staff. Further, the verification program's data system has shortcomings that have hindered VA's ability to operate, oversee, and monitor the program. Among other things, the system does not collect important data and has limited reporting and workflow management capabilities.
child_questions: question: What plans does the VA have regarding the system?
answer: VA plans to modify or replace the system, but has not directly tied this effort into its long-term strategic planning efforts to ensure that the new system meets the verification program's long-term information needs.

questions: question: How are service-disabled and veteran-owned small businesses helped by VA?
answer: VA is required to give contracting preference to service-disabled and other veteran-owned small businesses.
child_questions: question: Before contracting, what must the VA do regarding the ownership of these businesses?
answer: It must also verify the ownership and control of these firms to confirm eligibility.

question: What did reviews conclude about the program?
answer: Prior reports by GAO and VA's Office of Inspector General identified weaknesses in VA's processes and controls that allowed ineligible firms to be verified.

question: What did GAO assess in their report?
answer: For this report, GAO assessed (1) VA's progress in establishing a program for verifying firms' eligibility on a timely and consistent basis and (2) key operational and policy issues that VA would have to address should its verification program be implemented government-wide.
child_questions: question: In the creation of this report, what data did the GEO review, analyze, and collect?
answer: GAO reviewed VA's policies and procedures; compared its initial strategic planning effort with previously identified leading strategic planning practices; interviewed VA officials and veterans' organizations; and analyzed government-wide contracting databases.