Veterans
 

Benefits
 

Administration
 

Audit of the Veterans
 

Service Network
 

OFFICE OF AUDITS & EVALUATIONS
 

February 18, 2011 

09-03850-99 
ACRONYMS AND ABBREVIATIONS
 

ATT Application Turnover Transmittals 

BDN Benefits Delivery Network 

GAO Government Accountability Office 

IMS Integrated Master Schedule 

IT Information Technology 

PMAS Program Management Accountability System 

SEI Carnegie Mellon Software Engineering Institute 

VETSNET Veterans Service Network 

To Report Suspected Wrongdoing in VA Programs and Operations:
 

Telephone: 1-800-488-8244
 

E-Mail: vaoighotline@va.gov
 

(Hotline Information: http://www.va.gov/oig/contacts/hotline.asp)
 
Report Highlights: Audit of the Veterans 

Service Network 

Why We Did This Audit 

Since 1996, VA has been working to 

consolidate compensation and pension 

benefits processing into a single replacement 

system, called the Veterans Service Network 

(VETSNET). We conducted an audit of 

VETSNET to determine whether: effective 

controls have been implemented to address 

previously identified program governance 

deficiencies; schedule, cost, and 

performance goals for the program have 

been met; and effective change controls 

have been implemented to support the 

planning, testing, and implementation of the 

VETSNET suite of applications. 

What We Found 

VA has addressed prior program governance 

deficiencies by establishing oversight 

groups, risk management processes, and 

software development gate reviews to 

provide greater visibility and control of 

VETSNET program activities. Despite 

these improvements, VETSNET faces the 

continuing challenge of managing 

competing mandates and new systems 

initiatives that have repeatedly changed the 

scope and direction of the program. 

Such changes have adversely impacted 

achieving schedule, cost, and performance 

goals over the life of the VETSNET 

program. Specifically, work to meet 

original program objectives has been 

delayed and consolidation of compensation 

and benefits processing has been extended 

by nearly 5 years since 2006. In 2006, the 

total cost projection for VETSNET was 

$151 million through 2009. In 2009, VA 

reported a revised cost estimate of 

$308 million through 2012, more than two 

times the previous amount. Planned system 

functionality enhancements also remain 

unaddressed. 

Moreover, the competing priorities have 

resulted in changing business requirements, 

necessitating additional software releases to 

meet those requirements. Because software 

change controls and testing have not been 

adequate to ensure proper system 

functionality, software rework and rollback 

of installation packages have been required 

to correct defects, and planned functionality 

enhancements have been delayed. 

What We Recommend 

We recommend the Assistant Secretary, 

Office of Information and Technology, 

clarify goals, align resources, and establish a 

schedule for accomplishing the goals of 

VETSNET in the near term. We also 

recommend that improved processes be 

implemented to address software 

development deficiencies. 

Agency Comments 

The Assistant Secretary for Information and 

Technology and the Acting Under Secretary 

for Benefits agreed with our findings and 

recommendations. The OIG will monitor 

implementation of the action plans. 

 (original signed by:) 

BELINDA J. FINN
 

Assistant Inspector General
 

for Audits and Evaluations
 

i 
Table of Contents
 

INTRODUCTION ...........................................................................................................................1
 

RESULTS AND RECOMMENDATIONS.....................................................................................2
 

Finding 1 VETSNET Governance Improvements...............................................................2
 

Finding 2 VETSNET Schedule, Cost, and Performance Objectives Not Being Met..........6
 

Finding 3 Software Development, Testing, and Implementation Need Improvement......12
 

Appendix A Background ...................................................................................................... 17
 

Appendix B Scope and Methodology................................................................................... 19
 

Appendix C VETSNET Oversight, Development, and Implementation.............................. 21
 

Appendix D Development Phases and Gate Review Process............................................... 23
 

Appendix E Assistant Secretary for Information and Technology Comments.................... 25
 

Appendix F Acting Under Secretary for Benefits Comments ............................................. 30
 

Appendix G Projected Monetary Benefits in Accordance with IG Act Amendments ......... 32
 

Appendix H OIG Contact and Staff Acknowledgments....................................................... 33
 

Appendix I Report Distribution .......................................................................................... 34
 

ii 
Audit of the Veterans Service Network 

Objective 

Overview 

INTRODUCTION 

We conducted an audit of the Veterans Service Network (VETSNET) 

program to determine whether effective controls have been implemented to 

address program governance deficiencies identified in prior VETSNET 

assessments; schedule, cost, and performance goals for the program have 

been met; and effective change controls have been implemented to support 

planning, testing, and implementation of the VETSNET suite of applications. 

Since 1996, VA has focused on replacing the compensation and pension 

program functions within its aging Benefits Delivery Network (BDN) with a 

system called VETSNET. VA is developing VETSNET as a suite of 

applications to facilitate the administration of benefit programs and provide 

users with the data needed to make informed business decisions. Automated 

data exchanges with connecting systems to form “One VA” system will 

allow for faster retrieval of information critical to adjudicate and process 

benefit claims. Appendix C lists the numerous VETSNET stakeholder 

groups that are responsible for project oversight and development and 

ensuring the applications meet end-user requirements. 

Over the past 14 years, VA’s system development efforts have resulted in 

substantial improvements in the functionality and performance of VETSNET 

applications. In April 2008, VETSNET became the core business 

application and payment system for compensation benefits. Because of the 

functionality enhancements, more than four million veterans now receive 

compensation and pension payments through VETSNET. VA estimates this 

represents 99 percent of all veterans’ compensation claims and 94 percent of 

all veterans’ pension claims. Completing VETSNET development to process 

the remaining claims is a continuing challenge. 

Concerned about the slow pace of VETSNET development, VA contracted 

with the Carnegie Mellon Software Engineering Institute (SEI) in 2004 to 

perform an independent evaluation and provide recommendations for 

program management and process improvements.1 In 2006, the Government 

Accountability Office (GAO) evaluated VETSNET and noted that VA was 

generally implementing SEI recommendations while taking action to address 

identified weaknesses in overall management and software development.2 

1 Kathryn Ambrose; William Novak; Steve Palmquist, PE, PMP; Ray Williams (Team Lead); and 

Dr. Carol Woody, Report of the Independent Technical Assessment on the Department of Veterans 

Affairs VETSNET Program (Carnegie Mellon Software Engineering Institute, September 2005). 

2 Information Technology: Veterans Benefits Administration: Progress Made in Long-Term Effort to 

Replace Benefits Payment System, but Challenges Persist, U.S. Government Accountability Office 

(GAO-07-614, April 27, 2007). 

VA Office of Inspector General 1 
Audit of the Veterans Service Network 

RESULTS AND RECOMMENDATIONS 

Finding 1 VETSNET Governance Improvements 

VA has addressed prior program governance deficiencies by establishing 

oversight groups, risk management processes, and software development 

gate reviews to provide greater visibility and oversight of VETSNET 

program activities. Despite these improvements, the VETSNET Program 

Management Office faces the continuing challenge of managing competing 

external priorities and overlapping system development initiatives that have 

repeatedly changed the scope and direction of the program. 

Program 

Governance 

Improvements 

Risk Management 

Improvements 

GAO stated that since 2004, VA has taken a number of steps to improve 

VETSNET program management. SEI previously reported that stakeholders 

had not taken ownership responsibility for the project and did not 

comprehend total system and process operating costs. In response, VA 

developed a new governance structure for VETSNET, which the Under 

Secretary for Benefits approved in March 2006. This governance structure 

established a process for information technology (IT), business lines, and 

regional offices to share ownership and management. Within the new 

structure, the VETSNET Executive Board was expanded and reorganized to 

serve as a focal point and major governance mechanism. A number of teams 

at different levels were also established to manage the day-to-day activities 

associated with developing and implementing VETSNET. Appendix C lists 

these VETSNET oversight, development, and implementation entities. 

To further address governance issues, VETSNET has been subjected to 

Program Management Accountability System (PMAS) oversight. VA’s Chief 

Information Officer implemented PMAS to address IT performance 

management shortcomings. The goal of PMAS is to provide near-term 

visibility for troubled programs, allowing for earlier assistance and helping 

avoid long-term project failures. Under PMAS, software development 

projects must deliver smaller, more frequent releases of new functionality to 

customers. Since the adoption of PMAS, VA has implemented incremental 

releases of VETSNET every 6 months. The scope of functionality 

enhancements with each release has been reduced to meet the timelines for 

deliverables. 

Improvements have been made in the risk management area. SEI previously 

reported that documentation of VETSNET risk management activities was 

conducted as a formality and had minimal impact on program decisions. In 

response to SEI’s concerns, the VETSNET team, with contractor support, 

developed a risk management plan that was adopted in January 2007. The 

plan includes procedures for identifying, analyzing, and developing 

mitigation strategies for reporting and closing project risks. During 

VA Office of Inspector General 2 
Audit of the Veterans Service Network 

Competing 

Requirements 

Overlapping IT 

Projects 

fieldwork, the VETSNET Integrated Project Team met periodically to 

identify project risks and mitigation strategies, with special attention given to 

risks directly impacting current project activities. These status meetings 

addressed various risks including technical, programmatic, and supportability 

risks, as well as risks associated with cost, schedule, and staffing. Some 

resulting risk mitigation strategies included realigning resources to meet 

near-term priorities for software releases and reducing the planned scope of 

software functionality to meet established project timelines. 

Despite the improvements made, the VETSNET Program Management 

Office faces the challenge of managing competing external priorities and 

legislative requirements that have repeatedly altered the scope and direction 

of the VETSNET program. According to VETSNET program management 

officials, the new priorities and mandates, introduced to meet legitimate 

system needs, have required unplanned additions or adjustments to 

VETSNET functionality. Several major VETSNET releases that VA 

developed and deployed to address the emerging functionality requirements 

for benefit claims processing are as follows. 

	 A February 2009 VETSNET release included enhanced functionality for 

the Disability Evaluation System pilot program, which supports VA and 

Department of Defense interoperability agreements. Full implementation 

of this system will facilitate completion of disability ratings while service 

members are on active duty. 

	 An August 2009 VETSNET release provided new capabilities to furnish 

benefit payments under the Concurrent Retirement Disability Payment 

legislative mandate, a joint VA and Department of Defense project 

benefiting retired veterans. 

	 An August 2009 VETSNET release implemented new functionality to 

provide $250 in benefit payments to approximately 2.8 million recipients 

under the American Recovery and Reinvestment Act. 

	 Additional VETSNET releases implemented new functionality to deliver 

$198 million in benefits payments in accordance with Filipino Veterans 

Equity Compensation Act requirements, as well as to meet regulatory 

requirements under the Veterans Claims Assistance Act. 

As a further complication, VA has launched several high profile IT 

initiatives, such as the Paperless Veterans Benefit Management System, the 

Post-9/11 GI Bill Education Benefits Program, and the Chapter 31 

Vocational Rehabilitation and Employment Program that need to leverage 

VETSNET to make benefit payments. Several of these new IT initiatives 

entail implementing service-oriented architectures to provide Web interfaces 

for end users. Systems interdependencies require that some VETSNET 

applications, currently client-server-based, will have to be re-hosted as well. 

VA Office of Inspector General 3 
Conclusion 

Audit of the Veterans Service Network 

Such interdependencies increase the risk that VA may experience confusion 

in program priorities and delays in achieving the original program goals. 

Because of the many organizational units involved with these initiatives, 

VETSNET Executive Team and program management officials stated that 

they were not well informed of the status of other projects and their potential 

impacts on the VETSNET program. Accordingly, VA needs to develop a 

means of managing the resources, activities, and timing for addressing the 

complexities and functionality requirements imposed by the overlapping IT 

projects in a coordinated manner. 

Developing a fully Integrated Master Schedule (IMS) is one way to help 

meet this need. SEI previously reported that the VETSNET program lacked 

an IMS that would incorporate all critical areas of software development and 

identify the extent of work to be completed, as well as measure the progress 

of work performed. In response to the SEI report, the VETSNET Executive 

Team developed a high-level IMS, which is supported by some detailed 

work breakdown structures for VETSNET major releases. The most recent 

IMS, covering FYs 2010 to 2012, shows the timing of VETSNET major 

software releases, data conversion efforts, and other system development 

efforts that will leverage VETSNET to make benefit payments. 

However, due to the near-term focus of the VETSNET program, the highlevel IMS does not reflect all system interdependencies that could adversely 

impact the estimated completion dates for VETSNET major milestones. For 

example, the VETSNET Executive Team could not provide detailed work 

breakdown structures for other IT initiatives that will use common resources 

to support their system development efforts. 

Some program management officials acknowledged that program 

management resources were already being used to test Chapter 33 interface 

development, creating constraints on meeting some near-term goals for 

VETSNET. Further, it was indicated at a May 2010 VETSNET Integrated 

Project Team meeting that, given competing IT priorities, the scope of 

software changes for VETSNET Version Release 9 may need to be reduced 

to meet established software release timelines. 

VA has taken positive steps forward in addressing previously identified 

management deficiencies internal to the VETSNET program. However, such 

improvements have not been the solution to managing the external 

challenges imposed by competing priorities and IT interdependencies that 

have changed the focus and direction of VETSNET. Effectively integrating 

new requirements and managing interdependencies with IT development 

initiatives that rely on VETSNET resources will be key to sustained 

VETSNET progress and viability. 

VA Office of Inspector General 4 
Audit of the Veterans Service Network 

Recommendation 

Management 

Comments 

OIG Response 

1.	 We recommend the Assistant Secretary, Office of Information and 

Technology, update the Integrated Master Schedule to reflect project 

interdependencies and incorporate work breakdown structures of VA’s 

other Information Technology initiatives that will rely on Veterans 

Service Network resources. 

The Assistant Secretary, Office of Information and Technology, concurred 

with our findings and recommendation. The Assistant Secretary’s comments 

included as Appendix E provide a listing of interdependencies considered 

critical for the VETSNET program. The implementation plan will ensure 

incorporation of these items into the Integrated Master Schedule by April 1, 

2011. 

Management’s action is responsive to the recommendation and has provided 

an acceptable implementation plan. 

VA Office of Inspector General 5 
Audit of the Veterans Service Network 

Finding 2
 

Full BDN 

Replacement in 

Question 

VETSNET Schedule, Cost, and Performance Objectives 

Not Being Met 

Taken together, the competing priorities and overlapping system 

requirements have had adverse impacts on meeting VETSNET schedule, 

cost, and performance objectives. They have increased confusion about 

VETSNET focus and activities to achieve original BDN replacement goals. 

They have hindered accomplishment of critical data conversion activities to 

transition from the legacy to replacement system processing. Further, 

competition for attention and resources has prevented VA from addressing 

longstanding functionality defects that require substantial workaround 

processes to meet program performance goals. 

Ultimately, the competing priorities and changing system requirements have 

served to prolong the schedule and increase the costs needed to complete the 

VETSNET program overall. Originally scheduled to be fully operational in 

2006, VETSNET completion has now been delayed by nearly 5 years. 

Moreover, expanding the scope of the VETSNET program to become the 

primary payment system for VA’s other system development initiatives has 

increased estimated VETSNET development costs by more than three times 

the original estimates. 

The competing priorities and emerging systems requirements have led to a 

loss of focus regarding the long-term goals for the VETSNET program. The 

principal area of confusion is with regard to the disposition of the BDN 

mainframe system. Specifically, the VETSNET Executive Team has stated 

that VA’s long-term goal is to migrate all entitlements programs from the 

aging BDN system to the VETSNET Corporate Database and retire the 

legacy system. The VETSNET Office of Management and Budget 

Exhibit 300 and VA’s Fiscal Year 2011 Budget Submission documents 

similarly state the BDN mainframe system has exceeded its useful life and is 

scheduled for retirement in 2012. 

Conversely, the VETSNET Program Management Plan and VA 

Congressional briefing in March 2010 indicate that VA’s intent is to replace 

only BDN’s compensation and pension functionality and not the entire 

mainframe system. Some VA representatives likewise said it would not be 

practical to decommission BDN because the legacy system performs many 

critical and unique functions, which are not replicated in VETSNET. 

Inconsistency and a lack of clarity regarding the end goals of VETSNET in 

various documents created internally across VA components (that is, Office 

of Information and Technology and VA Program Offices) add to the 

confusion. VA’s recent capital planning documents, program management 

documents, and IMS do not reflect the level of effort needed to process all 

payments through VETSNET and fully decommission the BDN mainframe 

VA Office of Inspector General 6 
Audit of the Veterans Service Network 

Data Conversion 

Delayed 

system. For example, the documents do not state that to successfully 

decommission BDN, VETSNET functionality must be enhanced to process 

all compensation and pension cases, as well as process all Educational and 

Vocational Rehabilitation and Employment entitlement programs currently 

paid by the legacy system. 

An adverse impact of not fully replacing BDN is that the legacy system 

would continue to require costly maintenance each year after all entitlements 

cases have been migrated to VETSNET. VA has reported that the direct 

maintenance cost for BDN is approximately $7 million per year. 

Decommissioning this mainframe system would negate the need for the 

maintenance and provide significant cost savings to offset VETSNET 

development costs. In addition, a program official estimated that up to eight 

staff positions could be eliminated by migrating the legacy benefit programs 

that have not yet been moved from the BDN mainframe system to 

VETSNET. Decommissioning outdated mainframe legacy systems and 

eliminating redundancy are sound business practices. As funding for the 

VETSNET program has increased over the last several years, it is critical to 

utilize funds wisely and demonstrate to taxpayers a sufficient return on 

investment. 

VA has also been hindered in its ability to meet the original data conversion 

goals for the VETSNET program. After 14 years of development, 

VETSNET applications still require substantial enhancements before all 

records can be migrated from the BDN mainframe legacy system. VA’s 

system development efforts have nonetheless substantially improved the 

functionality and performance of the VETSNET applications in recent years. 

In April 2008, VETSNET became the core business application and payment 

system for compensation benefits. In April 2010, the VA successfully 

converted approximately 1.2 million active compensation and pension 

records from the BDN mainframe system to VETSNET, including most live 

cases, clothing allowances, and offset adjustments for insurance deductions. 

As of October 2010, approximately 380,000 legacy compensation and 

pension benefit records (including 54,000 active records) remained to be 

converted for VETSNET processing. VA anticipates converting the 

remaining legacy cases by October 2011. Until VA enhances VETSNET 

functionality to accommodate the remaining legacy cases, it will continue to 

use the mainframe system, preventing VA from eliminating redundant claims 

processing. 

One of the challenges faced in developing the functionality required to 

complete data conversion is that VETSNET must support many decisions 

and processes to administer a complex set of benefits. For instance, different 

categories of veterans and their families are eligible for various types of 

benefits and payments. Some benefits may also be paid to third party 

VA Office of Inspector General 7 
Audit of the Veterans Service Network 

Longstanding 

Performance 

Issues 

Unaddressed 

fiduciaries or when a veteran grants power of attorney to another individual. 

Compensation and pension cases not currently supported by VETSNET 

include: 

 Death and Terminated Pension benefits 

 Compensation and Pension cases involving more than nine dependents 

 Medal of Honor benefits 

 Offsets for hospital and nursing home reductions 

 Non-Monthly Pension benefits Veteran married to veteran benefit cases 

Competing legislative priorities and new system requirements only add to the 

complexities already inherent in the fundamental VETSNET program. 

Integrated system planning and scheduling, as previously discussed, along 

with effective resource prioritization will be critical for VA to maintain focus 

and continue to make progress in converting the remaining compensation 

and pension records from the legacy BDN as originally intended. 

Longstanding VETSNET functionality defects have gone unaddressed amid 

the competing demands for resources and attention. More specifically, end 

users have encountered functionality defects in the system, requiring manual 

updates and corrections to thousands of data tables. For example, VETSNET 

does not liquidate some accounts payable transactions or record certain 

accounts receivable transactions, requiring monthly data corrections to 

production systems. The manual workarounds to address these issues 

impede the ability to ensure accurate processing of benefit payments. Other 

ongoing VETSNET functionality defects include: 

 Rejected payments each month due to address formatting errors, 

requiring repayments by regional offices. 

 Inability to authorize certain compensation awards due to error messages. 

 Duplicate medical and income awards displayed to end users. 

 Incorrectly processed insurance transactions. 

 Failed accounting transactions due to incorrect document sequence 

numbers. 

 Incorrect negative balances contained in certain accounting transactions. 

During 2008 and 2009, VETSNET developers relied on more than 

900 master record change orders to compensate for the high number of 

defects and to correct the data errors in VETSNET production systems. VA 

has had to devote considerable resources each month to ensure that 

accounting transactions are properly recorded and data is accurately 

displayed within the production systems. VETSNET developers are tracking 

the accounting functionality defects and expect to fully remediate them by 

April 2011. The number of monthly master record changes should decline as 

the developers resolve the accounting functionality defects. 

VA Office of Inspector General 8 
Audit of the Veterans Service Network 

Overall Program 

and Schedule 

Cost Increases 

VA has developed a number of major software releases over the past few 

years to meet the changing priorities and unplanned requirements that have 

altered business requirements and therefore the program’s scope, direction, 

and cost . Table 1 provides a list of the eight different VETSNET software 

releases over the past 3 years. 

Table 1 VETSNET Software Releases 

Version 

Release Date Description 

VR1 August 2007 Enhance Compensation Functionality 

VR2 February 2008 Deliver Survivor Benefits 

VR3 August 2008 Deliver Income-Based Functionality 

VR4 March 2009 Enhance Awards Functionality 

VR5 August 2009 Deliver Filipino Equity Compensation 

VR6 February 2010 Enhance Functionality for Data Conversion 

VR7 March 2010 Software Defect Repairs 

VR8 August 2010 Enhance Functionality for Data Conversion 

The evolution of the system through such releases has adversely impacted 

the schedule, cost, performance goals over the life of the VETSNET 

program. In August 2004, VA reported that its goal was to deploy the 

VETSNET replacement system to support all compensation and pension 

cases by the end of December 2006. However, because of changing program 

requirements, VA now estimates that VETSNET will not support all 

compensation and pension claims until October 2011, approximately a 

5-year schedule delay in meeting program goals. 

Further, GAO stated that VA previously reported VETSNET total estimated 

system development costs of $89 million for fiscal years 1996 through 2006. 

In 2006, VA reported to Congress a revised total cost of $151 million for the 

VETSNET program through 2009. Due to the expanded scope of the 

program, in May 2009, VA reported to the House Appropriations Committee 

a revised total cost estimate of more than $308 million through 2012, more 

than two times the 2006 cost projection. Additionally, VA’s other IT 

initiatives increase the risk of yet further changes in VETSNET program 

priorities and additional delays in achieving original performance goals. 

VA Office of Inspector General 9 
Audit of the Veterans Service Network 

Conclusion 

Recommendations 

Management 

Comments 

Ideally, with complete development and implementation, VETSNET will be 

leveraged as the primary payment system for VA’s benefits delivery. 

However, changing priorities and overlapping systems requirements have 

had the adverse impact of diverting focus and resources from accomplishing 

original program objectives. As the program has evolved, confusion has 

arisen concerning full replacement of the legacy BDN mainframe system 

with VETSNET. Efforts to migrate thousands of data records from the 

legacy to the corporate database have been delayed. Manual workarounds to 

make up for longstanding functionality defects impede the ability to ensure 

accurate benefit payments processing. 

Ultimately, such performance shortfalls have served to prolong the 

VETSNET schedule by five years and increase the costs by more than twice 

the 2006 estimates. Clearly communicating VETSNET goals and 

reprioritizing resources to accomplish them will be critical to successfully 

put in place the application suite needed for integrated veterans claims 

processing and benefits delivery. 

2.	 We recommend the Assistant Secretary, Office of Information and 

Technology, update the capital planning, budgeting, and program 

management documents to clearly communicate the current disposition 

and future plans for the Benefits Delivery Network legacy system. 

3.	 We recommend the Assistant Secretary, Office of Information and 

Technology, define the level of effort and apply the resources required to 

complete data migration for all entitlement programs and decommission 

the Benefits Delivery Network legacy system. 

4.	 We recommend the Assistant Secretary, Office of Information and 

Technology, address longstanding accounting functionality defects to 

ensure that benefit payments are accurately recorded within the Veterans 

Service Network. 

5.	 We recommend the Assistant Secretary, Office of Information and 

Technology, prioritize resources to put the Veterans Service Network on 

track toward meeting the established completion schedule and program 

goals within anticipated cost parameters. 

The Assistant Secretary, Office of Information and Technology, concurred 

with our findings and recommendations. The Office of Information and 

Technology will use the timing of upcoming projects and their 

interdependencies as baseline planning for decommissioning the Benefits 

Delivery Network. 

VA Office of Inspector General 10 
Audit of the Veterans Service Network 

OIG Response 

The Office of Information and Technology is working with program offices 

to identify, prioritize, and resolve remaining VETSNET functionality 

defects. By October 2013, the Assistant Secretary plans to use baseline 

planning documents and achieve the program goal of VETSNET becoming 

the primary payment system for all VA’s benefits delivery. 

Management’s actions are responsive to the recommendations and have 

provided an acceptable implementation plan. 

VA Office of Inspector General 11 
Audit of the Veterans Service Network 

Finding 3	 Software Development, Testing, and Implementation 

Need Improvement 

Competing priorities, time constraints, and changes to business requirements 

have led to numerous software releases to meet near-term program goals. As 

a result, project goals have typically shifted and planned functionality for 

each software release has been reduced to fit established project timelines. 

Additionally, change controls and testing with each software release have not 

been adequate to ensure proper system functionality. As such, software 

rework and rollback of installation packages have been required to correct 

defects and planned functionality enhancements have been delayed. The 

software transmittal process also does not ensure that appropriate system 

documentation and installation packages are provided for system testing, 

resulting in rework and reduced cycle times for system testing. 

Changing	 Developing VETSNET software requirements has usually focused on shortRequirements	 term program priorities, resulting in frequently changing business 

requirements. SEI previously reported in 2005 that VETSNET project 

requirements were not stable, resulting in confusion and delays in the 

development of VETSNET. The report concluded that if software 

requirements are not clearly defined and stabilized, the end results are 

increased project costs, schedule slippages, and inability to deliver products 

that meet users’ needs. 

In response to the SEI report, VA implemented a gate review process to 

stabilize business requirements, add quality control, and improve the 

likelihood that software meets end user requirements. The objective of the 

Preliminary Design Gate Review is to develop an agreed-upon set of 

business requirements and functional specifications, so requirements can be 

“frozen.” Appendix D provides the software development lifecycle stages, 

descriptions, and objectives of each gate review. 

Despite implementation of the gate review process, test documents and 

internal reports reveal that VA continues to struggle with frequently 

changing functionality and business requirements during the development 

and testing cycles for each major release. The figure presents the significant 

number of changes to business requirements and functionality specifications 

after the Preliminary Design Gate Review for several major releases. 

As noted in the Figure, in some cases, the functionality requirements 

changed more than nine times during the design, development, and testing 

process. In general, major releases had to be reduced in scope and 

functionality to fit the project schedule. 

VA Office of Inspector General 12 
Audit of the Veterans Service Network 

Figure Requirements Changes after Preliminary Design Review 

80
 

70
 

60
 

50
 

40
 

30
 

20
 

10
 

0 

Software Testing 

Deficiencies and 

Rework 

Table 2
 

28 

61 

76 

50 

38 

VR2 VR3 VR4 VR5 VR6 

CHANGED 1-2 TIMES 

CHANGED 3-4 TIMES 

CHANGED 5-6 TIMES 

CHANGED 7-8 TIMES 

CHANGED 9 OR MORE TIMES 

TOTAL 

Frequently changing business requirements resulted in inadequate time to 

test the functionality for each software release. Table 2 identifies the 

significant number of functionality specifications and business scenarios that 

were not successfully tested during each planned major software release 

cycle. Internal documents and test results revealed that in a number of 

instances functionality that was not successfully tested was deferred to 

subsequent software releases. 

Functionality and Business Scenarios Test Results 

2307 605 21
 

Test Procedures Executed 

VETSNET 

Version 

Test 

Procedures 

Passed 

Test 

Procedures 

Failed 

Test 

Procedures 

(% Failed) 

VR2 236 156 40
 

Business Scenarios 

Business 

Scenarios 

Passed 

Business 

Scenarios 

Failed 

Business 

Scenarios 

(% Failed) 

VR3 1953 879 31
 264 254 49
 

VR4 2523 543 18
 258 79 23
 

VR5 2153 950 31
 223 68 23
 

VR6 2921 991 25
 266 33 11
 

Given inadequate time for testing, software functionality problems were 

typically encountered after major software releases, necessitating rework to 

correct defects. The Program Management Office compensated for the large 

number of test failures by relying on supplemental software releases and 

emergency change orders to implement functionality not successfully tested 

during the standard development cycle. Specifically, from February 2008 

through March 2010, the Program Management Office implemented 

16 supplemental releases to add system functionality and correct more than 

VA Office of Inspector General 13 
Audit of the Veterans Service Network 

Software 

Transmittal 

Deficiencies 

Table 3 

290 critical software defects associated with VETSNET releases. During the 

same period, the Program Management Office also initiated more than 

35 emergency software changes to correct critical software defects. 

Compared with the standard development process, the supplemental releases 

and emergency change orders allow developers to bypass standard gate 

reviews and some quality assurance checks. The inherent risk with the 

abbreviated process is that functionality defects will not be detected and 

corrected during development and testing. In March 2010, this risk was 

realized when one VETSNET sub-module, the Rating Board 

Automation 2000 installation package, was released into production with 

minimal testing via an emergency change order.3 Because the regional 

offices could not process certain transactions, VA was forced to “roll back” 

the installation package to an earlier version of the software. More thorough 

testing of emergency change orders could have identified the faulty 

installation package before its release into production. 

Reviews of documentation identified ineffective processes for ensuring 

appropriate system documentation and software installation packages are 

delivered for pre-production testing. Specifically, a large number of 

software transmittal packages, called Application Turnover Transmittals 

(ATTs), were delivered for pre-production testing with inaccurate system 

documentation and outdated software installation packages. Table 3 shows 

the total number of ATTs associated with VETSNET major releases and the 

number that were rejected for pre-production testing by the VA’s Release 

Management Division. 

Application Turnover Transmittals Rejected 

VETSNET 

Version 

Total 

ATTs 

ATTs 

Requiring 

Revisions 

ATTs 

Revised for 

Software 

Changes 

ATTs 

Revised due to 

Insufficient 

Documentation 

ATT 

Revisions 

Impacting 

Cycle 

Testing 

VR4 128 34 19 15 23 

VR5 98 32 15 17 18 

VR6 23 6 2 4 6 

VR7 10 6 3 3 6 

Totals 259 78 39 39 53 

As noted in the table, outdated software installations packages, insufficient 

system documentation, and changing functionality requirements have led to a 

significantly high rejection rate of approximately 30 percent, or 78 of 259, of 

all software packages submitted for pre-production testing. Changes to the 

3 The Rating Board Automation 2000 installation package supports the preparation of disability rating 

decisions. Appendix A contains more details on this application. 

VA Office of Inspector General 14 
Audit of the Veterans Service Network 

Conclusion 

Recommendations 

software migration packages have significantly reduced the time allocated 

for pre-production testing, thereby increasing the risk that insufficient testing 

will occur. 

In February 2010, this risk was realized as VA was forced to roll back 

installations of two VETSNET sub-modules—another Rating Board 

Installation 2000 installation package and the Control of Veterans Records 

System—until software defects could be resolved.4 On both occasions, the 

software transmittal packages were rejected, resulting in reduced cycle time 

for testing. Software “rollbacks” cause significant disruptions at VA 

regional offices and can adversely impact their ability to process 

compensation and pension claims using VETSNET. 

Stabilizing software requirements earlier in the development process and 

improving the effectiveness of software testing processes can greatly reduce 

the number of software defects that occur and must be corrected through 

supplemental releases and emergency change order processes. Implementing 

comprehensive test procedures for the supplemental release and emergency 

change order processes will reduce the risk that defects will not be 

remediated during software development and deployment. Finally, effective 

controls need to be implemented to ensure that accurate and up-to-date 

system documentation and software installation packages are provided for 

pre-production testing. 

6.	 We recommend the Assistant Secretary, Office of Information and 

Technology, implement controls to stabilize the functionality 

requirements process and enforce software “code freeze” during the 

Veterans Service Network software development process. 

7.	 We recommend the Under Secretary for Benefits implement controls to 

stabilize the functionality requirements process and enforce software 

“code freeze” during the Veterans Service Network software 

development process. 

8.	 We recommend the Assistant Secretary, Office of Information and 

Technology, implement full testing of all supplemental software releases 

and emergency software changes to identify and remediate functionality 

defects before Veterans Service Network applications are installed for 

production. 

9.	 We recommend the Assistant Secretary, Office of Information and 

Technology, implement controls to ensure that software transmittal 

4 The Veterans Benefits Administration uses the Control of Veterans Records System to electronically 

track veterans’ claims folders and control search mail—active claims-related mail waiting to be 

associated with a veteran’s claims folder. 

VA Office of Inspector General 15 
Audit of the Veterans Service Network 

Management 

Comments 

OIG Response 

processes provide accurate and up-to-date system documentation and 

software installation packages for pre-production testing. 

The Assistant Secretary, Office of Information and Technology, concurred 

with our findings and recommendations. The Assistant Secretary 

acknowledges that changes to functionality requirements can occur during 

design, development, and testing, resulting in many software exceptions. 

The Assistant Secretary stated that future supplemental and emergency 

software releases will be fully tested prior to installation, unless 

extraordinary situations require waivers from stakeholders. The Office of 

Information and Technology has conducted meetings with the Systems 

Integration Office to identify problems in the software transmittal process 

and implemented peer reviews to help improve process quality. 

The Acting Under Secretary for Benefits concurred with our findings and 

recommendation regarding implementing a software code freeze. The 

Acting Under Secretary agreed that controls to enforce gate reviews for 

requirements finalization, design acceptance, and code freeze development 

need to be strengthened. The VETSNET Integrated Product Team plans to 

monitor software exceptions and move them to subsequent software releases, 

unless mandated by legislative changes. In January 2011, VA began 

enforcing the gate reviews for requirements, acceptance, and development 

for VETSNET Release 10, which is scheduled for deployment in February 

2011. 

Management’s actions are responsive to the recommendations and have 

provided an acceptable implementation plan. 

VA Office of Inspector General 16 
Audit of the Veterans Service Network 

Appendix A 

VETSNET 

Objectives 

VETSNET 

Applications 

Background 

As originally envisioned, VETSNET technology was to provide faster, more 

flexible compensation and pension processing capability from initial claim 

application through review, rating, award, and benefits delivery. The 

VETSNET relational database also makes it possible to store more data for 

each veteran and make that data more accessible for inquiries and analysis. 

The VETSNET initiative was considered necessary to replace the 

compensation and pension functionality of the legacy BDN system that does 

not: (1) support compliance with Federal financial management regulations; 

(2) provide automated support for adequate control of payments processing; 

and (3) meet critical customer service needs, such as immediate response to 

payment inquiries and processing errors. Additionally, replacing BDN with 

VETSNET will eliminate concerns about attrition and a shortage of 

personnel qualified to maintain the legacy system. 

VETSNET includes a Windows-based graphical user interface at the regional 

offices, an Oracle relational database, and a middleware layer that links the 

end user to the corporate database. The corporate database is a resource 

shared with other VA systems. VETSNET consists of the following key 

applications used to process compensation and pension benefits. 

	 Share—Allows regional office employees to query legacy information 

such as the Beneficiary Information Locator System, the BDN, and other 

agencies’ information. Share, a Microsoft Windows-based client/server 

application, updates both legacy and corporate information with one 

transaction. 

	 Search and Participant Profile—Provides the ability to search and 

locate veterans, claims, and/or family member records by File Number, 

Social Security Number, or Taxpayer Identification Number. This search 

functionality includes a real-time interface between VETSNET and the 

Beneficiary Information Locator System. 

	 Modern Awards Processing Development—Provides a single 

capability for complete claims development, claims status monitoring, 

and case management using the features of existing applications. It also 

provides access to those applications that support the claims development 

process. 

	 Rating Board Automation 2000—Supports the preparation of disability 

rating decisions. This application is integrated with VETSNET 

specifications for award processing and provides support for the creation 

of text documents needed to document rating decisions. Current 

VA Office of Inspector General 17 
Audit of the Veterans Service Network 

capabilities include direct transfer of award data from this system to the 

BDN. 

	 Statement of the Case—Creates Word documents. By reading an 

existing rating document created through Rating Board Automation, the 

application presents the user with a list of issues to be included in the 

new or supplemental Statement of the Case. Adjudicative actions can be 

entered into the system, as well as pertinent laws related to Rating Board 

Automation. 

	 Awards—Provides the processing capability to generate and authorize 

compensation and pension awards. A link to the Finance and Accounting 

System affords complete accounting functions, and includes an interface 

with the Department of the Treasury. The Awards application collects 

data to support both information and budgetary requirements for the 

compensation and pension program. 

	 Finance and Accounting System—Supports fiscal and accounting 

transaction processing, including an interface with the Department of the 

Treasury. This application integrates with other applications and 

supports both the information gathering and budgetary requirements for 

the compensation and pension program. 

Long-Term The long-term goal of the VETSNET program is to integrate the corporate 

Program Goals database with other IT initiatives in the benefits portfolio to ultimately 

comprise “One VA” system. One of the challenges of developing the 

required functionality for VETSNET applications is that it must include 

decisions and processes to administer a complex set of benefits. 

VA Office of Inspector General 18 
Audit of the Veterans Service Network 

Appendix B Scope and Methodology 

This audit focused on evaluating the planning, development, and 

implementation of VETSNET. Our audit objectives were to determine 

whether: effective controls have been implemented to address program 

governance deficiencies identified in prior VETSNET assessments; schedule, 

cost, and performance goals for the program have been met; and (3) effective 

change controls have been implemented to support the planning, testing, and 

implementation of the VETSNET suite of applications. 

As background for our audit, we considered prior evaluations by Carnegie 

Mellon Software Engineering Institute (September 2005) and GAO (April 

2007), which stated that the VETSNET program faced many management, 

organizational, and technical challenges, but that none of these barriers were 

insurmountable. We focused on VETSNET program management and 

software development controls from January 2008 through April 2010 to 

determine the extent to which VA has overcome these issues. We performed 

limited testing on Release 7 as some system development documentation was 

not available during testing. 

We reviewed VA directives, handbooks, procedures, and supporting 

documentation to evaluate VETSNET business processes, software 

development, and testing procedures. We used the Carnegie Mellon 

Software Engineering Institute’s CMMI® for Development, Version 1.2, 

Improving Processes for Better Products and Information Systems Audit and 

Control Association’s Business Application Change Control as the basis for 

evaluating the effectiveness of program management and application change 

controls. 

We conducted interviews with Veterans Benefit Administration program 

officials, VETSNET Project Management Office staff and contractors, and 

Office of Information and Technology personnel to gather additional 

information about the progress and challenges of the VETSNET program. 

Specifically, we reviewed: 

	 System development life cycle documents, quality assurance reports, 

project performance metrics, post-implementation reports, and project 

milestone reviews to learn about software development progress. 

	 Functional requirements and test plans, scripts, and procedures to learn 

about quality control of VETSNET software releases and data conversion 

activities. 

	 Help Desk complaints and reports to identify issues regarding the 

functionality of VETSNET applications. 

VA Office of Inspector General 19 
Audit of the Veterans Service Network 

Data Limitation 

Reliability of 

ComputerProcessed Data 

Compliance with 

Government Audit 

Standards 

	 Software testing procedures, test results, and release plans to identify 

change control issues and their impact on VETSNET development and 

production. 

 The network architecture and integrated master schedule to identify 

system dependencies and interconnections with VETSNET. 

 Corrective action plans and supporting documentation to identify 

improvements since previous evaluations of the VETSNET program. 

	 Office of Management and Budget Exhibit 300 submissions, vendor 

contracts, and financial reports to learn about costs and budgets 

associated with the VETSNET program. 

We conducted our work at the following locations: 

 VETSNET Project Management Office – St. Petersburg, FL 

 St. Petersburg, FL Regional Office 

 Hines, IL Information Technology Center 

 Chicago, IL Regional Office 

 Waco, TX Regional Office 

 St. Paul, MN Regional Office 

 Austin, TX Information Technology Center 

For findings 1 and 2, we significantly relied on the information reported in 

SEI’s 2005 draft report and GAO’s 2007 audit report. We did not verify the 

accuracy of the data in these reports. We were not able to obtain 

documentation supporting the cumulative historical and projected program 

costs for VETSNET. The cumulative VETSNET program cost we cited is 

based on VA briefings to a Congressional subcommittee. 

We used computer-processed data such as data conversion reports and 

software development internal documents provided by the VETSNET 

Program Management Office to accomplish our audit objective. We tested 

the reliability of data by evaluating it for unusual items or inaccuracies. 

Because we obtained this data from source software development systems, 

we considered it sufficiently reliable to meet our audit objective. 

We conducted our audit work from November 2009 through October 2010. 

Our assessment of internal controls focused on those controls relating to our 

audit objectives. We conducted this performance audit in accordance with 

generally accepted government auditing standards. These standards require 

that we plan and perform the audit to obtain sufficient, appropriate evidence 

to provide a reasonable basis for our findings and conclusions based on our 

audit objective. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objective. 

VA Office of Inspector General 20 
Audit of the Veterans Service Network 

Appendix C 

VETSNET 

Governance 

VETSNET Oversight, Development, and Implementation 

The VETSNET quality management program is composed of stakeholders 

responsible for the processes, procedures, and reviews for developing a suite 

of applications to meet end user requirements. The VETSNET stakeholder 

groups are: 

VETSNET Executive Board—Provides strategic management and decision 

making for the VETSNET program. The Board is responsible for corporate 

commitment, including the appropriate level of management support to 

ensure the success of the program. The Board, chaired by the Deputy Under 

Secretary for Benefits, consists of: 

 Associate Deputy Under Secretary, Policy and Planning – Veterans 

Benefits Administration 

 Associate Deputy Under Secretary, Office of Field Operations – Veterans 

Benefits Administration 

 Deputy Chief Information Officer, Enterprise Development – Office of 

Information and Technology 

 Director, Benefits Program Executive Office – Veterans Benefits 

Administration 

 Chief Financial Officer, Office of Resource Management – Veterans 

Benefits Administration 

 Director, Compensation and Pension Service – Veterans Benefits 

Administration 

 Director, Office of Program Analysis and Integrity – Veterans Benefits 

Administration 

VETSNET Executive Team—Oversees and approves development and 

implementation of the remaining applications in the VETSNET program. 

Led by senior executives who are considered subject matter experts in 

compensation and pension processes, Executive Team members include: 

 Special Assistant for VETSNET 

 VETSNET Portfolio Manager – Office of Information and Technology 

 VETSNET Technical Architect – Office of Information and Technology 

 VETSNET Business Architect – Veterans Benefits Administration 

 Office of Business Process Integration – Veterans Benefits 

Administration 

VETSNET Integrated Project Team—Oversees the day-to-day activities 

necessary to develop and implement applications supporting the program. 

The Team meets regularly to monitor and control the progress of the 

VETSNET program. Program offices represented on the Team are: 

VA Office of Inspector General 21 
Audit of the Veterans Service Network 

Release 

Management 

Teams 

 Office of Enterprise Development – Office of Information and 

Technology 

 Office of Resource Management – Veterans Benefits Administration 

 Office of Field Operations – Veterans Benefits Administration 

 Office of Business Process Integration – Veterans Benefits 

Administration 

 Performance Analysis and Integrity – Veterans Benefits Administration 

 Enterprise Infrastructure Engineering – Office of Information and 

Technology 

As part of the VETSNET software development process, a number of 

“Release Teams” are dedicated to the design, development, testing, and 

implementation of major and supplemental releases for the VETSNET suite 

of applications. The Release Teams, consisting of various contractors and 

employees located in the St. Petersburg Regional Office and the Austin, TX 

and Hines, IL Information Technology Centers, are described below. 

VETSNET Executive Team—Responsible for overseeing and approving 

development and implementation of the remaining applications and the 

VETSNET program. 

Business Subject Matter Experts—Responsible for identifying and 

specifying business requirements consistent with the release scope approved 

by the Executive Team. 

Requirements Analysts—Responsible for working with subject matter 

experts to define business requirements and functional specifications 

documents for business approval. 

Development Teams—Responsible for the detailed design and development 

of functional components of a release, these teams are composed of both 

Government and contractor support staff. 

Testing Teams—Responsible for identifying test requirements, mapping 

specifications, and developing technical test plans. Testing teams collaborate 

closely with subject matter experts to ensure that timelines and both technical 

and business testing requirements are met. 

VA Office of Inspector General 22 
Design

Audit of the Veterans Service Network 

Appendix D Development Phases and Gate Review Process 

VA implemented a VETSNET gate review process as a quality control mechanism to improve 

the likelihood that major software releases will meet end user requirements. The process 

provides the VETSNET Executive Team with opportunities to review progress at pre-defined 

points within a software release’s development life cycle. Table 4 below provides a description 

of each gate review stage, along with expected outcomes for related system development 

activities. The software development group utilizes an “incremental” development approach for 

designing, developing, testing, and implementing major VETSNET releases. 

Table 4 VETSNET Gate Review Process 

Gate Reviews System Development Life Expected Outcomes Cycle Development Phases 

Project Initiation Review and 

Authorization—Ensures that 

release scope is defined, teams 

are established, and criteria are 

tailored to meet any specific 

need. Upon approval, the 

release is ready for 

requirements development. 

Requirements 

Functional Specifications 

Requirements and functional 

specifications are documented, 

reviewed, and approved. This stage 

includes developing the release scope 

and identifying risks and mitigation 

strategies. 

Preliminary Design Software design is developed, 

Review—Ensures that Design reviewed, and approved. Business 

requirements and functional requirements and functional 

specifications are documented specifications are frozen. 

and approved. The release is Test Management & Planning 

then ready for detailed design. 

Test Requirements 

Critical Design Review— The software release is developed and 

Ensures that the design is Assemble, built. Unit and integration testing is 

complete, meets requirements, 

and is approved. Test plan 

Development Test, & 

Build 

performed to determine whether 

functionality and business 

development begins and the 

release is ready for coding. 

Test Development 

Test Management & Planning 

(continues) 

requirements are met. 

VA Office of Inspector General 23 
Audit of the Veterans Service Network 

Gate Reviews System Development Life Expected Outcomes Cycle Development Phases 

Test Readiness Review— 

Ensures coding and unit testing 

is complete and that the code is 

ready for full functional and 

integration testing. 

Initial Cycle 

Defect 

Repairs 

Initial Cycle 

Test 

Execution 

Bug Fix 

Defect 

Repairs 

Bug Fix 

Test 

Execution 

Patch 

Priority software defects are resolved 

and baselines are established for 

formal software testing. 

Pre-Production Readiness The final software build is prepared 

Review—Ensures test results and released to the System 

showing that the release is Implementation Office. Regression 

ready for migration to the pre- Pre-Production and performance testing is performed production environment for 

packaging into a production 

release. 

to ensure functionality and business 

requirements are met. 

Operational Readiness Beta release and testing are performed 

Review—Ensures the release and feedback is collected from field 

is suitable for migration to the sites. Minor defects are corrected. A 

production environment. Test Go/No Go decision is made for full 

cases are developed and 

preliminary test plans are 

Production 

installation. The software build is 

installed onto production systems for 

prepared and reviewed. The user testing. 

final test plan is submitted. 

Post-Implementation The application is released nationwide. 

Review—Occurs after the The System Implementation Office 

release is in production for a documents and releases postperiod of time. The intent is to implementation issues to the 

review any issues identified development and performance group. 

during production usage and 

identify any lessons learned 

from the release. 

VA Office of Inspector General 24 
Audit of the Veterans Service Network 

Appendix E Assistant Secretary for Information and Technology 

Comments 

Department of Memorandum Veterans Affairs 

Date:	 January 24, 2011 

From:	 Assistant Secretary for Information and Technology (005) 

Subj:	 Draft OIG Report - Veterans Benefits Administration: Audit of the Veterans 

Service Network; Project #2009-03850-0184 

To: Assistant Inspector General for Audits and Evaluations (52) 

Thank you for the opportunity to review the Office of Inspector General (OIG) 

draft report, “Veterans Benefits Administration: Audit of the Veterans Service 

Network.” The Office of Information and Technology concurs with OIG’s 

findings and submits the attached written comments for each of the 

recommendations. 

If you have questions, please contact me at 202-461-6910, or have a member of 

your staff contact Steven G. Schliesman, Director, Secretary Initiatives Project 

Management Division, Product Development (005Q), at 732-982-7067. 

(Original signed) 

Roger W. Baker 

Attachment 

VA Office of Inspector General 25 
Audit of the Veterans Service Network 

Attachment- 005 Comments 

OIG DRAFT REPORT 

Veterans Benefits Administration: Audit of the Veterans Service Network 

(Project No. 2009-03850-0184) 

OIG Recommendations 

1. We recommend the Assistant Secretary, Office of Information and Technology, update 

the Integrated Master Schedule to reflect project interdependencies and incorporate work 

breakdown structures of VA’s other Information Technology Initiatives that will rely on 

Veterans Service Network resources. 

CONCUR. OI&T will update the Integrated Master Schedule to reflect project 

interdependencies. The following are currently identified as interdependencies and would be 

considered critical work items for this activity: 

A.	 Fix payment and accounting irregularities noted in the Financial and Accounting 

System (FAS). 

B.	 VETSNET client (excluding FAS) replaced by VBMS web front end. 

C.	 Conversion of Compensation and Pension (C&P) data off the Benefits Delivery 

Network (BDN). 

D.	 FAS becomes payment engine for Chapter 33 through iterative delivery. 

E.	 Enterprise Identity Management Solution replaces VETSNET Common Security 

System (CSS). 

F.	 Conversion of Chapter 33 data off BDN. 

G.	 VETSNET Corporate Data support for VBMS Correspondence Solution. 

H.	 Conversion of non-Chapter 33 Education data off BDN. 

I.	 FAS becomes payment engine for non-Chapter 33 Education benefits. 

J.	 Veterans Relationship Management (VRM) and Virtual Lifetime Electronic Record 

(VLER) anticipated needs through iterative delivery. 

TARGET COMPLETION DATE: Critical items will be incorporated into the Integrated Master 

Schedule by April 1, 2011. 

2. We recommend the Assistant Secretary, Office of Information and Technology, update 

the capital planning, budgeting, and program management documents to clearly communicate 

the current disposition and future plans for the Benefits Delivery Network legacy system. 

CONCUR. Contained in OI&T’s response to recommendation 1 are elements that would be 

used as the base planning information for the future plans of decommissioning the BDN. The 

capital planning, budgeting and program management documentation to achieve this would have 

to be formulated. 

TARGET COMPLETION DATE: Submission date for 2013 E300s. 

VA Office of Inspector General 26 
Audit of the Veterans Service Network 

3. We recommend the Assistant Secretary, Office of Information and Technology, define the 

level of effort and apply the resources required to complete data migration for all entitlement 

programs and decommission the Benefits Delivery Network legacy system. 

CONCUR. We understand the recommendation to include both the development costs 

necessary for data conversion, as well as data conversion costs. Such cost estimates have not yet 

been completed and would necessarily be components of the program planning for the BDN 

decommissioning, as addressed in our response to recommendation 2. 

TARGET COMPLETION DATE: Submission date for 2013 E300s. 

4. We recommend the Assistant Secretary, Office of Information and Technology, address 

longstanding accounting functionality defects to ensure that benefit payments are accurately 

recorded within the Veterans Service Network. 

CONCUR. Some progress has already been made in resolving these issues including: 

 Activation of the automated interface between VETSNET and FMS, which eliminated 

most manual posting of transactions by Hines Finance. 

 Software updates installed to eliminate address formatting errors. 

 Software updates installed to resolve error messages preventing award authorization. 

We do prevent award authorizations from occurring for brief periods during the 

creation of monthly pay files in order to prevent improper pay calculations. 

 Software updates installed to prevent duplicate display of award decisions to end users. 

 Increasing the control field size to resolve document sequence number issues. 

 Software updates installed to prevent negative balances caused by continuing to offset 

benefit payments after overpayment balances were recouped. 

We are working with VBA business sponsors to identify, prioritize and resolve other remaining 

defects and needed enhancements. 

TARGET COMPLETION DATE: All existing defects will be resolved by the planned date of 

BDN’s decommissioning, October 2013. 

5. We recommend the Assistant Secretary, Office of Information and Technology, prioritize 

resources to put the Veterans Service Network on track toward meeting the established 

completion schedule and program goals within anticipated cost parameters. 

CONCUR. With the final conversion of C&P off of BDN, the original completion schedule 

would be met. 

TARGET COMPLETION DATE: Complete 

6. We recommend the Assistant Secretary, Office of Information and Technology, 

implement controls to stabilize the functionality requirements process and enforce software 

“code freeze” during the Veterans Service Network software development process. 

VA Office of Inspector General 27 
Audit of the Veterans Service Network 

CONCUR. VA concurs that process controls (to enforce release gate reviews for requirements 

finalization, design acceptance and development code freeze) need to be more stringently 

enforced. The release process itself is robust, but the reluctance to extend/change the release 

schedules when gate deadlines are missed can result in the acceptance of too many exceptions. 

New functional requirements changes can occur during the design, development and testing 

process, resulting in the arrival of new functionality beyond the release’s first testing iteration. 

This reduces the testing available for this functionality and can affect other portions of the 

release in an unintended way. 

This has been recognized by the VA VETSNET Integrated Product Team (VIPT), and more of 

these exceptions are being turned back and scheduled for subsequent releases. More progress is 

needed, and, unless dictated by legislative changes, all exceptions should be refused and moved 

to subsequent releases going forward. 

TARGET COMPLETION DATE: March 31, 2011 

7. We recommend the Under Secretary for Benefits implement controls to stabilize the 

functionality requirements process and enforce software “code freeze” during the Veterans 

Service Network software development process. 

Please refer to VBA’s memo dated January 11, 2011. 

8. We recommend the Assistant Secretary, Office of Information and Technology, 

implement full testing of all supplemental software releases and emergency software changes 

to identify and remediate functionality defects before Veterans Service Network applications 

are installed for production. 

CONCUR. Process controls to enforce release gate reviews for requirements finalization, design 

acceptance and development code freeze are expected to reduce the frequency of supplemental 

and emergency releases. Future supplemental and emergency releases will be fully tested prior 

to installation into the production environments. In extraordinary situations where complete 

testing is not possible, the development manager will request a waiver with approval from the 

appropriate stakeholders. 

We have independent test coverage for up to four supplemental releases for in-scope software 

components (each six-month period). For production mission critical issues, our objective is to 

remedy the software problem being experienced in a timely fashion. Each supplemental release 

is assessed based on its content (arch, client, svcs, dB mods) to determine the testing required. 

TARGET COMPLETION DATE: March 31, 2011 

9. We recommend the Assistant Secretary, Office of Information and Technology, 

implement controls to ensure that software transmittal processes provide accurate and up-todate system documentation and software installation packages for pre-production testing. 

VA Office of Inspector General 28 
Audit of the Veterans Service Network 

CONCUR. Substantial progress has been made in this area over the last nine months. Meetings 

were held with the Systems Integration Office (SIO) to identify problem areas, and peer reviews 

of application turnover documents were instituted prior to submission. 

TARGET COMPLETION DATE: March 31, 2011 

VA Office of Inspector General 29 
Audit of the Veterans Service Network 

Appendix F Acting Under Secretary for Benefits Comments
 

Department of Memorandum Veterans Affairs 

Date: January 11, 2011 

From: Acting Under Secretary for Benefits (20) 

Subj: OIG Draft Report—Veterans Benefits Administration: Audit of the Veterans 

Service Network [Project No. 2009-03850-R1-0184]—VAIQ 7064618 

To: Assistant Inspector General for Audits and Evaluations (52) 

1.	 Attached is VBA’s response to the OIG’s Draft Report: Veterans Benefits
 

Administration: Audit of the Veterans Service Network.
 

2.	 Questions may be referred to Catherine Milano, Program Analyst, at 461-9216. 

(Original signed) 

Michael Walcoff 

Attachment 

VA Office of Inspector General 30 
Audit of the Veterans Service Network 

Veterans Benefits Administration 

Comments on OIG Draft Report 

Veterans Benefits Administration: Audit of the Veterans Service Network 

The Veterans Benefits Administration (VBA) concurs with the findings in OIG’s draft report 

and provides the following comments in response to the recommendation: 

Recommendation 7: We recommend the Under Secretary for Benefits implement controls to 

stabilize the functionality requirements process and enforce software “code freeze” during the 

Veterans Service Network software development process. 

VBA Response: Concur. VBA agrees that controls to enforce gate reviews for requirements 

finalization, design acceptance, and development code-freeze need to be more stringently 

enforced. The release process itself is robust, but the practice to extend/change the release 

schedules when gate deadlines are missed can result in higher risk for the project. 

VBA will enforce the gate review process via the VETSNET Integrated Project Team (VIPT). 

The VIPT will refuse all exceptions, unless dictated by legislative changes, and move those 

exceptions to a subsequent release. 

On January 7, 2011, VBA began enforcing the gate reviews for requirements, acceptance, and 

development with VETSNET Release 10 that is scheduled for deployment in February 2011. 

VBA requests closure of this recommendation. 

VA Office of Inspector General 31 
Audit of the Veterans Service Network 

Appendix G Projected Monetary Benefits in Accordance with IG Act 

Amendments 

Recommendation Explanation of Benefits Better Use of 

Funds 

# 3 Estimated $7 million annually $35,000,000 

could be used more efficiently 

by migrating all entitlement 

programs from the legacy 

mainframe system and 

decommissioning BDN. Over a 

5-year period, this amounts to 

$35 million. 

Total $35,000,000 

VA Office of Inspector General 32 
Audit of the Veterans Service Network 

Appendix H OIG Contact and Staff Acknowledgments
 

OIG Contact	 Michael W. Bowman 

Acknowledgments	 Carol Buzolich 

Elijah Chapman 

Katherine Gers 

Frederick Livingstone 

Richard Purifoy 

Gordon Snyder 

Felita Traynham 

VA Office of Inspector General 33 
Audit of the Veterans Service Network 

Appendix I Report Distribution 

VA Distribution 

Office of the Secretary
 

Veterans Health Administration
 

Veterans Benefits Administration
 

National Cemetery Administration
 

Assistant Secretaries
 

Office of General Counsel
 

Non-VA Distribution 

House Committee on Veterans Affairs 

House Appropriations Subcommittee on Military Construction, Veterans 

Affairs, and Related Agencies 

House Committee on Oversight and Government Reform 

Senate Committee on Veterans’ Affairs 

Senate Appropriations Subcommittee on Military Construction, Veterans 

Affairs, and Related Agencies 

Senate Committee on Homeland Security and Governmental Affairs 

National Veterans Service Organizations 

Government Accountability Office 

Office of Management and Budget 

This report will be available in the near future on the OIG’s Web site at 

http://www.va.gov/oig/publications/reports-list.asp. This report will 

remain on the OIG Web site for at least 2 fiscal years. 

VA Office of Inspector General 34