Bureau of Indian Affairs Schools: New Facilities Management Information System Promising, but Improved Data Accuracy Needed
Published: 2003-07-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Bureau of Indian Affairs (BIA) is responsible for providing over 48,000 children with a safe place to learn. In response to concerns that data in its old information system did not accurately reflect the condition of facilities, BIA acquired a new system, called the Facilities Management Information System (FMIS). GAO was asked to determine whether FMIS addresses the old system's weaknesses and meets BIA's management needs, whether BIA has finished validating the accuracy of data entered into FMIS from the old system, and how well the quality control measures are working for ensuring the accuracy of new data being entered into the system from individual schools.
section_title: What GAO Found
paragraphs: FMIS is designed to address the previous data system's shortcomings and appears to have the capability to meet BIA's management needs if the data that are entered into FMIS are correct and timely. The old system was hard to use and did not readily provide data for maintenance and repair efforts. FMIS's design appears to overcome these weaknesses. For example, FMIS has features that help facility managers make data more consistent, as well as tools for helping managers develop cost estimates for maintenance and construction projects. BIA's contractor has been correcting the data that were transferred to FMIS from the previous system, but issues such as software compatibility problems between the contractor's system and FMIS have delayed entry of some of the data for more than 1 year. BIA officials say that these problems are being addressed. They said the delay has not affected their ability to prioritize or fund repair and construction projects, and our review of the data indicated that most newly identified deficiencies will not need to be addressed for 2 to 5 years. Our review of data from 14 BIA schools and observations during site visits disclosed no instances in which these data problems resulted in an unsafe learning environment for children. Most measures for controlling the quality of new data BIA employees are entering into the system for individual schools are not working well. BIA has established a multilevel review process and training programs to help ensure that such data entries are complete and accurate, but BIA's contractor, in reviewing data at the end of this process, continues to find that nearly half of the proposed data entries coming through the system are inaccurate and incomplete. Data entries from one-third of 102 schools that entered data show a 100-percent error rate. As a result, BIA officials continue to rely on their contractor to ensure that FMIS reflects accurate and complete data on the condition of BIA's facilities.

section_title: Letter
paragraphs: The Bureau of Indian Affairs (BIA), which is responsible for ensuring that  over 48,000 Indian students have school facilities that provide them with a  safe and quality place to learn, works with Indian tribes to operate and  maintain nearly 2,200 buildings at 171 elementary and secondary schools.  During fiscal year 2002, BIA’s budget for operating, maintaining,  constructing, and repairing its school facilities was nearly $348 million.  For many years, BIA and tribal employees have been tracking the  condition of their school facilities using an automated information system.  Tribal Indian education organizations and BIA have had longstanding  concerns that the information system was inadequate for managing school  facilities and did not accurately reflect how many buildings are in good  condition or disrepair. As a result, BIA did not have accurate and complete  information to determine how much annual funding is needed at each  school site for heating, lighting, and other operating expenses, as well as  how much current and future funding is needed at each site for making  repairs and capital improvements, such as patching a roof or upgrading a  fire alarm system.
Recognizing the need to address these concerns, BIA is now in its third  year of implementing a new Facilities Management Information System  (FMIS) to replace its old information system. Prior to implementing FMIS,  however, and to ensure that this new system met BIA’s management  needs, BIA hired a consulting firm to identify weaknesses with the old  system and recommend improvements for designing a new one. Once the  new system was designed and BIA was prepared to implement it, BIA  hired an engineering firm to validate all of the data to be transferred from  the old system into FMIS to ensure that the data were accurate and  complete. To maintain the integrity of these data over time, BIA also  introduced various quality control measures for helping ensure that facility  management staff would accurately enter new data into FMIS. Because of  concerns about the quality of the data in the former system, the Congress  directed us to conduct a review in this area. After discussions with  cognizant congressional committees and several Indian education  organizations about the information that would be most helpful, we  focused our efforts on (1) whether BIA’s new facilities management  information system addresses the former system’s shortcomings and  meets BIA’s needs for managing school facilities; (2) the status of BIA’s  effort to validate the accuracy and completeness of the data being  transferred from the old system into FMIS; and (3) how well BIA’s quality  control measures are working to ensure that new data entered into FMIS  are accurate and complete.
To conduct our review of BIA’s new information system, we reviewed the  studies that evaluated BIA’s old information system and determined the  extent that BIA’s new system implements recommendations. We evaluated  data currently in the FMIS database and accompanied BIA’s contractor on  selected site visits designed to validate and update information in the  database. We also interviewed school officials at 8 BIA-funded schools in  Arizona, New Mexico, and South Dakota. We selected these schools to  obtain a mix based on differences in their size, geographic location, and  type of school, such as day school or boarding school. We also interviewed  officials from BIA’s Office of Facilities Management and Construction,  Division of Safety and Risk Management, and Office of Indian Education  Programs, as well as officials from various Indian education organizations.  See appendix I for a more complete description of our scope and  methodology.
We conducted our work between June 2002 and July 2003 in accordance  with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: BIA administers funding for the operation, maintenance, construction, and  repair of school facilities at 171 elementary and secondary schools in   23 states. These schools are located primarily in rural areas and small  towns and serve Indian students living on or near reservations. Many of  these schools include not only educational buildings, but also dormitories  and supporting infrastructure such as water and sewer systems. BIA  operates 64 of the schools directly while the others are operated by tribes  through separate grant or contract agreements.  We previously reported on  issues related to the condition of BIA school facilities in 1997 and 2001.
BIA’s Office of Facilities Management and Construction (OFMC) is  responsible for overseeing FMIS. At both BIA-operated and tribal-operated  schools, it is the responsibility of the facility managers to enter data about  the inventory and condition of their schools into the system. Prior to  acceptance into FMIS, these draft data entries are reviewed and approved  by facility managers at BIA agency and regional offices respectively,  before final review and approval by a BIA contractor and BIA’s central  office.
For 22 years BIA relied on its Facility Construction Operations and  Maintenance (FACCOM) system to maintain inventory data for its annual  O&M program, as well as “backlog” data that reflect repairs and  improvements needed outside of the annual maintenance program to  improve the facilities’ condition now and in the future. These data assist  BIA in monitoring the status of facilities repair and new construction  projects and identifying funding needs for O&M and renovation. However,  as BIA’s needs began to change, BIA managers realized that FACCOM had  limitations and acknowledged that there were serious concerns with the  accuracy and completeness of these data.
As shown in figure 1, BIA’s efforts to replace FACCOM began in 1995 when  one of its contractors issued a report about the FACCOM system’s  shortcomings and recommended actions for improvement. In 1995, BIA  entered into a contract with Anteon Corporation, a system developer, to  design the new management information system. Relying on government  standards, BIA worked with Anteon Corporation to design the new  system and address FACCOM’s shortcomings. In 1999, BIA contracted  with an engineering firm, Applied Management Engineering, Inc. (AME), to  conduct a survey at all school sites in order to validate the schools’  condition data and to verify the presence of buildings and their use.  According to BIA officials, after AME validated each school’s inventory  and condition data, and BIA approved it, the data were accepted into  FMIS.
From fiscal years 1995 through 2002, BIA spent nearly $12.5 million to  develop and begin implementing FMIS. These costs include about   $8 million for contractor expenses and over $2.6 million for BIA in-house  expenses, which covered the design of FMIS and ongoing technical  support. During fiscal years 1999- 2003, BIA spent about $13 million for the  AME contract covering the validation of inventory and condition data and  other engineering support activities. To operate FMIS, BIA expects to  spend about $1.7 million annually through fiscal year 2006 for contractor  expenses and about $250,000 for in-house expenses. In fiscal year   2007, BIA hopes to move to an annual steady rate of about $750,000 for  contractor costs and about $250,000 for in-house costs. To continue having  AME reassess and validate the schools’ inventory and condition data, BIA  projects to spend over $8.3 million from fiscal years 2004 through 2006 on  contract expenses.
section_title: FMIS’ Design Addresses Previous Information System’s Shortcomings and Appears to Have the Capability to Meet BIA’s Needs
paragraphs: Recognizing the FACCOM system’s shortcomings, BIA worked with its  system developer to design a new management information system that  would assist in resolving many of the weaknesses identified with the old  system, including those related to difficulty of use and accuracy of data.  FMIS is more user friendly and it is designed to meet facility managers’  needs at all levels within BIA by serving as both an information  management system and as a project management tool. FMIS incorporates  modules, including the inventory and backlog modules, which help facility  managers make decisions regarding the condition of the school facilities  to provide a safe environment for their students. The inventory module  contains information such as the physical characteristics and use of  buildings and is used to make funding decisions for annual operating  expenses and routine maintenance. The backlog module contains data that  tracks detailed information about the physical condition of a school’s  facility and is used to prioritize and fund repair projects, capital  improvements, and construction.
subsections: section_title: FMIS Is Designed to Resolve Shortcomings with the Previous Information System
paragraphs: FMIS is designed to better support the day-to-day activities of the facility  management staff by being more user friendly. FMIS is a Windows-based  system that provides a point-and-click feature, which makes it easy to  navigate the system without having to remember codes. This is important  for FMIS users, because some facility managers have little prior exposure  to computers. Facility managers we interviewed at the school sites and  agency and regional offices said that compared to FACCOM, FMIS is  better and easier to use. One facility manager in Arizona said FMIS is  easier to use because the system automatically sends messages to him  when changes are made to the data, allowing him to instantly see when  updates have been made to his school’s data. Another facility manager in  Arizona said FMIS’s automated functions, such as drop-down menus,  make it user friendly. Finally, according to BIA officials, FACCOM was  only accessible by about 3-4 percent of facility managers, which did not  include facility managers located at the schools. FMIS is designed to be  accessible by all of BIA’s facility managers, including those at the school  sites, via an Internet connection. Although most of the facility managers  we visited at the schools said FMIS was better compared to FACCOM,  many had been unable to access FMIS at school sites since December of  2001 due to a court order that shut down access to BIA’s Internet site.  These facility managers had to travel to agency or regional offices to enter  data or had to forward data to these offices for data entry.
FMIS is also designed to help BIA employees improve the accuracy of the  data, in part through automated mechanisms that help facility managers  consistently describe the category and rank of backlog entries and the  funding needed to address them. One difficulty under FACCOM was that  entries listed in the backlog were often categorized incorrectly with  inflated priorities, making it difficult to determine which projects needed  immediate attention. To address this problem, FMIS is designed to restrict  who can enter safety deficiencies, which are given first priority for  funding. In addition, FMIS is designed to only accept entries that have  been reviewed and approved by three different levels of BIA management  and the contractor. Further, BIA refined the definitions of how backlog  items should be categorized and ranked to help facility managers use the  definitions consistently. The definitions now include nine categories and a  ranking system for determining the priority of the items entered into the  backlog (see table 1). These nine categories describe whether the  deficiency at the school affects safety and endangers students’ lives;  violates an environmental, disability, or energy standard; is a maintenance  or capital improvement item; or requires an emergency repair. In addition,  the system ranks items using a scale from one through three—with one  describing the most severe deficiency. The backlog entries ranked as a   “1” will most likely be funded first because they are the highest priority.  FMIS requires facility managers to enter an associated category and rank  for all items entered into the backlog. Such a process, with its greater  specificity in how to categorize and rank deficiencies, can help facility  managers improve consistency in the data entered for all 171 schools.
U-1—an unforeseen event in which danger exists that could   reasonably be expected to cause death, physical harm,    or property damage.
S-1—serious deficiency that poses a threat to safety and    health, such as fire safety violations.
S-2—moderate deficiency such as poor lighting or trip or fall   hazards. (maintenance)
M-1—deficiency related to the structural, mechanical, or    electrical systems that render it inoperable, such as the   deterioration of a roof that causes interior building    damage.
M-2—deficiency related to the facility, systems, or grounds,   such as replacing worn door locks that are inoperable  M-3—functional facility equipment exceeds its normal life   expectancy.
H-1—serious code deficiency, such as the lack of accessible   door hardware.
H-2—violation of codes and standards, such as the lack of    code compliant accessible handrails.
X-1—serious code deficiency that poses a threat to life or    property, such as removing friable asbestos in occupied    areas.
X-2—code deficiency, such as removing asbestos floor tiles    from a building.
R-3—backlogs identified for future planning to determine the    life cycle needs beyond the 5-year plan.
C-1—to replace buildings with serious code/safety deficiencies  or to abate numerous high cost code violations that    meet or exceed the replacement cost rule.
C-2—to accommodate functional or programmatic needs, such  as replacing an undersized dining room to    accommodate the student population.
P-2—to change the functional space or to accommodate    programmatic space needs, such as retrofitting an    existing classroom into a computer laboratory.
E-2—violation of energy codes and standards, such as    upgrading or replacing inefficient heating systems.
E-3—deficiencies, which when corrected will reduce energy    use, such as replacing weather seals on exterior doors.
To help facility managers develop accurate and consistent cost estimates  to address backlog items, FMIS is designed to operate with a software  program that helps facility managers accomplish industry standard cost  estimates for replacement, renovation, or construction projects over  $5,000. A facility manager at a school site we visited said that this software  tool eliminates the need to make calculations by hand, and thus greatly  assists him in estimating accurate costs for school projects.
Another accuracy-related area that plagued FACCOM was that projects  would continue on the active backlog list even after completion. The FMIS  backlog module is designed with a “backlog completion screen” that  stores completion dates, costs, and narrative comments. This function  helps facility managers monitor the length of time that funded backlog  items remained on the backlog without being completed. Unlike FACCOM,  FMIS is designed with a tracking function that identifies the name of the  person who entered or updated a particular backlog item. Managers can  use this function to seek clarification or justification for items that have  not been completed within a reasonable timeframe.
section_title: FMIS Is Designed to Assist BIA in Making Funding Decisions for Annual Operating Expenses and Routine Maintenance
paragraphs: The FMIS inventory module, one of six modules in the system, is designed  to assist BIA in determining operations and maintenance funding for its  school facilities. Specifically, the inventory module is designed to manage  information about all of BIA’s school buildings, rooms, towers (such as a  water tower), and grounds, along with their associated inventory items,  such as stairs, sidewalks, or playgrounds. The inventory module also  details if the property is owned, operated, or maintained by BIA directly or  under contract or grant. The inventory module is designed with the  capacity to integrate with other FMIS modules in order to generate reports  and provide detailed documentation for federal funding purposes.
At the schools we visited, several facility managers said the inventory  function in FMIS helps them to better manage their school facilities. For  example, one school facility manager in Arizona said FMIS helps him more  accurately keep track of his school’s inventory and allowed him to enter  the information that is necessary for BIA to make good funding decisions.  Another facility manager in South Dakota said that because only one staff  person can enter information into the inventory module, his school is able  to maintain consistency in inventory changes and additions.
section_title: FMIS Is Designed to Help BIA Prioritize and Justify Funding for Repair Projects and Capital Improvements
paragraphs: The FMIS backlog module is designed to help BIA officials prioritize and  make funding decisions for needed repair projects and capital  improvements. The backlog collects and tracks condition data related to  deficiencies, capital improvements, or construction for specific inventory  items, such as classrooms, sidewalks, or utility systems. These data are  entered into FMIS by a facility manager, by safety officers as a part of a  safety inspection, or by BIA’s contractor. Although the backlog module  can store information about any identified deficiency, the only items that  are reported as part of BIA’s backlog are those with an estimated cost of  more than $1,000 to fix. These deficiencies, which may be grouped  together to form repair, replacement, or construction projects are  maintained in the backlog until funded and complete. During our site  visits, many of the facility managers said the backlog module helped them  to better manage their facilities. For example, one school facility manager  in Arizona said FMIS’s ability to store digital pictures was helpful because  a picture of a deficiency could be sent to the regional facility manager and  reviewed without the facility manager traveling nearly 284 miles to the  school.
Facility managers at schools use the information in the backlog module to  justify funding needs for repair projects and capital improvements at their  schools. BIA management officials allocating funding among the schools  said the data in the backlog module allow them to determine which  deficiencies are related to student safety and need to be addressed  immediately, and which are related to capital improvements, such as roof  replacement, that are planned for the future. BIA officials said they use the  backlog data to help improve the physical condition of their schools in  order to provide a safe and healthy learning environment for the students.
section_title: Inventory and Backlog Data Have Been Validated, but Not Transferred to FMIS
paragraphs: BIA’s engineering contractor has corrected the inventory and backlog data  that existed in the old data system, but BIA has not transferred all  corrected information to the new FMIS. The contractor completed its  validation of the inventory data in February 2003, and BIA plans to transfer  the corrected data into the FMIS by August 2003. BIA officials expect that  the corrected inventory data, in conjunction with improvement to the  existing funding formula, will enhance their ability to better match  funding with annual expenses for utilities and routine maintenance at each  school site. For the backlog data, BIA’s contractor is in the third year of  the second cycle review of the condition of BIA-funded schools as  planned. In fiscal year 2002, the contractor visited 33 schools and  identified corrections to add, delete, or adjust the FMIS data. BIA,  however, had not entered the results of these condition assessments into  FMIS for over 1 year. BIA officials attribute the long delays in correcting  the FMIS condition data to a revised process for verifying contractor data  and to software compatibility problems that they say are being addressed.
subsections: section_title: Contractor Has Finished Correcting Inventory Data; BIA Plans to Transfer the Data into FMIS Later in 2003
paragraphs: The FMIS inventory module contains data on BIA facilities, including  almost 2,200 separate buildings that are occupied by or used for BIA- funded schools. More than 50 percent (1,146) of the buildings are used  directly by children, as shown in table 2.
Accurate and up-to-date inventory data are crucial to the operation of the  entire FMIS because other modules rely on inventory data for planning  and prioritizing the work and for identifying and prioritizing funding  needs. For example:    School facility managers access inventory data when planning and  scheduling routine maintenance of their facilities, grounds, and  equipment. For example, one FMIS module acts as a scheduling tool to  inform facilities managers about work, such as preventive  maintenance, that needs to be done to buildings, equipment, and other  physical assets listed in the inventory.
BIA’s Office of Facilities Management and Construction uses inventory  data in the formula that determines the amount of operations and  maintenance funding allocated to each school location. Distribution of  this funding is calculated using a formula that includes such inventory  data as the square footage of rooms in each building and systems that  support the facility such as heating and cooling systems. Funding  distributions have been a particular source of contention for Office of  Indian Education officials, who told us that inaccuracies in the  inventory data have led to inequities in how the money is apportioned.  Accurate data that are collected using a methodology that is consistent  from site to site is a necessary component for demonstrating the  fairness of the process.
BIA’s engineering contractor, AME, has remained on schedule in its effort  to improve the accuracy of the inventory information currently in FMIS. In  conducting this effort, AME visited each school and collected inventory  data using a standardized, industry-based approach to help ensure that  information on all facilities is uniformly collected and recorded. AME  completed the first phase of this effort in 2000, when it visited each school  to verify and update the inventory data listed in the FACCOM before its  transfer to FMIS in that year, according to a BIA official. This phase, which  focused on the more general aspects of the inventory, was aimed at such  matters as identifying which buildings were still in use, the use of the  facility, and who owns it. AME completed the second phase of the  improvement effort in February 2003. This second phase, which took  longer than the first phase, involved a more extensive measurement of the  buildings and updated the drawings of floor space, grounds, infrastructure,  and utility lines. BIA’s current plans call for replacing the existing data in  FMIS with this updated and corrected data by August 2003.
Our preliminary review of the new data generated during the second phase  of AME’s work indicates that the inventory figures may change  considerably for some schools. At our request, AME officials provided  revised square footage data for more than 90 buildings (such as classroom  buildings, dormitories, multipurpose buildings, and offices) at 13 different  schools. Overall, the revised measurements for these buildings decreased  the total square footage by about 3 percent, but the range in increases and  decreases at each school varied significantly. For 9 of the schools the  decrease in square footage ranged from less than 1 percent to more than  13 percent; increases for the remaining 4 schools, ranged from less than   1 percent to almost 18 percent. We do not know if these results will be  typical for all schools. One BIA official indicated, however, that some  schools were likely to experience greater changes than others in the  square footage that would qualify for O&M funding.
BIA officials said that the improved data, along with improvements to the  funding formula, will help ensure the various schools that their share of  O&M funding was objectively and accurately determined. However,  whether this corrected inventory data will be transferred to FMIS in time  for making fiscal year 2004 funding decisions is uncertain. BIA may yet  face some implementation problems as it moves into the final months of  putting this information in place. For example, during 2002, BIA attempted  to run the O&M funding formula using the existing FMIS inventory data for  the first time and experienced problems with the data and software. While  the agency has had a year to work out these problems, introduction of the  updated inventory data may hold its own unforeseen problems. If such  problems are encountered and remain unresolved, a BIA official told us  that the agency would continue to use the data currently in the system to  allocate the O&M funding for fiscal year 2004.
section_title: Contractor Is on Schedule in Validating Backlog Data, but BIA Is Behind Schedule in Incorporating the Updated Information
paragraphs: The backlog data in FMIS reflects actions and funding needed to improve  the condition of facilities and infrastructure at the various schools now  and in the future. Most of the items listed in the backlog provide detail for  repairs needed over the next 5 years to correct what is wrong with a  facility such as a leaky roof, the presence of asbestos, or a violation of  handicapped codes and standards. However, FMIS also includes entries  for capital improvements that will need to be addressed beyond 5 years to  upgrade specific building components such as replacing lighting and  power systems, siding, and carpeting as well as future construction to  replace, renovate, and add buildings to accommodate program needs.  Accurate and up-to-date backlog data are important because FMIS  contains formulas that use these data to allow BIA to make informed  decisions not only about which projects are in greatest need of attention,  but also how much money is needed to fund them each year. As of  October 2002, for example, BIA schools had a backlog of unfunded repairs  and improvements with an estimated cost over $640 million (see table 3);  FMIS shows that almost two-thirds of this amount may be needed within  the next 5 years.
AME currently validates the backlog for each school in a 3-year cycle.  During its first review, in 1999, AME conducted a 100-percent validation of  the backlog data prior to transferring that data from the old FACCOM  system to the new FMIS backlog module. In that review, AME updated the  backlog data by confirming entries already in the system, updating the  costs estimated to conduct the work, deleting entries for duplicate items  or completed work, and identifying new entries. Results of the validation  effort, as shown in table 4, increased the backlog by more than   $265 million; almost 28 percent of the total backlog of $960 million that  existed in 1999.
AME has since started the second review of each school, which consists of  updating this information for a certain percentage of the facilities each  year. AME updated 20 percent of the facilities in each year 2001 and  2002. For 2003, AME is on track to increase the percentage of facilities  reviewed each year to 34 percent to comply with recent changes in the  law. These updates involve visually inspecting the architectural,  structural, mechanical, and electrical components of each facility to  determine if action is still needed and to update the estimated costs. In  addition, AME identifies new deficiencies, including the extent to which  handicapped accessibility requirements are met, and verifying estimated  costs.
Our review of some updates conducted during fiscal year 2002 indicates  that AME’s reviews will continue to result in substantial changes in  backlog data. We obtained data for 14 of the schools reviewed between  February and April 2002. For the 14 schools, the AME update resulted in  a net increase of almost $11 million (see table 5) in the unfunded FMIS  backlog of more than $39 million, an increase of about 28 percent. Part of  the change involved modifications to deficiencies already in the backlog  inventory, such as revising cost estimates and deleting projects that had  been completed or no longer needed but were still listed in the backlog as  ongoing. However, a large part of the change involved adding new  deficiencies to the backlog. In all, there were almost 650 new backlog  entries, and more than 75 percent of these entries were for deficiencies  identified at school facilities and dormitories, where children are the  primary occupants. For example, there were more than 200 entries for  dormitories with an estimated cost of almost $4 million. None of these  entries were for urgent or safety deficiencies that needed immediate  attention; most were maintenance deficiencies that will need attention  over the next 2 to 5 years, such as repairing lighting and plumbing systems,  carpeting, and ceilings.
While the contractor’s field assessments are proceeding on schedule, there  have been significant delays in incorporating the 2002 updated backlog  data into FMIS. In fiscal year 2001, the first year of the updates, the  contractor assessed the condition of 39 schools and transferred the data  from its information system to the FMIS without problems, according to  BIA and contractor officials. However, officials said that in fiscal year  2002, the untimely transfer of data from the field assessments to the  contractor’s information system and software compatibility problems  between the contractor’s information system and FMIS delayed the update  of backlog data for 33 schools for over a year. These implementation  problems occurred for such reasons as the following:    BIA added a new function to the FMIS, which took 6 months to  implement, rather than the 2 months that they had planned, according  to a BIA official. This new function involved using the FMIS, for the  first time, to generate the O&M funding amounts to be distributed to  the various schools.
A change to the backlog category and ranking system in FMIS created  duplicate entries that took time to resolve. In the 2002 assessments,  some deficiencies that were already in the system were recategorized,  and when the update was transferred to FMIS a duplicate entry was  created instead of overwriting the old entry. According to BIA and  contractor officials, this software compatibility problem has largely  been addressed. As of April 2003, data for 27 of the 33 schools had been  entered into FMIS.
A BIA official told us that delays in introducing these updates into the  FMIS backlog could have some impact on their ability to prioritize or fund  repair and capital improvement projects, but not a significant impact for  two reasons. First, the deficiencies that receive the highest points in the  project ranking system are safety deficiencies, which are identified and  updated in an annual safety inspection by BIA’s safety officers and not  AME. Second, the most critical deficiencies at the schools were identified  in the first assessment in 1999, and during this second assessment, AME is  finding very few deficiencies that would be funded within 1 to 2 years that  were not already in the system. However, one official acknowledged that  since the FMIS ranks the schools for major repair and capital  improvement projects based on the points applied to each deficiency, if  AME’s assessment indicated an increase in the severity of a deficiency and  that change was not reflected in the system, that school could be ranked  lower than if the data were up to date.
Our site visits to 8 BIA-funded schools did not disclose any instances  where serious problems were not being addressed. While facilities  management staff and school principals pointed out problems with their  facilities, we did not observe that the children were in an unsafe learning  environment with obvious safety or repair issues. The problems we  observed were either of a less serious nature, or if serious, were being  addressed in some form. For example, at a boarding school in Arizona the  principal said that the fire alarm system for the school building and  dormitory had been improperly installed and had to be replaced. While  waiting for the funding for a new system, which had been approved, they  had to use funds from the school budget to hire extra people to stand fire  watch 24 hours a day.
section_title: Control Measures in Place for Ensuring Data Quality Are Largely Ineffective
paragraphs: The ability of FMIS to provide accurate and complete data depends on BIA  employees entering correct and timely information, but review processes  and training programs BIA has established for ensuring data quality have  been largely ineffective. Although adherence to federal control standards  are a major part of providing reasonable assurance that the objectives of  the agency are being achieved, half of the entries proposed by BIA  employees are incorrect or incomplete and are flagged by BIA’s  contractor. Discussions with BIA employees indicate that some employees  are unclear about their responsibilities in maintaining the accuracy of  FMIS data, and the high error rates in data entry indicate that additional  training is needed in some locations to improve performance. BIA has not  analyzed the information it has available about the content and origination  of the data errors to determine the type of additional training that might be  needed or to target locations with the highest error rates for technical  assistance. Further, BIA has not established criteria or performance goals  that define its expectations for the accuracy and completeness of the data  for employees that enter and review this information. BIA’s Office of  Facilities Management and Construction, which manages FMIS, did not  until recently have authority to establish criteria or performance goals for  agency and regional office personnel that are responsible for reviewing  and approving data entries by school facility managers. BIA’s OFMC still  does not have similar authority over school facility managers who  originate most data entries. Under BIA’s current organization, such action  would have to be taken by the Office of Indian Education Programs.
subsections: section_title: Despite BIA Review Process, Its Contractor Rejects Half of All FMIS Data Entries
paragraphs: BIA established a multilevel review process as a control to prevent  problems related to inaccurate and incomplete data entries to BIA’s  information system. In this process, each entry that school facility  managers propose for the backlog is first reviewed and approved by BIA  facility management personnel at an agency and a regional office before it  is sent to BIA’s contractor, AME, for review and approval, with final  approval by BIA’s central office. After approval has been obtained from  each level, the status of the entry is changed from “draft” to “accepted” in  FMIS, according to BIA.
Although BIA established this multilevel review process to improve the  quality of the data entered into FMIS, AME continues to reject half of the  proposed entries because they are inaccurate and incomplete. For  example, between August 2001 and December 2002, out of more than   650 entries to the backlog made by facilities management staff from   102 schools, more than 300, or almost 50 percent, were rejected by the  contractor. BIA documents show that the incidence of the errors among  the 102 schools was widespread. In all, 73 of the 102 schools entering data  had one or more entries disapproved, and 33 of them had all entries  disapproved (see table 6).
section_title: Roles and Responsibilities May Be Unclear to Employees Reviewing Data Entries
paragraphs: Facility management staff at the regional and agency offices appear to  have the necessary background to fulfill their responsibilities for screening  and correcting inaccurate and incomplete data entries; however, it may be  unclear to some reviewers what their role is in this process. Federal  control standards require that employees have the requisite knowledge,  skills, and abilities to perform their job appropriately and have clear roles  and responsibilities outlined in their job descriptions. Among the facilities  management staffs in the agency and regional offices that we interviewed,  most had an engineering background or had significant experience in  facilities management and they were aware that maintaining accurate and  complete data in the FMIS was important. However, facilities managers we  interviewed at two regional offices each had a different view about their  role in the review process. One said that it was important that the backlog  entries are consistent and that two staff had been designated to review the  FMIS entries from the schools and agencies in the region before being sent  on to the central office for review by the contractor. The second manager  said he reviewed the entries so that he knew what new additions were  being proposed for the backlog; he did not consider it his role to critique  the entries for accuracy and completeness.
section_title: Training Program May be Insufficient to Prepare Users
paragraphs: Although most BIA locations have staffs that have received training to use  FMIS, the extent of the errors indicates that employees may not be  receiving the kind of training needed to create accurate and complete data  entries. Standards for internal control in the federal government include a  commitment to competence, which includes the provision that employees  receive the appropriate training necessary to improve their performance.  BIA has developed a training program that is intended to provide FMIS  users with sufficient information to operate the system. All FMIS users  receive 40 hours of training before they are given a password that allows  them to access the system, according to BIA; a review of BIA’s training log  indicates that about 70 percent of the schools have at least one staff  member onsite that has received this training.
While training appears to be adequate in terms of providing staff with the  basic skills needed to use the computer-based applications, what appears  to be lacking in the training program is more specific instruction and  guidance on the kinds of information that are needed to enter an accurate  and complete deficiency to the backlog. BIA officials acknowledged that a  user manual to provide this type of guidance was lacking and should be  developed. BIA officials said developing the FMIS training has been a  challenge because the facilities management staffs have different levels of  knowledge about computers—for some facility managers, FMIS training  was their first exposure to using a computer. The training programs were  developed to meet the needs of this diverse group of users, according to  the contractor that developed and provides the training. In our site visits  we asked users about the training that they had received for using FMIS;  almost all of the staff that we interviewed said that they were pleased with  the training they received and believed that it had prepared them for using  FMIS. In addition, they said that when they did have problems they  contacted central office with questions and/or problems with using the  system, the response was prompt and very helpful.
However, the engineering contractor indicated that the varying levels of  experience and expertise is a difficulty affecting staff’s ability to input data  successfully. He said that there are more than 180 sites with education  buildings for which data must be entered and the level of knowledge and  expertise about facilities and the kinds of information needed for an FMIS  entry varies widely—particularly in those sites where the turnover rate of  facilities management staff is high. We were unable to obtain  comprehensive data on the turnover rate for the facility management staff,  but among the schools that we visited, the facilities managers’ length of  employment at their current location ranged from 2 years to 17 years.
section_title: BIA Does Not Use Information on Error Rates to Improve Employee Performance
paragraphs: BIA officials have information on the number and reasons that data entries  are rejected at each location, but said that they had not used this  information to provide performance counseling to employees or modify  the training and guidance in this regard. Standards for internal control in  the federal government include the provision that employees receive the  feedback necessary to improve their performance. Analyzing the extent  and content of data errors would be helpful to determine the type and  amount of additional training and guidance needed to improve employee  performance at schools, agencies, and regional offices and target them  appropriately for technical assistance. BIA data we reviewed indicated  that the reasons for disapproval generally fell into one of four groups and  were consistent with historical data entry problems experienced under the  old FACCOM system.  For example,     more detailed description was needed for the deficiency; a roof repair,  for example, required specific information about the kind of roof and  its size;    questionable cost estimates involving labor rates or material costs;   duplicate entry for a deficiency already in the backlog; and   wrong category and/or rank for the entry such as categorizing the  replacement of asbestos floor tiles as maintenance rather than  environmental, which are funded from different sources.
Of these problems, most of the rejected backlog entries generally related  to insufficient detail to accurately estimate the cost to address the  deficiency, according to an AME official who reviewed and rejected many  of these entries. The kind of detail that is needed to successfully enter a  deficiency can be seen in an example involving the repair of a leaking roof.  To adequately estimate the cost for this repair, information is needed  about the size of the area that needs repair or replacement, the  composition of the roof (such as asphalt shingles or tile), and other  associated components (such as whether skylights or gutters are present  and whether they also need to be replaced). We reviewed one entry that  had been rejected by the contractor because it came through the review  process without information about the roof’s size or its composition.
section_title: Employees Not Held Accountable for FMIS Data Integrity
paragraphs: The ineffectiveness of BIA’s guidance, training, and review processes to  minimize inaccurate or incomplete data entries by its employees suggests  that accountability is another issue that deserves attention. Federal  standards require that agencies clearly establish authority and  responsibility for achieving agency goals and hold their employees  accountable for performing their assigned responsibilities in a competent  manner. During our review, the organization of BIA was such that the  office that manages FMIS did not have line authority to establish  performance criteria and standards for the BIA employees that entered  and reviewed the FMIS data.
In April 2003, BIA announced a new organization plan. This plan may offer  OFMC greater opportunity to establish performance criteria and standards  for facility managers at agency and regional offices. BIA officials told us  the reorganization would not provide OFMC with line authority for facility  managers at schools. However, the Director of the Office of Indian  Education programs said that his office would work with OFMC to  establish comparable performance criteria and standards for school  facility managers.
section_title: Conclusion
paragraphs: FMIS is designed to assist BIA employees improve the quality of  information used to manage school facilities, but the quality of the  decisions that BIA makes for managing the operations and maintenance,  repair, and construction of its facilities is directly dependent upon BIA  employees entering correct and timely information. Currently, the FMIS  data that BIA uses for making its decisions are improving as the data are  updated by its contractor and entered into FMIS—to date the inventory  data have been updated and the contractor is in the third year of assessing  the condition of the schools and updating the backlog for the second time.  However, challenges remain in BIA’s efforts to improve the quality of data  entered by its employees. Although BIA has implemented controls for  ensuring the accuracy and completeness of the FMIS data entered and  reviewed by BIA employees, they do not work effectively. Without the role  of the contractor as a reviewer of new entries by field staff and in  conducting site visits to verify and update the data, the quality of the FMIS  data could quickly become inaccurate and out of date. BIA has not taken  the necessary steps to hold its staff accountable for data accuracy or to  use the available information on why the problems exist to develop  training programs and target technical assistance where it is needed. Such  actions are needed if BIA is to rely on its employees, rather than the  contractor, to ensure that it provides a safe and quality learning  environment for Indian children.
section_title: Recommendation for Executive Action
paragraphs: To better enable BIA to rely on its employees for maintaining accurate and  complete information in the FMIS, we recommend that the Office of the  Assistant Secretary of Indian Affairs    establish data standards for accuracy and completeness of FMIS data  and related performance criteria for BIA employees who are  responsible for entering and reviewing the data and    analyze available error data and use this information to provide its  employees with the necessary training, guidance, and technical  assistance to improve performance.
section_title: Agency Comments on Our Evaluation
paragraphs: We provided a draft of this report to the Department of the Interior for its  review and comment. Interior’s comments are provided in appendix II.  In  its written comments, Interior agreed with our findings and  recommendations and said that BIA is establishing a special working  group to develop a plan to address our recommendations.  In addition, BIA  will consider our comments and observations as it continues to develop  and implement the FMIS.
We will send copies of this report to the Secretary of the Department of  Interior, relevant congressional committees, Indian education  organizations, and other interested parties, and will make copies available  to others upon request. In addition, the report will be available at no  charge on GAO’s Web site at http://www.gao.gov. Please contact me at   (202) 512-6778 if you or your staff have any questions about this report.  Other major contributors to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this study were to determine (1) whether the Bureau of  Indian Affairs (BIA) new facilities management information system (FMIS)  addresses the former system’s shortcomings and meets BIA’s needs for  managing school facilities; (2) the status of BIA’s effort to validate the  accuracy and completeness of the data being transferred from the old  system into FMIS; and (3) how well BIA’s quality control measures are  working to ensure that new data entered into FMIS are accurate and  complete.
To determine the extent to which FMIS was designed to address  weaknesses of the previous data processing system and how the new  system meets BIA’s facility management needs we reviewed contractor  reports, BIA documentation on the FMIS, and interviewed contractors and  BIA headquarters, regional, agency, and school facility management staff.  First, we reviewed the needs assessment studies conducted by  independent contractors to identify old system weaknesses and the  recommendations made for addressing the system problems. We then  reviewed the FMIS documentation to determine whether the system  addressed weaknesses identified in the needs assessment. In addition, we  conducted interviews with some of the contractors hired by BIA to build  and implement the system. We also interviewed BIA officials at the Office  of Facilities Management and Construction, the Division of Safety and  Risk Management, and the Office of Indian Education Programs about  improvements in the new system and how it meets their management  needs. Finally, to understand how well school facility management staff  received FMIS, we conducted site visits to 8 schools in Arizona, New  Mexico, and South Dakota. We selected these schools to obtain a mix  based on their differences in size, geographic location, type of school   (i.e., grade level, day school, or on-reservation boarding school), and  whether it was BIA-operated or tribal-operated. At these schools, we  interviewed facility managers, education line officers, principals, and tribal  officials. We also interviewed facility management staff at two regional  and five agency offices that provide facility management services to the  schools.
To determine the extent to which the FMIS inventory and backlog data are  accurate and complete we used three methods. First, we obtained data  from two sources (1) a copy of the FMIS database and (2) a copy of the  contractor file of backlog data from the fiscal year 2002 condition  assessments of 14 schools. The data in these files were assessed for  reliability, which included looking for missing data, the relationship of one  data element to another, values beyond a given range, and dates outside of  valid time frames. We determined that the data were sufficiently reliable  for the purposes of this report. We also calculated summary statistics of  the data from these files. Second, we evaluated BIA’s multilevel review  process. For this analysis, we obtained data on the number of backlog  entries made by facility managers and the number that had been accepted  and rejected by the contractor. In addition, we reviewed the log of entries  that had been rejected by the contractor to understand the reasons for  these rejections. We also interviewed the contractor and BIA central office  and regional and agency facility management staff about this review  process. Finally, we accompanied BIA’s engineering contractor on site  visits to 2 schools in Arizona where we evaluated their methodologies for  data collection and validation of the inventory and backlog data.
To determine how well BIA’s internal control measures are working for its  FMIS we first reviewed our standards on internal controls to identify  controls that apply to an organization’s management of an information  system. We then compared the BIA controls with those identified in our  reports to evaluate the effectiveness of the BIA controls. Additionally, we  interviewed staff at the BIA central office about the internal controls they  had in place for the FMIS. We also interviewed staff at the regional and  agency offices and the schools about one of the controls—the  effectiveness of the training they received.
section_title: Appendix II: Comments from the Department of the Interior
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the individuals mentioned above, Jessica Botsford, Maya  Chakko, Terry Dorn, David Gill, Barbara Johnson, Nathan Morris, Stan  Stenerson, and Michelle Zapata made significant contributions to this  report.