Following terrorist attacks against the U.S. embassy in Beirut, Lebanon, in 1983, State began an embassy construction program—known as the Inman program—to protect U.S. personnel. However, State completed only 24 of the 57 planned construction projects, in part due to poor planning, systemic weaknesses in program management, difficulties acquiring sites, schedule delays, cost increases, and subsequent funding limitations. Following the demise of the Inman program in the early 1990s, State initiated very few new construction projects until after the two 1998 embassy bombings in Africa. Following those attacks, the Secure Embassy Construction and Counterterrorism Act of 1999 required State to maintain a list of diplomatic facilities to be scheduled for replacement based on their vulnerability to attack. In response, State initiated the Capital Security Construction Program to construct new, secure facilities overseas. At that time, State determined that diplomatic facilities at more than 180 posts—more than half of U.S. overseas missions—needed to be replaced to meet security standards. In 2016, State reported that from 2000 through 2014, it moved over 30,000 staff into more secure facilities. The Secure Embassy Construction and Counterterrorism Act of 1999 calls for new diplomatic facilities to be sufficiently sized to ensure that all U.S. government personnel at a post are located on a single secure site and that those facilities are set back not less than 100 feet from the site’s perimeter boundary. Before constructing a new embassy, State must certify to Congress that, among other things, the facility incorporates adequate measures for protecting classified information and activities as well as personnel working in the facilities. OBO contracts with architectural and engineering firms (design firms) to develop designs meeting security and other project requirements. These design firms submit their designs for reviews by OBO and DS to ensure conformance with building code and security standards, respectively. DS, in consultation with the Office of the Director of National Intelligence, must certify that the design meets security standards prior to the start of construction. While this certification occurs in the design phase of a project, DS also has other roles in the process, such as participating in site selection, ensuring OBO contractors have necessary security clearances, and ensuring facilities are securely constructed. To address some of the performance problems experienced during the Inman program, OBO implemented reforms to its business processes in structuring the new Capital Security Construction Program. Among the most prominent reforms were the development of the SED to expedite the planning, contract award, design, and construction of new diplomatic compounds and use of the design-build (DB) project delivery method, which combines responsibility for design and construction under a single contract and allows contractors to begin basic construction before the design is fully completed. Initially there were three common SED classes—small, medium, and large—based on the size of a post. For planning purposes, each size had predefined schedules and costs associated with them. The SED itself was a set of documents providing prototypical plans (for a medium SED), specifications, and design criteria, and explaining how to adapt those to a particular site and project. The SED was not a complete design but rather a standardized template for the structural, spatial, and security requirements of a new embassy compound to guide a contractor’s final design. Compound elements described by the SED generally included the main office building; U.S. Marine Security Guards’ living quarters; a warehouse; a utility building; compound access control buildings and perimeter walls; and parking facilities. The SED also allowed for the standardization of building components such as security windows and doors. Figure 1 shows the prototypical facilities defined by the SED. The main office building within the SED was organized around two parallel wings connected by a central lobby. Occasionally site conditions such as size, shape, or topography required deviating from that typical configuration; OBO refers to such projects as SED “derivatives.” From 2001 through 2015, OBO constructed more than 50 embassies using the SED approach. Figure 2 shows examples of both the typical SED and derivative SED office building configurations. In 2006, we reported that OBO had made significant progress constructing new diplomatic compounds using the SED approach. We found that the average time to design and construct the 18 new embassies completed from 1999 to 2005 was about 3 years (36.7 months). This was nearly 3 years faster than embassies built during the Inman era, even though the newer facilities were significantly larger and more complex. We also found that reforms implemented by OBO, including the switch to the SED and the DB contract delivery method, had reduced project completion times, although it was difficult to quantify the effects of any single reform. In 2007, OBO reported that the SED, combined with DB project delivery, was expected to reduce overall delivery time—from site selection to occupancy—by 34 percent. In 2008, State’s Inspector General found that OBO’s continued use of the SED, in conjunction with the DB delivery method, was generally effective. Additionally, State’s Inspector General found that the SED permitted faster certification of project designs and accreditation by DS because the standardized design specifications were fully vetted for conformity to security standards. OBO took some actions to incorporate sustainability principles in the SED to meet federal energy mandates to reduce energy and water consumption. In 2006 OBO committed, in concert with 20 other federal agencies, to seek common strategies for planning, acquiring, siting, designing, building, operating, and maintaining federal facilities in an energy efficient and environmentally sustainable manner. In 2008, OBO established Leadership in Energy and Environmental Design (LEED) certification as a design standard for its SED projects. In 2009, OBO documentation indicates OBO elevated its sustainability requirement for SED projects from LEED Certified to the higher certification level of LEED Silver. In 2011, OBO announced a new project approach it termed Design Excellence, intended to deliver embassies that (1) best represent the U.S. government overseas, (2) are functional and secure, (3) incorporate sustainable design and energy efficiency, (4) are cost-effective to operate and maintain, (5) have greater proximity to host-government counterparts and users via more centrally-located urban sites, and (6) better respond to the unique needs and context of specific posts. OBO subsequently phased out the SED as the basis for embassy designs, and according to OBO officials, SED specifications, standards and guidance were incorporated into OBO’s Design Standards and Design Guide. In 2013, OBO renamed its approach “Excellence in Diplomatic Facilities” (Excellence) to convey what OBO officials have said is a more holistic effort to improve every aspect of OBO’s operations, including real estate acquisition, security methods and technologies, cost management, construction management, and facilities management. Table 1 shows 23 new construction contracts awarded since Excellence was approved and through fiscal year 2015 at a total value of $3.67 billion, according to State data. Of these, OBO reports 6 as being Excellence projects; the other projects include projects with certain Excellence features in terms of site, permit, or other requirements; SEDs; and derivative SEDs. OBO officials assert that although Excellence was approved in 2011, OBO never planned to award any Excellence construction contracts until fiscal year 2014. No Excellence projects had been completed as of the end of fiscal year 2016 (September 30, 2016). See appendix IV for a timeline illustrating the history of OBO’s building program from 1998 through 2016. Although combining the SED with a DB project delivery method enabled OBO to accelerate the construction of new embassies, concerns raised by various stakeholders about the aesthetics, quality, location, and functionality of SED facilities prompted OBO to take some steps to improve the SED concept and eventually transition to Excellence. These steps included the introduction in 2008 of the design-build with bridging method (bridging), whereby OBO first contracts with a design firm to develop a project-specific, partial design that a construction contractor and its design firm then completes. After a nearly yearlong review, in April 2011 OBO approved a series of recommendations and planned actions to implement Excellence. A significant change announced at this time was OBO’s increased use of design-bid-build (DBB) as another delivery method alongside bridging. Generally under DBB, OBO first solicits and contracts with a design firm to develop a 100-percent design, which is then used to solicit bids from prospective construction contractors. During the SED era, OBO predominately used a DB project delivery method. DB integrates design and construction responsibilities into a single contract. Under this model, the DB contractor is responsible for design and construction and thus bears the risks, such as added cost, for any design problems because it (not OBO) hires the design firm to bring the design to completion. According to industry experts, DB is generally recognized as the best project delivery method for supporting accelerated delivery, in part because the DB contractor may undertake some construction while design is still in progress. Under OBO policy, in the SED approach OBO provided the DB contractor with the SED prototypical design—to include standard site and building plans, technical specifications, design criteria, and instructions for its adaptation for a particular project and contract requirements. The contractor’s design firm would then use the SED documentation to develop a 100-percent design adapted for a site at a particular post, becoming the architect-of-record. According to the AIA, in general, the architect-of-record for a project prepares the bulk of the design and construction drawings and assumes professional responsibility for the design. Although the DB contractor’s design firm completed the project design, OBO’s policy was to hire its own design firm beforehand to conduct project development activities such as due-diligence planning surveys, site studies, and other analyses needed to inform the project’s design. Figure 3 provides an overview of the embassy construction process under OBO’s implementation of DB. According to former senior OBO officials, the OBO Director who had implemented the SED viewed OBO’s mission as needing to build secure embassies as fast as possible and within a fixed budget, given the large number of facilities that State needed to replace. They stated that the then Director’s commitment was that OBO would combine a standardized design with the DB delivery method to speed design and construction and limit the costs to build each embassy to no more than $100 million (based on a large SED). OBO also maintained at that time that the SED generally would take no more than 36 months to build—inclusive of the time for contract acquisition, design, and construction—depending on the post size. According to these former senior OBO officials, those estimates did not always reflect the budget and time needed to build some SED embassies. They also stated that adapting the SED to the unique requirements of some posts—such as a very large consular services operation—was challenging and that the SED did not always account for quality and long-term maintenance and operations cost considerations. In addition, one former OBO official stated that although the emphasis in the SED approach on speed and cost control enabled OBO to promote that it had moved a certain number of people into secure facilities each year, this was an indicator of performance related to a single goal: project delivery. He noted that OBO did not use any performance indicators related to design and construction quality to evaluate the new SED facilities being built. Although the SED approach enabled OBO to accelerate the construction of new embassies intended to meet rigorous new security requirements, some stakeholders raised concerns about the aesthetics, quality, location, and functionality of those facilities. Aesthetics. One design firm we spoke with said there were criticisms that SED embassies were “cookie cutter” facilities that looked like fortresses. In 2010, then U.S. Senator John Kerry and former Secretary of Defense William Cohen—key advocates of Excellence—reported newly constructed embassies were not sending the right message. They described new embassies as cold concrete facilities at a forbidding distance hidden away from city life, with little regard for the local surroundings, undermining U.S. diplomats’ message and mission. They asserted that State was constructing a standardized “embassy in a box,” uniform in appearance, quickly assembled, and fortress-like. Quality. According to some former senior OBO officials, OBO’s emphasis on speed and cost under the SED approach resulted in some poor-quality buildings. According to these officials and one design firm we interviewed, the time and budget pressures sometimes resulted in OBO and its contractors making trade-off decisions such as using less costly and lower-quality building systems or materials. For example, one former official reported that the SED approach resulted in some projects where contractors used lesser-quality exterior stone or metal cladding on building exteriors. In some projects, he indicated that contractors installed heating, ventilation, and air-conditioning systems that were minimally acceptable under the SED but not the best solution for the post’s geographic climate. Location. OBO officials commented that in some cases the 10-acre lot specified by the SED required siting the embassy too far from urban centers where foreign government offices and other embassies are located. This issue also arose in a 2007 report to State entitled The Embassy of the Future. Guided by a commission composed of former U.S. ambassadors, among others, the report recommended that State avoid constructing embassies in locations remote from urban centers. It also noted that although the appearance of embassies as influenced by security requirements deserves careful consideration, their location is of higher importance. Functionality. One former OBO official stated that because the SED was “very complete” as a standardized design concept and was based on a completed embassy in Africa, its design was not always conducive to being site adapted and applied to other regions in the world. For example, the design criteria for the heating and cooling systems generally specified by the SED may not always have been the best for climates that are very hot, cold, or humid. Some design firms we spoke with echoed that assessment, saying that project size, site shape or topography, regional climate, or special post needs could render the SED difficult to apply. OBO’s most recent former Director has also stated that the SED did not always permit OBO to meet posts’ varied needs. Former OBO officials told us that, in some cases, functional elements such as warehouses were eliminated from SED projects to deliver them on time and within budget. Additionally, current OBO officials emphasized a need to acknowledge the frequency of which the scope of SED projects were reduced to keep projects on time and within cost, without a corresponding reduction to schedule and budget. Issues with the SED approach have been documented in past OBO and GAO studies. For example, in 2008, OBO initiated a “look back” study to examine shortcomings with the early SED projects (2001-2007). The study identified deficiencies in newly completed SEDs stemming from building functionality issues, construction flaws, maintenance issues, and de-scoped facilities. Our 2010 review also examined functionality at 22 new embassy compounds where construction began in or after fiscal year 1999 and was completed by September 30, 2009. Officials at 21 of the 22 posts reported that the design of some spaces within their facility did not fully meet their functional needs, with an average of five functionality- related issues per post. We reported that in some cases, functionality challenges resulted in the need to conduct costly follow-on projects after posts occupied the embassy. OBO officials assert that by using the SED approach, a deficiency in one project was effectively built into each active project as it was a standardized approach to project delivery. By comparison, OBO officials assert that by designing each project individually, the Excellence approach provides OBO with the ability to more quickly identify and make changes or improvements from project to project. OBO took some steps to improve the exterior look of embassies prior to adopting the Excellence approach. According to former senior OBO officials we spoke with, OBO recognized there were some legitimate criticisms about the aesthetics and architecture of the embassies built under SED. Those officials indicated that one of OBO’s interim Directors initiated a study to improve future embassy projects such that they better fit in with the streets and spaces around an embassy. These officials cited OBO’s 2011 Embassy Perimeter Improvement Concepts & Design Guidelines as a direct effort to improve the exterior appearance of embassies by using various design techniques and landscaping so that they would look less “fortress-like.” OBO had previously prepared a report, in 2008, that reviewed its embassy construction process and the SED. To address some of the problems found in this report—such as the need to balance SED standardization with unique post conditions—OBO’s then-Director approved the use of DB with bridging in 2008. Generally under this delivery method, OBO first contracts with a design firm (the bridging architect) to develop a project-specific, partial design package (bridging design) that conveys State’s design vision and a higher level of detail for key design requirements. Upon completing a project’s bridging design, OBO’s procedure is to separately contract with a DB contractor to complete the design and build the project. Therefore, unlike the SED, each bridging design is project-specific, customized, and separately contracted to a design firm. According to senior OBO officials, as well as construction contractor and design firm officials, the current extent of design represented by Excellence bridging documents varies by project but generally approximates an overall 35- to 50-percent design. Those officials indicated that bridging designs include multiple design disciplines whereby elements such as architectural design may be developed to a far greater extent than others, such as electrical design. Under this method, the DB contractor and its design firm are responsible for completing the design. Figure 4 provides an overview of the embassy construction process under bridging. In 2009, we reported that by providing more design detail up front, OBO believed bridging would more effectively translate project requirements to contractors, speed the design security certification process, and enable construction to begin sooner. OBO documentation also indicates that OBO believed bridging would better define the desired look and quality for projects than the SED alone could achieve and provide less room for the contractor to make interpretations and change OBO’s vision of the project. However, according to OBO documentation, the effort, cost, and time to produce a contract solicitation with bridging documents can be significantly more than that required for a typical DB contract using the SED. In 2009, the American Institute of Architects (AIA) submitted a report to OBO entitled Design for Diplomacy, New Embassies for the 21st Century. Informed by a task force composed of architects, engineers, former ambassadors, staff from the U.S. General Services Administration (GSA), OBO design professionals, and others, the AIA recommended that OBO “adopt Design Excellence as a mandate to advance a new generation of secure, high performance embassies and diplomatic facilities that support the conduct of American diplomacy.” It outlined several actions it viewed as necessary to realize the benefits of design excellence. AIA officials we spoke with said that AIA never expressly advocated that the SED be completely abandoned, because that approach might remain appropriate for some projects. However, these officials noted that as more SED projects were built, AIA’s members (i.e., architects) believed that SEDs were not the optimal choice for most projects, as the standardized design was not always conducive to adapting to different climates, countries, or unique post functions. In April 2010—a year before formally instituting Excellence—OBO released “Guiding Principles of Design Excellence in Diplomatic Facilities” (Guiding Principles) to its Industry Advisory Panel. See appendix V for a summary of these principles. At that time, State also announced OBO’s intent to create the Design Excellence approach, with the goal to produce diplomatic facilities outstanding in all respects, including security, architecture, construction, sustainability, operations and maintenance. OBO’s “Guide to Excellence in Diplomatic Facilities” (Guide to Excellence) released in July 2016, states that the new approach will result in innovative, new American landmarks around the globe. OBO’s establishment of Excellence was informed by a nearly yearlong review—begun in June 2010—by seven internal OBO working groups overseen by a steering committee composed of OBO’s senior managers and chaired by OBO’s then Deputy Director (who later served as OBO’s Director from June 2012 through January 2017). OBO also sought assistance from GSA, which assigned GSA’s Director of Design Excellence to subsequently participate as an external advisor to the Steering Committee. OBO’s working groups were tasked with examining OBO policies and procedures and providing the steering committee with recommendations as to how best to integrate design excellence into all of OBO’s activities. This review resulted in over 60 recommendations and a series of planned actions that were approved in an April 2011 decision memo (Excellence decision memo) as a means to implement Excellence. The review also identified some specific changes to OBO’s processes in the areas of (1) site selection; (2) project delivery method; (3) design standards and guidelines; (4) hiring of outside architectural and engineering design firms; (5) design reviews; and (6) life-cycle cost analysis, among other areas. See appendix VI for a table describing the approaches OBO identified to achieve its goals under the Excellence approach. A significant change under Excellence since 2011 has been OBO’s shift to an increased use of the DBB delivery method alongside bridging. Generally under DBB, OBO first solicits and contracts with a design firm to develop a 100-percent design. Under this method, OBO then uses the completed design to solicit bids from prospective construction contractors. According to OBO documentation, OBO selects a project’s delivery method, either bridging or DBB, based on an evaluation of local context, project complexity, construction factors, and urgency. According to OBO officials, the timing of a construction award (i.e. the planned fiscal year when OBO expects to receive funding to make an award) is also a key determining factor regarding delivery method. Figure 5 provides an overview of the embassy construction process under DBB. Changes made under Excellence provide OBO with greater design control, but carry trade-offs. Key elements under the Excellence approach include (1) allotting funding and time for developing custom designs; (2) hiring leading design firms for projects and promoting innovation in design; (3) conducting peer reviews of designs; and (4) using bridging and DBB project delivery (rather than DB). We found that OBO now funds the development of customized designs and provides up to 24 months for front-end design work. OBO also seeks to hire leading U.S. design firms to develop those designs for each project. New design firms OBO has hired for Excellence projects have faced some challenges, and OBO only recently began assessing their performance. OBO also requires design reviews by industry advisors. This shift to more design-focused delivery methods—from DB to bridging and DBB—has design, schedule, and cost trade-offs. OBO’s staff had split opinions regarding the Excellence approach compared to the SED approach. OBO’s Excellence approach—using bridging and DBB—represents a new investment to develop innovative, project-specific designs. Previously, the SED approach combined with DB delivery made use of the same standard design, which DB contractors’ design firms would adapt to a specific site. Thus OBO did not contract with design firms to develop customized designs. One senior OBO official said that OBO’s intent under Excellence is to “own the quality of each project” and that contracting for project-specific designs provides control over the design process to avoid what OBO reports were quality issues with some SED projects. OBO reports it has awarded 24 new embassy or consulate design contracts— for either 100-percent designs or partial bridging designs—during fiscal years 2011 through 2015. The first design contract solicited as an Excellence project, according to OBO, was awarded in January 2013 for the new U.S. embassy to be built in Mexico City. Design-related activities include both actual project design, which entails the preparation of plans, drawings, and specifications; and project development, which included due diligence efforts such as boundary, utility, and soil surveys. OBO officials stated that they could not distinctly segregate the cost for project designs from project development costs needed to complete a project’s design under any project delivery method, including DB using the SED. Furthermore, according to OBO officials, because such costs were funded out of a central pot of money during the SED era, they cannot be broken out of those earlier contracts. Table 2 shows the amount of funding OBO has authorized for both project design and development activities from fiscal year 2011 through 2015 at a total value of over $400 million. OBO identified 16 of these 24 project designs as being Excellence projects. OBO’s Excellence approach—using bridging and DBB—also represents a new investment in up-front design time, potentially up to 2 additional years (compared to the SED approach) to develop custom, innovative designs before OBO contracts for construction. OBO maintains that this additional design time is integrated into its projects such that planned construction contract award dates are not affected and remain consistent with OBO’s overall Capital Security Construction Program schedule for constructing new secure facilities. In other words, by starting projects earlier, OBO asserts that it can still meet the Capital Security Construction Program schedule for delivering new embassies. However, because the first set of Excellence construction projects were awarded in fiscal year 2014 and are still in progress, it is currently unknown whether OBO will deliver new facilities consistent with the overall program schedule. OBO’s process under both bridging and DBB is now to award contracts to design firms to produce custom project designs, and OBO planning documentation generally estimates up to 2 years (24 months) for the design process of a generic new embassy. OBO documentation generally indicates that when there is sufficient time available for conducting planning and design activities before awarding a construction contract, OBO is inclined to utilize DBB, which OBO asserts often results in a superior end product. OBO officials cautioned that its generic timeframes are just a starting point, and that every project can encounter unique challenges. Figure 6 shows OBO-generic timelines under the prior SED approach (using DB)—which were established based on the size of a post—in comparison with OBO’s generic schedules for Excellence bridging and DBB projects, the latter two providing up to 24 months of additional design work before construction begins. One of OBO’s Guiding Principles for Excellence is that OBO will hire leading U.S. design firms based on their design achievements and portfolio of work. OBO’s intent in hiring leading design firms is, in part, to promote the innovation of American architecture, engineering, and design disciplines as well as U.S. technology, manufacturing, and product design. According to OBO, selection of these firms is based, in part, on their achievements in the design field and work on projects similar in scale and complexity to an embassy project. OBO’s guidance indicates that material advances and new technologies can result in the delivery of better diplomatic facilities and that OBO must invest in innovation. For most of its design projects, OBO utilizes an Indefinite Delivery/Indefinite Quantity (ID/IQ) contract mechanism—under which five design firms have been hired—to task design firms to develop project designs, when needed. In some instances, OBO prefers to issue project-specific solicitations and contracts for more unique and challenging projects, such as the London and Mexico City embassy designs. Under the SED approach, design firms typically conducted due diligence and project development (i.e., planning activities) for OBO to ensure projects were ready for design and construction by the DB contractors. Under Excellence, OBO contracts with design firms to develop Excellence designs before awarding a construction contract, according to OBO. Four of the five firms hired by OBO under its current ID/IQ are new to embassy construction work. Officials we spoke with—including DS officials, design firms, construction contractors, and former OBO officials—identified some adjustment challenges facing design firms new to OBO that have never designed an embassy before. For example, having to become familiar with State’s unique security requirements for diplomatic facilities is a challenge for those design firms. DS officials reported the firms require a great deal of work to bring them up to speed on State’s security requirements. To mitigate this situation, DS has been conducting “101 Certification Workshops” for new design firms so that they understand DS’s approach to the security certification of new embassy designs. Those officials stated that conducting those workshops and reviewing customized Excellence designs has increased DS’s workload. Newer design firms have also been challenged by such issues as lack of sufficient staff with required security clearances, information systems, or office space to independently and securely perform the contracts, according to DS and design firm officials. Some of the new-to-OBO design firms further indicated that they have contracted—as partners or subcontractors—design firms that have worked on past OBO projects to assist the new-to-OBO firms in navigating State’s standards and process. OBO did not begin conducting performance evaluations of its design firms until recently. Recommendations from the 2011 Excellence decision memo indicated that OBO would (1) measure performance of its designers for Design Excellence and project performance, and (2) use the federal contractor performance reporting system to promote consistency, increase data integrity, and motivate contractor performance. The Federal Acquisition Regulation requires agencies to conduct contractor performance evaluations. Such evaluations are intended to provide essential information regarding whether to award future contracts to these design firms. We found that OBO had not been conducting contract performance evaluations of the design firms contracted to deliver Excellence designs. OBO officials acknowledged they had not been recording performance evaluations. As a result of our inquiry and a subsequent request from State’s contracting office, OBO has trained staff and as of August 2016 had initiated design firm evaluations for six of its Excellence projects. The Excellence approach entails greater involvement by OBO’s industry advisors. In 2012, OBO made changes to its industry advisory body, previously called the Industry Advisory Panel. It is now called the Industry Advisory Group (IAG) and OBO increased the number of members from up to 9 advisors representing industry organizations— such as AIA, Associated General Contractors of America (AGC), and Design-Build Institute of America (DBIA)—to up to 35 members, who must, among other criteria, be members of professional organizations and trade groups involved in property management issues, but who represent the companies employing them (not those organizations or industry groups). OBO officials indicate the change was made to allow OBO to have broader industry representation. In September 2014, OBO established a new policy mandating OBO senior management and industry peer design reviews, called OBO senior management and IAG design reviews. The policy requires that two such design reviews be conducted for each new project and that OBO’s Director will designate three members from its IAG or other adjunct professionals to serve as reviewers. These reviews are intended to assist OBO in making certain that projects are well-conceived and can be realized in an efficient and cost-effective manner. The first design review occurs during the Concept Design Phase. The design firm awarded the design contract must submit three viable concepts that it assesses as achievable within the project budget, according to OBO policy. The design firm must explain the factors that influenced each of the three proposed designs, including any opportunities and constraints. During the concept design review, OBO senior management and IAG panel members may raise and discuss any concerns about the proposed concepts or issues affecting scope, schedule, or cost. After considering the IAG panel’s recommendations, OBO selects one concept to be designed to a greater level of detail. The second set of design reviews, outlined in OBO policy, occurs during the Schematic Design Phase, when the selected concept has been more fully designed. The schematic review examines aspects of the proposed design—though not a final design—against the project’s requirements, approved schedule, and estimated construction contract price. The review examines site context and surroundings; the proposed building systems, including security systems and sustainability features; the exterior and interior design elements and materials; and how the local environment and construction labor may impact the design. The general purpose is to highlight opportunities to strengthen the project before the design progresses further and is completed. Following both OBO senior management and industry concept and schematic design reviews, the contracted design firm makes a presentation to OBO’s Director. OBO’s Director may either approve the proposed schematic design to be used for a project or indicate necessary changes to the design for its subsequent approval. After OBO’s Director has approved the schematic design for a project, the final design must be in keeping with the approved design. Senior OBO officials indicated industry advisory design reviews do not add additional time in OBO’s process, as they occur within the overall time allotted for design. See appendix VII for a figure depicting where IAG design reviews occur within OBO’s overall design process. The first Excellence project to go through the industry design review process was the new U.S. embassy planned for Mexico City. Through April 2016, OBO had conducted a total of 27 industry design reviews on 14 Excellence projects. Figure 7 shows the three design concepts that underwent an IAG design review and the schematic design of the selected concept for the new consulate in Hyderabad, India. Design firms we spoke with had varying views on the utility of OBO’s industry advisory reviews. For example, two design firms reported that although the contractual requirement to develop three design concepts for review and consideration adds some value, making it a formal requirement and holding a structured, peer-reviewed process adds additional time, cost, and work. Another design firm new to OBO found the process valuable, particularly when peer reviewers (members of the IAG design review panel) already have OBO experience and can provide advice on potential embassy design pitfalls. OBO’s Guide to Excellence indicates that different delivery methods have design, schedule, and cost implications that must be evaluated relative to the characteristics of each project and that OBO’s Director must approve the delivery method for each project. Since 2011, OBO has generally been using both bridging and DBB as delivery methods to have more control over project designs, according to OBO officials. OBO officials believe that greater design control under Excellence will improve embassies’ appearance in representing the United States, functionality, quality, and operating costs. Table 3 lists various design, schedule, and cost trade-offs inherent in the DB, bridging, and DBB project delivery methods identified by industry studies and experts we interviewed. Construction contractor representatives we spoke with reported seeing these issues play out in the execution of OBO’s Excellence approach. AGC’s representative told us that the more customized designs under Excellence create increased risks for design problems or errors that could result in cost and schedule increases. Two OBO construction contractors we spoke with reported that OBO’s bridging and DBB projects cost more and take longer from start of design to completion when compared with a SED DB project. In part, these contractors said they had found some problems with some aspects of the designs, which took time to resolve with OBO and OBO’s contracted design firms. Those contractors also said OBO’s Excellence projects tend to specify more unique materials or custom-made products, which also adds to construction costs. These contractors also stated that they are tracking more change orders and redesign work on current OBO projects, which further indicates the potential for cost and schedule growth under Excellence. Finally, contractors we spoke with said that while OBO now uses bridging to develop a partial customized design for some OBO projects, their own firms’ design costs for a bridging project will not be lower than a similar SED DB project. They said this is because their own design firms are still responsible for the design and must validate any design information that OBO’s bridging architects develop. OBO staff expressed a wide range of opinions in response to our survey request for comments regarding any specific benefits or challenges brought about by Excellence in their specific area of expertise. Some 421 respondents provided comments covering diverse topics that we evaluated and grouped into 20 categories. Staff often held opposing views regarding a wide range of Excellence issues such as developing Excellence standards or procedures, facilitating stakeholder input, and focusing on maintenance and sustainability. Staff providing comments generally provided more negative narratives than positive ones. Table 4 summarizes the results of our analysis. For examples of specific benefits and challenges cited, see appendix III. We asked OBO staff to characterize Excellence compared with the SED approach in terms of producing diplomatic facilities that are outstanding in all respects, including security, architecture, construction, sustainability, operations and maintenance. Of the 339 staff expressing an opinion, 157 (46 percent) identified the SED as generally more effective, 109 (32 percent) identified Excellence as generally more effective, and 73 (22 percent) believed they were equally effective. We analyzed responses by length of service in OBO. Some 288 staff with 5 years or less experience responded to the question. These respondents were hired around or after the introduction of Excellence in 2011. Most respondents with 5 years or less experience, 204, did not provide an opinion about Excellence or SED. Of the 84 respondents with 5 years or less who did express an opinion, 37 (44 percent) indicated Excellence was more effective, 31 (37 percent) reported SED was more effective, and 16 (19 percent) found both equally effective. On the other hand, many OBO staff with 6 or more years of experience who responded (255 out of 395) offered an opinion. Of the 255 staff with more than 5 years’ experience who had an opinion, 72 (28 percent) indicated Excellence was more effective, 126 (49 percent) reported SED as more effective, and 57 (22 percent) found both equally effective. We also selected offices for further analysis based on size. Of the respondents expressing an opinion to the question from particular offices within OBO (specifically not including those with “No opinion/no basis to judge” or who provided no response), the four largest offices found SED generally more effective. Staff from Facility Management were most closely divided. Of the 62 Facility Management staff who had an opinion, 24 (39 percent) said that the SED program is generally more effective than the Excellence program, while 20 (32 percent) said that Excellence is generally more effective, with the remaining 18 (29 percent) reporting one program as effective as the other. A larger percentage of Construction Management and Design and Engineering staff reported SED as generally more effective. Of the 79 Construction Management staff who had an opinion, 44 (56 percent) said that the SED program is generally more effective, compared to 20 (25 percent) who said Excellence is generally more effective. Of the 58 Design and Engineering staff who had an opinion, 28 (48 percent) said that the SED program is generally more effective, compared to 18 (31 percent) who said Excellence is generally more effective. The greatest division among the large offices was in Security Management, with 21 out of 29 (72 percent) who had an opinion reporting SED was generally more effective than Excellence, compared with 3 out of 29 (10 percent) reporting Excellence was generally more effective. These offices are among those most directly involved in the planning, design, construction, maintenance, and security at U.S. facilities worldwide. Other offices were more supportive of Excellence. Of the 24 respondents from the Office of Project Development and Coordination who had an opinion, 13 (54 percent) reported that Excellence is generally more effective than SED. All remaining offices were more narrowly split, with 35 out of 87 (40 percent) reporting Excellence as more effective than SED and about 32 out of 87 (37 percent) reporting SED more effective. The OBO Front Office firmly supported Excellence as more effective than SED with five of six (83 percent) of respondents who had an opinion saying so. Some 403 staff provided narrative comments comparing SED with Excellence. We categorized the comments, some of which fell into more than one category. The most common positive comment regarding Excellence cited aesthetic or architectural improvements, while the most common negative comment noted higher costs under Excellence compared to SED. The tally for the categories is in table 5 below. Figure 8 lists some selected comments comparing the SED and Excellence approaches. When asked whether Excellence had generally improved the Capital Security Construction Program (i.e., the embassy construction program), OBO staff who responded were more evenly divided. Of the 470 respondents expressing an opinion, 174 (37 percent) generally agreed that Excellence improved the program, while 161 (34 percent) respondents generally disagreed, and 135 (29 percent) neither agreed nor disagreed. While OBO has established some policies and other guidance to implement Excellence, it lacks tools to fully evaluate the performance of the new approach. OBO continues to document changes in its policies, procedures, standards, and other guidance. In our survey, OBO staff generally were evenly split on the sufficiency of OBO’s efforts in these areas. However, OBO has not defined performance measures specific to Excellence goals at either the strategic or project level, such as greater adaptability to individual locations, functionality, or environmental sustainability. OBO also lacks a centralized database to broadly manage Excellence by enabling, for example, effective reporting on projects’ design and construction costs and schedules. Without performance measures specific to Excellence and sufficient systems to collect and analyze relevant data, OBO will not be able to demonstrate whether the performance of Excellence projects over time justifies the increased emphasis on and investment in their designs. While OBO has created or updated some policies and other guidance to implement Excellence, it has taken more time to do this than OBO estimated in 2011. Key guidance deliverables in OBO’s 2011 Excellence decision memo—identified as “critical elements” by OBO—were to be produced within the first year after Excellence was approved. However, it took more time than OBO estimated to issue some of those key elements. For example, OBO replaced the SED with the new OBO Design Standards in 2013 and released its Guide to Excellence in 2016, despite its initial plan to release these documents within a year of the memo. New or updated policies issued in support of implementing Excellence were also not in place until nearly 2 years or more after Excellence was approved in 2011. For example, in recent years OBO has finalized several policies, such as the following: 2013 Site Selection: This new policy emphasizes criteria for urban sites; attributes of the preferred site include (1) considering American values in promoting a sense of openness, accessibility, and transparency through location; (2) proximity to key host-government facilities, embassies of other countries, and businesses and cultural centers; and (3) an urban setting that provides connectivity to public transportation and infrastructure, making the mission accessible to visitors and clients. 2014 OBO Senior Management and Industry Advisory Group Design Reviews: This new policy requires two reviews by external industry advisors and approval of Excellence designs by OBO’s Director. 2015 OBO Core Project Team: This new policy requires OBO’s Design Manager to be an integrated team member—with OBO’s Project Manager, Project Director, and Construction Executive—to ensure that decisions about a project’s design are integrated at project inception and maintained through project completion. 2016 Architect/Engineer Team Selection: This revision to an existing policy governs the evaluation and award of design contracts to design firms. The revision expands the evaluation panel from at least three to up to seven members, with key changes being the addition of an OBO Director’s designee; a representative with a connection to the post or regional bureau; and an external advisor (a federal employee from another agency). A report commissioned by OBO provided insight into the challenges faced in implementing Excellence. In June 2014, OBO modified an existing task order with one of its design firms to require the firm to participate in a roundtable discussion about Excellence and identify ways Excellence might be more effectively communicated and managed. The design firm subsequently delivered a report, based on the roundtable and its own experiences, with the firm’s findings and recommendations on how to improve Excellence. The roundtable included OBO officials, DS, the Office of Logistics Management, and contractors. The resulting consultant’s report noted that OBO had transformed its design approach, design guidelines, project requirements, and preferred project delivery methods. According to the report, this transformation presented significant opportunities. For example, the introduction of highly regarded design firms new to working with OBO offered tremendous potential for innovation and overall quality of building design and performance. However, the report also identified challenges with Excellence, including the following: There did not appear to be an entity responsible for instituting and managing the significant organizational change that the Excellence approach imposed. The Excellence process had altered the internal practices at OBO and its offices; with every project, its implementation has resulted in nuances in the definitions of the Design Standards that led to variations in their implementation, document submittal requirements, milestone analysis, and security risk assessments. While the new Design Standards were more comprehensive, they were very difficult to navigate, particularly as a Portable Document Format (PDF) file with over 8,000 pages. Senior management reviews and the IAG peer review process extended project schedules, and their purpose appeared unclear. The consultant’s report made numerous recommendations to OBO, including that OBO (1) seek to manage and clarify change internally; (2) assign dedicated staff to be responsible for instituting change; (3) utilize more standardization for project requirements, while acknowledging the recommendation may seem counterintuitive in light of OBO’s move away from the SED; (4) further define the Excellence program to capture the new standards and processes OBO was instituting; and (5) train OBO personnel and modify internal systems and practices to be compatible with the new OBO project delivery methods and design standards. In discussing OBO’s implementation of Excellence with us, senior OBO officials stated that they continue to work to improve OBO’s processes. They noted that the development or updating of OBO policies and procedures takes considerable time because numerous OBO technical offices must weigh in on any needed or proposed changes and that OBO management must then review and approve those changes (or send them back for revisions). Senior OBO officials also maintained that it was difficult to “describe a program at the same time that you are implementing it.” One former senior OBO official stated that OBO’s priority during the transition to Excellence was trying to implement the new program and that they may have lagged in establishing policies and procedures to document changes to OBO’s processes. In our survey seeking feedback on Excellence, we asked OBO staff whether they agreed or disagreed with seven statements about OBO’s establishment and communication of strategic direction, policies, and guidance for doing their daily jobs. OBO staff who responded agreed most strongly with statements about the provision of policies and standards for their daily jobs. They divided more narrowly on statements about strategic vision and guidance (see table 6). The largest percent of OBO staff who responded generally agreed with the statement: “Since 2011, OBO has provided clear and comprehensive technical standards and guidelines related to my job.” The largest percent of OBO staff who responded generally disagreed with the statement: “Since 2011, OBO has provided clear and comprehensive strategic or long-term guidance to implement its planning, design, construction, and maintenance approach.” While OBO has established Excellence and taken some steps to implement it, OBO has not established strategic or project-level performance measures to evaluate and communicate the effectiveness of the Excellence approach in delivering embassies under the Capital Security Construction Program. Performance measures are essential tools for managers to evaluate progress toward a program’s objectives. GAO’s Standards for Internal Control in the Federal Government state that agencies’ internal controls should include the establishment and review of performance indicators. Furthermore, State’s Foreign Affairs Manual indicates that State must maintain effective systems of internal controls that incorporate GAO’s internal control standards. In addition, the AIA 2009 report stated that “OBO should be willing to evaluate and explain the benefits of integrating security with design excellence, and the potential benefits to life-cycle costs, design, operations, maintenance, public image, and public diplomacy. OBO’s ability to explain the benefits will require some empirical evidence of claims made for those tangible items such as cost benefit and operations.” Both OBO’s 2011 approval of Excellence and 2016 Guide to Excellence assert that a design excellence program will provide the best value for the U.S. taxpayer. According to GAO’s Business Process Reengineering Assessment Guide, if an agency decides to initiate a reengineering project, it should develop and communicate a compelling business case to customers and stakeholders that supports this decision. Such a business case should contain critical performance measures relating to the organization’s core business processes, such as cost, quality, service, and speed. As an agency completes its process redesign work, the business case should be updated to present a full picture of the benefits, costs, and risks involved in moving to a new process. Without meaningful performance indicators, an agency has no way of knowing if the new process has produced the desired results and whether those results compare favorably or not to the previous process. OBO’s strategic plan does not define how OBO intends to evaluate the performance of the Excellence approach. State’s 2010 press release announcing Excellence and OBO’s 2011 Excellence decision memo both noted that a comprehensive strategic plan was to be implemented in 2011 and would act as a roadmap for developing Excellence policies and procedures. OBO senior officials told us that a 2010 presentation— briefed to the then Secretary—was OBO’s strategic plan for Excellence implementation. The briefing document does not say how Excellence is to be evaluated—one of the functions of a strategic plan—nor does it outline any performance indicators to show how OBO would assess and report on the extent to which Excellence facilities are any more safe, secure, functional, sustainable, or more effective in better supporting U.S. diplomacy than the SED facilities. State’s department-level fiscal year 2014-2017 strategic plan is largely silent on Excellence. Its single Capital Security Construction Program- related performance indicator is the relocation of 6,000 U.S. government employees into more secure and functional facilities by September 30, 2017. OBO used a similar performance indicator under the SED approach. This indicator provides no performance assessment on the extent to which Excellence facilities are any more functional, sustainable, or effective in supporting U.S. diplomacy. Furthermore, the projected target may be low relative to past performance, since 6,000 employees moved by September 2017 equates to an average of 1,500 employees relocated per fiscal year (2014 through 2017). However, State reports that from 2000 through 2014 it moved over 30,000 people into more secure facilities—which equates to an average of over 2,100 people per year (based on actual performance). As a result, it is unclear whether State’s target is an appropriate measure given OBO’s past performance. Excellence is briefly discussed in OBO’s bureau-level Functional Bureau Strategy for fiscal years 2015–2017. It states that OBO is implementing design innovation and that Excellence will introduce improved use of functionality, sustainability, and security for diplomatic facilities. That strategy document includes bureau-level design and construction related performance indicators, among others related to other OBO operations. Those indicators include the following, among others: average duration and cost growth for capital construction design standards are met and updated on an annual basis incorporating lessons learned and other feedback from stakeholders from prior years; and percent of new embassy and consulate compounds designed to achieve LEED Silver certification. While the first set of indicators can quantitatively measure performance and enable OBO to report on the efficiency of project delivery under Excellence (as OBO did previously under the Inman program and under SED), those schedule and cost indicators do not address new aspects of Excellence, such as lower operating costs or better support for U.S. conduct of diplomacy. In addition, the latter two indicators in the list above are, if anything, even less useful for assessing Excellence performance. First, OBO updates its design standards annually and conducts design reviews to ensure that projects meet those standards. Thus, it is unlikely OBO would fail to meet this performance indicator. Second, according to OBO documentation, LEED Silver certification has been an OBO design standard since 2009, before Excellence. Thus, to meet design standards, every Excellence embassy built—with an emphasis on greater sustainability—should be at least LEED Silver, so the indicator should be 100 percent, or very near to it. Furthermore, the LEED indicator assesses only the performance implied by the design itself, not the actual building operations and maintenance performance and whether the actual utility usage and costs are equal to or less than initially estimated in the designs. While no Excellence projects can be evaluated yet, as none have been completed, without additional performance indicators relevant to the goals of the Excellence approach, OBO has no way of knowing if its new process is achieving the desired results. Furthermore, it lacks an important tool for reporting on the Excellence approach to congressional overseers, the public, and other State stakeholders such as other U.S. diplomatic agencies that must help pay some of the costs for constructing and maintaining new embassies. OBO also lacks post-specific performance measures to track and evaluate the long-term performance of its embassies. According to Office of Management and Budget guidance, more than 80 percent of a building’s total cost over its lifespan can consist of ownership costs such as operations, maintenance, and energy usage. When combined with front-end costs such as design and construction, these costs embody a project’s “life-cycle costs.” OBO has attempted to address long-term operations and maintenance costs on the front-end by, for example, committing to include LEED Silver certification in its design standards since 2009, according to OBO officials. Other sustainability “stretch” initiatives OBO considers desirable (though not required) under Excellence include trying to achieve LEED Platinum certification, increasing use of renewable energy sources, reducing greenhouse gas emissions, and achieving net-zero energy and water consumption on its compounds, whereby enough renewable energy or water is generated to meet a post’s requirements. OBO design standards also require design firms to incorporate operations and maintenance cost analysis into embassy designs through sustainability studies. From 2001 through 2015, in locations where OBO has constructed a new diplomatic compound, OBO reported it has constructed 26 new LEED-certified embassy or consulate office buildings, 19 of which were SEDs. However, despite the additional emphasis now focused on operations and maintenance on the front end during design, OBO has no post-specific performance measures related to operations and maintenance cost performance after a new embassy is constructed. One reason for this is the lack of available or reliable data. OBO officials stated that although some embassies do have utility meters on site, getting data from there back to Washington, D.C., is challenging. While OBO does have a data system in place to capture some operations and maintenance information, such as utility usage, it is dependent upon manual entry of data at each specific post. According to OBO officials, this lowers data reliability, and differences in data entry compliance by posts over time make historical analysis of operations and maintenance costs difficult. Also, while some posts have building-level meters for the main office building, other posts have compound-level utility meters that track data for multiple buildings, making broader data comparisons difficult across posts. OBO is taking some steps to address this situation. According to OBO’s 2016 Guide to Excellence, OBO is in the process of developing project and portfolio operations and maintenance cost assessment procedures to account for these costs over the estimated life of embassies. In 2016 OBO initiated an effort to develop a methodology and process to better assess the full life-cycle cost of its projects. OBO’s July 2016 statement of work on its Life-Cycle Cost Assessment effort shows that OBO intends to develop a methodology and plan to assess the total cost of ownership of projects and facilities, which takes into account the costs of (1) acquisition, (2) design, (3) construction, and (4) operations and maintenance. According to OBO officials, this effort represents a gradual shift in OBO’s orientation, whereby OBO’s portfolio is expected to reflect less emphasis on new construction and greater attention to maintenance, repair, and renovations. Therefore, decisions must be made regarding what metrics should be tracked. OBO is also working with State’s Office of Management, Rightsizing, Policy, and Innovation on a pilot effort—called “MeterNet”—with the intent to install more metering systems on embassy compounds and to transmit performance data back to Washington. Under MeterNet, State intends to automate and improve the collection of data on electrical energy usage (both utility and renewable sources), water usage, and fuel consumption. According to OBO officials, MeterNet should relieve posts of manual data entry and also enable OBO to more accurately monitor, collect, and analyze more reliable data on sustainable energy and water performance. OBO anticipates that this in turn will enable facility managers to manage energy consumption data across State’s facilities, as well as analyze and track energy usage trends over time, such as energy per square foot or overall electricity demand. OBO has also been working with the Department of Energy to address challenges with its existing utility data system. According to OBO officials, OBO has not yet determined how MeterNet will interface with OBO’s existing data systems. The steps OBO has taken to improve monitoring of post-specific operations and maintenance costs are at a very early stage. Until OBO clearly defines a process to assess the performance of its projects after construction and establishes reliable data systems to track and report this performance, OBO will lack an essential tool for determining whether completed projects—whether Excellence or SED facilities—are performing as intended by their designs from either a sustainability (e.g., energy and water usage) or an operating and maintenance cost standpoint. OBO currently lacks easily accessible data to provide overall project management information. During our review, we requested a variety of data related to OBO’s embassy construction projects from January 2001 through September 2015, such as contract award amounts, site acquisition costs, delivery method, completion dates, and other data. However, OBO was unable to easily provide such information. According to OBO officials, while these data did exist and could be retrieved, the data were not available in any centralized data source. Rather, each OBO office maintained separate data relevant to its own operations, and so consolidated and current data to provide overall project information were unavailable. OBO offices consolidate certain project management information in periodic project performance reviews, whereby individual offices and project teams report on cost, schedule, and scope for specific embassy projects. However, while this can facilitate some data retrieval from a specific ongoing project, according to OBO officials it can be difficult and time consuming to find information on older, completed projects for which there is no longer an active project team. We reported on similar data issues in September 2014, when OBO could not provide all of the real property files we requested, at that time also citing lack of centralized data, maintenance by different groups within OBO, and difficulty of retrieval. According to federal internal control standards, quality information is an essential tool for agency management to achieve an agency’s objectives. According to these standards, a process should exist that uses the agency’s objectives and related risks to identify the information requirements needed to achieve the objectives and address the risks. Furthermore, such data should be sufficiently relevant, reliable, and accessible to agency management. Additionally, OBO’s 2011 Excellence decision memo cited the need for a comprehensive information technology platform that would integrate and make available all OBO project information; promote effective review, communication, and decision making; and support the maintenance and operations of completed facilities. No such system existed at OBO in October 2015 at the time of our data request. In response to our request, OBO began assembling a wide range of project management data to fulfill our request as well as to better provide information to Congress. We received these data in the form of a spreadsheet 10 months later in August 2016. OBO officials attributed this delay to the aforementioned difficulty of retrieving historical project data as well as having to address concurrent information requests from Congress and State’s Inspector General. The database we received covers projects from January 2001 through September 2015, includes many elements of the information we requested, and also includes some other information useful to OBO management. According to OBO officials, these data were compiled by OBO office units and project managers based on the latest documentation available. OBO recently established an initiative—termed the Ideal Operational State—to explore long-term ways to centralize and standardize data collection across OBO’s operations. According to OBO officials, this Excellence-related initiative is intended to provide a long-term data solution that will allow for better program management across OBO’s business activities as well as better tracking of project metrics such as cost and schedule performance. The study group tasked with assessing OBO’s current information technology systems and potential market alternatives held a kickoff in May 2016 and, after a series of working sessions and vendor evaluations, recommended a series of actions to OBO’s senior management, including an upgrade and modification of existing OBO management software. OBO management approved action on these recommendations in October 2016. Until OBO develops an effective, centralized data system capturing essential and reliable project management data as well as cost and schedule performance across its project portfolio, not only will OBO management lack a critical tool for consolidating key project data, assessing performance, and guiding strategic oversight and internal control, but it will also be hampered in responding to oversight queries by Congress, GAO, and State’s Inspector General. At the heart of OBO’s changes under the Excellence approach is the premise that greater design focus and control will produce more innovative, functional, and sustainable embassies that are just as secure as those built using the SED but that will be more cost efficient to operate and maintain. From fiscal year 2011 through fiscal year 2015, OBO has allotted hundreds of millions of dollars to fund more customized designs rather than applying a standardized design to build new embassies. Though a greater upfront investment in design may yield embassy improvements, it carries with it increased risk to project costs and schedule. While OBO is attempting to manage this risk, without strategic or project-level performance measures specific to the goals of Excellence, OBO cannot fully assess the merits of this new approach. Furthermore, as projects initiated during Excellence’s implementation come to fruition and begin operations, such measures will be essential to any long-term assessment of their performance. Establishing reliable data systems to measure, record, and report on building performance can help OBO management evaluate all costs that occur over a building’s lifespan. Further, centralized project management data are also needed to allow OBO to quantify and assess design and construction costs under Excellence for each project. While OBO has begun efforts to establish such systems, it will take time to complete these initiatives and collect these crucial data. Nevertheless these steps are essential to creating safe and lasting buildings that best represent the United States while ensuring that projects make efficient use of resources and assess the value of shifting to the Excellence approach rather than continuing to use the SED. To better assess OBO’s performance, we recommend that the Secretary of State take the following four actions: 1. Determine whether the existing OBO program performance measure and annual target of moving 1,500 people into safe, secure, and functional facilities is still appropriate or needs to be revised. 2. Establish additional performance measures applicable to the new goals of the Excellence approach in support of the Capital Security Construction Program. 3. Finalize the mechanisms OBO will use to better track and evaluate the actual operations and maintenance performance of its buildings— whether Excellence or SED—and document through appropriate policies, procedures, or guidance. 4. Finalize the mechanisms OBO will use to centrally manage project management data (to include project cost and schedule information), currently termed the Ideal Operational State, and document through appropriate policies, procedures, or guidance. We provided a draft of this report to State for comment. State provided technical comments on the draft, which we incorporated as appropriate. State also provided written comments that are reproduced in appendix VIII. In its written comments, State concurred with our four recommendations and described actions planned or under way to address each of them. State said it will 1. Perform a comprehensive evaluation of its performance measure and annual target of moving 1,500 people into safe, secure, and functional facilities and determine whether that target remains appropriate. 2. Develop new metrics applicable to the Excellence approach. 3. Finalize the mechanisms it will use to better track and evaluate the actual operations and maintenance performance of its buildings, stating that this will occur after its life cycle cost analysis methodology project produces its final report. 4. Finalize the mechanisms OBO will use to centrally manage project management data, noting that State expects the ultimate product of this multiyear effort to provide a comprehensive framework for managing project data. We are sending copies of this report to the appropriate congressional committees and the Secretary of State. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact either Michael J. Courts at (202) 512-8980 or at courtsm@gao.gov or David J. Wise at (202) 512-5731 or at wised@gao.gov. Contact points for our Office of Congressional Relations and Office of Public Affairs can be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IX. This report examines (1) reasons for the Department of State’s (State) shift to its Excellence approach, (2) key elements and trade-offs of the new approach, and (3) the extent to which State has established guidance and tools to implement and evaluate its Excellence approach. To conduct this review, we obtained and analyzed information from State policy, planning, funding, and reporting documents, administrative memos, and select project documentation. We also interviewed officials from State’s Bureau of Overseas Buildings Operations (OBO); Bureau of Diplomatic Security (DS); Office of Management, Rightsizing, Policy, and Innovation; and Office of Acquisitions Management. Within OBO, we spoke with officials from offices responsible for site acquisition, planning, project development, design and engineering, cost management, construction management, facility management, policy and program analysis, and financial management. We also interviewed officials from a variety of architecture and engineering (design) firms and construction contractors that have worked for State. Additionally, we met with experts from several industry groups. In general, we did not review acquisition plans, the complete contracts for each project, or the terms and conditions that could have impacted cost, schedule, and performance of any project. To identify the reasons for State’s shift to its Excellence approach, we analyzed relevant industry studies and OBO assessments from before the introduction of Excellence. We also examined the outputs from OBO’s 2011 Excellence working groups as well as other Excellence documentation, such as OBO’s “Guiding Principles of Design Excellence in Diplomatic Facilities” and OBO’s 2011 memo approving the Excellence approach. Also, because the decision to adopt Excellence was made in 2011—and the work leading up to the decision was undertaken in 2010— we interviewed key former OBO officials with direct experience with OBO’s efforts to improve the Capital Security Construction Program at that time, including some who served on OBO’s management steering committee for Excellence. To examine the key elements and trade-offs of the new approach, we collected and analyzed OBO policy and procedures directives, administrative memos, budget documentation, project authorization documents, design standards, and design-related documentation. We discussed changes in OBO’s process with relevant officials from OBO, DS, and the Office of Acquisitions Management. We also discussed these changes with officials from design firms and construction contractors that had previously worked, or are currently working for State. Furthermore, we consulted industry studies and spoke with experts from industry groups, including the American Institute of Architects, the Associated General Contractors of America, the Bridging Institute of America, and the Design-Build Institute of America to determine the trade-offs inherent to different delivery approaches. To determine the extent to which State has established guidance and tools to implement and evaluate its Excellence approach, we examined changes to OBO’s policies and procedures directives, design standards, standard operating procedures, and other guidance since 2011. We compared these changes to goals and recommendations from OBO’s approval of Excellence and also reviewed an OBO-sponsored study of its implementation progress. Additionally, we reviewed strategic planning documentation, to include State’s strategic plan, OBO’s Functional Bureau Strategy, and State’s Annual Performance Report. We also consulted federal standards for internal control and business process reengineering guidance. We also met with officials from OBO and the Office of Management, Rightsizing, Policy, and Innovation to discuss efforts to evaluate embassy buildings and to improve OBO’s data management. To supplement our findings, we conducted a web-based survey of OBO staff from July 15 through August 12, 2016, soliciting their views on the sufficiency of OBO’s strategic vision, policies, procedures, and technical guidance for the Excellence approach as well as any particular efficiencies or challenges brought about by the Excellence approach. This survey was sent to 1,511 OBO staff, 705 (47 percent) of whom responded. We do not make any attempt to extrapolate our findings to the remaining 53 percent of eligible employees who chose not to complete our survey. The results of our survey provide measures of employees’ views at the time they completed the survey in July and August 2016. Because we surveyed all OBO staff, the survey did not involve sampling errors. To minimize nonsampling errors, and to enhance data quality, we employed recognized survey design practices in the development of the questionnaire and in the collection, processing, and analysis of the survey data. To minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses that should be qualitatively the same, we conducted pretests with six OBO employees. To ensure that we obtained a variety of perspectives on our survey, we randomly selected one employee from each of the following offices to pretest the survey: Area Management; Construction, Facility, and Security Management; Design and Engineering; Planning and Real Estate; Program Development, Coordination and Support; and Security Management. Based on feedback from these pretests, we revised the survey in order to improve the clarity of the questions. An independent survey specialist within GAO also reviewed a draft of the questionnaire prior to its administration. To reduce nonresponse, another source of nonsampling error, we followed up by e-mail with employees who had not responded to the survey to encourage them to complete it. To analyze open-ended comments provided by those responding to the survey, we conducted a content analysis in two steps. In the first step, analysts read the comments and jointly developed categories for the responses. In the second step, each open-ended response was coded by one analyst, and then those codes were verified by another analyst. Any coding discrepancies were resolved by the analysts agreeing on what the codes should be. Additionally, many comments touched upon findings we developed through our separate audit work. We have included some of these comments for illustrative purposes in appendix III. Respondents generally provided more negative comments than positive ones; however, where possible, we have tried to present a balanced set of positive and negative comments. In some cases, we edited responses for clarity or grammar. Views expressed in the survey may not be representative of all OBO staff views on given topics. We conducted this performance audit from August 2015 to March 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We conducted a web-based survey of the Department of State’s (State) Bureau of Overseas Buildings Operations (OBO) staff from July 15 through August 12, 2016, soliciting their views on the sufficiency of OBO’s strategic vision, policies, procedures, and technical guidance for the Excellence approach to the design and construction of U.S. embassies and consulates, as well as any particular efficiencies or challenges brought about by the approach. We sent the survey to 1,511 OBO staff, 705 (47 percent) of whom responded. We do not make any attempt to extrapolate the findings to the remaining 53 percent of eligible employees who chose not to complete our survey. The results of our survey provide measures of employees’ views at the time they completed the survey in July and August 2016. The questions we asked in our survey are shown below. Our survey comprised both fixed-choice and open-ended questions. In this appendix, we include all survey questions and aggregate results of responses to the fixed-choice questions and the number of responses provided to the open-ended questions. We do not provide text of the responses to the open-ended questions. For a more detailed discussion of our survey methodology, see appendix I. For our summary analysis and selected examples of comments provided in response to open-ended questions, see appendix III. Numeric Responses to GAO’s Survey of State’s Embassy Construction Program 1. About how long have you worked with OBO?  Less than 1 year  1 to 5 years  6 to 10 years  11 to 20 years  More than 20 years Provided no answer to this question 76 respondents 218 respondents 150 respondents 170 respondents 89 respondents 2 respondents 2. Which of the following categories best describes your position?  Foreign Service  PSC  Other: _______________________ Provided no answer to this question 264 respondents 214 respondents 134 respondents 86 respondents 7 respondents 3. How many new NEC/NCC projects have you worked on or supported as an OBO employee? Please do not include annexes or where you were covering for someone else.  None 127 respondents 162 respondents 100 respondents  1 to 3 72 respondents 239 respondents  4 to 6 5 respondents  7 to 12  More than 12 Provided no answer to this question 4. Where is your current posting?  Headquarters (SA-6)  Overseas Provided no answer to this question 501 respondents 196 respondents 8 respondents 5. In which OBO office do you currently work? Operations (OPS):  Area Management  Fire Protection  Safety Health and Environmental Resource Management (RM):  Financial Management  Office of the Executive Director  Policy and Program Analysis 8 respondents 25 respondents 5 respondents Other:  Other: ____________________________ 52 respondents  42 respondents 3 respondents I’d rather not identify my office  Provided no answer to this question 6. Do you agree or disagree with the following statements about the program direction of OBO’s construction program? What perspectives or specific examples can you provide to illustrate your answers? 7. Since 2011, what, if any, benefits or efficiencies—related to how OBO now plans, designs, constructs, and maintains NECs/NCCs—have been introduced within your specific area of expertise (i.e., site acquisition, planning, cost estimating, project management, scheduling, design, construction, security management, and facility management)? We received 373 written responses to this question. 8. Since 2011, what, if any, challenges or inefficiencies—related to how OBO now plans, designs, constructs, and maintains NECs/NCCs—have been introduced within your specific area of expertise (i.e., site acquisition, planning, cost estimating, project management, scheduling, design, construction, security management, and facility management)? We received 387 written responses to this question. 9. To the extent you have knowledge, how would you characterize the Excellence program (roughly 2011 to present) as compared to the SED program (roughly 2001 to 2011) in terms of producing diplomatic facilities that are outstanding in all respects, including security, architecture, construction, sustainability, operations and maintenance? 1. The Excellence program is generally more effective than the SED program. 2. The SED program is generally more effective than the Excellence program. 3. The Excellence program is generally as effective as the SED program. 4. No opinion/no basis to judge Provided no responses to this question 9a. What specific examples regarding security, architecture, construction, sustainability, or operations and maintenance can you provide to illustrate your answer? We received 403 written responses to this question. 10. What additional information, if any, would you like to share in order to further elaborate on any of the responses you provided above? We received 338 written responses to this question. Of the 705 respondents, 550 provided comments in response to at least one open-ended question in our survey of Department of State Bureau of Overseas Buildings Operations (OBO) staff. For specific questions, we analyzed and categorized respondents’ comments and have reproduced selected comments below to characterize the results of that analysis. In addition, since many of the comments touched upon findings we developed through our separate audit work, we have also included some of those comments for illustrative purposes. Respondents generally provided more negative comments than positive ones; however, where possible, we have tried to present a balanced selection of positive and negative comments. In some cases, we edited responses for clarity or grammar. Views expressed in the survey may not be representative of all OBO staff views on given topics. Table 9 summarizes the results of our analysis to categorize comments expressing opinions on the benefits and challenges of OBO’s Excellence approach to the design and construction of new embassies and consulates. The two text boxes that follow contain selected narrative responses on, respectively, the top four most-cited benefits and the top four most-cited challenges of the Excellence approach. Selected Survey Comments on the Top Four Most-Cited Benefits of the Excellence Approach Development and/or improvement of standards, processes, procedures, templates, documents, etc.: The 2016 OBO Design Standards have greatly improved the design process. We’ve started doing constructability reviews during design, which is helpful at making sure the designers are realistic in what they propose and that projects can be implemented in the specific region. Improved coordination; facilitating input from different stakeholders/internal teams; consensus and leadership around program objectives: The different offices in OBO seem to work more closely together through the life of the project. There is still some “stove-piping,” but it generally works better than previous years. Seems to be better coordination between OBO and the embassies and consulates. Increased focus on maintenance/sustainability/life cycle analyses: OBO has recently directed its attention to the large role Facility Management undertakes during and after the construction of [an embassy]. It takes OBO 3 to 5 years to get a constructed, but OBO Facility Management has the responsibility for operations and maintenance for following 50 years of life. Facility Management now has a role in design and is working toward obtaining a meaningful role in the construction phase. Operations and Maintenance (O&M) are finally being brought into the full realm of planning, design, and construction of new facilities by current management. The greatest expense to the is not in the planning or the design or the construction of new facilities, it is in the O&M of facilities over their 50-year average life cycle. OBO must bring greater focus as an organization on not just design excellence but on the “Total Ownership Cost” by incorporating the facility management experts and into every single facet in order to minimize the O&M costs to the taxpayer over that 50-year life cycle. This will require resources, not currently at hand. Greater control/flexibility of designs and site selection: My understanding is that previous to 2011 most projects were SED design-build. While this is a very efficient delivery method for some projects, it is not necessarily the best fit for every project. I believe that the Excellence Initiative provides the flexibility to review and select the most appropriate design and delivery method for each project taking into account the unique budget, schedule, and site-specific parameters each project has. Since 2011, we have co-led the effort to search for and legitimize smaller sites in downtown or urban locations. Prior to 2011, the practice was to find overscaled sites outside of the urban core. The Excellence initiative allows for greater flexibility and customization to a specific site (relative to SED), allowing for greater efficiencies. Those efficiencies can, and often do, lead to reduced construction and operational costs. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on the Top Four Most-Cited Challenges of the Excellence Approach Lack of, inadequate, or inconsistent application of policies/procedures/standards/systems; uncertain impact of new policies, etc.: The transition from “design-build” back to “design-bid-build” construction has been poorly implemented. Project design documents have not fully achieved the transition to the level of quality and detail required for overseas “design-bid-build” construction. The effort to move from standard design documents has further made achieving the level of quality for construction and security more difficult. Clear, applicable, specific, and enforceable standards have been watered down or replaced with less specific and in some cases tentative suggestions. Senior OBO management has difficulty objectively articulating design excellence goals or even attempting to measure results. Contract performance in particular is difficult to measure or, in some cases not obtained due to lack of quantifiable criteria. New/slow/problematic processes and/or requirements resulting from more complex and varied projects: Challenges have been introduced in realizing design intent in most of the underdeveloped countries that construction occurs in. Designs are complex and the materials exotic for the location. Inefficiencies have been introduced in requiring senior management reviews of projects without fully defining what the intent of such reviews are. Challenges have been introduced in the constructability of building features. Challenges introduced with reliance on outside architect firms to develop plans and drawings and to judge design intent or answer questions. My required time spent on projects has at least doubled because of the policies put in place since 2011. In terms of improvements, more aesthetically pleasing facilities are being produced, but in my opinion the amount of effort required to attain good design is very disproportionate to the effort required to achieve this goal. This is due to a poorly organized process and a severe lack of communication throughout the organization. Schedule challenges; extended timelines: Schedules are consistently moved forward with longer timelines to accomplish the design portion of the project. Planning and design take much more time and effort than before. Assurance of meeting security criteria is challenging, since every new embassy is a “one of a kind” project. More daunting is the process by which the Front Office approves design concepts. That alone has added 3 to 6 months to the planning and design schedule. Budget challenges; high costs: Since no two posts have the same size, plan, finish materials, exterior components, or operating systems, I see no efficiencies whatsoever. The push for high initial budgets is to be able to cover the costs associated with the fancier design elements. Site acquisition is more difficult, and more costly than ever. Only urban locations and not cheaper bigger suburban sites are even considered. Project design costs are exponentially higher through the Excellence program. Higher costs mean fewer projects being planned for or constructed in any given year. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Excellence costs more compared with SED (“budget challenges”) Excellence takes more time compared with SED (“schedule challenges”) Excellence introduces new/ slow/ problematic processes and/or requirements resulting from more complex and varied projects Under Excellence (compared with SED): Lack of, inadequate, or inconsistent application of policies/ procedures/ standards/ systems; poor communication of policies, etc. Legend: — = No contrasting comments w ere identified in this category. Selected Survey Comments Regarding the SED and Excellence Approaches The SED model streamlined many processes which apparently translated into a more expedient overall delivery of new facilities. The Excellence program successfully addresses many of the SED program drawbacks (“embassies looking similar and like fortresses”), however, at a price (arguably longer and more expensive projects). The decision clearly needs to be made whether it is worth that price. While the SED program was severely limiting, it is my belief that there is a middle ground between the SED and the Excellence program—where OBO has a set of standards, specifications, and requirements that are clearly communicated to the design firm while allowing them to customize the footprint and design features of the building. OBO has buildings in its portfolio that were built using “SED Criteria” that are not “prison-like” and forbidding. It can be done in a thoughtful and efficient way that would appeal to people architecturally. In plain words—OBO didn’t have to throw the baby out with the bathwater. On the positive side, designing unique facilities improves the aesthetics of the U.S. presence abroad and sets an example for building system efficiency and innovative systems to the world. On the negative side, each unique design requires re-inventing the wheel and creates additional challenges for the designer to integrate physical and technical security, as well as accounting for building maintenance and upkeep. These designs are often difficult to implement overseas, and the state-of-the-art systems are difficult to maintain in some countries with unique equipment and a significant increase in facilities staff. Unique designs are more difficult and frequently more expensive to implement, creating an impact on time and cost. The Excellence program designs buildings that are architecturally pleasing, but in order to achieve nice aesthetics and achieve the security mandates that embassies must adhere to, I believe a cost premium exists in order to meet the aesthetics of the embassies designed under the Excellence program. I think a compromise can be achieved, and in that compromise some cost savings could be realized, but in my opinion the pendulum has swung completely opposite that of the SED, and the evidence is in a cost comparison between the embassies designed under the SED program to those designed under the Excellence program. The designs of are more challenging or difficult to introduce, implement, and execute. Every design is unique and different, takes longer to evaluate, and requires more coordination; more issues are encountered that need to be resolved. The construction of is also more challenging or difficult to execute. The distinct construction materials used are more expensive, take longer to procure and transport, and are more difficult to install. Therefore, the schedule is longer, especially if there are more issues encountered during design and construction, and the cost of construction and contingencies are much higher. SEDs were all about schedule and budget and when either of these were threatened, things were de-scoped or glossed over. Many times those were support annexes (shops, storage facilities) that allowed the facilities to be properly maintained or morale/ welfare/recreation facilities or amenities, meaning that these things had to be added after the fact, putting a great deal of pressure on posts. Excellence does a better job in terms of incorporating architectural features, energy efficiency and sustainability design, and delivering a fully realized compound. I believe that the Excellence program produces a better product and platform for diplomacy; however, most of us also acknowledge that the SED program gave us the ability to execute faster when needed. They should not be mutually exclusive. There are principles in the SED that can be applied and probably should not have been just “thrown out.” It should have been a good solution for an expeditious need, in a place where appropriate. We also know that it did not work well everywhere, and in places with stringent code, zoning, or other restrictions (including the desire to operate in an urban context), it was definitely not a good solution. As long as the Front Office doesn’t focus solely on aesthetics, the Excellence program should produce some outstanding facilities that have far more flexibility than the SED, better functionality, and improved suitability for the country in which they reside. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Many responses to open-ended questions in the survey touched upon issues we reviewed through our separate audit work. We have included some of these responses for illustrative purposes in the text boxes that follow. In some cases, we edited comments for clarity or grammar. Selected Survey Comments on Concerns with SED The SED program was a failure for site specific implementation. It is generally known around OBO that there was no such thing as a real SED building. Due to the technical requirements and the site specific requirements, all SED compounds required adaptation, usually at significant cost and time. The SED had plenty of problems. Yes, the main issue was the boring and unimaginative concrete box; however, it did produce embassies. There is a middle road that opens up the options and opportunities for improved design without going overboard. The SED program was effective in quickly and efficiently providing safe and secure facilities to perform diplomatic activities overseas using the delivery method. However, the designs were not very attractive and didn’t represent the best in American architecture. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on the Desire to Improve Embassies under the Excellence Approach The Excellence initiative is a better way to design the Department’s projects and buildings. It takes advantage of innovation in the design generated by architects and engineers from different points of view. The SED program produced more and more generic buildings that translated into more or less a template mentality. The architecture is less fortress-like, more approachable to local populations, more culturally sensitive (more diplomatic). It is more attractive and something that Americans can be prouder of, as well as being greener and increasingly sustainable. From my limited experience with OBO, it appears as though the Excellence program attracts better design firms to the program. Better design firms should result in a better end product. The Excellence program also appears to result in more excitement from the host country and allows for embracing culturally specific designs. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on Industry and Senior OBO Management Design Reviews The incorporation of the Industry Advisory Reviews gives design firms real-time feedback from their peers on not only how to support the Excellence program, but how to save costs and make buildings more efficient during the design process. I think that the new processes are moving forward, and we are seeing some positive results with stronger designs and hopefully, construction. For example, there were positive reactions by both the staff and private sector architects to the Industry Advisory Reviews that I attended. Too much time and effort is spent on reviews by OBO Front Office and outsiders. Front Office seems to be inconsistent in what it wants in a design and will change its mind from one review to the next. We are hiring professional design firms. They should be able to do their jobs. The design process has been significantly impacted by the introduction of numerous senior management and Industry Advisory Group reviews. These reviews are expensive and time consuming. An independent evaluation of their value versus their cost is warranted. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on OBO Guidance Related to the Excellence Approach I do believe OBO as a whole is trying to maintain and keep up with outdated policies. In my observations OBO is very understaffed or doesn’t have the right mix of professionals necessary to maintain policy and regulations. There is no mention anywhere of changes to the security program brought on by Design Excellence. In my opinion, no one has ever thought through what it means to construct anything other than a SED. There is no Design Excellence guidance for security professionals. The Guiding Principles are widely available to staff and clearly reflect the vision of the senior leadership. The Design Excellence guide took 2 years to write; then they wanted the operations and maintenance piece added but gave us initially only 2 weeks. Case in point. Just yesterday an OBO notice came out on the Excellence Initiative. No doubt it had something to do with the timing of this survey. OBO has been working to modify policies and procedures directives (PPDs), etc.; however, it takes time, and it is difficult to get people to take the time away from their regular duties to step back and look at policies. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on the Need for Excellence-Specific Performance Metrics I have not seen any performance metrics that compare before/after the Excellence Initiative. What are the performance metrics used to measure this ? Senior OBO management has difficulty objectively articulating Design Excellence goals or even attempting to measure results. I am not aware of any metrics for evaluating the “Excellence” of a project and the many facets involved in assessing a building. What metrics does OBO employ to measure “effectiveness” or “efficiency,” specifically, strategically? Answer: none. Each project takes as long as it takes. The project costs what it costs. Several presentations have been given and a few documents produced to try to articulate Design Excellence; however, much of the concept is subjective, due to the emphasis on architecture, and cannot be measured or effectively monitored for performance. There are no real performance measures; total operations and maintenance costs are unknown. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Selected Survey Comments on the Need for Better Project Data and Systems Project database does not have a user manual or guide, either. Who does this? Why create a database and never provide a user guide or how-to manual for the user of the data? We are supposed to have a new and better projects database. We used the same databases/networks and computers for the last 10+ years. Not enough server space to store the size/volume of electronic files/docs that we produce. The “Ideal Operational State” research/data collection effort, however, we hope, will resolve some of these technological issues. The Ideal Operational State effort is an excellent opportunity to manage the Life Cycle Cost Analysis of the entire business process life cycle. The existence of the Ideal Operational State, while itself a very good thing, indicates the need for more strategic alignment of process with and definition of a vision. Legend: OBO = Bureau of Overseas Buildings Operations; SED = Standard Embassy Design. Intent Embassies and consulates have two essential purposes: to be safe, secure, functional, and inspiring places for the conduct of diplomacy, and to physically represent the U.S. government to the host nation. The site and location of an embassy have practical as well as symbolic implications. OBO will develop sites that best represent the U.S. government and its goals, and enhance the conduct of diplomacy. Whenever possible, sites will be selected in urban areas, allowing U.S. embassies to contribute to the civic and urban fabric of host cities. OBO will evaluate designs on the basis of their success in skillfully balancing requirements, and on how well the design represents the United States to the host nation. Designs are to be functionally simple and spatially flexible to meet changing needs and be enduring over time. An official embassy style will be avoided. Designs will be cost-effective. Each design will be responsive to its context, to include the site, its surroundings, and the local culture and climate. The designs will make use of contextually appropriate and durable materials and incorporate the latest in security and safety features. The grounds should be functional and representational spaces. They will be sustainable, include indigenous plantings, and incorporate existing site resources, such as mature trees. The facilities will incorporate advanced methods, systems, technologies, and materials appropriate to the facility and local conditions, including the site, climate, natural hazards, security, and the practical reality of construction, maintenance, and operations in the host nation. The safety and security of staff and visitors is paramount. Designs and construction will meet or exceed all security and safety standards and specifications. Buildings and grounds will incorporate sustainable design and energy efficiency. Construction, maintenance, and operations practices will be sustainable. OBO will hire leading American architects and engineers. Selection will be based on the quality of their design achievements and portfolio of work. The selection methodology will be open, competitive, and transparent. Construction professionals will be engaged throughout the process to ensure the best possible design and implementation. OBO is committed to selecting the most qualified building contractors with a record of delivering high quality projects. Operations and maintenance Operations and maintenance professionals will be engaged throughout the design and construction Historically, architecturally, or culturally significant properties and collections process. Buildings and sites will be economical to operate and maintain and will utilize equipment and materials that are durable, dependable, and suitable. Designs will be based on life-cycle analysis of options that take into account longterm operations and maintenance. Embassy buildings and grounds are an opportunity to showcase the best of American and host nation art and culture. OBO is committed to integrating art into its facilities such that each property will be both an individual expression of Excellence and part of a larger body of work representing the best that America’s designers and artists can leave to later generations. OBO is committed to preserving the State Department’s historical, cultural, and architectural legacy. The Secretary of State’s Register of Culturally Significant Property is the official listing of important diplomatic architecture overseas and properties that figure prominently in our country’s international heritage. Appendix VI: Approaches to Achieve Excellence; U.S. State Department, Bureau of Overseas Buildings Operations Approaches to Achieve Excellence; U.S. State Department, Bureau of Overseas Buildings Operations (OBO) Strategy General Intent Holistic approach to project Project teams include all key stakeholders such as users, tenant agencies, and OBO disciplines, as delivery well as members of the architectural, engineering, and construction contractor teams. Information technology (IT) OBO’s IT platform integrates and makes available all project information, promoting effective review, Architect and engineer (A-E) selection Design process Construction contractor selection Best value contract award communication, and decisionmaking during project development, construction, maintenance and operations. OBO uses either Design/Build or Design/Bid/Build. Neither delivery method is a default. Context, complexity, construction environment, and urgency are evaluated when selecting a method for each project. OBO recognizes the representational and symbolic importance of site location. OBO has revised site scoring criteria to acquire sites in urban areas. OBO considers redevelopment of strategically- located U.S. government owned sites. OBO contracts with the most talented A-E firms, whether long-established or emerging new firms. The selection process focuses on the portfolio of work, team members, and past performance. OBO effectuates high-quality design through design processes such as on-site design charrettes, on-board working sessions, constructability and maintainability reviews, senior management approvals, and peer reviews. The Guiding Principles outline the fundamental design goals of all of our projects. These include the integration of purpose, function, security, safety, flexibility, sustainability, maintainability, and art. OBO uses sustainability principles and life-cycle cost analysis to ensure that facilities provide the lowest overall long-term cost of ownership, consistent with quality and function. OBO is working to expand the pool of contractors and reach out to new emerging firms to promote competition and ensure the best outcome. OBO is using the Best Value method, which includes factors such as past performance and team qualifications, as well as consideration of lifecycle costs in the evaluation process. Early contractor involvement OBO involves construction contractors in early stages, particularly on long-term and complex projects, to ensure the best outcome and reduce risk. Operations and maintenance Reference guides and training programs are developed for each major project. The guide includes Revised architectural and engineering design guidelines Recognizing Excellence information such as design intent, systems information, maintenance requirements, and troubleshooting. This ensures that facilities are operated and maintained properly and that future modifications to the building are in keeping with the original design intent. A Guide to Excellence in Diplomatic Facilities has been released on the OBO website (www.state.gov/obo). The guide is comprehensive and highlights how Excellence goals and priorities will be achieved in each phase of a project. The Guide is a basic “how to” manual. Design requirements have been revised to support Excellence. The requirements emphasize high- performance buildings, flexibility and best design practices, while moving away from a fixed solution. An Excellence awards program is in development. The program will recognize projects that exemplify Excellence. Michael J. Courts, (202) 512-8980 or courtsm@gao.gov David J. Wise, (202) 512-5731 or wised@gao.gov. In addition to the contact named above, Leslie Holen (Assistant Director, International Affairs and Trade), Michael Armes (Assistant Director, Physical Infrastructure), David Hancock, John Bauckman, Eugene Beye, Melissa Wohlgemuth, and Elisa Yoshiara made key contributions to this report. Technical assistance was provided by David Dayton, Jill Lacey, Alex Welsh, and Neil Doherty.

Summary: In 1998, terrorists bombed the U.S. embassies in Nairobi, Kenya, and Dar es Salaam, Tanzania, killing over 220 people and injuring 4,000. In 1999, State began a new embassy construction program, administered by OBO, which to date has received $21 billion, according to State. OBO's primary goal was to provide secure, safe, and functional workplaces, and it adopted SED with a streamlined, standard design for all embassies. In 2011, OBO replaced the SED with the Excellence approach, which makes use of customized designs for each embassy. GAO was asked to review the implementation of Excellence. This report examines (1) the reasons for State's shift to the Excellence approach, (2) key elements and tradeoffs of the new approach, and (3) the extent to which State has established guidance and tools to implement and evaluate its Excellence approach. GAO analyzed information from State policy, planning, funding, and reporting documents and interviewed State and industry officials. GAO also surveyed OBO staff about, among other things, the sufficiency of OBO's policies, procedures, and technical guidance for the Excellence approach. GAO will examine project cost and schedule issues in a subsequent report. In 2011, the U.S. Department of State's (State) Bureau of Overseas Buildings Operations (OBO) established the Excellence approach in response to concerns regarding the aesthetics, quality, location, and functionality of embassies built using its Standard Embassy Design (SED). The SED utilized a standard prototypical design for new embassies and consulates along with a streamlined delivery method combining responsibility for design and construction under a single contract. Under the Excellence approach, OBO now directly contracts with design firms to develop customized embassy designs before contracting for construction. OBO officials believe that greater design control under Excellence will improve embassies' appearance in representing the United States, functionality, quality, and operating costs. Excellence consists of several key elements and involves trade-offs. For example, OBO now allots time and funding to develop customized designs and hires leading design firms to produce them. These design firms have faced initial adjustment challenges designing U.S. embassies, and OBO only recently began evaluating their performance as required by federal regulation. OBO's new approach poses cost and schedule trade-offs since, for example, OBO now has greater design control but may also be responsible if design problems are identified during construction. GAO's survey found that OBO staff who responded held split or conflicting opinions on Excellence compared with SED. While OBO has established guidance to implement Excellence, it lacks tools to fully evaluate the performance of this new approach. Performance measures are essential tools for managers to evaluate progress toward a program's goals, as noted in Standards for Internal Control in the Federal Government . However, OBO has not established performance measures to specifically evaluate and communicate the effectiveness of Excellence in delivering embassies. Moreover, OBO's bureau-wide strategic measures do not address Excellence priorities, such as greater adaptability to individual locations, functionality, or sustainability. OBO also lacks a reliable system to monitor operating performance, such as building energy usage, and a centralized database to broadly manage the Excellence program, to include effectively reporting on projects' design and construction costs and schedules. Without performance measures and reliable systems to collect and analyze relevant data, OBO cannot fully assess the value of shifting to the Excellence approach and away from the SED. GAO is making four recommendations that State take several steps to strengthen performance measures and reporting, monitoring mechanisms, and data systems for the Excellence approach. State concurred with all four recommendations.