Source: https://www.anao.gov.au/work/major-projects-report/major-projects-report-2016-17
Timestamp: 2020-01-26 20:38:36
Document Index: 669479705

Matched Legal Cases: ['art 1', 'art 2', 'art 3', 'art 4', 'art 3', 'art 1', 'art 2', 'art 3', 'art 3', 'art 4', 'art 4', 'art 2', 'art 2', 'art 3', 'art 2', 'art 2', 'art 1', 'art 3', 'art 3']

2016-17 Major Projects Report | Australian National Audit Office
Auditor-General Report No. 26 of 2017–18
Due to the complexity of material and the multiple sources of information for the 2016-17 Major Projects Report, we are unable to represent the entire document in HTML. You can download the full report in PDF or view selected sections in HTML below. PDF files for individual Project Data Summary Sheets (PDSS) are also available for download.
2. The Australian National Audit Office (ANAO) has reviewed 27 of Defence’s Major Projects in this tenth annual report (2015–16: 26). The objective of the report is ‘…to improve the accountability and transparency of Defence acquisitions for the benefit of Parliament and other stakeholders.’1
3. The Capability Acquisition and Sustainment Group (CASG) within the Department of Defence (Defence), manages the process of bringing new capabilities into service.2 In 2016–17 CASG provided support to the Australian Defence Force (ADF) through the acquisition and sustainment of required military equipment and supplies3, and expended some $6.2 billion on major and minor capital acquisition projects.4
4. The February 2016 Defence White Paper established the Government’s priorities for future capability investment for the next 20 years and provided for additional spending of over $29 billion across the next decade. More recently, the 2017–18 Defence Portfolio Budget Statements indicated that the Defence budget would total approximately $200 billion over the coming decade, for investing in Defence capability.5 Additionally, the Government commenced its $89 billion investment in Australia’s future shipbuilding industry in April 2017.6
5. Major Projects are selected for review based on the criteria included in the 2016–17 Major Projects Report (MPR) Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit (JCPAA).7 They represent a selection of the most significant Major Projects managed by Defence.
6. The total approved budget for the Major Projects included in this report is approximately $62.0 billion, covering nearly 59 per cent of the budget within the Approved Major Capital Investment Program of $105.9 billion.8 The selected projects and their approved budgets are listed in Table 1, below.
Table 1: 2016–17 MPR projects and approved budgets at 30 June 20171,2
16 004.9
9 090.1
3 462.5
3 363.5
3 091.9
Hawkei3
61 973.4
Note 2: Air to Air Refuelling Capability was removed from the MPR program in 2016–17.
Note 3: Hawkei and CMATS are included in the MPR program for the first time in 2016–17.
9. The exclusions to the scope of the review noted above are due to a lack of Defence systems from which to provide complete and accurate evidence9, in a sufficiently timely manner to facilitate the review. This has been an area of focus of the JCPAA over a number of years10, and it is intended that all components of the PDSSs will eventually be included within the scope of the ANAO’s review.
10. Separate to the formal review, the ANAO has undertaken an analysis of key elements of the PDSSs—including cost, schedule, progress towards delivery of required capability, project maturity, and risks and issues. Longitudinal analysis across these key elements of projects has also been undertaken.
12. The ANAO has reviewed the PDSSs prepared by Defence as a priority assurance review under section 19A(5) of the Auditor-General Act 1997. The criteria to conduct the review are provided by the Guidelines approved by the JCPAA, and include whether Defence has procedures in place designed to ensure that project information and data was recorded in a complete and accurate manner, for all 27 projects.
Part 1 comprises the ANAO’s review and analysis (pp. 1–59);
Part 2 comprises Defence’s Commentary, Analysis and Appendices (not included within the scope of the Independent Assurance Report by the Auditor-General) (pp. 61–107);
Part 3 incorporates the Independent Assurance Report by the Auditor-General, the Statement by the Secretary of Defence, and the PDSSs prepared by Defence as part of the assurance review process (pp. 109–381); and
Part 4 reproduces the 2016–17 MPR Guidelines endorsed by the JCPAA, which provide the criteria for the compilation of the PDSSs by Defence and the ANAO’s review (pp. 383–409).
Figure 1: 2016–17 Report structure
Note: To assist in conducting inter-report analysis, the presentation of data in the PDSSs remains largely consistent and comparable with the 2015–16 MPR.
Section 1—Project Summary: including description; current status, including financial assurance and contingency statement; context, including background, uniqueness, major risks and issues, and other current sub-projects;
Section 2—Financial Performance: including budgets and expenditure; variances; and major contracts in place (in addition to quantities delivered as at 30 June 2017);
Section 3—Schedule Performance: providing information on design development; test and evaluation; and forecasts and achievements against key project milestones, including Initial Materiel Release (IMR), Final Materiel Release (FMR)11, Initial Operational Capability (IOC) and Final Operational Capability (FOC)12;
Section 6—Project Maturity: provides a summary of the project’s maturity, as defined by Defence13, and a comparison against the benchmark score;
17. The Statement by the Secretary of Defence was signed on 3 January 2018. The Secretary’s statement provides his opinion that the PDSSs for the 27 selected projects ‘… comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2017’.
18. The Secretary also ‘acknowledge[s] the difference of view between Defence and the ANAO in relation to the AIR 87 Phase 2 – Armed Reconnaissance Helicopter PDSS’. Further detail is provided in paragraphs 20 to 25 below (see Conclusion by the Auditor-General).
19. In addition, the Statement by the Secretary of Defence details significant events occurring post 30 June 2017, which materially impact the projects included in the report, and which should be read in conjunction with the individual PDSSs. These include: Joint Strike Fighter, AWD Ships, MRH90 Helicopters, Hawkei, ARH Tiger Helicopters, Battlefield Airlifter, Overlander Light, Additional MRTT, CMATS, ANZAC ASMD 2B, Additional Chinook, HATS, Battle Comm. Sys. (Land), Maritime Comms, Hw Torpedo, Collins R&S, ANZAC ASMD 2A and BMS.
20. The Auditor-General has been unable to provide an unqualified Independent Assurance Report for 2016–17 as a number of matters were identified, in the course of the ANAO’s review, that resulted in the qualification of progress and performance as reported in one Project Data Summary Sheet (PDSS).
21. The review Guidelines define a project as the acquisition or upgrade of Specialist Military Equipment. The Guidelines provide that the scope of Defence reporting includes the performance of selected major equipment acquisitions and associated sustainment activities, where applicable.
22. The ARH Tiger Helicopters PDSS has been prepared on the basis of the Defence acquisition project14, which is narrower than the scope established in the Guidelines.
The project maturity score in Section 6.1 of the ARH Tiger Helicopters PDSS reports a total of 69 out of a maximum of 70 (98.6 per cent) at the time of transition from acquisition to sustainment in April 2017. Noting the caveats, capability deficiencies and obsolescence issues at the declaration of FOC in April 201615, 16, and considering that only two of the nine caveats applying at FOC have been lifted by the Capability Manager (in July 2017), this score does not accurately or completely represent the project’s maturity as at 30 June 2017. The Auditor-General’s conclusion has had regard to the July 2017 events.
23. In addition, a material inconsistency has been identified in the forecast information. Section 4.1 in the ARH Tiger Helicopters PDSS reports that materiel capability delivery performance is at 100 per cent, indicating that materiel capability delivery performance has been met. Rate of effort continues to be lower than planned17, and expert analysis commissioned by Defence in April 2016 indicates that the program will remain incapable of fully meeting expectations relating to reliability, availability, maintainability and rate of effort.18
24. The Auditor-General also drew attention to these matters in the Independent Assurance Report for 2015–16.19
25. With the exception of the matters above, the Auditor-General has concluded in the Independent Assurance Report for 2016–17 that ‘…nothing has come to my attention that causes me to believe that the information in the 27 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2016–17 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.’
26. Additionally, in 2016–17, a number of administrative issues were observed in the course of the ANAO’s review, as summarised below:
a change to the basis of financial reporting and the application of incorrect exchange rates when managing contracts (Section 2 of the PDSS). See further explanation in paragraphs 1.41 to 1.43 and paragraph 2.25;
a lack of oversight, non-compliance with corporate guidance and the use of spreadsheets20 in the management of risks and issues (Section 5 of the PDSS). See further explanation in paragraphs 1.44 to 1.50;
outdated policy guidance for the project maturity framework21 (Section 6 of the PDSS). See further explanation in paragraphs 1.51 to 1.57; and
an increase in the number of MPR projects which have achieved significant milestones with caveats. See further explanation in paragraphs 1.58 to 1.60.
27. As discussed, the ANAO has undertaken an analysis of key elements of the Defence PDSSs—including cost, schedule, progress towards delivery of required capability, project maturity, risks and issues, and in particular, longitudinal analysis across these key elements of projects. Table 2, below, provides: summary data on Defence’s progress toward delivering the capabilities for the Major Projects covered in this report; and compares current data against that reported in previous editions of the MPR. This section also contains a summary analysis of the three principal components of project performance: cost, schedule and capability.
$32.1 billion (51.7 per cent)
$4.1 billion (96.6 per cent)
$21.5 billion (34.7 per cent)
-$1.6 billion (-2.6 per cent)
793 months (29 per cent)
149 months (6 per cent)
1 531 / 1 890 (81 per cent)
Refer to paragraphs 27 to 42 in Part 1 of this report.
Note 1: The data for the 27 Major Projects in the 2016–17 MPR compares the data from projects in the 2015–16 MPR and 2014–15 MPR.
Note 2: Slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project. However, Figure 10 reports in-year schedule reductions.
Note 3: Based on the 23 repeat projects from the 2013–14 MPR, 23 repeat projects from the 2014–15 MPR, 25 repeat projects from the 2015–16 MPR respectively, and one new project (CMATS) that had slippage in 2016–17.
Note 4: The figures represent the total of the reported maturity scores divided by the total benchmark maturity score, in the PDSSs across all projects.
Note 7: Defence has advised that Joint Strike Fighter will not deliver one element of capability at FOC (which equates to approximately one per cent). However, across all 27 Major Projects this percentage rounds to zero per cent.
28. Cost management is an ongoing process in Defence’s administration of the Major Projects. While all projects reported that they could continue to operate within the total approved budget of $62.0 billion, CMATS is seeking approval for a significant Real Cost Increase22 which is anticipated to be considered by government in February 2018. In addition, ARH Tiger Helicopters was provided a heavily caveated Final Operational Capability (FOC) in April 2016 without having delivered all of the capabilities required as part of the acquisition project.23 Delivery of outstanding requirements has been transferred from acquisition to sustainment management within CASG.
29. The approved budget for Major Projects included in this MPR has increased by $21.5 billion (34.7 per cent) since Second Pass Approval, as detailed in Table 3, below. However, as the MPR predominantly focusses on the approved capital budget, the ongoing costs of Project Offices (acquisition), training, replacement capability, etc., are not reported here.
Table 3: Budget variation post Second Pass Approval by variation type1, 2
Price Indexation – materials and labour (net) (to July 2010) 2
Exchange Variation – foreign exchange (net) (to 30 June 2017)
Note 1: Variations greater than $500 million are included in this table. For the breakdown of in-year variation, refer to Table 10 of this report.
Note 2: Prior to 1 July 2010, projects were periodically supplemented for price indexation, whereas the allocation for price indexation is now provided for on an out-turned basis at Second Pass Approval.
Source: ANAO analysis of the 2016–17 PDSSs.
30. Delivering Major Projects on schedule continues to present challenges for Defence24; affecting when the capability is made available for operational release and deployment by the Australian Defence Force, as well as the cost of delivery.25
31. The total schedule slippage26 for the 27 selected Major Projects, as at 30 June 2017, is 793 months (2015–16: 708 months) when compared to the initial schedule. This represents a 29 per cent (2015–16: 26 per cent) increase since approval. Table 4 below includes details of in-year and total schedule slippage by project. While the table shows a six per cent in-year slippage for 2016–17, the removal of a completed project (Air to Air Refuel) has removed 64 months of slippage. The effect of this project exiting the review explains the difference between the total schedule slippage in 2016–17 (85 months) and the total in-year slippage amount (149 months).
Table 4: Schedule slippage from original planned Final Operational Capability1
ARH Tiger Helicopters 2
Note 1: Refer to footnote 26.
Note 2: FOC for ARH Tiger Helicopters was declared with caveats. That is, not all capabilities required by government were delivered by the acquisition project.
Note 4: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A is expected to achieve FMR and MAA closure in quarter one 2018.
32. Platform availability has contributed to the slippage experienced within some projects. For example, the submarine programs have been impacted by changes to docking schedules, following government commissioned reviews. Significant delays have also been experienced by those projects with the most developmental content: AWD Ships, MRH90 Helicopters, ARH Tiger Helicopters, CMATS and ANZAC ASMD 2B. Additionally, delays to operational test and evaluation activities have led to further delays to the LHD Ships and LHD Landing Craft projects.
33. Table 5, below, provides details of total schedule slippage by project, for projects that have exited the MPR. Compared to the 793 months total schedule slippage for the current 27 Major Projects, the 14 projects which have exited the MPR have reported accumulated schedule slippage of 601 months, as at their respective exit dates. Again, schedule slippage was more pronounced in projects with the most developmental content.
34. Additional ANAO analysis (refer to Figure 7, on page 46) has compared project slippage against the Defence classification of projects as Military Off-The-Shelf (MOTS), Australianised MOTS or developmental.27 These classifications are a general indicator of the difficulty associated with the procurement process. This analysis highlights, prima facie, that the more developmental in nature a project is, the more likely it will result in project slippage, as well as demonstrating one of the advantages of selecting MOTS acquisitions.28
35. Figure 8 (on page 47) provides analysis of projects either completed, or removed from the MPR review, and shows that a focus on MOTS acquisitions has assisted in reducing schedule slippage. Figure 8 was requested by the JCPAA in May 2014.29
36. Longitudinal analysis indicates that while the reasons for schedule slippage vary, it primarily reflects the underestimation of both the scope and complexity of work, particularly for Australianised MOTS and developmental projects (see page 84 in Part 2).
37. The third principal component of project performance examined in this report is progress towards the delivery of capability required by government. While the assessment of expected capability delivery by Defence is outside the scope of the Auditor-General’s formal review conclusion, it is included in the analysis to provide an overall perspective of the three principal components of project performance.
38. The Defence PDSSs report that 23 projects in this year’s report will deliver all of their key capability requirements. Defence’s assessment indicates that some elements of the capability required may be ‘under threat’, but the risk is assessed as ‘manageable’. The three project offices experiencing challenges with expected capability delivery (2015–16: three) are MRH90 Helicopters, Battlefield Airlifter and LHD Landing Craft. One project office (Joint Strike Fighter) is currently unable to deliver all of the required capability by FOC.
39. Defence’s presentation of capability delivery performance in the PDSSs is a forecast and therefore has an element of uncertainty. In 2015–16, the ANAO developed an additional measure of the status of current capability delivery progress to assist the Parliament—Capability Delivery Progress—which is a tally of the capability delivered as at 30 June 2017, as reported by Defence. Tables 6 and 7 below provide two worked examples of the ANAO’s methodology, utilising the performance information provided in the relevant PDSS.
Table 6: Capability Delivery Progress assessment – CMATS
No. of elements delivered at 30 June 2017
Transition of Amberley, East Sale, School of Air Traffic Control and Edinburgh from Australian Defence Air Traffic System (ADATS) to Civil Military Air Traffic Management System (CMATS).
No sites have been transitioned.
Transition of Oakey, Nowra, Tindal, Darwin, Townsville, Williamtown, Pearce, Richmond and Gin Gin from ADATS to CMATS.
Completion of vehicle deliveries for all five production periods as detailed in Section 1.1.
All vehicles have been delivered.
40. Table 8 below, summarises expected capability delivery as at 30 June 2017—as reported by Defence and using the ANAO’s Capability Delivery Progress measure.
2014–15 MPR (%)
2016–17 MPR (%) Adjusted 3
Not delivered at FOC 2
Note 1: Defence has advised that Joint Strike Fighter will not deliver one element of capability at FOC, of a total of 79 elements required for the project (which equates to approximately one per cent). However, across all 27 Major Projects this percentage rounds to zero.
Note 2: In addition, ARH Tiger Helicopters had a small number of elements not delivered at FOC. However, as there is a total of 28 321 elements across all 27 Major Projects, these percentages round to zero.
Note 3: Excluding the six projects with the largest number of elements for delivery (i.e. Overlander Medium/Heavy, Hawkei, Bushmaster Vehicles, Overlander Light, Battle Comm. Sys. (Land), and BMS), results in an increase to the proportion of capability ‘not yet delivered’ to 46 per cent (from 30 per cent) and ‘not delivered at FOC’ to two per cent (from zero per cent). These six projects disproportionately weight the calculation of Capability Delivery Progress due to a large number of physical elements for delivery. These six projects represent 27 876 deliverables out of a total of 28 321 deliverables for all 27 Major Projects.
41. The ARH Tiger Helicopters platform was provided a caveated FOC and Defence faces ongoing risks and issues in relation to delivering the remaining capabilities.30 It is also impacted by technological obsolescence, related to delays in delivery, which impact future use. The impact of these issues has not translated into Defence’s assessment of future capability delivery performance, although they could reasonably be assumed to have a long term effect on capability. Refer to paragraphs 17 to 25 for further detail.
42. In addition to reporting on expected capability delivery, Defence has continued the practice of including declassified information on settlement actions for projects. Prior settlements for projects within this report related to MRH90 Helicopters, LHD Ships, ARH Tiger Helicopters and Maritime Comms.
1.1 This chapter provides an overview of the review’s scope and approach, as implemented by the Australian National Audit Office (ANAO), for the review of the 27 Project Data Summary Sheets (PDSSs) prepared by the Department of Defence (Defence). This chapter also provides the results of the Major Projects Report (MPR) review.
1.2 In 2012 the Joint Committee of Public Accounts and Audit (JCPAA) identified the review of the PDSSs as a priority assurance review, under section 19A(5) of the Auditor-General Act 1997 (the Act). This provided the ANAO with full access to the information gathering powers under the Act. The ANAO’s review of the individual project PDSSs, which are reproduced in Part 3 of this report, was conducted in accordance with the auditing standards set by the Auditor-General under section 24 of the Auditor-General Act 1997 through its incorporation of the Australian Standard on Assurance Engagements (ASAE) 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Australian Auditing and Assurance Standards Board.
1.3 The following forecast information is excluded from the scope of the ANAO’s review: capability delivery, risks and issues, and forecast dates. These exclusions are due to the lack of Defence systems from which to provide complete and accurate evidence31, in a sufficiently timely manner to complete the review. Accordingly, the Independent Assurance Report by the Auditor-General does not provide any assurance in relation to this information. However, material inconsistencies identified in relation to this information, are required to be considered in forming the conclusion.
1.4 The ANAO’s work is appropriate for the purpose of providing an Independent Assurance Report in accordance with the above auditing standard. However, the review of individual PDSSs is not as extensive as individual performance and financial statement audits conducted by the ANAO, in terms of the nature and scope of issues covered, and the extent to which evidence is required by the ANAO. Consequently, the level of assurance provided by this review, in relation to the 27 major Defence equipment acquisition projects (Major Projects), is less than that provided by the ANAO’s program of audits.
1.5 Separately, the ANAO undertakes analysis of key elements of the PDSSs and examines systemic issues and provides longitudinal analysis for the 27 projects reviewed.
1.6 The review was conducted in accordance with the ANAO Auditing Standards at a cost to the ANAO of approximately $1.8 million.
resolution of matters described in the Auditor-General’s prior year (2015–16) qualified Independent Assurance Report, relating to the ARH Tiger Helicopter PDSS and the LHD Landing Craft PDSS32;
the ongoing review of the maturity of the Enterprise Risk Management Framework (currently undergoing reform), and the completeness and accuracy of major risk and issue data in order to pilot bringing risks and issues into the scope of the Independent Assurance Report in the 2018–19 MPR (Section 5 of the PDSSs);
the impact of acquisition issues on sustainment to ensure the PDSS is a complete and accurate representation.
1.9 This review informed the ANAO’s understanding of the systems and processes supporting the PDSSs for the 2016–17 review period. It also highlighted issues in those systems and processes that warrant attention.
1.10 Consistent with previous years, key developments in acquisition governance processes are covered in the ANAO’s review in order to inform the planning process. While some initiatives are mature, others require further progress prior to achieving their intended impact.
Independent Assurance Review Boards
1.11 First introduced in 2008, the Gate Review (acquisition) process33 was designed to provide the Defence Senior Executive with assurance that all identified risks for a project are manageable, and that costs and schedule are likely to be under control prior to a project passing through the various stages of its life cycle. Gate Reviews were introduced for sustainment in 2013–14.
1.12 Since July 2016, Gate Reviews have been referred to as Independent Assurance Reviews34, with corporate policies and procedures updated for the revised processes under the modified Capability Life Cycle. The process has also introduced a contestability function, to focus on project business cases prior to government approval. Sixteen of the projects included in this report had an Independent Assurance Review conducted during 2016–1735, which formed key corroborative evidence for the ANAO’s review.
1.13 Defence advised in November 2017 that ‘Gate Review’ is now a description for a separate process that leads to Gate submission (to the Investment Committee) including the CASG Independent Assurance Review and the Capability Manager Gate Review.
1.14 First established in 2008, the Projects of Concern process was implemented to focus the attention of the highest levels of government, Defence and industry on remediating problem projects. The process has continued to play a role across the portfolio of MPR projects. As at 30 June 2017, two MPR projects were continuing projects of concern:
43. AWD Ships, a project of concern since June 2014, due to increasing commercial, schedule and cost risks, including difficulties and delays in shipbuilding36; and
44. MRH90 Helicopters, a project of concern since November 2011, due to technical issues preventing the achievement of milestones on schedule.37
1.15 In August 2017, the Ministers for Defence and Defence Industry announced38 that the Civil Military Air Traffic Management System (CMATS) project was being placed on the list due to substantial challenges getting into contract.39 The challenges revolve around issues with ensuring value for money for the taxpayer.
1.16 The Quarterly Performance Report (QPR) introduced in 2014, aims to provide senior stakeholders within government and Defence with a clear and timely understanding of emerging risks and issues in the delivery of capability to the Australian Defence Force’s end-users.40 Defence has advised that the report is provided to the Minister for Defence and the Minister for Defence Industry on a quarterly basis, with reports starting to cover the broader remit of the Capability Acquisition and Sustainment Group (CASG) deliverables, as recommended by the First Principles Review.41
1.17 In 2016–17, further to the two MPR projects of concern noted above, the June 2017 QPR also identified five MPR projects as Projects of Interest42:
Joint Strike Fighter, due to the inability to deliver one element of capability required for FOC;
LHD Ships, due to propulsion issues and other system defects, which have impacted on the forecast dates for FMR and FOC;
CMATS, due to ongoing contract negotiations and the request for approval of a Real Cost Increase (this project has since been declared a Project of Concern);
Battle Comm. Sys. (Land), due to additional validation and verification requirements not originally captured in the schedule, which have impacted on the forecast dates for FMR and FOC; and
1.18 The ongoing issues highlighted above for Joint Strike Fighter, LHD Ships, CMATS, Battle Comm. Sys. (Land) and UHF SATCOM align with the results of the ANAO’s review. Delays to progress have impacted the delivery schedule of four43 of these projects (see Table 4, on page 12).
1.19 The longstanding issue for Defence in maintaining complete and accurate records of government approvals for Major Projects, led to the introduction of Joint Project Directives (JPDs) (from March 2010).44 JPDs state the terms of government approval and are used to inform internal documentation such as Materiel Acquisition Agreements (MAAs)45 between CASG and the Service Chiefs.46
1.20 However, the initiative started slowly, with Defence taking over two years to begin to produce the first JPDs.47 Further, JPDs are regularly finalised after the MAAs they are intended to inform and, as a result, care is required to ensure that JPDs properly reflect the relevant government decision, and that MAAs are appropriately aligned with the relevant JPD.48
1.21 In 2016–17, 15 of the 16 MPR projects approved from 1 March 2010, have completed a JPD.49 However, the ANAO requires access to original approval documents to validate the requirements of projects. At this time, validation based on internal Defence documentation is not always possible.
1.22 The ANAO will continue to take JPDs into account in its review program in future years. However, the extent to which they can be relied upon will be dependent on the completeness and accuracy of JPDs, in relation to recording the detail of government approvals.
1.23 Product Delivery Agreements (PDAs) are being developed to replace the existing MAAs and Materiel Sustainment Agreements (MSAs).50 PDAs will be a higher level document (reviewed annually) that combine the MAA and MSA for each program. In August 2017, Defence advised that the development of the PDA templates was still ongoing.
1.24 Defence’s business systems rationalisation is aimed at consolidating processes and systems in order to provide a more manageable system environment.51 The Monthly Reporting System (MRS), which provides much of the data for the PDSSs, is to be replaced by the Project Performance Review for acquisition, and the Sustainment Performance Management System for sustainment.52 As reported to the JCPAA on 31 March 2017, Defence stated that there was a ‘need to get a single unified system of accountability and reporting inside the organisation’. Defence intends to rely on interfaces with existing systems, such as Open Plan Professional (OPP – the scheduling tool), rather than create another ‘system’.53
1.25 In September 2017, Defence advised that 33 Defence projects, 15 of which are included in the MPR, are participating in a pilot of the Project Performance Review. The pilot is still in its formative stages and development work and reviews will continue into 2018. Defence has advised that the MRS is still the mandated reporting system and will continue to be used until late 2018. The ANAO will review the progress of the pilot during the next reporting period.
1.26 The following sections outline the results of the ANAO’s review, which inform the overall conclusion in the Independent Assurance Report by the Auditor-General for 2016–17.
the contingency budget, which is established to provide adequate budget to cover the inherent cost, schedule and technical risks and uncertainties of the in-scope work of the project and any contingency events that may arise during the conduct of a project.54
1.29 In 2016–17, the ANAO reviewed the financial framework as it applied to managing project budgets and expenditure, including: contingency, project financial assurance and the reporting environment.
1.30 The project financial assurance statement was added to the PDSSs to enhance transparency by providing readers with information on each project’s financial position (in relation to delivering project capability) and whether there is ‘sufficient remaining budget for the project to be completed’.55
1.31 In 2016–17 the CMATS project is seeking approval for a significant Real Cost Increase which is anticipated to be considered by government in February 2018.
1.32 Defence has continued to subject a sample of project financial assurance statements to a third-party agreed-upon procedures engagement. The third party engagement supports Defence in assessing the project financial assurance statements, by reporting on projects’ compliance with an internal Defence instruction issued by the Chief Finance Officer.56
1.33 Projects selected for the 2016–17 third-party engagement, in support of the financial assurance statement assurance process, were:
additional procedures—Joint Strike Fighter and UHF SATCOM; and
standard procedures—LHD Ships, Additional MRTT and Battle Comm. Sys. (Land).
1.34 Defence has advised that the third-party engagement ‘found no adverse factual findings that would indicate any financial issues with the PDSS [project financial assurance statements] for the five selected projects’.
1.35 In conclusion, for the 2016–17 Major Projects Report, the Acting Chief Finance Officer’s representation letter to the Secretary on the project financial assurance statements was unqualified. The project financial assurance statement is restricted to the current financial contractual obligations of Defence for these projects, including the result of settlement actions and the receipt of any liquidated damages, and current known risks and estimated future expenditure as at 30 June 2017.
1.36 The purpose of the project contingency budget is ‘to provide adequate budget to cover the inherent risk of the in-scope work of the project’.57 Defence policy requires project offices to maintain a contingency budget log to identify and track components of the contingency budget.58
1.38 The five project offices which had contingency funds applied in 2016–17 were MRH90 Helicopters (supportability and performance risks), ANZAC ASMD 2A and 2B (gain share, combat management system, dockyard facilities and training facilities), Additional Chinook (building upgrade, missile warning system, improved vibration control and improved seating), and UHF SATCOM (software review and system security).
1.39 The ANAO’s examination of the contingency statements as at 30 June 2017 also highlighted that:
the clarity of the relationship between contingency application and identified risks continues to be an issue. Of the 25 project offices that have a formal contingency allocation59, eight projects (Joint Strike Fighter, P-8A Poseidon, LHD Ships, Hawkei, CMATS, ANZAC ASMD 2B, UHF SATCOM and ANZAC ASMD 2A) did not explicitly align their contingency log with their risk log, by including risk identification numbers as required by PRMM version 2.4;
the method for applying contingency varied, with 23 project offices using the ‘expected costs’ of the risk treatment (as required by PRMM version 2.4), with HATS using a proportionate allocation of the likelihood of the risk eventuating (the method outlined in PRMM version 2.2); and
there were seven project offices that did not meet all the requirements of PRMM version 2.4 in terms of keeping a record of review of contingency logs, however, the ANAO observed that the information required could be located in other documents.
1.41 Defence advised projects at the start of the year to change reporting of expenditure to a cash basis (previously accrual). This resulted in significant changes to financial disclosures for 14 projects during ANAO site visits. Financial reporting was reverted to an accrual basis at 30 June for the purposes of consistency in reporting across years.60
1.42 Defence prepares, on a cash basis, all financial data related to projects and capital programs provided within the Defence Portfolio Budget Statements, Portfolio Additional Estimates Statements and annual report.61 Therefore financial data in the PDSSs may not be consistent with that reported in the 2016–17 Defence annual report.
1.43 The ANAO also observed that incorrect exchange rates in system-generated commitment reports were used for projects managing contracts in foreign currencies, which meant that contract values had not been calculated correctly and required manual adjustment.
1.44 While major risks and issues data in the PDSSs remains excluded from the formal scope of the Auditor-General’s Independent Assurance Report, material inconsistencies identified in relation to this information are required to be detailed in the report.62 The following information is included to provide an overall perspective of how risks and issues are managed within Defence and the selected Major Projects.
1.45 Risk management has been a focus of the MPR since its inception. The CASG risk management environment consists of multiple policies and varying implementation mechanisms and documentation. There are multiple group-level (i.e. CASG), sub-group (i.e. Divisional) and project-level risk management documents. The primary focus of the ANAO’s examination of risk management is at the project level, in order to assure the PDSS.
1.46 The ANAO first became aware of a comprehensive risk reform being pursued in CASG following the provision of a consultant’s report which responds to recommendations contained in the First Principles Review.63 This report recognises the opportunity for improvement and makes a number of observations, including:
four of the eight CASG divisions have documented how risk management is conducted;
there does not appear to be a positive risk culture where risk is appropriately identified, assessed, communicated and managed; and
‘a general perception that culture in Defence and more broadly Defence industry was that the truth is not clearly represented or documented in risk reports. The justification and evidence of this, is that risk reports often do not align with reality and the issues that emerge’.
1.47 At the Group level, Deputy Secretary CASG issued a directive in May 2017 establishing a CASG Risk Management Reform Program to implement a risk management model that is situated within Defence’s risk management framework, to be implemented over two years. The first phase of the reform was expected to be completed by September 2017. Defence advised in October 2017 that this is yet to be completed. The ANAO will continue to monitor the implementation of the reform as part of future reviews, but will not be able to consider including risks and issues in scope until the 2018–19 MPR, when the reform is expected to be complete.
1.48 In 2016–17, the ANAO again examined project offices’ risk and issue logs at the Group and Service level, which are predominantly created and maintained utilising spreadsheets and/or Predict! software.64 Overall, the issues with risk management that the ANAO observed related to:
variable compliance with corporate guidance, for example, five out of 27 Major Projects did not update their Risk Management Plan in line with PRMM version 2.4;
risk management logs and supporting documentation of variable quality, particularly where spreadsheets are being used65; and
1.49 The ANAO has previously observed that Defence’s use of spreadsheets as a primary form of record for risk management is a high risk approach. Spreadsheets lack formalised change/version control and reporting, thereby increasing the risk of error. This can make spreadsheets unreliable corporate data handling tools as accidental or deliberate changes can be made to formulae and data, without there being a record of when, by whom, and what change was made. As a result, a significant amount of quality assurance is necessary to obtain confidence that spreadsheets are complete and accurate at 30 June, which is not an efficient approach. The ANAO’s review of CASG’s 27 project offices indicates that 14 utilise spreadsheets66 as their primary risk management tool, 11 utilise Predict! and one utilises a bespoke SharePoint based tool.67
1.50 Defence advised the JCPAA in March 2017 that there are ‘too many systems and too much variation in the way [Defence] apply risk management in [the] organisation.’68 While some project offices will experience greater challenges with risks and issues administration—often reflecting project complexity, scale and timing—it is important that Defence ensure that risk management systems and processes are used appropriately and consistently with the Defence Enterprise Risk Management framework. This is particularly important for higher cost/risk developmental projects.
1.51 Project Maturity Scores have been a feature of the Major Projects Report since its inception in 2007–08. The DMO Project Management Manual 2012, defined a maturity score as:
The quantification, in a simple and communicable manner, of the relative maturity of capital investment projects as they progress through the capability development and acquisition life cycle.69
1.52 Maturity scores are a composite indicator, cumulatively constructed through the assessment and summation of seven different attributes. The attributes are: Schedule, Cost, Requirement, Technical Understanding, Technical Difficulty, Commercial, and Operations and Support, which are assessed on a scale of one to 10.70 Comparing the maturity score against its expected life cycle gate benchmark provides internal and external stakeholders with a useful indication of a project’s progress.
1.53 The ANAO has previously raised inconsistency in the application of Project Maturity Scores as an issue. However this year, Defence has been more consistent in applying this guidance.
1.54 The policy guidance underpinning the attribution of maturity scores would benefit from a review for internal consistency and the relationship to Defence’s contemporary business. For example, allocating approximately 50 per cent of the maturity score at Second Pass Approval, regardless of acquisition type, is often inconsistent with the proportion of project budget expended, and the remaining work required to deliver the project.
1.55 Further, the existing project maturity score model does not always effectively reflect a project’s progress during the often protracted build phase, particularly for developmental projects. During this phase it can be expected that maximum expenditure will occur, and that many risks will be realised, some of which will only emerge as test and evaluation activities are pursued through to acceptance into operational service. For example, the ARH Tiger Helicopters project had capability deficiencies and obsolescence issues at FOC (declared on 14 April 2016), but the maturity score prepared for the 2015–16 MPR did not accurately represent the project’s maturity as at 30 June 2016, and the maturity score prepared for the 2016–17 MPR does not accurately represent the project’s maturity as at 30 June 2017. Refer to paragraphs 17 to 25 for further detail.
1.56 The policy guidance underpinning maturity scores was due for review in September 2012.71 In May 2016, the JCPAA recommended ‘that the Department of Defence work with the Australian National Audit Office to review and revise Defence’s policy regarding Project Maturity Scores in time for the new approach to be implemented in the next Major Projects Report.’72 In response, Defence engaged a contractor to develop a more appropriate methodology to support the presentation of the Project Maturity Score graphs. However, due to the immaturity of the processes and systems referred to, CASG is not yet in a position to test or apply such a methodology and has not proposed an approach to the ANAO.
1.57 In October 2017, the JCPAA recommended ‘that the Department of Defence commence discussions with the Australian National Audit Office on updating Project Maturity Scores, with a view to advising the Committee on a way forward prior to the first sitting week of 2018.’73
1.58 In 2016–17, the ANAO noted a continuing trend of Major Projects which have achieved significant milestones with caveats.74 Table 9 below lists the current MPR projects which have achieved a major milestone with caveats.75
Table 9: Caveated projects
Milestone (Year)
Status of Caveats (as at 30 June 2017)
IMR (2014) and IOC (2015)
Capability requirements;
FOC (2016)
Unresolved–both lifted in September 2017
IMR and IOC (2016)
Supply support deficiencies;
Unresolved—both lifted in August 2017
Availability of supplies; and
Availability of industry and Defence staff.
Unresolved—two lifted in July 2017
1.59 At JCPAA hearings on 31 March 2017, Defence confirmed that caveats are an infrequent event.76
1.60 The ANAO will continue to monitor the declaration and resolution of caveats in future reviews. Additionally, from 2017–18, projects which have been removed from the MPR which still have outstanding caveats are required to report on the status of these caveats in the Statement by the Secretary of Defence until their final status is accepted by the Capability Manager.77
percentage of time elapsed (Time Elapsed)—which measures the percentage of time elapsed from original approval to the forecast Final Operational Capability (FOC)78; and
percentage of key materiel capabilities delivered79 (Capability Delivery Progress)—which measures the total capability elements delivered as a percentage of the total capability elements across all Major Projects.
2.5 The following sections of this chapter provide analysis relating to the three principal components of project performance. This includes in-year information, longitudinal analysis and the results of project progress for the year-ended 30 June 2017. The first piece of analysis, in Figure 2 below, sets out each project’s Budget Expended and Time Elapsed.80
Note: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A is expected to achieve FMR and MAA closure in quarter one 2018.
2.6 Figure 2 shows that for most projects (21 of 27), Budget Expended is broadly in line with, or lagging, Time Elapsed.81 This relationship is generally expected in an acquisition environment predominantly based on milestone payments. However, due to the varying complexity, stages and acquisition approaches across the portfolio of projects, further analysis of these simple performance measures is required to provide an overall picture of key variances.
2.7 Where Budget Expended is significantly lagging Time Elapsed the project schedule may be at risk, i.e. expenditure lags may indicate delays in milestone achievement. However this is not the case for the two projects where the Budget Expended is over 20 per cent less than the Time Elapsed in 2016–17, as detailed below:
Joint Strike Fighter (Budget Expended 10 per cent, Time Elapsed 54 per cent)—a large scope increase ($10.5 billion) for the purchase of additional aircraft was approved in April 2014, with the project yet to enter into main production contracts, as aircraft development continues; and
Additional Chinook (Budget Expended 70 per cent, Time Elapsed 99 per cent)—the variance reflects cost savings achieved through the integration of a number of previously post production modifications (including some of the Australian unique modifications) on the production line and progressive price reductions in the Foreign Military Sales (FMS) case.
2.8 Where Budget Expended leads Time Elapsed the project budget may be at risk, i.e. expenditure increases may indicate real cost increases. However, for the five projects where Budget Expended leads Time Elapsed by 10 per cent or more, the actual reasons are related either to early procurement of major equipment due to production timing, or schedule delays caused through platform availability, as detailed below:
P-8A Poseidon (Budget Expended 52 per cent, Time Elapsed 42 per cent)—most of the expenditure on equipment is in line with aircraft production over the coming three financial years. Three aircraft have now been delivered to Defence.
Growler (Budget Expended 61 per cent, Time Elapsed 45 per cent)—expenditure reflects aircraft production costs (which represent a large proportion of project costs) having occurred before a large decrease in annual expenditure over the following years as work continues on the Mobile Threat Training Emitter System. All aircraft have now been delivered to Defence. The variance is also exacerbated by the length of time between Initial Operational Capability (IOC) (July 2018) and FOC (June 2022) with most of the major equipment being delivered by 2018.
MH-60R Seahawk (Budget Expended 58 per cent, Time Elapsed 48 per cent)—the project has taken delivery of all 24 aircraft. The variance is caused by the time between final aircraft delivery and FOC, which is being used to implement Australian unique modifications and modify navy vessels to operate with the MH-60R Seahawk.
LHD Ships (Budget Expended 90 per cent, Time Elapsed 80 per cent)—most of the budget has been expended. The Final Materiel Release (FMR) and FOC milestones have been further delayed in 2016–17 due to the unavailability of the LHD Ships to conduct operational test and evaluation activities as a result of issues with the propulsion pods and ongoing remediation of other systems.
Collins R&S (Budget Expended 89 per cent, Time Elapsed 77 per cent)—most of the materiel has been acquired and expenditure undertaken. In addition, originally planned installation dates have been extended based on submarine availability, reducing the proportion of time elapsed.
2.11 The practice of issuing caveated milestones will require Defence to exercise appropriate judgement for the capability disclosures within the MPR, in order to prepare project PDSSs that provide an accurate depiction of performance to readers of the PDSS while also ensuring that classified data is prepared in such a way as to allow for unclassified publication. Additionally, the ANAO may need to monitor and report on projects ‘in sustainment’, when projects complete tasks defined, and funded, for delivery in acquisition.
For example, the ARH Tiger Helicopters acquisition received caveated FOC and requires additional funding to address outstanding issues. The ANAO’s performance audit82 identified that the funding required to remediate the ARH Tiger Helicopters was beyond the scope of the already approved $2 033.0 million for the acquisition project. Expert analysis commissioned by Defence indicates that the issues arising from the developmental nature of the ARH Tiger Helicopter platform and sub-optimal sustainment arrangements will endure.83
2.12 The JCPAA agreed to removing the ARH Tiger Helicopters project from the 2017–18 MPR, instead requiring that the status of projects achieving FMR/FOC with caveats be reported in the Statement by the Secretary of Defence until their final status is accepted by the Capability Manager.
2.13 The practice of Defence issuing caveats to milestones is discussed further in Chapter 1, in paragraphs 1.58 to 1.60.
2.14 Figure 3, below, sets out each project’s Budget Expended against Project Maturity84 and shows that Budget Expended lags Project Maturity for the majority of projects (18 of 27). This relationship is expected for two reasons:
where there is a larger proportion of Military Off-The-Shelf (MOTS) projects. MOTS products are generally in-service with other military forces, and will generally have benefited from significant development and testing, prior to selection by Defence, resulting in a higher level Project Maturity score.
2.15 Budget Expended lags Project Maturity with a variance of 20 per cent or more in 12 projects. As expected, these projects are generally classified as either MOTS or Australianised MOTS. The exceptions are Joint Strike Fighter, which is expected to be classified as MOTS by the time of aircraft delivery; Hawkei, which is still conducting testing and other reviews prior to the start of full-rate production; and CMATS, which remains in negotiation with the prime contractor ahead of signing the main acquisition contract. There are no instances where Budget Expended leads Project Maturity by 20 per cent or more.
2.16 The variances are, in part, the result of Defence’s project maturity framework attributing approximately 50 per cent of total Project Maturity at Second Pass Approval (the main investment decision by government).85 This reduces the value of project maturity assessments during the early stages of acquisition.
2.17 Defence’s focus on typically lower risk MOTS acquisitions in recent years, has assisted in meeting schedule timelines across projects.86 Analysis of the available performance information highlights that the selection of MOTS projects assists in reducing risk during project acquisition, where Project Maturity is more advanced at Second Pass Approval than developmental projects.
Second Pass Approval and 30 June 2017 approved budget
2.18 Figure 4, below, compares each project’s approved budget at Second Pass Approval and its approved budget at 30 June 2017.
2.19 The total budget for the 27 projects at 30 June 2017 was $62.0 billion, a net increase of $21.5 billion, when compared to the approved budget at Second Pass Approval of $40.5 billion (detailed analysis of this variance is included in Table 3, on page 11).
2.20 Figure 4 indicates relative budget variations from Second Pass Approval of $500 million or more for six projects. The list below describes the components of these variations:
Joint Strike Fighter—increase of $13.3 billion, comprising $10.5 billion for 58 additional aircraft in 2013–14, $2.4 billion for exchange rate variation and $0.4 billion for price indexation;
AWD Ships—increase of $1.9 billion, comprising $1.2 billion for a Real Cost Increase87 in July 2015 to complete the project, $1.2 billion for price indexation, offset by a $0.4 billion decrease for exchange rate variation and a $0.1 billion decrease for transfers to facilities projects in 2013–14;
P-8A Poseidon—increase of $1.7 billion, comprising $1.3 billion for four additional aircraft in 2015–16 and $0.4 billion for exchange rate variation;
MRH90 Helicopters—increase of $2.8 billion, comprising $2.4 billion for 34 additional aircraft in 2005–06 and other minor scope changes, $0.7 billion for price indexation, offset by a $0.3 billion decrease for exchange rate variation;
Growler—increase of $0.8 billion, comprising $0.9 billion for exchange rate variation, $0.2 billion in 2014–15 for the Mobile Threat Training Emitter System and weapons, and in 2016–17, $0.1 billion for Advanced Mobile Threat Training Emitter System scope, offset in 2015–16 by a $0.2 billion decrease for transfers to facilities projects and $0.1 billion for the return to the Defence budget of surplus funds for re-allocation and in 2016–17 by a $0.1 billion decrease for the reduction of project contingency associated with aircraft production being returned to the Defence budget for re-allocation; and
Bushmaster Vehicles—increase of $1.0 billion, comprising $0.8 billion for 715 additional vehicles in 2007–08 (437 vehicles), 2011–12 (70 vehicles) and 2012–13 (208 vehicles) and other minor scope changes and $0.1 billion for price indexation.88
Figure 4: Projects’ Second Pass Approval and 30 June 2017 approved budget ($m)
Note 1: [] indicates that the budget for the project at 30 June 2017 is less than the original budgeted cost. However, for Overlander Light this reflects a transfer of $2.2 billion to Overlander Medium/Heavy on separation of the original project into two phases in December 2011.
2.21 The following figures and tables illustrate the budget performance for the 27 selected projects by way of:
in-year budget variations by project (see Table 10, below); and
expenditure forecasting performance against actual expenditure for 2016–17 (see Figure 5, on page 42).
2.22 Table 10, below, sets out the in-year budget variations for each project. Overall, the approved budget for the projects as at 30 June 2017 decreased by $1 641.2 million, or 2.7 per cent, compared to their approved budget as at 30 June 2016. This was driven by net real decreases of $162.3 million, and exchange rate variation decreases of $1 478.7 million.
2.23 Real Variations89 primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability and budgetary adjustments such as administrative savings decisions. In 2016–17, the two projects with more significant Real Variations were90 :
Growler—variation of $2.6 million reflecting approval for the Advanced Mobile Threat Training Emitter System scope, offset by the return to the Defence budget of surplus funds for re-allocation; and
ARH Tiger Helicopters—variation of -$165.0 million reflecting the return to the Defence budget of unspent funds at closure of the acquisition project.91
2.24 Exchange rate variations result from projects’ exposure to foreign currencies and movements in foreign exchange rates against the Australian dollar.92 Budget adjustments aim to maintain the relative buying power of the project budget. Movements in the US dollar and the Euro are the main influences. Projects with larger movements in foreign exchange in 2016–17 included:
Joint Strike Fighter—movement of -$733.5 million, or 4.4 per cent decrease in budget;
P-8A Poseidon—movement of -$257.4 million, or 4.7 per cent decrease in budget; and
Overlander Medium/Heavy—movement of -$102.1 million, or 2.9 per cent decrease in budget.
Table 10: In-year (2016–17) budget variations by project
Bushmaster Vehicles 1
Maritime Comms 1
60 875.8
(1 478.7)
(1 641.2)
Source: ANAO analysis of the 2015–16 and 2016–17 PDSSs.
2.25 Accurately forecasting and managing budget expenditure is an important element in the management of a portfolio of projects. Figure 5, below, sets out the expenditure forecasting performance of each project against actual expenditure in 2016–17. It should be noted that the PDSSs report expenditure on an accrual basis, while the budget figures are reported on a cash basis for the first time in 2016–17.93 In total, actual expenditure for the 27 projects at 30 June 2017 was $4 076.1 million when measured on a cash basis.94 This cash expenditure is compared to $4 137.3 million per the PDSSs (accrual basis), which was 1.5 per cent, or $61.2 million higher than the cash expenditure. This is compared against an initial Portfolio Budget Statements (PBS) forecast expenditure of $4 803.7 million, a mid-year Portfolio Additional Estimates Statements (PAES) forecast of $4 417.7 million, and a final forecast of $4 282.7 million (Final Plan, approved during May 2017). The main factors contributing to the variances on a cash basis were changes to delivery and payment schedules, and foreign exchange fluctuations.
2.26 Figure 5 highlights that notable in-year underspends occurred in the following projects:
AWD Ships (accrual expenditure of $532.7 million and cash expenditure of $575.0 million compared to $725.5 million PBS, $675.7 million PAES and $674.0 million Final Plan estimates)—the remainder of the variance is due to the deferral of payments from 2016–17 to 2017–18 and savings against indexation estimates and direct project costs;
MRH90 Helicopters (accrual expenditure of $104.4 million and cash expenditure of $131.0 million compared to $174.4 million PBS, $180.8 million PAES and $175.5 million Final Plan estimates)—the remainder of the variance is due to the deferral of payments across a number of project deliverables;
Growler (accrual expenditure of $168.2 million and cash expenditure of $128.1 million compared to $242.0 million PBS, $157.4 million PAES and $165.8 million Final Plan estimates)—the remainder of the variance is due to the deferral of payments for the Advanced Mobile Threat Training Emitter System, which was originally scheduled to occur in 2016–17 but has shifted to 2017–18 because of a delay in contract signature;
MH-60R Seahawk (accrual expenditure of $123.7 million and cash expenditure of $78.2 million compared to $230.0 million PBS, $183.9 million PAES and $141.0 million Final Plan estimates)—the remainder of the variance is due to delays in deliveries and subsequently payments for some FMS deliverables and ANZAC and AWD integration activities;
Hawkei (accrual expenditure of $24.3 million and cash expenditure of $25.3 million compared to $95.2 million PBS, $55.8 million PAES and $55.4 million Final Plan estimates)—the remainder of the variance is due to delays in the completion of testing and the resulting delay of milestone payments; and
Battlefield Airlifter (accrual expenditure of $48.6 million and cash expenditure of $33.1 million compared to $147.6 million PBS, $72.8 million PAES and $60.7 million Final Plan estimates)—the remainder of the variance is due to reductions in FMS case payments and delays in contracting for a number of project deliverables, including spare parts, support equipment and maintenance.
2.27 Figure 5 also highlights that notable in-year overspends occurred in the following projects:
Joint Strike Fighter (accrual expenditure of $745.3 million and cash expenditure of $669.0 million compared to $725.7 million PBS, $644.2 million PAES and $613.4 million Final Plan estimates)—the remainder of the variance is due to the acceleration of payments due to earlier aircraft production; and
P-8A Poseidon (accrual expenditure of $1 145.0 million and cash expenditure of $1 145.3 million compared to $1 046.8 million PBS, $1 089.6 million PAES and $1 108.6 million Final Plan estimates)—the remainder of the variance is due to the acceleration of payments due to earlier aircraft production. This is partially offset by deferral of air to air refuelling clearance activities and support system spares purchases and the associated payments.
Figure 5: In-year (2016–17) projects’ forecast expenditure performance compared to actual expenditure ($m)
Sources: ANAO analysis of the 2016–17 PDSSs, cash basis expenditure data provided by Defence and Defence Portfolio Budget Statements.
2.28 Defence data continues to show that schedule performance is a key issue in delivering and sustaining equipment.95 Project schedule slippage can effectively introduce or exacerbate an existing capability gap, or require an extension to the planned withdrawal date for those platforms being replaced.96
2.29 Figure 6, below, sets out each project’s Time Elapsed against Project Maturity.97 Time Elapsed lags Project Maturity for 15 of 27 projects. The 15 projects are classified as either MOTS or Australianised MOTS, except Joint Strike Fighter, which is expected to be classified as MOTS by the time of aircraft delivery, as well as Hawkei and CMATS, which are new developmental projects.
2.30 For the nine projects where Time Elapsed lags Project Maturity by 20 per cent or more, this typically reflects projects at an early stage of acquisition processes, including proceeding through design processes and awaiting significant amounts of their major equipment to be constructed and delivered. There are two significant exceptions to this:
Growler, where the EA-18G aircraft have been accepted and transferred to Australia, but project scope was increased in April 2017 to include a phased array radar threat emitter which is used in training; and
MH-60R Seahawk, where the majority of equipment has been delivered but the project needs to test and integrate a number of ADF Mission System Options and modify Navy vessels to operate with the helicopter.
2.31 For the 12 projects where Time Elapsed leads Project Maturity, there were no instances where this difference was significant (20 per cent or more).
Note 2: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A is expected to achieve FMR and MAA closure in quarter one 2018.
2.32 Figure 7, below, illustrates the total schedule slippage98 since Second Pass Approval for the 27 selected projects. It also depicts the acquisition type and places projects in order of government approval. Figure 8 illustrates the total schedule slippage for the 14 projects that have exited the review.
2.33 Figures 7 and 8 show that the continued focus on MOTS and Australianised MOTS acquisitions is, prima facie, contributing to a reduction in schedule slippage in the Major Projects portfolio. However, it is not always possible to acquire the necessary capability in this manner, and decisions on whether to undertake developmental projects should be considered on a risk basis. In this context, the consideration of risk includes not just the project specific attributes related to procurement, but also any compromises to the capabilities that would have been acquired through a developmental acquisition program.
2.34 The 2008 Audit of the Defence Budget (Pappas Review) identified technical risk as the largest source of post Second Pass Approval schedule slippage for ‘post Kinnaird’ projects99, and also observed that schedule slippage causes cost escalation.100 The challenge of gaining a full understanding of the complexities of developmental aspects of projects at Second Pass Approval is evident by the extent of slippage over time.
2.35 Figures 7 and 8 illustrate that older projects, which achieved Second Pass Approval prior to 2005, have experienced the most slippage. These projects tended to be more developmental (complex) in nature and typically experienced schedule slippage in the past, and have often continued to do so. This demonstrates an ongoing trend of slippage in historically late projects, which is more pronounced in older projects. This trend is also visible, but less prominent, in newer projects.
2.36 While it is not possible to predict the full extent of slippage a project will experience, this analysis has been provided to highlight changes since Kinnaird. Seven post Kinnaird and seven pre Kinnaird projects have exited the MPR. Total slippage of the seven post Kinnaird projects is 5.8 years. Total slippage of the seven pre Kinnaird projects is 44.3 years. Five of the seven post Kinnaird projects were MOTS acquisitions and all of the seven pre Kinnaird acquisitions were Australianised MOTS or Developmental.
Note 3: BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A is expected to achieve FMR and MAA closure in quarter one 2018.
Note 4: The following projects have had additional scope approved following Second Pass Approval: Joint Strike Fighter, P-8A Poseidon, MRH90 Helicopters, Bushmaster Vehicles, Overlander Light and Additional MRTT.
2.37 The figures and tables that follow illustrate:
the original and 30 June 2017 forecasts for achieving FOC;
Original and 30 June 2017 Final Operational Capability forecasts
2.38 Figure 9, below, presents information on the selected projects’ original and 30 June 2017 forecasts for achieving FOC. The total schedule slippage for the 27 Major Projects to date is 793 months compared to the initial prediction when approved by government. This represents a 29 per cent increase on the approved schedule.101 Of the 27 projects in the 2016–17 report, 21 have experienced schedule slippage.
2.39 Total schedule slippage across the Major Projects was 793 months in 2016–17. This is 85 months higher than the figure of 708 months reported in the 2015–16 report. The difference is mainly due to significant slippage in P-8A Poseidon (additional aircraft purchase decision), LHD Ships (technical difficulties delaying test), Additional MRTT (additional Government Transport and Communication capability), CMATS (ongoing contractual negotiations), and LHD Landing Craft (LHD Ships availability for operational test and evaluation activities).102 These projects, combined, added 123 months of the 149 months schedule slippage in 2016–17, but were offset by the exit of Air to Air Refuel, which reduced the accumulated slippage by 64 months.
2.40 The reasons for schedule slippage often require a deep understanding of project technical elements and a realistic assessment of the capacity of the private sector to deliver in the expected timeframe. A project office’s ability to gain access to the platform for upgrading can also result in schedule delay (for example, the two Collins submarine projects and Hw Torpedo).103
2.41 A closer examination of the reasons for schedule slippage demonstrates the importance of initial assessments of project complexity. A key factor is whether a project is MOTS, Australianised MOTS or developmental.104 Two projects, MRH90 Helicopters105 and ARH Tiger Helicopters106, were originally misclassified as MOTS. The projects were reclassified by Defence to Australianised MOTS (i.e. more developmental) subsequent to Second Pass Approval. Both projects have experienced extended schedule slippage.107
Figure 9: Projects’ original and 30 June 2017 FOC forecasts
Note 1: [] indicates that the forecast FOC date for the project at 30 June 2017 is earlier than the original FOC date.
Source: ANAO analysis of the 2016–17 PDSSs.
2.42 In 2016–17, there was schedule slippage of 149 months in the forecast achievement of FOC across the 27 Major Projects. In-year project performance, measured by slippage over the last 12 months, may not reflect the project trend. However, Figure 10 below, shows recovery of previously reported slippage for two projects, and that one project anticipates delivery ahead of the original schedule:
Hw Torpedo and Collins RCS—the projects currently expect to achieve FOC two months ahead of the 2016 forecast schedule in December 2018; and
Growler—the project currently expects to achieve FOC one month ahead of the original schedule.
2.43 In-year schedule slippage occurred for the following ten projects108 (the explanation provided, drawn from the 2016–17 PDSSs, may also include the reasons for prior slippage):
AWD Ships—the variance reflects minor rescheduling of the FOC milestone;
P-8A Poseidon—the delay reflects additional schedule required following the approval to purchase an additional four aircraft in March 2016;
LHD Ships—the delay reflects technical issues that have impacted the availability of the LHDs to conduct test activities. These technical issues have also delayed the rectification of outstanding acquisition activities;
Bushmaster Vehicles—the variance reflects a minor delay to the declaration of FOC by the Capability Manager;
Additional MRTT—the delay to FOC reflects adjustments to the project schedule to account for the inclusion of the Government Transport and Communications capability following contract signature in August 2016;
CMATS—the delay to FOC reflects ongoing delays in negotiations of the primary acquisition contract with the forecasts remaining uncertain at this time. A final forecast FOC date is not expected to be available until the primary acquisition contract is signed;
Additional Chinook—the delay in the FMR and FOC milestones relates to delayed delivery of Aircraft Survivability Equipment training and certification of the Crashworthy Pilot Seat;
Battle Comm. Sys. (Land)—the further delay continues to relate to the need to clarify the forecast dates for FMR and FOC with the capability manager following Full Design Acceptance in December 2016;
UHF SATCOM—FOC has been delayed as a result of FMR number 2 being rescheduled to January 2019; and
LHD Landing Craft—the project has incurred further delay as final operational test and evaluation trials are yet to occur. The trials are currently anticipated to occur in the second quarter of 2018.
Figure 10: In-year (2016–17) schedule changes to achieving FOC
Note: Defence’s PDSSs indicate that 14 of the 27 Major Projects Report projects did not record changes to their Final Operational Capability dates this year.
2.44 Figure 11, below, shows the accumulated schedule slippage over time of the Major Projects included in the MPR reports from 2007–08 to 2016–17.109 Table 11 provides the details of the specific projects included in the analysis. The figure shows that 24.2 per cent (16.0 years or 192 months) of the total schedule slippage across the Major Projects covered in the 2016–17 report (66.1 years or 793 months) is made up of the slippage from the three remaining projects reported in the 2007–08 Major Projects Report.110
2.45 Further disaggregation according to a project’s Second Pass Approval date in Table 12, on page 54, shows that 54 per cent (2015–16: 69 per cent) of the total schedule slippage across the 2016–17 Major Projects is made up of projects approved prior to July 2005.
Figure 11: Longitudinal schedule slippage across years for projects in the 2016–17 MPR (in years)
Note 1: The total schedule slippage in 2016–17 across the 27 projects is 793 months. BMS does not have IOC or FOC milestones. These were to be linked to Work Packages B-D which received government approval in September 2017 under LAND 200. Work Package A is expected to achieve FMR and MAA closure in quarter one 2018.
Table 11: Projects included in Figure 11 analysis by Major Projects Report
Table 12: Project slippage by project approval
No. of months between Approval and 30/6/17 FOC date
No. of months slippage between Original FOC and 30/6/17 FOC date
Note 1: These figures do not add horizontally due to the exclusion of schedule reductions over the life of the project. Refer to footnote 26.
2.46 Defence defines capability as the power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.111 An operational effect is achieved by combining the nine Fundamental Inputs to Capability – organisation; command and management; personnel; collective training; major systems; facilities and training areas; supplies; support; and industry112 – and undertaking designated operations.
2.47 In acquiring Defence platforms and systems, a range of documentation (including capability definition, operational concept, function and performance specification, and test concept documents) is developed, which establishes the detailed requirements/performance attributes to be achieved.
2.48 Since the 2009–10 MPR, capability reporting113 has been based on Defence’s prediction of the final capability that would be achieved on the basis of deliverables and/or activities completed. This assessment of capability performance (Expected Capability) was measured against the Materiel Release Milestones (MRMs) and Completion Criteria specified in each project’s Materiel Acquisition Agreement (MAA). As the ANAO has previously noted, this data involved ‘…making certain assumptions in forecasting achievements and is therefore subjective in approach…’.114
2.49 For example, for the LHD Landing Craft project, Defence predicted and reported that 99 per cent of elements of capability have a ‘high level of confidence of delivery’, and accordingly reported a predominantly ‘green’ capability pie chart graphic in the PDSS prepared for the 2015–16 MPR. However, as reported to the JCPAA on 17 March 2016 during public hearings, trials to test the ability to transport a M1A1 Main Battle Tank are required prior to the achievement of Final Operational Capability.115
Subsequent trials conducted in May 2016 were unsuccessful. Carrying the M1A1 on the LHD Landing Craft requires the operation of the craft in an overload state. In consideration of the unsuccessful trials, the 2015–16 PDSS depicted that one per cent of capability for the LHD Landing Craft is ‘under threat, considered manageable (Amber)’, based on Defence’s consideration of the landing craft’s functional performance specification.
Defence updated the JCPAA during public hearings on 31 March 2017, advising that: ‘We were going to be doing trials in late 2016. We are now putting them back off. With the issues with the two LHDs at the moment, we will be putting that back off until we can actually resolve those, because you actually want to transit out the back of the ship in a docked position and we are not in that position right now to do that’.116 The LHD Landing Craft PDSS prepared for the 2016–17 Major Projects Report now advises that these tests are not expected until the second quarter of 2018, and continues to report one per cent of capability ‘Amber’. If the tests are successful, it is anticipated that Defence will produce a 100% green capability graphic to reflect increased LHD Landing Craft capability.
2.50 Over time, the JCPAA has sought the use of a more robust measure of capability performance. For example, in JCPAA Report 442, the Committee recommended:
ANAO and DMO should provide a specific proposal to the Committee preferably by the end of August 2014 in line with submission of the 2014-15 MPR Guidelines.117
2.51 Defence has not developed this measure, reporting that the difficulties relate to the varied nature of projects being managed, the inherent subjectivity of the content, and the lack of a system that tracks at a sufficient level of detail the progress of inputs to capability.118
2.52 Noting with concern the ANAO’s assessment that delivery of capability estimates were in some cases overly optimistic, the Committee recommended that Defence further review the procedure for development of expected capability estimates.119
To ensure consistency with project level risk information and to improve reliability, the Committee recommends that the Department of Defence review the procedure for development of expected capability estimates for future Major Projects Reports.120
2.53 Defence has not yet developed an alternative method of capability assessment which would enable an improvement in capability reporting in the MPR, or put a proposal to the ANAO.
2.54 In October 2017, the JCPAA recommended ‘that the Department of Defence review the procedure for the development of expected capability estimates for future Major Projects Reports. The outcomes of this review should be provided to the Committee within six months of the tabling of this report. Further, the Committee requests that Defence provide a progress report within three months of the tabling of this report.’121
2.55 In light of the above, in 2015–16 the ANAO developed a measure of key materiel capabilities delivered (Capability Delivery Progress). This presents a current assessment of the capability delivered which differs from Defence’s prediction of final capability. The information used in forming the ANAO’s assessment is primarily based on Section 4.2 of the PDSS, which sets out the capability elements required to achieve Initial Materiel Release and Final Materiel Release, combined with other information in the PDSS reporting the delivery of equipment/achievement of these requirements toward FOC.
2.56 Noting that a system of capability reporting with a robust methodology applicable to materiel acquisition does not exist within Defence, the information presented below is a more meaningful reflection of project progress than an end-state prediction.
2.57 Figure 12, below, sets out each project’s Capability Delivery Progress against Project Maturity.122 It shows that Capability Delivery Progress lags Project Maturity for the majority of projects (16 of 27). This relationship is expected as projects will typically develop confidence in the ability to deliver their scope and capability through testing and demonstration of capability components (for example, design reviews and acceptance tests) prior to delivery of the majority of equipment.
2.58 Figure 12 also shows that Capability Delivery Progress lags Project Maturity by 20 per cent or more in 10 projects, and for six of these, Capability Delivery Progress lags by 50 per cent or more.
2.59 As noted in paragraph 2.16, Defence’s project maturity framework attributes approximately 50 per cent of total project maturity at Second Pass Approval.123 These differences further indicate that Defence’s project maturity framework is not appropriately structured to assign project maturity progress throughout the project life cycle, particularly within the acquisition phase, which is predominantly the longest and most expensive component.
2.60 Figure 12 also highlights a continuing issue with the level of specification of capability elements. For the projects that show little or no Capability Delivery Progress, this can be attributed to Defence’s high level description of requirements in the capability elements. This indicates that it would be worthwhile for Defence to undertake additional work to track project progress. In respect of the three projects that show no capability delivery at 30 June 2017, progress is as follows:
Hawkei—this project is progressing through design processes and Low Rate Initial Production was approved in August 2017;
CMATS— this project is in early stages of procurement, and is progressing through early design processes ahead of signing the primary acquisition contract. Additionally, in August 2017 this project was designated a Project of Concern; and
2.61 Further, Figure 12 indicates that:
three projects are still to deliver any of their capability (refer to paragraph 2.60 for detail);
19 projects are still to deliver part of their capability;
three projects, Bushmaster Vehicles, Overlander Light and Battle Comm. Sys. (Land), have delivered essentially all of their capability with only minor items remaining;
one project, ARH Tiger Helicopters, had not delivered all of its intended capability at FOC, and remediation has been ongoing since FOC.
Based on the procedures I have performed and the evidence I have obtained, except for the effects of the matters described in the Bases for Qualified Conclusion paragraphs, nothing has come to my attention that causes me to believe that the information in the 27 Project Data Summary Sheets in Part 3 (PDSSs) and the Statement by the Secretary of Defence, excluding the forecast information, has not been prepared in all material respects in accordance with the 2016–17 Major Projects Report Guidelines (the Guidelines), as endorsed by the Joint Committee of Public Accounts and Audit.
I have undertaken a limited assurance engagement of the PDSSs, reporting on the status of the projects selected by the Joint Committee of Public Accounts and Audit, and the Statement by the Secretary of Defence, for the year-ended 30 June 2017. The following forecast information was excluded from the scope of this engagement:
The project maturity score in Section 6.1 of the ARH Tiger Helicopters PDSS reports a total of 69 out of a maximum of 70 (98.6 per cent) at the time of transition from acquisition to sustainment in April 2017. Noting the caveats, capability deficiencies and obsolescence issues at the declaration of Final Operational Capability in April 2016, this score does not accurately or completely represent the project’s maturity as at 30 June 2017. This represents a departure from the Guidelines. The lifting of two of the nine caveats in July 2017 was a result of events occurring prior to 30 June 2017 and, accordingly, my conclusion has had regard to the caveats being lifted.
In addition, a material inconsistency has been identified in the forecast information. Section 4.1 in the ARH Tiger Helicopters PDSS reports that materiel capability delivery performance is at 100 per cent, indicating that materiel capability delivery performance has been met. Rate of effort continues to be lower than planned, and expert analysis commissioned by Defence in April 2016 indicates that the program will remain incapable of fully meeting expectations relating to reliability, availability, maintainability and rate of effort.
The Secretary of Defence is responsible for the preparation and presentation of the PDSSs for the 27 selected projects, and the Statement by the Secretary of Defence, in accordance with the Guidelines. This responsibility includes the design, implementation and maintenance of internal control relevant to the preparation of PDSSs that are free from material misstatement, whether due to fraud or error. The Guidelines provide that the PDSSs and supporting evidence, provided to the ANAO for review, are complete and accurate.
My responsibility is to express an independent limited assurance conclusion on the PDSSs and Statement by the Secretary of Defence, based on the procedures I have performed and the evidence I have obtained. I conducted the engagement in accordance with the auditing standards set by the Auditor-General under section 24 of the Auditor-General Act 1997 through its incorporation of the Australian Standard on Assurance Engagements ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, issued by the Auditing and Assurance Standards Board. ASAE 3000 requires that I comply with relevant ethical requirements and that I plan and perform my procedures to obtain limited assurance about whether the PDSSs and the Statement by the Secretary of Defence are prepared in all material respects in accordance with the Guidelines.
The attached Project Data Summary Sheets (PDSS) for the 27 major projects included in this report have been prepared in accordance with the Guidelines developed by Defence in consultation with the Australian National Audit Office (ANAO) and endorsed by the Joint Committee of Public Accounts and Audit (JCPAA). In making this statement, I acknowledge the difference of view between Defence and the ANAO in relation to the AIR 87 Phase 2 - Armed Reconnaissance Helicopter PDSS.
I am confident that the PDSS for this project is an accurate reflection of the acquisition of this capability as at 30 June 2017, and is compliant with the 2016-17 Major Projects Report Guidelines.
Project Status as at 30 June 2017
In my opinion, the Project Data Summary Sheets comply in all material respects with the Guidelines and reflect the status of the projects as at 30 June 2017.
Significant Events Occurring Post 30 June 2017
In stating this opinion, I acknowledge the following material events have occurred post 30 June 2017:
AIR 6000 Phase 2A/B Joint Strike Fighter
Fleet Release of Block 3F aircraft software was achieved for the US services in October 2017.
The project achieved Initial Materiel Release and Initial Operational Release for NUSHIP Hobart in September 2017. NUSHIP Hobart was commissioned as HMAS Hobart on 23 September 2017.
AIR 9000 Phase 2/4/6 Multi-Role Helicopters
The acceptance of aircraft 47 was achieved in July 2017. MRH-90 rate of effort (ROE) has significantly improved with the system currently achieving one hundred per cent of its planned ROE. Additionally, the project schedule is currently under review with FMR and FOC to be updated to align with delivery of the remaining materiel components and completion of planned Test and Evaluation of the Special Operations capability.
AIR 87 Phase 2 Armed Reconnaissance Helicopter
In August 2017, following the German Airbus Tiger crash, the Australian fleet of 22 Tiger Helicopters was grounded. The German crash investigation is ongoing. However, preliminary information released by Airbus Helicopters has allowed Defence to conduct a risk assessment, which allowed the resumption of flying operations in November 2017.
Land 121 Phase 4 conducted the Critical Design Review (CDR) in July 2017 with Contractor Thales, which concluded the Capability Delivery Activity Stage 1 Engineering and Manufacturing Development (EMD). The Hawkei vehicle displayed improved performance and continued reliability growth. However, at CDR the vehicle had not achieved all requirements, and some reliability concerns remained. These reliability concerns related to select aspects of vehicle performance such as mean time between failure, which are being resolved by Thales and will be jointly evaluated by the Project Office and Capability Manager. A Reliability Demonstration Test (RDT) program has been developed to prove resolution of the remaining reliability issues in accordance with the Contract. The RDT will reschedule IMR by four months, from quarter four 2018 to quarter one 2019. Successful completion of the RDT will enable progression into Production Reliability Acceptance Test (PRAT) which will progress to October 2018. The Low Rate Initial Production was agreed to commence in August 2017 in parallel to the RDT program.
To offset these reliability concerns the project office has also renegotiated the acquisition contract to increase the extant warranty provisions (at nil cost) ensuring value for money for this Commonwealth procurement. Post-CDR Maintenance Evaluation for the vehicle platform commenced in October 2017, with Integral Computing System (ICS) Maintenance Evaluation commencing in February 2018.
The ICS will integrate all C4I and vehicle systems through a common Generic Vehicle Architecture interface. This is a leading edge military systems capability, being a significant developmental project in itself. The ICS is configurable, with a common baseline configuration for four-door vehicles planned to enable role-specific configuration. CDR specifically for the ICS was achieved in July 2017. ICS Developmental Test and Evaluation requirements for Stage 1 were met prior to Commonwealth Stage 1 acceptance. As the ICS has now completed initial baselining, a Contract Change Proposal (CCP013) is implementing the ICS procurement - essentially being the ‘ICS Contract’. This significant contract will exceed $AUD100m due to the nature of the procurement across the Hawkei vehicle fleet. CCP013 was signed in November 2017 with the Commonwealth concluding that the offer from Thales satisfied the value for money requirement.
Two caveats raised at IMR and IOC (supply support deficiencies and training requirements) were lifted by Air Force in August 2017. Additionally, the acceptance of aircraft eight and nine was achieved ahead of schedule in September and October respectively.
Operational Release was declared by Army on 29 September 2017. A further two caveats raised at FOC (training requirements and external air transportability capability requirement) were lifted by Army in September 2017.
In September 2017, MRTT number 6, the first of two additional KC-30A Multi-Role Tanker Transport aircraft was accepted by the Commonwealth and transferred onto the State Register to commence service with Air Force.
AIR 5431 Phase 3 Civil Military Air Management System
Airservices received the Final Offer from Thales on 25 September 2017. A joint analysis will be carried out by Defence and Air Services to determine if the offer is a viable solution. Government consideration of the Real Cost Increase is expected to occur in February 2018. Further, the project was declared a Project of Concern in August 2017.
AIR 9000 Phase 5C Additional Medium Lift Helicopters
Final Operational Capability was declared by Army in July 2017.
JP 9000 Phase 7 Helicopter Aircrew Training System
Formal declaration of IMR will now occur in quarter one 2018 due to finalisation of Materiel Acquisition Agreement elements which do not directly impact commencement of Piloting Courses. This includes registration of aircraft #7, the 15th aircraft to be delivered, acceptance of the Aircraft Replica Trainer which is being managed through a recovery plan, and administration related to the IMR process.
The project has advised that Final Materiel Release has been delayed to quarter one 2018 and Final Operational Capability will be delayed until quarter three 2018.
SEA 1442 Phase 4 Maritime Communications Modernisation
The project achieved Integration Detailed Design Review and Support System Detailed Design Review in October 2017.
The project has advised that submission of the Advanced Processor Build Spiral 4 - Very Shallow Water documentation to Navy was delayed to December 2017, with the limitations expected to be lifted in quarter one 2018.
The project has advised that achieving SUBSCUT Operational Release (OR) and Special Forces Exit and Re-Entry (SF E&RE) IOR will be delayed from December 2017, with a revised schedule for completion to be established in 2018.
SEA 1448 Phase 2A & 2B ANZAC Anti-Ship Missile Defence
The project has delivered all materiel requirements for the materiel release of ship eight (HMAS Stuart), with Capability Manager acceptance forecast for January 2018. The slight delay to materiel release eight has seen FMR and FOC delayed to quarter one of 2018.
Following Government approval of LAND 200 Tranche 2 (Work Packages B-D) in September 2017, LAND 75 Phase 4 (Work Package A) is expected to achieve Final Materiel Release and MAA closure in quarter one 2018.
Civil Military Air Management System
!Part 4. JCPAA 2016–17 Major Projects Report Guidelines
The JCPAA 2016–17 Major Projects Report Guidelines part is available to download in a separate PDF file, or in the complete report PDF which is available to download at Related documents on this page.
1 Joint Committee of Public Accounts and Audit, Report 468, Defence Major Projects Report (2015–16), October 2017, Executive Summary, p. 1.
2 Defence describes CASG’s role as ‘purchases and maintains military equipment and supplies in the quantities and to the service levels that are required by Defence and approved by the Government’. Department of Defence, About CASG, available from < http://www.defence.gov.au/casg/AboutCASG/> [accessed 18 October 2017].
3 Department of Defence, Defence Annual Report 16–17, Chapter 3, Annual Performance Statements, p. 33.
4 ibid., Chapter 11, Financial Statements, p. 180.
5 Department of Defence, Defence Portfolio Budget Statements 2017–18, May 2017, p. 5.
6 The Minister for Defence Industry, the Hon. Christopher Pyne MP, Historic milestone for Australia’s shipbuilding program, 26 April 2017. A performance audit to assess the effectiveness to date of Defence’s planning for the mobilisation of its continuous shipbuilding programs in Australia, is expected to be tabled in 2018.
7 The 2016–17 Major Projects Report Guidelines were endorsed by the JCPAA in November 2016 and are included in Part 4 of this report.
8 Based on information provided to the ANAO by the Directorate of Capital Investment Program, Department of Defence.
9 For example, Defence project risk management records can be managed in spreadsheets, where the risk to the completeness and accuracy of records is too high to be included within the scope of the review.
10 JCPAA Report 468, Defence Major Projects Report (2015–16), October 2017, Recommendation 1, p. vii.
11 IMR and FMR are milestones that Defence utilises to mark the completion and release of acquisition project supplies required to support the achievement of IOC and FOC respectively. They are defined in the relevant MAA (Materiel Acquisition Agreement). See Department of Defence, Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, pp. B2–B3.
12 IOC and FOC are the points when the first or final subset of a capability system that can be operationally employed is realised. They are capability states endorsed at project approval at Second Pass, and reported as having been reached by the Capability Manager. See Department of Defence, Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, pp. B2–B3.
13 The project maturity framework—outlined in the Department of Defence’s Defence Materiel Standard Procedure (Project Management), DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010—is a methodology used to quantify the maturity of projects as they progress through the acquisition life cycle.
14 An acquisition project can be closed at Defence’s discretion.
15 The caveats, capability deficiencies and obsolescence issues were discussed in ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, pp. 25–33 and pp. 50–53.
16 Defence has advised that where FOC is declared with caveats, the Capability Manager will have considered other Defence capabilities that can substitute while the caveats are resolved, and the Capability Manager will have considered the capability risk acceptable.
17 This shortfall in rate of effort has been reflected in the impairment of the value of this asset in Defence’s 2016–17 financial statements.
18 Department of Defence, Houston Review into Army Aviation, April 2016.
19 The Auditor-General was unable to provide an unqualified Independent Assurance Report as a number of matters were identified, in the course of the ANAO’s review, that resulted in the qualification of progress and performance as reported in two PDSSs, including the PDSS for the ARH Tiger Helicopters. See ANAO Report No.40 of 2016–17, 2015–16 Major Projects Report, paragraphs 20–23 and pp. 129–131.
20 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error. See paragraph 1.49 for further detail.
21 Refer to footnote 13.
22 Defence has advised that: ‘Real Cost Increases [are] attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget: excluding ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; and ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors.’
23 Defence has subsequently advised that the two caveats relating to the Electronic Warfare System and Identification, Friend or Foe have been remediated. Army is managing the remediation of the remaining seven caveats.
24 See Defence’s analysis on page 82 in Part 2 of this report.
25 M Thomson, Australian Strategic Policy Institute, Indexation, inflation and the cost of defence projects, 25 June 2015, available from <http://www.aspistrategist.org.au/indexation-inflation-and-the-cost-of-de...> [accessed 20 October 2017].
26 As noted in Note 2 of Table 2, slippage refers to the difference between the original government approved date and the current forecast date. These figures exclude schedule reductions over the life of the project. In November 2017, Defence raised with the ANAO, for the purposes of calculating total schedule slippage, the feasibility of identifying what the proportion of slippage represented by the expanded scope of projects is (for example with respect to the P-8A Poseidon and Additional MRTT projects). The ANAO will test the feasibility of such an approach in the context of the next MPR.
27 See Table 2 in Part 2 of this report for Defence classifications.
28 Off-The-Shelf: Systems, hardware or software that already exists or is confirmed in service for an equivalent purpose and requires no, or minimal, change. Sometimes expressed as commercial off-the-shelf or military off-the-shelf. Department of Defence, Interim Defence Test and Evaluation Manual, October 2016, Annex 1A, Definitions, p. iii.
29 JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 5, p. 31.
30 Refer to footnote 23.
31 Refer to footnote 9.
32 The Auditor-General was unable to provide an unqualified Independent Assurance Report for 2015–16 as a number of matters were identified, in the course of the ANAO’s review, that resulted in the qualification of progress and performance as reported in two PDSSs, for the ARH Tiger Helicopters and LHD Landing Craft. See ANAO Report No.40 of 2016–17, 2015–16 Major Projects Report, paragraphs 20–23 and pp. 129–131.
33 ANAO Report No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, paragraph 13, pp. 15–16, found that while Defence had improved the effectiveness of the program, there remained opportunities for further improvement and rigour.
34 These reviews are not carried out within frameworks issued by the Australian Auditing and Assurance Standards Board.
35 Independent Assurance Reviews were conducted for: P-8A Poseidon, MRH90 Helicopters, Growler, MH-60R Seahawk, Overlander Medium/Heavy, LHD Ships, ARH Tiger Helicopters, Battlefield Airlifter, Overlander Light, Additional MRTT, CMATS, Additional Chinook, HATS, Battle Comm. Sys. (Land), Maritime Comms, and LHD Landing Craft. Seven projects had reviews scheduled for late 2017. This includes an Independent Assurance Review of the Hawkei project conducted in October 2017—its implications continue to be considered by Defence.
36 Issues in the project were discussed in ANAO Report No.22, 2013–14, Air Warfare Destroyer Program.
37 Issues in the project were discussed in ANAO Report No.52, 2013–14, Multi-Role Helicopter Program.
38 The Minister for Defence Industry, the Hon. Christopher Pyne MP, and the Minister for Defence, Senator the Hon. Marise Payne, Projects of Concern Update, 18 August 2017.
39 See the CMATS PDSS in Part 3 of this report.
40 Department of Defence, Quarterly Performance Report, June 2017, p. 4.
41 ANAO Report No.2 2017–18, Defence’s Management of Materiel Sustainment, July 2017, paragraph 16, p. 9, found that the contents of Quarterly Performance Reports are neither complete nor reliable, and may not include additional information available to Defence that is critical to the reader’s ability to understand the status of significant military platforms.
42 These are CASG acquisition projects that have variances significant enough—in the areas of schedule, cost, and/or capability performance—to warrant attention from senior management. Department of Defence, Quarterly Performance Report, June 2017, p. 12.
43 LHD Ships, CMATS, Battle Comm. Sys. (Land) and UHF SATCOM.
44 The Project Directive is a tasking statement from Vice Chief of the Defence Force and defines the Project, in terms of fundamental inputs to capability, together with the resources necessary to deliver the project and is developed in accordance with the exact parameters agreed by government. Department of Defence, Interim Capability Life Cycle Manual, April 2016, Annex A, p. 92. The mechanism for providing the directive is via the Capability Life Cycle (CLC) management tool, which records the Government decision in relation to a project. The accountabilities and responsibilities of specific roles within the CLC are defined in the Interim Capability Life Cycle Manual. Where necessary, the Joint Force Authority may provide a specific documented directive.
45 MAAs are being phased out gradually by Product Delivery Agreements (PDAs). Projects in this MPR have an approved MAA.
46 For further information on Joint Project Directives see ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 11.1 to 11.54, pp. 219–232.
47 ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraph 11.53, p. 232.
48 ibid., paragraph 11.54, p. 232.
49 Joint Strike Fighter (Stage 2), P-8A Poseidon, Growler, MH-60R Seahawk, Overlander Medium/Heavy, Battlefield Airlifter, Bushmaster Vehicles, Overlander Light, Additional MRTT, CMATS, HATS, Battle Comm. Sys. (Land), Maritime Comms, BMS and LHD Landing Craft. As at 30 June 2017, the JPD for the Hawkei project was still in draft.
50 A PDA is an agreement between the Sponsor and Lead Delivery Group which specifies the scope, resourcing, priorities and performance and preparedness requirements for support of a capability system throughout its life, to support performance measurement. Department of Defence, Interim Capability Life Cycle Manual, April 2016, Annex A, Definitions, p. 91.
51 Business system weaknesses, such as project offices having inconsistent record keeping and methods of tracking project progress were highlighted by the Committee in JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, paragraph 3.116, p. 39.
52 See ANAO Report No.2 2017–18, Defence’s Management of Materiel Sustainment, July 2017, for further detailed information on this system.
53 Mr K Gillis, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 31 March 2017, p. 11.
54 Department of Defence, Defence Materiel Manual Project, DMM (PROJ) 11-0-003, DMO Project Controls Manual, August 2014, Glossary, p. 41.
55 JCPAA Report 436, Review of the 2011–12 Defence Materiel Organisation Major Projects Report, May 2013, paragraph 3.4, p. 14.
56 The Statement of Work for this engagement dated 18 October 2017 references the Department of Defence, Defence Materiel Instruction (Finance) DMI(FIN) 01-0-044, Project Financial Assurance Statements, February 2015, which Defence advised on 27 October 2017 was ‘no longer current as it is an artefact of the previous DMO agency’. A response to the ANAO’s request made on 13 November 2017 for the reason Defence continues to reference this DMI, is still pending.
57 Department of Defence, DMM (PROJ) 11-0-002, DMO Project Risk Management Manual 2013, July 2013, Chapter 9 – Management of Contingency Budgets in DMO Acquisition Projects, p. 108.
58 The manual requires that the Project Contingency Budget Log is kept up to date for the proper overall management of risk and that it is submitted for internal review at Additional and Budget estimates.
59 The ARH Tiger Helicopters acquisition project was closed in April 2017 and the Collins R&S project does not have a formal contingency allocation.
60 See page 75 in Part 2 of this report for more information.
61 Department of Defence, Defence Annual Report 16–17, Chapter 5, Corporate Governance, p. 65.
62 The ANAO will continue to work with Defence to bring risks and issues into the scope of future MPR reviews.
63 Department of Defence, Strategy to remodel the management of risk in Acquisition and Sustainment in Defence, February 2017.
64 Predict! is a risk management tool used by Defence to manage risks and issues.
65 Spreadsheets lack formalised change/version control and reporting, increasing the risk of error.
66 The 14 projects are: MRH90 Helicopters, Growler, MH-60R Seahawk, Battlefield Airlifter, Additional MRTT, CMATS, ANZAC ASMD 2B, Additional Chinook, HATS, Battle Comm. Sys. (Land), UHF SATCOM, Collins R&S, ANZAC ASMD 2A and BMS.
67 The Joint Strike Fighter project utilises SharePoint.
68 Mr K Gillis, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 31 March 2017, pp. 12–13. Defence also observed the importance of ‘the intellectual rigor that somebody applies to risk management, not the tool’.
69 Department of Defence, DMM (PROJ) 1-0-001, DMO Project Management Manual 2012, April 2012, Glossary, p. 75. This manual has since been superseded by (PM) 002 CASG Project Management Manual which does not refer to project maturity.
70 See Appendix 4 in Part 2 of this report and footnote 13 for further detail.
71 Department of Defence, DMSP (PROJ) 11-0-007, Project Maturity Scores at Life Cycle Gates, September 2010, p. 9, with a stated 24 month review period.
72 JCPAA Report 458, Defence Major Projects Report (2014-15), May 2016, Recommendation 3, p. 50. The JCPAA sought an update from Defence in the course of public hearings on 31 March 2017.
73 JCPAA, Report 468, Defence Major Projects Report (2015-16), October 2017, Recommendation 2, p. vii.
74 In May 2016, Defence described caveats to the ANAO as ‘alerts’ to operational decision makers about risks to be taken into account when making decisions about the use of the ARH Tiger Helicopter in particular operational circumstances. ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, p. 25.
75 Wedgetail, which has exited the MPR, achieved FOC with caveats in 2015.
76 Defence evidence, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 31 March 2017, p. 3.
77 This requirement was agreed to by Defence and included in the 2017–18 Major Projects Report Guidelines endorsed by the JCPAA in September 2017.
78 Refer to footnote 11 for the definition of IMR and FMR milestones, and footnote 12 for the definition of IOC and FOC milestones.
79 See paragraphs 2.48 to 2.56 for further explanation.
80 A project’s budgeted cost and schedule data is at 30 June 2017, and may differ from originally approved budgets and schedules.
81 A project’s budget expended is cash based. In cases where pre-payments have been made, but have not been expensed/amortised, cash paid by a project will be greater than the accrued expenditure.
82 The caveats, capability deficiencies and obsolescence issues were discussed in ANAO Report No.11 2016–17, Tiger—Army’s Armed Reconnaissance Helicopter, September 2016, pp. 25–33 and pp. 50–53. See also paragraphs 17 to 25 in Part 1 of this report.
83 Department of Defence, Houston Review into Army Aviation, April 2016.
84 The JCPAA has recommended that a capacity to publish Project Maturity Scores be maintained by Defence until they are no longer required by the JCPAA. JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, Recommendation 8, p. 39.
85 The JCPAA has recommended that Defence work with the ANAO to review and revise its policy regarding Project Maturity Scores. JCPAA Report 458, Defence Major Projects Report (2014–15), May 2016, pp. 49–50, and JCPAA Report 468, Defence Major Projects Report (2015–16), October 2017, pp. 9–10.
86 See paragraphs 2.32 to 2.36 and Figure 8, on page 47, for more information. However, acceptable MOTS solutions may not always be available.
87 See Note 3 of Figure 4, below, for further information.
88 The components for this project do not add up to $1.0 billion due to rounding differences.
89 Real Variations include ‘Scope’ changes attributable to changes in requirements by Defence and government; ‘Transfers’ which occur when a portion of the budget and corresponding scope is transferred to or from another approved project or sustainment product in Defence; ‘Budgetary Adjustments’ made to account corrections resulting from foreign exchange or indexation accounting estimation errors; ‘Real Cost Increases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been approved by government to increase the Project’s budget; and ‘Real Cost Decreases’, attributed to any negotiated Foreign Military Sales or commercial contracts, where funds have been handed back to the Defence portfolio.
90 In addition, Collins R&S had a real variation of $0.1 million reflecting minor rounding changes to the PDSS.
91 FOC for ARH Tiger Helicopters was declared with caveats. Defence has subsequently advised that the two caveats relating to the Electronic Warfare System and Identification, Friend or Foe were lifted in July 2017. Army is managing the remediation of the remaining seven caveats. Defence is considering additional expenditure potentially through a Capability Assurance Program pending replacement of the platform in the mid-2020s. Department of Defence, 2016 Integrated Investment Program, February 2016, p. 21.
92 Australian Government arrangements for foreign exchange variation involve ‘no win/no loss’ supplementation. As a matter of policy, unless specifically approved, individual entities are not permitted to ‘hedge’ against foreign exchange risk.
93 Refer to footnotes 60 and 61 for more detail.
94 Cash basis expenditure data was provided to the ANAO by the Chief Finance Officer Group, Department of Defence.
97 Refer to footnote 84 for more detail.
98 Refer to footnote 26.
99 M Kinnaird, Defence Procurement Review 2003, August 2003.
100 G Pappas, Department of Defence, 2008 Audit of the Defence Budget, April 2009, p. 76.
101 In instances where a Major Project has multiple segments/capabilities with separate Final Operational Capability (FOC) dates, the ANAO has used the project’s current lead/main capability FOC for calculating schedule performance. Defence’s approach is to use the final FOC date for a project listed in the 2016–17 PDSSs. These approaches, both valid, led to a small difference in the calculated percentage by which the Major Projects’ total schedule has slipped for the 2016–17 MPR (ANAO—29 per cent; Defence—29 per cent).
102 Refer to footnote 26.
103 See the Collins RCS, Hw Torpedo and Collins R&S PDSSs in Part 3 of this report.
104 ANAO Report No.6 2013–14, Capability Development Reform, October 2013, paragraphs 9.1 to 9.4, pp. 198–199.
105 Further information on MRH90 Helicopters can be found in ANAO Reports No.48 2008–09, Planning and Approval of Defence Major Capital Equipment Projects, June 2009, pp. 84, 90 and 133; No.52 2011–12, Gate Reviews for Defence Capital Acquisition Projects, June 2012, pp. 86–87 and pp. 130–133; and No.52 2013–14, Multi-Role Helicopter Program, June 2014.
106 Refer to footnote 82 for more detail.
107 At the 31 March 2017 hearings of the JCPAA, Defence acknowledged that in respect of the ARH Tiger Helicopter acquisition ‘…it had assumed that it was an off-the-shelf acquisition and that it was more mature than it was’. Defence advised the Committee that this is ‘…one of the fundamental lessons that Defence has learnt from this Tiger program’. Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 31 March 2017, p. 9.
108 In the Statement by the Secretary of Defence in Part 3 of this report, the Secretary also makes reference to additional information on achieved milestone dates for AWD Ships, Overlander Light and Additional Chinook.
109 Tables 4 and 5, on pages 12 and 13 respectively, report on the slippage for each project that has been in the MPR since 2007–08.
110 The three projects are ARH Tiger Helicopters, Bushmaster Vehicles and Collins RCS.
111 Defence Instructions (General), DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, Annex B, p. B1.
112 Source 1: Department of Defence, Interim Capability Life Cycle Manual, April 2016, pp. 11–12.
Source 2: Department of Defence, DI(G) OPS 45–2, Capability Acceptance into Operational Service, November 2012, paragraph 1, p. 1.
113 As per the 2016–17 MPR Guidelines, a project is defined as the acquisition or upgrade of Specialist Military Equipment, which normally excludes facilities and other Fundamental Inputs to Capability. The 2016–17 MPR Guidelines also note that the MPR may report on associated sustainment activities (where applicable).
114 ANAO Report No.17 2010–11, 2009–10 Major Projects Report, November 2010, p. 35.
115 Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 17 March 2016, p. 5.
116 Mr K Gillis, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 31 March 2017, p. 14.
117 JCPAA Report 442, Review of the 2012–13 Defence Materiel Organisation Major Projects Report, May 2014, pp. 37–39.
118 Source 1: Ms S McKinnie, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 27 February 2015, p. 10.
Source 2: Mr H Dunstall, Official Committee Hansard, Joint Committee of Public Accounts and Audit Hearing, 27 February 2015, p. 10.
Source 3: The CEO DMO Mr Warren King advised the JCPAA that ‘Landing on a method to have an easily auditable statement of what the capability is that we have delivered is really a complex issue and still there is, I think, work to be done’. Commonwealth of Australia, JCPAA, Defence major projects report 2012–13, 20 March 2014, pp. 1–3.
119 JCPAA Report 458, Defence Major Projects Report (2014-15), May 2016, pp. 48–49.
120 JCPAA Report 458, Defence Major Projects Report (2014-15), May 2016, p. 49.
121 JCPAA, Report 468, Defence Major Projects Report (2015–16), October 2017, Recommendation 1, p. vii.
122 Refer to footnote 84 for more detail.
123 Refer to footnote 85 for more detail.