taking into account the agency’s current commitments and capacity to deliver? together) the skills and capabilities it needs to plan and achieve the desired outcomes • evidence that the agency is realistic about the complexity of the project deliverables and how they can be managed • key roles within the project identified with individuals named 1.9 Is the agency able to from experience with this project and other projects? • evidence that the agency has processes in place to incorporate lessons learnt from this project, and their components, into wider best practice • evidence that the agency learns from the experiences of others 1.10 Does this project have a risk management framework? • defined roles, responsibilities and processes for managing risk across the agency and within the project, with clearly defined routes for escalating risk to senior management 2. Business Case: Scope and Stakeholders Areas to probe Evidence expected 2.1 Is the scope of the project understood? • a description of the project as far as it is known—what is in scope? 2.2 What is the full extent of the project envisaged and why? • description of all project components and outputs, explaining how each will contribute to the required outcomes 2.3 What will constitute success? • a definition of key critical success factors and how the required quality of performance will be measured; • a description of the main outcomes and an analysis of the leading and lagging indicators; • a relationship between project outcomes and government targets, or major policy initiatives; • projected performance over the life of the project, with key performance targets and measures agreed with stakeholders; and • the project can be evaluated in a practical and affordable way. 2.4 Who are the stakeholders and do they support the project? • a list of principal stakeholders and statements of their needs and support for the project • a plan for communicating with and involving stakeholders in appropriate ways • clear lines of accountability for resolving any conflicting stakeholder requirements • recognition of the need to involve external delivery partners, industry and the supplier
2.5 Is the proposed project affordable? • an estimate of the project cost, and the of costing (for example, based on previous experience and/or comparison with similar projects) • an estimate of of the project versus potential alternatives • sources of funding and amount of project funding currently included in forward estimates (if any) 2.6 What assumptions is the project based on? What are the major constraints for the project? • description of factors already underway that could affect the required outcomes 2.7 Has the supplier market been considered? • engagement with the market to determine capability to meet the need and, where appropriate, to identify suitable delivery options • where suppliers/partners are already in place, evidence that their ability to deliver has been considered 2.8 Have project controls been determined, especially constituent projects will be connected with other agencies, jurisdictions or the private sector? • overall project controls (progress tracking, issue identification and resolution, impact assessment) defined • interdependencies between project defined, with plans for managing them • for collaborative projects, accountabilities for different parties defined and agreed • parties in the delivery chain identified and an approach to them working together established 2.9 What else could affect success? • the main identified at the outset, with nominated risk owners; options for mitigating these risks considered; and the need recognised for contingency plans and business continuity plans 3. Review of Current Phase Areas to probe Evidence expected 3.1 Does the project need to comply with broader government or departmental timing requirements? • links to government or departmental commitments on delivery 3.2 Is the project on track? • the project report and plan updated; • milestones achieved as planned; • a plan for benefits measurement and achievement is on track; • the risk register is current; • highlight reports for constituent work-streams; • resources and funding is used to-date; • issues being resolved; • confidence from delivery partners that future milestones and plans are realistic; and • interdependencies with projects being managed
3.3 Have options for potential ways forward been identified? • listing of high-level options for meeting the business need • options analysis or feasibility studies, if available. 4. Management of Intended Outcomes Areas to probe Evidence expected 4.1 Have the required outcomes from the project been identified? • a current list of the main outcomes and desired benefits, linked to strategic outcomes and to the deliverables from specific projects 4.2 Are the planned outcomes still achievable, or have any changes in scope, relationship or value been properly agreed, and has the business case been reviewed? Is the project time critical? • outcomes identified, together with their relationships to each other; • credible plans for the achievement of outcomes; and • ongoing commitment from stakeholders to the outcomes and their achievement • ministerial agreement to scope changes 4.3 Are key stakeholders confident outcomes will be achieved when expected? Is it on track to deliver? • confirmation of time-critical issues and a realistic schedule to achieve them; • mechanisms for collecting performance data are in place and a plan for evaluating the impact of the project are in operation; • the project board is confident that planned milestones will result in good quality deliverables that will deliver the necessary outcomes; and • a commitment from key stakeholders that project deliverables will achieve the desired outcomes. 4.4 Is there a plan for achieving the required outcomes? • a benefits management plan and a plan to ensure that outcomes are delivered in terms of performance measures and/or key performance indicators; • plans to identify appropriate baseline measures against which future performance will be assessed; • plans to carry out performance measurement against the defined measures and indicators; • where planned outcomes have not been achieved, evidence the problems have been identified and plans are in place to resolve them; and • clarity on how the objectives from the projects link to the outcomes of the program 5. Risk Management Areas to probe Evidence expected 5.1 Have the major risks been identified? • a current list of the major strategic, political and/or reputation, environmental and legislative risks to the overall project, analysed by likelihood and impact; • early warning indicators identified;
• consideration of the risks of success, e.g. take-up or usage greater than expected and contingencies identified; and • evidence of regular review of mitigation options and contingency plans. 5.2 How will risks be managed? • evidence that the to the success of the project (e.g. take-up or usage greater or less than expected) have been considered and mitigation strategies and contingencies identified • evidence of regular review of risks, mitigation options and contingency plans • details of the risk allocation (to whom allocated and why) with high-level plans for managing risks • action to manage the risks identified and, where appropriate, action taken 5.3 Have assurance measures for the project been put in place? • ‘critical friends’ to the project (for example probity auditors, internal audit, specialists and/or peer reviewers) appointed, with evidence that they challenge assumptions, decisions and risks • evidence that advice from ‘critical friends’ is on • Gateway Reviews, health-checks and/or policy reviews incorporated into plans 5.4 Is a contingency plan and business continuity plans? • decisions about contingency and necessary business continuity arrangements made with appropriate plans; • the project’s effects on public services is analysed and decisions taken about those for which contingency arrangements will be needed; and • milestones relating to contingency measures in plans and the milestones being achieved as expected. 5.5 Have lessons from similar projects been considered? • details of issues identified from previous, or similar projects and application to the current project; and • P3M3 or Agency Capability Reviews and Capability Improvement Plans. 6. Readiness for Next Phase (Business Case) to probe Evidence expected 6.1 Affordability: Are the funds to reach the next phase of the project available? • budget provision for initiating project(s) and preparing the high level business case • costings from Finance to validate estimates, where appropriate 6.2 Are the required internal/external resources available, suitably skilled, and committed to undertake the work? • resource plan and sources identified showing who needs to be involved, when, and what they must deliver as well as any additional training needs required for their role • key identified (Senior Responsible Official, Project Steering Committee, and Project Manager) • evidence that these resources will be available when needed throughout the next phase
• agreed project management process and project organisational structure, including a design methodology where appropriate • senior management’s commitment to the project and key role in decision-making 2.2 Has a Steering Committee, or equivalent, been established to oversee the project? • clear articulation of the role and decision-making power of the Steering Committee • details of the process for providing information to the Steering Committee and frequency of meetings • details of the type of information to be provided to the Steering Committee, such as budget reports, risk management reports and action items 3. Stakeholders Areas to probe Evidence expected 3.1 Have all the likely stakeholders been identified and are their interests clearly understood? • internal and external stakeholders identified and documented • stakeholders’ roles and responsibilities defined and agreed • end-users for the project identified and documented • evidence that the decision-making process includes all the relevant stakeholders and is both efficient and effective • analysis of potential conflicts between stakeholder groups and contingency plan for dealing with such conflicts • results of consultations documented as part of the communications strategy 3.2 Do stakeholders support the preferred option? (This includes the potential or recommended delivery approach and mechanisms.) • documentation supporting the consultation, involvement, support and endorsement of stakeholders 3.3 Are stakeholder issues being addressed? These include: • communications • public relations • environmental issues • personnel • statutory processes • a communication plan for dealing with various stakeholders showing responsibilities • for dealing with statutory consultation requirements • flexibility and ability to to changes in stakeholder interests and concerns • governance arrangements clearly articulate opportunities for stakeholder concerns to be aired and addressed 4. Risk Management Areas to probe Evidence expected
2.13 Is the evaluation strategy (including how to demonstrate value for money) accepted by stakeholders and compliant with the Commonwealth Procurement Rules? • evaluation criteria and approved by stakeholders • key evaluation criteria linked to business objectives and given appropriate weighting • separated financial and non-financial aspects of the evaluation • evaluation criteria and weightings included in tender documents • for construction projects, adherence to building code, occupational health and safety guidelines, and sustainability requirements 3. Review of Current Phase Areas to probe Evidence expected 3.1 Is the project managed in accordance with its governance framework, stakeholder engagement plan, project plan and other critical documents? • the project running to schedule and costs within budget, as shown in project budget and timetable reports • a governance framework in place and the project is managed accordingly 3.2 What caused any variations such as over-runs or • reconciliations made against the budget and timeframe, according to risk allowances • reconciliations supported by variance reports which explain the reasons for, and actions taken in response to, variances 3.3 What actions are necessary to prevent variations recurring in other phases? • show an analysis and plans in the project management documentation, which is continually reviewed and updated 3.4 Were there any assumptions made at Gate 1, or earlier reviews that have not been verified? • a log of outstanding assumptions, and show plans to verify them; these should be categorised and managed as issues where applicable 4. Risk Management Areas to probe Evidence expected 4.1 Are the major risks identified, understood, financially evaluated and considered in determining the procurement strategy? • a log of major issues and risks • interdependencies with other projects and programs identified • for construction projects, occupational health and safety risks identified • for IT-related projects, risks relating to IT innovation, infrastructure, security and take-up identified • each risk assessed financially and included in business case as a sensitivity or contingency • an escalation process in place for upward referral of risks • a log of issues and regularly reviewed by project team; evidence of appropriate action taken
1.2 Are the requirements of the business case complete or does the tender decision require the business case to be amended and/or • a reassessment of the business case requirements, including strategic, economic, financial, commercial and project management factors • an updated business case if required 1.3 Does the recommended way forward meet the business need? • key objectives revisited against final tender and proposed solution 1.4 Has the most appropriate option been selected? • a cost-benefit and risk analysis against final bid information and results of evaluation, including sensitivity analysis • the preferred option complies with the requirements/ specifications and evaluation criteria 1.5 Does the commercial arrangement represent value for money, with an appropriate level of quality over the whole life of the project? • market assessment, other agency benchmarks and previous experience • results of evaluation, demonstrating how for money is achieved • an assessment of the supplier’s funding arrangements, if appropriate • if a PPP arrangement is proposed, PPP guidelines should have been addressed 1.6 Is the agency realistic about its ability to manage the change? • a documented understanding of cultural implications, where appropriate • comparison with other similar projects 1.7 Does the business case still demonstrate affordability when incorporating the proposal? • an updated business case incorporating bid information, including: • changes from established budget • returns and value recalculated with a new benefits plan • compared with budget and pre-tender estimates 1.8 Is an agreed plan for the realisation of anticipated benefits? • a benefits management strategy and plans, including: • critical success factors • a list of the individuals responsible for delivering and achieving benefits identified • the agreed process for measuring benefit • a post-implementation review plan identifying areas to be assessed • payment mechanisms linked to benefits realisation, where appropriate 1.9 Have suitable stakeholders been involved and, if so, what has been their involvement? • involvement of stakeholders and business or user representatives in the quality and proposal reviews • stakeholder representation on the evaluation team • approval by the Project Steering Committee and Senior Responsible Official 2. Assessment of the Proposed Solution Areas to probe Evidence expected
• evidence that the agency and supply team will work together as an integrated project team • evidence that an exit strategy exists 2.9 Are resources available for the supplier to fulfil its obligations within the contract? • a plan for implementing the new contract, identifying the quantity, type and quality of resources required • formal management acceptance of resource requirements 2.10 Have the technical implications been assessed? • evidence demonstrating that the proposal is technically acceptable and has considered relevant implications such as sustainability and legacy systems etc. 2.11 Does the project have resources with the appropriate skills and experience to achieve the intended outcomes of the investment? • plans for building client capability, with nominated staff allocated to the major roles • internal and external commitment to provide the resources required • job descriptions for key project staff • a skills appraisal undertaken, with plans for addressing shortfalls, if any 2.12 Is the proposed procurement within financial approvals and is there adequate budget to accommodate the procurement? • documentation confirming that required financial approvals exist • documentation detailing the required budget and timing of milestone payments, referring to the relevant budget measure 3. Review of Current Phase Areas to probe Evidence expected 3.1 Is the project under control? (i.e. running to plan and budget) • project running to schedule and cost within budget • project governance structures fulfilling their roles and actively oversighting the project 3.2 What caused any variations? • reconciliations against budget and timetable • advice provided to the Project Steering Committee or Senior Responsible Official, where appropriate 3.3 What actions are necessary to prevent variations recurring in other phases? • the analysis and plans are included in project documentation; documentation is continually updated and reviewed 3.4 Have all the assumptions from Gate 0, 1 and 2 reviews been validated? • validation of all assumptions. Those that cannot be validated should be: • stated in the risk management plan/issue register • assessed • discussed with potential suppliers 3.5 Have all the required agency procurement and technical checks been undertaken? • tender management review and approval processes
• supplier has insurances • the supplier has realistic implementation and risk management 5.4 Does the agency’s plan reflect the supplier’s plans? • the updated project management plan reflects tender proposals • personnel and responsibilities have been defined and organised on the agency side • supplier’s personnel have been vetted to meet project requirements • a process exists for resolving issues, and is agreed with supplier • evidence that all plans have been reviewed, agreed and included in the contract 5.5 Are the long-term contract, administration plan and performance measurement process complete? • a long-term plan with a contract management strategy and a detailed service delivery plan exists • key performance measures agreed with supplier • an analysis of the project plan showing that resource requirements are identified, planned, budgeted and available when required • required resources and sourcing identified • defined roles and responsibilities 5.6 Are all mechanisms and processes for the next phase in place? • the project plan confirms arrangements for management, monitoring, transition and implementation 5.7 Are the service management plan, administration and service level arrangements complete? • a documented service management strategy and plan for the agency • defined and agreed service level management, service levels, service quality and measure-ment between the agency and supplier • responsibilities for each party agreed • defined and agreed standards for services • defined and agreed monitoring, reporting and review mechanism 5.8 Is the management process for service change complete? • change control procedures (both technical and business) defined, agreed and included in the contract • management process and responsibilities are defined and agreed 5.9 Is there an acceptance strategy or commissioning strategy, as applicable? • an acceptance/commissioning strategy and plan exist with fully documented and timetabled decision paths • decision-makers are clearly identified and informed their role in acceptance • the acceptance criteria is agreed with supplier • a validated acceptance testing plan, including technical and business components 5.10 Is there an implementation strategy? • an implementation strategy and plan exist • users, stakeholders and the agency’s business management are involved in developing the strategy • where applicable, plans for transition to new ways of working
• compliance with statutory requirements (e.g. occupational health and safety, data protection) • contractual issues resolved and recorded 2.3 Have all the stakeholder issues been addressed? • progress reports completed detailing resolution of issues, and circulated as part of the communication plan for stakeholder information 2.4 Are all testing and commissioning/ acceptance (or transition) procedures complete? • commissioning/testing team with relevant skills and experience • commissioning/testing plans, results and analyses of products and/or services against acceptance criteria • ratified test reports and logs • confirmed ‘end-to-end’ testing, including changed or new business processes • testing, taking into account future modules or deliveries • there are missing or incomplete items; agreed corrective action is documented • for IT-enabled projects, information assurance tests 2.5 Have all parties accepted the commissioning/test results and any required action plans? • evidence of acceptance of procedures and results by supplier and agency 2.6 Are there workable and tested contingency and back-out plans for roll-out, implementation and operation? • a fully documented approach with timetables, for key aspects and events, for example, IT components and decisions about roll-out • decision-makers should be informed and clearly identified in the documentation • roles and responsibilities listed; resources allocated and staff trained • commissioning/testing represented the expected scenario(s) • plans for the transition to new ways of working, where applicable • for handover to facilities management, where applicable • training and relevant supporting material, if required 2.7 Have the supplier and all internal and external parties agreed on the implementation plans? These could include: • management of change • migration and data transfer • all required plans in the contract • all parties, or their representatives, are aware of their responsibilities and have agreed to them
2. Review of Operating to probe Evidence expected 2.1 Is the service/facility operating to defined parameters? • documented operating parameters, updated as needs change • updated service agreements, where applicable • service delivery is measured against defined parameters • evidence to show that the measures used to address poor or non-performance are effective • the facility is safe to operate and maintain • sustainability targets are met or exceeded • client/stakeholder satisfaction and experiences are assessed 2.2 Have the project documentation, training material and training program been delivered and kept up to date? • new staff trained and existing staff updated • there is no backlog of training material to be updated • responsibilities for updating training material and documentation defined • the occupational health and safety record updated as required 2.3 Are the contractual relationships satisfactory? • project records demonstrating effective contract management procedures • supplier and client staff who are enthusiastic, mutually respectful and united in their objectives • improvements to the contract • regular reviews between supplier and client • action plans • measurement of contract improvements • reports on work done and for expected work 2.4 Are plans for continued contract management? • ongoing forward resource plans, as required • succession planning for key roles • skills appraisal and plans for addressing shortfalls 2.5 Are plans for ongoing risk management up to date? • an updated risk register, risk reporting and management • business continuity/contingency plans are updated as required • occupational health and safety aspects considered 2.6 Have the contract management procedures been successful to date? • contract management reports, which demonstrate successful project implementation • project documentation should record plans for change management and dispute management 3. for Ongoing Improvements in Value for Money to probe Evidence expected
based 3.1 What is the scope for improved value for money? Can more be done for less? Could the provider deliver better service quality at the same price? Can maintenance costs be reduced? • an analysis of value for money to date against scenarios for future service use • commercial intelligence the supplier’s track record on other projects and/or comparison with supplier’s offering similar services • details of efficiency gains, both expected and achieved 3.2 Has the agency benchmarked its contract-related processes by comparing them with other equivalent organisations involved in similar relationships? • benchmarking assessments of processes such as: • demand management • service planning and development • service quality • investment decisions/project justification • benefits management 3.3 Are commercial mechanisms providing appropriate incentives? • evidence could include: • payments to the provider are dependant on benefits derived from implementing a particular programme of change • the provider has incentives to deliver and to ensure that individual investments are well planned, achievable and will deliver value • a business case with robust benefits identified on each occasion • target incentive mechanisms, where work is • the provider is incentives to submit optimum resource estimates for a task, with equitable sharing of risks and benefits, between the agency and supplier, in exceeding or undercutting those original estimates 4. Plans for Ongoing Improvements in Performance to probe Evidence expected 4.1 Is the agency setting realistic targets for continuous improvement from this service? • understanding and use of key techniques such as Balanced Scorecard, ongoing stakeholder analysis, benchmarking etc. 4.2 Is the agency tracking its progression to improved performance and the flow of results through key milestones and the business planning • documented performance information which is clearly linked to planned outcomes and enables ready assessment of effectiveness, efficiency and service quality • core sets of performance information that meet multiple purposes, are consistent and complementary • ongoing assessment of the appropriateness of performance information • responsibilities for performance management are defined and understood by the agency and supplier
• direct links exist between planning and results • ongoing monitoring of performance and periodic evaluation • integration with corporate and business planning • consistency with the principles set out in the operational strategies of the business 4.3 Does the agency have performance measures to cover all aspects of the contract? • performance measures relating to: • cost and value obtained • performance and client satisfaction • delivery improvement and added value • delivery capability • benefits realised • relationship strength and responsiveness 4.4 Do the performance measures selected offer clear and demonstrable evidence of the success (or otherwise) of the contract? • performance measures chosen: • are meaningful and visible to an agency’s management group • are properly reflect user and stakeholder perceptions • can be used to identify the need for support or remedial action as part of contract management activity 4.5 Are performance measures related to delivery or capability improvement tracked against an existing baseline? • a baseline is established in the business case for the contract • performance measures are tracked against that baseline 4.6 Are there performance assessment measures for: • ongoing service delivery • the desired results of individual programmes of change or improvement, the implementation of projects • the overall results or effect of the deal? What does the agency want to have achieved by the end of the contract period? • service level agreement approaches and related measures • investment appraisal and benefits management techniques are constructed on a case-by-case basis • objectives identified during project scoping and in preliminary business analysis activity; they should be rooted in the organisation’s long-term business strategy • for construction projects, evidence that user satisfaction has been or will be monitored as required 5. Readiness for the Future Areas to probe Evidence expected 5.1 Is there an ongoing need for the service? • an updated business case, linked to the current business strategy
5.2 If the service will be needed in the future, what is its likely scope? • the options appraisal to include some or all of the following: • ‘do nothing’ • retain the of the existing contract • split the scope of the existing contract • broaden the scope of the existing contract • completely rethink the requirement for the contract • consider single/multiple sources of supply • combine new services with similar/ complementary services 5.3 Are any major issues with the current contract that could affect the approach to retendering the service? Factors to consider include: • the of the provider cope with the range of services provided or were there significant weaknesses? • the flexibility of the contract— how adaptable was the relationship to both foreseen and unexpected changes in the nature and of demand? • clients’ reaction and adjustment to outsourcing—how well did users adapt to services provided by a third party? Was management confident that the provider could be relied on to provide the service? Is the organisation now ready for a greater dependence on outsourcing? • an exit strategy—will the retendering be straightforward or is a danger the client is now ‘locked in’? Have agreements been made to ensure the handover is as smooth as possible? • an updated risk management plan and issue register/ log • exception reporting from regular client/ provider progress meetings • reports from contract and service management functions • an exit strategy and details of handover arrangements
• the benefits and costs of delivering the service through alternative delivery models (for example, using sourcing) 6. Review of Organisational Learning and Maturity Targets 6.1 Does the agency have a well-defined and effective process implemented for identifying lessons learnt from the project, embedding them internally and sharing them across government? • a mechanism for capturing and recording the initial data • an internal evaluation of lessons learnt • mechanisms and policy for making information available within and outside the organisation • a process for providing feedback to agency project teams • feedback into the business case • agency staff participation in knowledge-sharing forums 6.2 Has there been a review of how well the project was managed? • evidence of a formal review at project closure 6.3 Are suppliers encouraged to from experience? • incentives for suppliers to improve project delivery • a commitment to long-term relationships with integrated project teams