Gate 5 Benefits Realisation - focuses on measuring the project’s success to date in achieving its objectives, expectations for the future and building in remedial action to deal with any potential Gateway for Programmes There are three different review stages that occur for programmes, these are: Critical Stage First stage review Conducted before or soon after Government approval. It assists entities in defining the programme by examining the business need and formulation of the business case. Can also be conducted whenever the priority or the scope of the programme changes significantly. Mid stage review Focus is on assessing the programme execution with the number of these reviews being determined by the complexity, timeframe and risks attached to the programme. Final stage review Focus is on programme closure, including programme controls, records management and the identification and application of lessons learned as well as the delivery of the intended outcomes and benefits. Blended Gateway Reviews Programmes do not always fit neatly into the existing structured Gateway process for projects (from Gates 0 to 5) because programmes can represent a series of interrelated projects with a common objective, or a broad framework or policy concept that may result in a series of largely independent smaller projects (potentially all at different stages of implementation). The ‘blended’ review approach (where a programme review is combined with the review of a constituent project that may be critical to the overall success of the programme) will help to reduce the review burden on entities while simultaneously providing programme strategic alignment and project milestone delivery assurance. Identifying the critical factors in a multi-project programme, including the issues that need to be addressed to realise benefits, is an important component of the blended review approach. Programme reviews will assess the significance of any one project to the overall success of the programme, and where the single project could benefit from a focused review, the overall programme review will accommodate a combined focus of programme and project review. For example: Gates 0 and 1 at a first-stage programme review; or blend Gates 2, 3 and 4 reviews in mid-stage programme reviews.
Programme/project timeline, showing critical path, dependencies and key milestones Risk matrix and risk management approach Performance measures and reporting regime Evaluation and communication plan List of other entities involved in the programme/project List of stakeholders List of relevant files Details of the consultative process showing who was consulted and key submissions/discussion papers Governance including minutes from any steering or programme/project management committees Issues log (if available) Organisation chart for relevant areas of the entity Details of the implementation team List of entity contacts and phone numbers for the programme/project Appendix B: Skills profile of an Assurance reviewer Review Team Leaders and Review Team Members will be matched to reviews on their skills and experience, and the requirements of the review. A reviewer accreditation process is operated on behalf of Finance by ARU, which aims to develop a flexible team allowing progression and development of Review Team Leaders and Review Team Members. Applications for inclusion on the ‘Assurance Reviewer Services Multi Use list’ (MUL) must meet the following conditions for participation: Provide proof satisfactory to Finance, that you have the skills, expertise and experience in at least three of the following Categories of Expertise (A – H): Public sector senior executive experience Portfolio/Programme/Project Management expertise and experience such as: Programme/Project / Portfolio management Business analysis Business change management Risk management Benefits realisation Business and Policy expertise and experience such as: Policy design and implementation Financial analysis Procurement Legal and contracting ICT expertise and experience such as: IT project management Systems integration Enterprise architecture System delivery and management Electronic documents and records management systems Infrastructure expertise and experience such as: Construction / property project management
where the Gate is integrated with the ICT Investment Approval/Two Stage processes ensure that the review supports development of the first pass business case; check that financing has been adequately assessed for the project; and check that the project is realistic, properly resourced and has authorised work plans through to the next stage of the project. Project Documentation Required for the Business Need Review To undertake the review, the review team may require access to: the NPP or Cabinet Submission/Decision; the entity’s Portfolio Budget Statement, business plan or equivalent documents. These documents should set out the entity’s outcomes and outputs, strategic direction, high level aims and objectives; a project brief that provides information about: objectives—a high-level description of the purposes, outcomes sought, key deliverables and timelines; critical success factors—the main criteria against the success of a project will be measured. This could include a benefits management plan showing that the project is required, the expected benefits and the business changes and enablers that will be required for the anticipated benefits (these should be linked to the objectives); background—an outline of the key drivers for the project, showing how it will contribute to policy outcomes and/or the business strategy; scope—a high-level view of the boundaries of the project, the main assumptions, possible constraints and dependencies; stakeholders —and how they will be engaged; finance—the assessed costing and budget for the project and its components; organisation—the way in which the project is to be organised, led, managed and monitored; risks—the main risks identified so far and how they will be managed; issues—a strategy for capturing and resolving issues; and evaluation – an approach for measuring results and achieving outcomes. a work plan for the short to medium term, which: identifies the components of the project; shows the main deliverables and milestones for each component; shows the contribution each component makes to the overall project outcomes; and contains estimates of resource requirements. These documents, and other information the review team deems relevant, will be requested at the Planning Meeting or during the course of the review. Areas to be Considered in the Business Need Review This section contains topics that would commonly be considered when conducting a Gate 0— Business Need Review. Review teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Teams may determine that additional topics are considered critical to the assessment of the project. Policy and Context
to probe Evidence expected 1.1 Is the project innovative in planning to succeed? • New approaches have been explored and existing systems challenged. • Confirmation that the project is structured to responsiveness and flexibility in achieving outcomes. 1.2 Is there a clear understanding of the required outcomes of the project and are they soundly based? • A description of the project policy drivers and outcomes and how they contribute to the business strategy and objectives . • An outline of the required outcomes, benefits derived and their relationship to outputs. • Confirmation that the way forward is likely to achieve the required outcomes, the expected benefits and the business changes and enablers required for the anticipated benefits. 1.3 Does the project demonstrate a clear link with wider government objectives? • An analysis which shows the relationship between the project and major government policies and objectives. • A demonstrated link between strategic objectives and outcomes and the deliverables of the project. 1.4 Is the business strategy to which this project contributes robust? • A clear direction set out in the business strategy, which is approved by all key stakeholders. 1.5 Does the project, programme or policy require new governance arrangements e.g. cross-portfolios • For projects, programmes or policy, confirmation that all parties involved know how they are engaged in the project and are committed to its delivery • Clear governance arrangements to ensure sustainable alignment with the business objectives of all organisations involved. 1.6 Does the project align with government policy and initiatives, the broad environment of the entity and the entity’s strategic objectives? • Documented links between the proposed project and the entity’s policy, environment and strategy. 1.7 Is the governance framework fit for the purpose? Is commitment to key roles and responsibilities for this project? • Documented commitment from top management and, where appropriate, minister(s) and a clear understanding of their continuing roles in achieving successful outcomes. • Clear allocation of key roles of responsible minister, SRO, and project managers. • Confirmation that all parties understand their roles in the project and are committed to the delivery of the project. 1.8 Are the required skills and capabilities for this project available, taking into account the entity’s current commitments and capacity to deliver? • Confirmation that the entity has brought together (or has credible plans for bringing together) the skills and capabilities it needs to plan and achieve the desired outcomes. • Confirmation that the entity is realistic the complexity of the project deliverables and how they can be managed. • Documentation of allocation to individual officials of key roles within the project .
• Interdependencies with other projects are being managed. 3.3 Have options for potential ways forward been identified? • A listing of high-level options for meeting the business need is maintained. • Options analysis or feasibility studies, are maintained, if available. Management of Intended Outcomes to probe Evidence expected 4.1 Have the required outcomes from the project been identified? • A current list of the main outcomes and desired benefits, linked to strategic outcomes and to the deliverables from specific projects. 4.2 Are the planned outcomes still achievable, or have any changes in scope, relationship or value been properly agreed, and has the business case been reviewed? Is the project time critical? • Outcomes identified, together with their relationships to each other. • Credible plans for the achievement of outcomes. •Documentation showing ongoing commitment from stakeholders to the outcomes and their achievement. • Documentation showing ministerial agreement to scope changes. 4.3 Are key stakeholders confident outcomes will be achieved when expected? Is it on track to deliver? • Confirmation of time-critical issues and a realistic schedule to achieve them. • Mechanisms for collecting performance data are in place and a plan for evaluating the impact of the project is in operation. • Confirmation that the project board is confident that planned milestones will result in good quality deliverables that will achieve the necessary outcomes • A commitment from key stakeholders that project deliverables will achieve the desired outcomes. 4.4 Is there a plan for achieving the required outcomes? • A benefits management plan and a plan to ensure that outcomes are delivered in terms of performance measures and/or key performance indicators. • Plans identify appropriate baseline measures against which future performance will be assessed. • Plans to carry out performance measurement against the defined measures and indicators; • planned outcomes have not been achieved, confirmation that the problems have been identified and plans are in place to resolve them. • there is clarity on how the objectives from the projects link to the outcomes of the programme. 5. Risk Management
1.8 Is there an agreed plan for the realisation of anticipated benefits? • A benefits management strategy and plans. • Critical success factors • A list of the individuals responsible for delivering and achieving benefits identified. • The agreed process for measuring benefit. • A post-implementation review plan identifying areas to be assessed. • Payment mechanisms linked to benefits realisation, where appropriate. 1.9 Have suitable stakeholders been involved and, if so, what has been their involvement? • Involvement of stakeholders and business or user representatives in the quality and proposal reviews • Stakeholder representation on the evaluation team. • Approval by the project steering committee and SRO. 2. Assessment of the Proposed Solution Areas to probe Evidence expected 2.1 Does the proposed solution still meet the business needs and government and entity objectives? • The recommended bid fully complies with all requirements. • Documented consultation with stakeholders during evaluation. 2.2 Have the suppliers proposed any alternatives or options in addition to a fully compliant bid? • An assessment of options showing how they are beneficial to the project’s outputs/outcomes and are within the scope of the tender documents. 2.3 Will the proposed bid deliver the business need described in the business case? • Analysis showing the proposal is defined in terms of the required outcomes. • Confirmation that the business can achieve the necessary organisational and business process changes. • Confirmation that the proposed services and service levels as defined in the contract will meet the agreed business requirements. 2.4 Has the proposed solution affected the strategy for business change? • An updated plan for managing the business change on the basis of the proposed solution, which is agreed by project steering committee, and SRO. • Analysis of differences from the original plan. 2.5 Has the proposed solution affected the expectations of business benefits? • An updated plan for benefits realisation and benefits profiles. • Analysis of differences to the original position. • Changes documented and agreed with users and stakeholders. 2.6 Are the entity and supplier prepared for the development (where there are new systems and processes), implementation, transition and of any new services? • The proposed supplier’s development and implementation plans are included in the bid and presented to the appropriate entity endorsement and governance bodies/committees. • The entity’s implementation plan is agreed by users and other internal/external stakeholders. 2.7 Are there plans and processes to address future issues, both business and technical? • A strategy for managing change is agreed by all parties, including the supplier.
2.8 Is there clear allocation and understanding of responsibilities between all parties, in addition to any contractual obligations? • Responsibilities defined and allocated among the entity officials, the supplier organisation and personnel. • For projects, a Memorandum of Understanding (or equivalent), defining responsibilities, relationships and interaction between entities. • Documentation identifying internal relationships and interactions (describing ‘who does what’) with the supplier. • A statement from the supplier outlining reciprocal arrangements, including senior management roles. • If a single supplier, a description of how the project management team will manage the supply chain. • If multiple suppliers, details of how the entity will manage the interaction. • Documentation confirming that the entity and supply team will work together as an integrated project team. • A documented exit strategy exists. 2.9 Are resources available for the supplier to fulfil its obligations within the contract? • A plan for implementing the new contract, identifying the quantity, type and quality of resources required. • Formal management acceptance of resource requirements. 2.10 Have the technical implications been assessed? • Documentation demonstrating that the proposal is technically acceptable and has considered relevant implications such as sustainability and legacy systems etc. 2.11 Does the project have resources with the appropriate skills and experience to achieve the intended outcomes of the investment? • Plans for building client capability, with nominated staff allocated to the major roles • Confirmation of internal and external commitment to provide the resources required. • Job descriptions for key project staff. • A skills appraisal undertaken, with plans for addressing shortfalls, if any. 2.12 Is the proposed procurement within financial approvals and is there adequate budget to accommodate the procurement? • Documentation confirming that required financial approvals exist. • Documentation detailing the required budget and timing of milestone payments, referring to the relevant budget measure(s). 3. Review of Current Phase Areas to probe Evidence expected 3.1 Is the project under control? (i.e. running to plan and budget) • Confirmation that project is running to schedule and costs are within budget. • Confirmation that project governance entities/structures are fulfilling their roles and actively oversighting the project. 3.2 What caused any variations? • Reconciliations against budget and timetable criteria. • Advice provided to the project steering committee or SRO, where appropriate.
4.1 Are all project elements ready for service? • Updated schedules. • Occupational health and safety file • Handover certificates. • Testing and commissioning data. • for transition in place. • Plans for ‘operate contract’/service phase available. • Business continuity and contingency plans in place, if required. • Technical documentation available, including: • drawings; • operating manuals; • maintenance manuals; • instructions and • information assurance documentation. 4.2 Is the entity ready to adopt new ways of working, where applicable? • New business processes have been thoroughly worked out, tested and are ready to be implemented. • Information and support is available (e.g. client information at call centres). • Where applicable, members of the public/end-users are aware of the new service and can obtain further information. 4.3 Is the long-term contract management process in place? • Detailed and aligned plans, roles, responsibilities and organisation in place for client and supplier, with reporting arrangements at appropriate levels. • An appropriate number of suitably qualified are appointed by client and supplier, with continuity planned, skills appraised and plans made for addressing shortfalls, if any. • Staff managing the contract are aware of and trained for their contract management role; they are familiar with the contract’s aims and purpose. • Plans for managing service delivery, changes to the contract, and the relationship with the supplier. 4.4 Is there a process to manage and measure performance? • Performance management plans in place. • The performance enhancement process is agreed with the service provider and documented in the contract before awarding of the contract. • The means of measuring performance is agreed with the service provider/partner. 4.5 Is there a process to manage measure and report benefits? • Benefits management plans are in place, linked to intended project outcomes, where applicable. • Means of measuring benefits are agreed with the service provider/partner. • Reporting lines are clear, with stakeholder involvement clarified. • For collaborative projects, all parties understand and agree to their responsibilities and arrangements for benefits realisation. 4.6 Have ongoing operation and maintenance been considered in detail? • Issues and ongoing costs relating to maintenance (buildings, physical and/or IT infrastructure and applications as appropriate) are addressed and monitored against expectations.
1.4 Have all the governance and stakeholder issues been addressed? These include: • the statutory process • communications • external relations • environmental issues • personnel. • Updated operational communications strategy, communications plan and issues log. • Governance structure reciprocated in both client and provider organisation. • Issues addressed at the appropriate level in client and provider organisations. and • Staff empowerment to make decisions and stakeholder involvement. 1.5 Are the users satisfied with the operational service? • Details of user groups, outputs and the feedback process. • Indication of advance preparation for changes in way services are delivered. • User-friendly to the services provided. 2. Review of Operating Phase Areas to probe Evidence expected 2.1 Is the service/facility operating to defined parameters? • Documented operating parameters, updated as change. • Updated service agreements, where applicable. • Service delivery is measured against defined parameters. • Confirmation that the measures used to address poor or non-performance are effective • The facility is safe to operate and maintain. • Sustainability targets are met or exceeded. • Client/stakeholder satisfaction and experiences are assessed. 2.2 Have the project documentation, training material and training programme been delivered and kept up to date? • New trained and existing staff skills updated. • Updated training material . • Responsibilities for updating training material and documentation defined. • The occupational health and safety record updated as required. 2.3 Are the contractual relationships satisfactory? • Project records demonstrating effective contract management procedures. • Supplier and client staff who are enthusiastic, mutually respectful and united in their objectives. • Improvements to the contract. • Regular reviews between supplier and client. • Action plans exist. • Measurement of contract improvements. • Reports on work done and plans for expected work. 2.4 Are there plans for continued contract management? • Ongoing forward resource plans, as required; • Succession planning for key • Skills appraisal and for addressing shortfalls 2.5 Are plans for ongoing risk management up to date? • An updated risk register, risk reporting and management.
4.2 Is the entity tracking its progression to improved performance and the flow of results through key milestones and the business planning • Documented performance information which is clearly linked to planned outcomes and enables ready assessment of effectiveness, efficiency and service quality. • Core sets of performance information that meet multiple purposes, are consistent and complementary. • Ongoing assessment of the appropriateness of performance information. • Responsibilities for performance management are defined and understood by the entity and supplier. • Direct links exist between planning and results. • Ongoing monitoring of performance and periodic evaluation. • Integration with corporate and business planning. • Consistency with the principles set out in the operational strategies of the business. 4.3 Does the entity have performance measures to cover all aspects of the contract? • Performance measures in relation to: • cost and value obtained; • performance and client satisfaction; • delivery improvement and value; • delivery capability; • benefits realised; and • relationship strength and responsiveness. 4.4 Do the performance measures selected offer clear and demonstrable evidence of the success (or otherwise) of the contract? • Performance measures chosen have the following attributes: • are meaningful and visible to an entity’s management group; • properly reflect user and stakeholder perceptions; and • can be used to identify the need for support or remedial action as part of contract management activity. 4.5 Are performance measures related to delivery or capability improvement tracked against an existing baseline? • A baseline has been established in the business case for the contract. • Performance measures are tracked against that baseline. 4.6 Are there performance assessment measures for: • ongoing service delivery • the desired results of individual programmes of change or improvement, the implementation of projects • the overall results or effect of the deal? What does the entity want to have achieved • Service level agreement approaches and related measures. • Investment appraisal and benefits management techniques are constructed on a case-by-case basis. • Objectives are identified during project scoping and in preliminary business analysis activity; and should be rooted in the organisation’s long-term business strategy. • For construction projects, confirmation that user satisfaction has been or will be monitored as required.
4. Review of current phase Review current status of the programme in terms of budget, schedule and progress with outcomes, against the performance measures set out in the relevant Portfolio Budget Statements, the approved business case and benefits management plans. How to use this section for: First Stage Programme Review This section would not normally apply but some of the topics may need to be considered. Mid Stage Programme Review All areas need to be investigated thoroughly to confirm the programme remains on track and issues are being managed effectively. Final Stage Programme Review Confirm the expected outcomes have been achieved against the performance indicators and targets, as set out in the relevant Portfolio Budget Statements, and the Business Plan and that no outstanding issues remain. Areas to probe Evidence expected 4.1 Are the programme’s key milestones compliant with broader government or entity timing requirements? • Timelines for processes are identified and lead times are factored into the programme schedule. 4.2 Is the programme on track in relation to planning and/or delivery? • The programme report and plan are updated. • Milestones are achieved as planned. • The plan for benefits measurement and achievement is on • The risk register is current. • There are highlight reports for constituent work • The resources and funding used to date. • Issues being resolved. • Documented confidence from delivery partners that future milestones and plans are realistic. • Confirmation that interdependencies with other programmes are being managed satisfactorily. 4.3 Have problems occurred and if so how have they been resolved? • Issues and details of action are documented. • Governance framework with escalation routes to senior management, ministers and responsible interdepartmental committees and central entities. • The programme plan is updated to reflect changing issues and • Recommendations from previous Gateway reviews are addressed. Key focus areas to probe Evidence expected 4.4 Have options for potential ways forward been identified? • Documentation of various solutions including policy, and non-asset options. • A comparison of retention maintenance and replacement indicative costs. • Options analysis or feasibility studies. 4.5 Have lessons learned been shared? • Confirmation of lessons learned and shared across the entity and across programmes. 5. Assessment of intended outcomes and benefits Review the management of intended outcomes and benefits by analysing governance arrangements for leading, managing and monitoring the programme, including roles and