Spreadsheet: 017b5546-0a7d-4db4-9355-b8644e71da1e
Sheet: Introduction

City of Portland
Technology Oversight Committee Quality Assurance Guidelines
1         About this document
This document defines the Quality Assurance (QA) services and deliverables required by the City of Portland Technology Oversight Committee of selected quality assurance vendors to support its role to provide oversight to selected city project.  These guidelines will enable all quality assurance vendors to provide services in a standard and consistent manner across projects.
This document leverages aspects of community best practice found in the State of Oregon DAS IT Investment and Planning1 and State of New Mexico Department of Information Technology2 as well as other examples of QA services on projects within the city like the Public Safety Systems Revitalization Program3.  See Section 5 - Reference Materials for more information.
For the purposes of this document “Contractor” refers to the firm on selected and under contract to provide quality assurance services to the Technology Oversight Committee.
2         Authority
As directed by council resolution 36844 the City of Portland Office of Management and Finance(OMF) is responsible for creating an independent citizen oversight committee for specific City of Portland technology projects.  The Bureau of Technology Services Administrative Rule BTS-1.07 Technology Project Oversight defines the purpose of the citizen committee.
“City technology and communications projects can have widespread financial and operational implications to the City’s business practices. The City will adopt an approach to these projects that applies project management, citizen oversight and quality assurance. One component of project oversight will be addressed through ongoing review from an independent citizen Technology Oversight Committee (TOC). One component of quality assurance will be addressed by having projects contract for services
with a qualified, external quality assurance firm.”
3         Introduction to the QA Guidelines
3.1       Purpose
The purpose of the QA guidelines is to establish common, repeatable standards for the delivery of QA services and deliverables for all QA efforts done on behalf of the Technology Oversight Committee.
3.2       Quality Assurance Approach
To ensure that the appropriate quality management and risk management activities are conducted these guidelines are based on the Project Management Institute’s (PMI) Standard as described in the Project Management Body of Knowledge (PMBOK), Fourth Edition, 2008.  This includes activities of Quality Assurance and Quality Management that determine the quality policy, objectives, and responsibilities, and implements them by means such as quality planning, quality control, and quality assurance.
For the purpose of this document, the term “quality standards” shall refer to both Project “process” and “product” quality standards.   “Process” quality standards shall cover organizational influences, management support, decision drivers, project management, schedule, resourcing, experience, and others.   “Product” quality standards shall cover product content, design, development, deployment, environment, technology, security, maintainability, and others.
3.2.1       Definitions
Quality Assurance
“Quality Assurance” means the project is adhering to project management disciplines, planned and performed according to its project plans and that such adherence can be verified by an independent examination of project documents and other evidence.
Quality Management
“Quality Management” is defined as “a subset of project management that includes the process required to assure that the Project shall satisfy the needs for which it was undertaken.”  Quality Management consists of activities in quality planning, quality assurance, and quality control
Verification
“Verification” means the project is adhering to project management disciplines, planned and performed according to its project plans and that such adherence can be verified by an independent examination of project documents and other evidence.
Independent
“Independent” means autonomous and impartial verification and validation assessment of a project’s adherence to project management plans and compliance with business requirements. These independent assessments are performed by an entity that is not responsible for developing the product or performing the activity being evaluated.
3.3       Overview of Contractor Tasks, [NAME] and Processes
The QA Contractor shall perform the following tasks:
1.       Develop Quality Assurance Management Plan
2.       Conduct Initial Review and Assessment
3.       Provide Monthly [NAME] and Reports
4.       Post implementation Assessment Report
By virtue of the Contractor successfully completing these tasks the Technology Oversight Committee is able to provide oversight to help ensure the City of Portland that the project team and Contractors are applying best practices in project and quality management. This includes the delivery of products that meet business requirements for the project in respect to schedule, cost, scope, functionality, security, and other relevant quality standards.
3.3.1       Develop Quality Assurance Management Plan
The Contractor is responsible for developing a QA Management Plan for the task order that defines the deliverables, timelines, frequency, and [NAME] personnel assigned for the duration of the task order.  Following the acceptance of the QA Management Plan by the city the next report will usually be the Initial Assessment Report provided by the Contactor.  The Contractor should specify in the QA Management Plan a list of inputs from the City that is needed to perform quality assurance.  This would include but is not limited to; all project documentation, formal solicitations (RFP) and contracts as well as access to key staff such as the project team, business users, and subject matter experts to conduct interviews and meetings.
3.3.2       Initial Review and Assessment
The Contractor is responsible for conducting an initial review of project risks and activities to produce an initial assessment of the project.  The initial assessment should include:
·         A Risk Management assessment by which the [NAME] evaluates the effectiveness of the risk management strategy and planning for the project.  This includes specific assessment of the Risk Management metrics and criteria documented in Section 6 of this document.
·         A Verification of Project Deliverables by which the Contractor evaluates if deliverables are appropriately scoped and detailed with respective to the current phase, included but not limited to: Project Management Plan, Project Requirement, Project Budget, Project Schedule and Project Communications.  This includes specific assessment of the Project Deliverables metrics and criteria documented in Section 6 of this document.
·         A Project Management assessment that evaluates the projects organization, roles and responsibilities and management oversight to ensure the project has engaged all stakeholders at the appropriate level and is provided clear authority, managerial support and business ownership. This includes specific assessment of the Project Management metrics and criteria documented in Section 6 of this document.
·         A Verification of Product Deliverables by which the [NAME] verifies if product deliverables are of acceptable quality and delivered on agreed upon schedule and cost. This includes specific assessment of the Product Deliverables metrics and criteria documented in Section 6 of this document.
3.3.3       Provide Monthly [NAME] and Reports
The Contractor shall provide monthly reports to the Technology Oversight Committee.  These reports will appraise the current status of the project and track issues, trends and recommendations.  The monthly reports shall reflect the current phase of the project and should be tailored to the project respective of risk, scope, schedule and budget.
The Contractor should include the baseline established in the initial assessment deliverable and track the progress of the metrics at each monthly report.  This report shall also include an executive summary of project progress and a forward looking calendar of significant deadlines, decisions and milestones that need to be met to keep the project on schedule.
Monthly reports shall be delivered to the Technology Oversight Committee following the format established in Section 6 which includes a written report and a presentation of report information.  Reports shall be submitted to the City Project Manager, Bureau Director, CTO and CAO 10 business days prior to the monthly meeting to provide review and feedback.  The final version shall be provided to the Technology Oversight Committee no less than five business days prior to the monthly meeting.
3.3.4       Post Implementation Assessment Report
Ninety days following the completion of the project the Contractor should provide a post implementation report.  This report will assess if the business and technical objectives were achieved based on project scope and acceptance criteria.  The Contractor shall validate if prior established project outcome metric and success criteria are being met and as a result the intended value of the project has been realized.
4         Reporting Schedule
Unless specified otherwise in the task order the Contractor will follow a monthly cycle for reporting QA status. Monthly reports shall be delivered to the Technology Oversight Committee following the format established in Section 6 which includes a written report and a presentation of report information.  Reports shall be submitted to the City Project Manager, Bureau Director, CTO and CAO 10 business days prior to the monthly meeting to provide review and feedback.  The final version shall be provided to the Technology Oversight Committee no less than five business days prior to the monthly meeting.
Week 1-2 Contractor conducts interview, verifies and drafts assessment and reports.
Week 3 - Draft report and presentation distributed for review by City staff.
Week 4 - Contractor delivers report and presentation to the Technology Oversight Committee.
5      Reference Materials
These documents provided accepted best practice and guided the formation of the Quality Assurance Guidelines for the Technology Oversight Committee.
A Guide to the Project Management Body of Knowledge (PMBOK Guide) Fourth Edition, 2008 Project Management Institute. http://www.pmi.org/en/PMBOK-Guide-and-Standards/Standards-Library-of-PMI-Global-Standards.aspx
1 http://www.oregon.gov/DAS/EISPD/ITIP/IT_Investment_Oversight.shtml
2 www.doit.state.nm.us/docs/project_oversight/proj_mgmt_templates/QAIVVGuidelines.doc
3 https://www.portlandonline.com/omf/index.cfm?c=44868
6         Initial Review and Monthly Quality Assurance Templates and Metrics
For each assessment area defined in section 3.3.2 Initial Review and Assessment a rating should be provided to describe the Contractors evaluation of that aspect of the project.  Reports delivered to the TOC should include both written reports and a formal presentation of findings.  Reports should contain but are not limited to the following topics:
·         Executive Summary of report findings
·         Detailed project area assessments using the format provided below.
·         A summary of all project assessment areas ratings metrics ratings that have changed since the last evaluation period.
·         The status of prior recommendations
As defined in section 3.3.2 Initial Review and Assessment these should include but are not limited to:
·         Risk Assessment
·         Verification of Project Deliverables
·         Project Management Assessment
·         [NAME]
Assessment areas shall be evaluated using a series of metrics that contribute to the overall rating for the area.  Rating and Metrics associated with an Assessment area shall be rated using the following Green, Yellow, Red color coded system to highlight status.
GREEN   -   All scope, budget, schedule or quality assurance issues are manageable by the project team and are being resolved in an appropriate timeframe.
YELLOW  - All scope, budget, schedule or quality assurance issues are manageable but one or more require escalation to the projects steering committee for management intervention and resolution following the timeframe for escalated issue resolution.
RED – Scope, budget, schedule or quality assurance issues have been escalated to project governance but intervention and resolution have not yet occurred resulting in significant risk to scope schedule or budget. Issues at this level are likely to require intervention form Bureau leadership, CTO/CAO or the Technology Oversight Committee.
The following worksheets of this workbook contains the Quality Assurance Reporting Template.