Clean Air Data City of York Council Internal Audit Report 2018/19

Business Unit: Economy and Place Directorate Responsible Officer: Assistant Director for Planning & Public Protection Service Manager: Trading Standards Manager Date Issued: 03/08/2018 Status: Final Reference: 10530/007

| | P1 | P2 | P3 | |---|----|----|----| | Actions | 0 | 0 | 2 | | Overall Audit Opinion | High Assurance | Summary and Overall Conclusions

Introduction The council is responsible for monitoring the quality of air within its area under the Environment Act 1995 and is required work towards meeting air quality objectives for key pollutants outlined in the Air Quality Standards Regulations 2010. Under this legislation, the results of this monitoring must be compiled by the council into an Annual Status Report sent to the Department for Environment, Food & Rural Affairs (DEFRA). Should an area fail to meet air quality objectives by an agreed date, the authority is required to declare an Air Quality Management Area and draw up an Air Quality Action Plan to be submitted to DEFRA.

Poor air quality also remains a high profile area of public protection, with significant health, economic and planning implications for local populations. For these reasons, air quality results are also reported as part of the council’s annual performance indicators and council data is made available to the public through the York Open Data Platform and Jorair – a separate website maintained by the council.

There have been significant savings targets over the last five years for air quality monitoring but this income has been supplemented with funding from a range of external sources, including a Public Health grant of £50,000 per annum and planning fees of £6,000 in the year 2017/18.

As a result, air quality in York is now monitored via 9 real-time air quality monitoring stations and 233 diffusion tubes. The Public Protection team within the authority retains liability for the collection, processing, integrity checking and reporting of data to DEFRA, but some of the ongoing data management functions are outsourced due to staff resources within Public Protection. In addition, analysis of diffusion tube samples needs to be undertaken by an accredited laboratory.

Objectives and Scope of the Audit The objective of the audit was to provide assurance to management that procedures and controls within the system will ensure that:

- There is a strategy and plan in place to manage the air quality in York
- There is a robust procedure in place to collect and report on air quality data

The audit did not review the performance of the council in meeting national air quality standards, but instead focused on the performance management of the system itself, including a review of the accuracy of calculated final figures.

Key Findings Overall, a sound control environment was found to be in place within the air quality management team. An experienced and knowledgeable team is in place and the overall system relies heavily on the professional expertise of these staff, representing a potential risk if key staff were to leave. However, the availability of a joint-Local Authority officer networking group provides support to maintain the team’s current function. A review of the strategy used to manage the air quality in York found there was a clear set of procedures within the current system to ensure that an appropriate and timely plan was produced. This means the team are fully discharging the council’s statutory duties by completing both the Annual Status Report and Air Quality Action Plan (AQAP) with all the required content, as outlined in DEFRA’s mandatory guidelines. The only area of potential improvement identified during testing for the current strategy was the reintroduction of the AQAP Steering Group. The reinstatement of an AQAP Steering Group that meets annually would ensure the long-term strategy for the city of York is kept updated and remains appropriate for improving the quality of air. Nevertheless, the majority of the controls within the system are functioning effectively, with the inclusion of multiple two-stage reviews in combination with strong external checks ensuring all published documents are completed to a high standard.

Audit testing performed on the final air quality data reviewed did not uncover any issues with processing or collection that directly affected the accuracy of the final data reported within York. The combination of an annual technical audit, clear expectations outlined in outsourced contracts, as well as internal and external reviews guarantee there is a robust system in place to ensure the appropriateness of the final figures. To standardise the process of reviewing the annual air quality results before submission to the Business Intelligence Hub, a peer-review could be implemented. Nevertheless, the separation of duties between officers within the air quality management team and the peer-review process in place for all other elements of the calculation ensures there is a reduced likelihood that inaccurate results would be publicly published. Overall, the current system in place ensures the final air quality figures produced accurately reflect the quality of the air within York, with no significant data quality errors identified.

**Overall Conclusions**

It was found that the arrangements for managing risk were very good. An effective control environment appears to be in operation. Our overall opinion of the controls within the system at the time of the audit was that they provided High Assurance.

1. Reinstatement of the AQAP Steering Group

| Issue/Control Weakness | Risk | |---------------------------------------------------------------------------------------|----------------------------------------------------------------------| | The Air Quality Action Plan (AQAP) Steering Group has not met annually for two years and the group has now informally disbanded. | Outdated air management strategies or inappropriate policies may continue to be used or are implemented by City of York Council, negatively impacting on the quality of air within the city. |

Findings

During testing it was found that the AQAP Steering Group had not met since 2015 and the responsibility for updating the AQAP had been informally transferred to the Principal Air Quality Officer through the Annual Status Report (ASR). DEFRA guidelines suggests each Local Authority Steering Group should meet annually, in addition to the reviews conducted during compilation of a ASR in their statutory guidelines, LAQM.TG(16).

Without the AQAP Steering Group, air quality strategies and policies in place cannot be adequately considered inter-departmentally within City of York Council on a regular basis. This prevents the AQAP from being kept ‘live’ during irregular and often extended intervals between the development of new AQAPs. Although the Principal Air Quality Officer does evaluate the continued suitability of the measures included in the AQAP during compilation of the ASR, the approach taken is largely piecemeal with individual departments consulted independently.

The function of the AQAP Steering Group is intended to also encompass a secondary internal peer-reviewing process for any new or amended draft AQAP (new AQAPs are generally drafted once every five years). With the group currently disbanded, this means there are no internal staff members remaining outside the air quality management team with the experience necessary to adequately review and challenge the content of the AQAP.

Agreed Action 1.1

The Air Quality Action Plan (AQAP) Steering Group will be reinstated and will meet annually to review City of York’s air quality management strategy.

| Priority | Responsible Officer | Timescale | |----------|---------------------|-----------| | 3 | Corporate Director of Economy and Place | 05/11/2018 | 2. Peer-Review of air quality data before submission to the Business Intelligence Hub

| Issue/Control Weakness | Risk | |---------------------------------------------------------------------------------------|----------------------------------------------------------------------| | Audit testing found that the spreadsheet containing the annual air quality results is not peer-reviewed prior to submission to the Business Intelligence Hub (BIH). | Inaccurate results are published by the BIH for the annual air quality performance indicators, leading to reputational damage and decisions made that may negatively impact on air quality within the city. |

Findings

Unlike the other spreadsheets containing air quality indicator information submitted to the Business Intelligence Hub (BIH), the spreadsheet containing the annual air quality results is not peer-reviewed prior to submission. Although the final annual air quality results go through numerous internal checks to ensure their accuracy prior to publication, there is no peer-review to confirm the data has been accurately transferred from the annual results spreadsheet to the Key Performance Indicator spreadsheet. No errors were identified during the audit, but without a peer-review process in place to match the system for other air quality indicators, there is a risk that data entry errors may be reported publicly via the BIH.

Agreed Action 2.1

A consistent, formalised peer-review process will be implemented within the air quality management team across all air quality indicators reported to the Business Intelligence Hub.

| Priority | 3 | |----------|---| | Responsible Officer | Public Protection Manager | | Timescale | 05/11/2018 | Audit Opinions and Priorities for Actions

Audit Opinions

Audit work is based on sampling transactions to test the operation of systems. It cannot guarantee the elimination of fraud or error. Our opinion is based on the risks we identify at the time of the audit.

Our overall audit opinion is based on 5 grades of opinion, as set out below.

| Opinion | Assessment of internal control | |--------------------|------------------------------------------------------------------------------------------------| | High Assurance | Overall, very good management of risk. An effective control environment appears to be in operation. | | Substantial Assurance | Overall, good management of risk with few weaknesses identified. An effective control environment is in operation but there is scope for further improvement in the areas identified. | | Reasonable Assurance | Overall, satisfactory management of risk with a number of weaknesses identified. An acceptable control environment is in operation but there are a number of improvements that could be made. | | Limited Assurance | Overall, poor management of risk with significant control weaknesses in key areas and major improvements required before an effective control environment will be in operation. | | No Assurance | Overall, there is a fundamental failure in control and risks are not being effectively managed. A number of key areas require substantial improvement to protect the system from error and abuse. |

Priorities for Actions

| Priority | Description | |----------|-----------------------------------------------------------------------------| | Priority 1 | A fundamental system weakness, which presents unacceptable risk to the system objectives and requires urgent attention by management. | | Priority 2 | A significant system weakness, whose impact or frequency presents risks to the system objectives, which needs to be addressed by management. | | Priority 3 | The system objectives are not exposed to significant risk, but the issue merits attention by management. | Where information resulting from audit work is made public or is provided to a third party by the client or by Veritau then this must be done on the understanding that any third party will rely on the information at its own risk. Veritau will not owe a duty of care or assume any responsibility towards anyone other than the client in relation to the information supplied. Equally, no third party may assert any rights or bring any claims against Veritau in connection with the information. Where information is provided to a named third party, the third party will keep the information confidential.