The Audit of Performance Information in the Public Sector (International Syllabus only)

YouTube video

1 Introduction

Performance information is information about the performance of an organisation against set criteria such as a budget or benchmark. For example, information may be produced about how many units of a product are produced or how efficient an employee is in terms of units produced per hour. In terms of the public sector this could be number of operations performed in a hospital, or how many times the grass is cut in a council park, or how many school children reach a target literacy level.

The audit of performance information in the public sector is examining central and local government departments in terms of three main criteria:

Effectiveness: whether the objectives of the department have been met.
Efficiency: how well inputs are turned into outputs.
Economy: obtaining best value for money (ie lowest possible cost for the right quality and quantity or resources at the right time).

For example:

A government department is responsible for repairing street-lights. The performance of that department can be determined using the '3Es' criteria.

Effectiveness: Whether each light was repaired within the time limits set by the government department.
Efficiency: How many resources were used to fix each light in terms of materials, labour etc. potentially against some standard.
Economy: The financial budget for fixing lights was not exceeded during a specific period.

2 The use of performance information

In many countries, government departments produce statistical information concerning the performance of that department. For example, a local council in the UK (a local council is responsible for provision of services such as refuse collection, street lighting, road maintenance, etc) will produce information not only on its financial performance but also on other areas such as the amount of refuse recycled or time taken to respond to requests to repair streetlights.

2.1 Objectives

Improvement. Targets are set and performance measured against those targets to try and determine how management can be improved. For example, if recycling targets are not being met then councils may use performance information to determine which categories of waste are not being recycled and then introduce new systems to (hopefully) improve actual recycling against target.
Monitoring. Performance information is used to report on the achievement of targets. In this sense, local government may have to report back to central government to obtain the necessary funding for next year’s budgets. Similarly, performance information can be used as a management tool to determine the amount of performance related pay.
Reporting. Including performance information in accounts or other external reports partly as a means of showing progress against targets but also as a means of advertising how well those targets have been met. For example, meeting a recycling target will be reported not simply to show that the target has been met but also to generate good publicity for the council. Taxpayers will therefore be happy with the activities of the council and ultimately this means that the councillors in charge of the council are re-elected(!).

From the audit perspective, this means that auditors will have to treat performance information with a high degree of scepticism. There will be management bias to show good results not only to maintain or increase funding but also so councillors retain their posts.

In other words, a lot of performance information may be available, but it may be produced to meet the three objectives above, not because there is any standard (such as an IFRS) which requires production in a specific format. Comparability between different information providers (that is different councils, schools or hospitals etc.) will be difficult as the actual information produced will the different and different standards will be used to produce that information anyway.

2.2 Auditing

There are two types of audit:

‘Performance audit’ – this ‘value-for-money’ audit examines efficiency and effectiveness having regard for economy. It focuses on government policy or management processes and its main aim is to lead to improvements.
‘Audit of performance information’ – this provides assurance that the performance reported against predetermined objectives is reliable, accurate and complete (in all material respects). It focuses on:
- Data reliability – is it accurate and complete? The auditor must assess the systems that produce the performance information.
- Content quality – is it relevant (ie linked to objectives), comparable (eg with past periods) and verifiable (ie supported by documentation)? Is it well-defined (ie clear and unambiguous) and timely (to be useful)? Does it avoid ‘perverse incentives’ (ie not encourage wasteful behaviour)?
- Compliance with reporting requirements – is the report clear? Is there sufficient explanation of inputs, outputs and outcomes? Has actual performance been evaluated?

3 Planning the audit of performance information

To audit that performance information the auditor needs to know how that information (actual performance and target performance) was produced. There are five stages to the production of performance information:

Defining a measurement objective
Formation of relevant indicators
Data collection
Data analysis
Reporting

These stages are explained below along with typical examination procedures that can be used by the auditor.

Audit procedures for performance information in the public sector can include:

Tests of controls on the systems used to generate performance information (eg managers dating and signing off tasks as they are accomplished).
Performing analytical review to evaluate trends and gauge the consistency of the information (for example, compare to the prior period).
Discussion with management and others responsible for the reporting process about problems both with performance and its adequate recording.
Review of minutes of meetings where performance information has been discussed. For example, council meetings, school governors’ meetings, health authority meetings.
Inspection of performance information source documentation, probably on a sample basis.
Recalculation of quantitative performance information measures and their summarisation onto performance reports, again on a sample basis.

The example of refuse collection is used to illustrate each section. In many countries, the residents of each house are provided with a large bin on wheels into which household refuse is placed. The local council provides a refuse collection service, so each week the bins are emptied into a refuse collection lorry which takes that refuse to a tip. Some councils also provide a recycling service where recyclable refuse is placed in a separate bin and taken for sorting and recycling.

Stage	Explanation	Examination procedures
Defining a measurement objective	The process or activity is identified and level of detail to be measured confirmed. For example, the number of houses to collect refuse from each day may be considered as an indicator although some allowance might be necessary for different types of houses.	Ensure the measurement objective can actually be measured. Review the proposed indicator to ensure it is achievable and at the correct level of detail.
Formation of relevant indicators	Having decided what to measure, this stage defines how to measure the process or activity. To ensure that the measure is appropriate, it must be SMART: Specific to the process or activity. Measurable, that is quantifiable. Achievable. Relevant to the process or activity Time bound, that is measured over a specific period of time. For the refuse collection the objective can be: “Bins to be emptied from all properties in the council area once each week with 1,000 bins emptied each day by each work team”.	Confirm number of houses in the council area – by reference to detailed maps. Ensure 1,000 bins emptied each day is achievable. Ensure all bins can be emptied each week by dividing number of properties by 5,000 (1,000 bins over 5 days) and there are at least that number of refuse teams.
Data collection	Data is then collected on each of the SMART indicators identified. For refuse collection, each collection team has a pre-determined route covering about 1,000 houses and a shift supervisor signs a document confirming refuse bins have been emptied. Also confirm that residents can complain (probably online) if their bin is not emptied.	Review signature sheets confirming that bins have been emptied. Access online complaints system and determine from this how many bins were not emptied. If necessary, write to a number of residents in the council asking them to confirm their refuse bins were emptied.
Data analysis	The data collected is analysed and compared to the indicators. Reasons for performance being above or below standard are identified. For refuse collection, compare results of collections each week (signature sheets) to confirm all bins have been emptied – or where bins are not emptied calculate the % missed. Identify any areas or streets where collections are normally below the performance indicator.	Re-calculate the council’s statistics of bins being emptied and confirm accuracy of non-emptied bin % (complaint information already obtained above).
Reporting	Finally a report is produced on the whole process or activity, with recommendations made for improvement or change to the activity, the indicators being used or both. For refuse collection, from the results above, determine what remedial action is necessary. For example, this may mean changing the number of refuse collection teams or targeting specific teams with training where their % of bins not emptied exceed other teams.	Obtain and review report. Ensure that the recommendations are congruent with the problems identified in the previous section. Where necessary, discuss actions with the managers in the council to determine whether further remedial action is necessary.

4 Reporting on performance information

In some countries, including the UK, reports on performance information are generally included within the report on the financial results of the government or other department. The audit may also be carried out by a government auditor (literally auditors employed by the government in effectively an internal audit role) using specific government reporting standards. This means that care must be used in interpreting the reports partly due to possible bias (no truly independent report) and lack of knowledge of the different reporting standards used. In terms of non-financial performance information there may not be any reporting standard anyway, potentially making the report even more subjective.

A typical report is summarised below.

If you remember the basic elements of an auditor's report (ISA 700), most of the sections of a report on performance information can at least be “guessed”.

Section of report	Content	Implication for auditors
Title and addressee	Indicates that it is an independent assurance report and to whom it is addressed – in this case normally the members of the government department.
Subject matter information and applicable criteria	Identifies what has been audited and the standards used. In many countries there will be a specific financial reporting framework for government departments.	The auditor will need to understand the financial reporting framework. The auditor will also state clearly what information has been audited, normally be identifying the page numbers in the report where the performance information is shown.
Relative responsibilities	The report states, for example, that the chief financial officer prepares the report and the auditor audits the information in that report.
Summary of work performed	In the absence of a more specific ISAE, ISAE 3000 applies. The summary must convey how, based on the procedures performed, the auditor may detect material misstatement.	This summary is likely to be more detailed for a limited assurance engagement than a reasonable assurance engagement in order to convey the limitations on the nature, timing and extent of procedures.
Opinion/Conclusion	Conclusions on the reported performance information and whether objectives have been achieved may provide express reasonable (positive) or limited (negative) assurance.	If not governed by national regulation, the type of conclusion must be agreed with the organisation's regulating body. A conclusion must be modified if there is a scope limitation (qualified or disclaimed) or material misstatement (qualified or adverse).
Findings and recommendations	This is a separate section of the report where the performance information is reported on. The auditor will state findings related to the content and relevant data systems and whether or not the council has fulfilled its reporting responsibilities (ie complied with reporting requirements). It will also provide recommendations based on findings.	This might include, for example, identified misstatements (and whether they were corrected) and recommendations for further/future action (eg to improve internal control over data collection and/or reporting of a performance indicator).