Audit Tips

October 24, 2023

   Audit Tips   

In 2022-23, the Pacific Association of Supreme Audit Institutions (PASAI) commissioned the Canadian Audit and Accountability Foundation to support the design and implementation of performance management systems for the Office of the National Auditor of the Federated States of Micronesia (FSM) and the Pohnpei Office of the Public Auditor.

This project aimed to develop performance monitoring systems (PMS) to enable the two Supreme Audit Institutions (SAI) to monitor and report on their performance against their strategic plans. The purpose of a PMS is to enable a SAI to track the implementation of its strategic plan while also capturing the operational level key performance indicators (KPI) and establish targets, baselines and milestones for key activities linked to the achievement of outputs and outcomes.

PASAI and CAAF identified the following key learnings resulting from the project that apply to a vast range of legislative audit offices:

	Design and implement a robust multi-year strategic plan One of the central principles of a well-governed legislative audit office, as outlined in the INTOSAI-IDI Strategic Management Handbook for SAIs, is the importance of conceptual and operational consistency between the audit office strategic framework and the results framework (or program logic model), which provides a chain of results progressing from basic activities to ultimate outcomes. All the oversight and guiding documents of the audit office should use a consistent language both internally and externally and form the basis of the PMS.
	Start modestly and improve gradually Legislative audit offices often have already done considerable work to establish and conceptualize performance indicators. They can identify indicators that are cost-effective, practical and relevant to their goals and objectives. They don’t have to be perfect from the get-go. Once a baseline is established it could be expanded and eventual improvements of the indicators could be implemented.
	Leverage existing administrative databases Basic indicators such as number of reports, hours of training provided, etc., are often already collected and reported for operational reasons. They represent low-hanging fruits for legislative audit offices to use. Audit offices can review good reporting practices and identify potential KPIs for which they already collect information.
	Use surveys and questionnaires Surveys provide an opportunity to assess perception of various stakeholders receiving the services of legislative audit offices. Their stakeholders are usually audited organizations, legislators and the public. Each question of a survey of auditees or other stakeholders could be turned into an indicator for professionalism, transparency, efficiency or effectiveness. Although sometimes subjective and imperfect, they are cost-effective and generate new and fresh data going beyond mere administrative outputs. As a legislative audit office matures, these indicators can be displaced by more objective measurements or kept as complementary evidence.
	Use a results framework to develop new indicators as the legislative audit office matures The PMS should be based on a strategic plan and grounded on the outcomes, strategies, and activities outlined in a results framework developed for this purpose. A well-designed results framework establishes a hierarchy of steps to attain stated objectives, i.e., activities, outputs, immediate outcomes, intermediate outcomes, ultimate outcomes. Legislative audit offices can use such a tool to identify gaps and invest efforts into filling them by gradually defining more sophisticated KPIs requiring an increasing level of proficiency.
	Make leadership the keystone of a successful PMS Leadership is the essential ingredient for the successful implementation of a Performance Measurement System. Leaders of legislative audit offices can not only demonstrate their commitments by adopting a conceptually sound PMS but also assign resources for the data collection and commit to transparency of results through internal and external reporting. The PMS can therefore become a valid gauge of progress, or an early-warning system indicating that corrective actions must be taken.
	Be a model for the government A PMS functions not only as a management tool but also as a mechanism of accountability. By using its PMS, linking it to a strategic framework, measuring its results and reporting them, legislative audit offices can develop a “best-in-class” approach to governance and accountability. They can use the expertise they developed in their own organization to demonstrate the usefulness of this approach and to develop the know-how to assess government departments’ efforts to move in this direction.

Measuring their progress and achievements at the output and outcome level is a critical supporting element of legislative audit offices’ strategic results frameworks and a core part of the strategy formulation process. A PMS will allow legislative audit offices to continuously assess and compare whether operational realities still allow and support the realization of the strategic intent. This will help assess progress with the implementation of planned outputs and outcomes as specified in the results framework of the audit office.

These tips are based on an article written by Yves Genest, CAAF Vice-President, Research and Strategic Initiatives, for the International Journal of Government Auditing.

Liked it?

If you want to learn more about assessing the performance of an audit office see our Discussion Paper on The Impact of Performance Audits: Defining, Measuring and Reporting Impact.

Also don’t miss our Voices from the Field on Reporting on the Financial Impact of Performance Audits and our webinar on the same topic.

See more Audit Tips