• Cart
Log in

Log in

home page banner blank


Practice Guide to Auditing Oversight


Selecting Audit Criteria

Audit criteria represent the standards expected to be met by an audited organization. Audit criteria are a key contributor to an audit’s strength and potential impact. Audit procedures focus on determining whether criteria are met or not met. Suitable criteria are clear, concise, relevant, reliable, neutral, understandable, and complete.

Finding suitable criteria is a challenge for any performance (value-for-money) audit, not just for audits of oversight. Each audit is unique due to the auditor’s mandate, audit focus, audit objectives, and the way the organization being audited approaches the audit’s subject matter. However, the governing bodies of agencies, boards and authorities usually share many organizational and operational aspects and many studies have been published on board governance. As a result, guidance already exists about the audit criteria that can be used to audit oversight in Crown corporations or agencies. The criteria presented as examples in this section are largely derived from the work of the Canadian Council of Legislative Auditors (CCOLA) Governance Study Group and the Office of the Auditor General of Canada.

Examples of audit criteria and sub-criteria that can be used to audit oversight structures/systems and their results/effectiveness in agencies, boards and authorities are presented in Table 8. The criteria and sub-criteria are divided into 11 categories:

  1. Oversight roles and responsibilities
  2. Independence
  3. Skills and knowledge
  4. Sufficient and appropriate information
  5. Risk management
  6. Performance monitoring
  7. Compliance
  8. Corrective actions
  9. External reporting
  10. Performance assessment
  11. Government oversight

These categories correspond to the audit objective topic numbers 2 to 12 in Table 7. Objective topic 1 in Table 7, the overall oversight framework, is very broad and would need, in practice, to be supported by a selection of criteria taken from these 11 sub-categories.

Auditors are not expected to use all of the suggested criteria. Rather, they can pick and choose those that are most relevant to the scope of the audit and document the rationale for their selection. They can also develop additional criteria where needed, in order to conclude on their audit objective(s).

Auditors should always use their professional judgment to select audit criteria and to determine whether the expectations defined by the criteria are reasonable given the nature and operational constraints of the audited organization. The reasonableness of potential criteria is, in part, a function of the degree to which they represent a balance between cost, risk, and effectiveness. For example, it would not be reasonable to expect an organization to adopt an unproven, costly control measure to mitigate a minor risk.

Table 8 – Examples of Audit Criteria that Can Be Used to Audit the Oversight of Agencies, Boards and Authorities

Topic

Structures and Systems

Results and Effectiveness

1. Oversight roles and responsibilities

Criterion: The oversight body and its committees have clearly defined oversight roles and responsibilities.

Sub-criteria:

  • The oversight body has clearly defined oversight roles, responsibilities, and authorities.
  • Each committee of the oversight body has terms of reference that clearly define its areas of responsibility and level of authority, and that have been approved by the board.
  • The roles and responsibilities of the audit committee, set down in its terms of reference, include:
    • maintenance of an effective internal/external audit function,
    • maintenance of a suitable risk management and internal control framework,
    • meeting frequency and core agenda items,
    • committee authority, and
    • reporting to the oversight body.

Criterion: The oversight body and its committees fulfill their assigned oversight roles and responsibilities.

2. Independence

Criterion: The oversight body and its committees have established systems and procedures to ensure that members have, and can demonstrate, the independence necessary to perform their oversight responsibilities objectively.

Sub-criteria:

  • The oversight body has established clear policy and guidance about independence requirements. Specific prohibitions are listed and guidance covers the various forms of independence threats (self review, self-interest, advocacy, familiarity, and intimidation) and how they are to be addressed.
  • Oversight body members have to sign an annual independence declaration that requires them to disclose any known independence threats and confirm their understanding of the organization’s independence policy.

Criterion:

  • Members of the oversight body and its committees comply with applicable independence policies.
  • Independent oversight body members hold regular in camera meetings without management in attendance.
  • The internal audit function reports to the oversight body or its audit committee, and its independence from management is supported by the oversight body.

3. Skills and knowledge

Criterion: The skills, knowledge and experience required of oversight body members have been identified and communicated.

Sub-criteria:

  • The oversight body has profiled the skills and knowledge required of individual directors and for the oversight body as a whole to ensure effective oversight of the corporation. The oversight body has shared this profile with the responsible minister.
  • An orientation program has been developed to provide all new oversight body members with information on:
    • the roles and responsibilities of the oversight body and its committees;
    • the organization’s mandate, vision, mission, and strategic plan;
    • the organization’s compliance regime; and
    • the organization’s accountability framework.

Criterion: Oversight body members have the skills, knowledge and experience they require to effectively discharge their oversight responsibilities.

Sub-criteria:

  • The skills and knowledge of oversight body members are aligned with those described in the oversight body profile.
  • The oversight body has access to and uses outside expertise when necessary to fill gaps in its skills and expertise profile.
  • Committee members have the qualifications, skills, and competencies necessary to effectively fulfill the committee’s role and responsibilities, as defined in its terms of reference.
  • All oversight body members receive sufficient, appropriate training and guidance to provide them with a working knowledge of their corporation and the environment within which it operates.

4. Sufficient and appropriate information

Criteria:

The oversight body has defined the information and knowledge it needs from management (on performance, compliance, risk management, financial management, etc.) to effectively exercise its oversight role and communicated these needs to management.

The oversight body has established a process to periodically review the quality and quantity of information it receives from management and external sources.

Criterion: The oversight body and its committees have sufficient relevant and reliable information to fulfill their oversight responsibilities.

Sub-criteria:

  • The oversight body ensures that it receives sufficient and appropriate information on a timely basis to support oversight body decision making overall.
  • The oversight body ensures that it receives appropriate (credible, complete, timely) financial, performance, and risk information to allow it to:
    • fully assess the corporation’s performance at regular intervals;
    • ensure that pertinent legislation, regulations, corporate bylaws, and board policies are being complied with; and
    • ensure that key risks are being adequately managed.
  • Where additional information is required to make an assessment or a decision, the oversight body requests such information from management and/or external sources, and ensures that it is obtained on a timely basis. The oversight body defers decisions when appropriate information has not yet been received.
  • Periodically, the oversight body looks critically at the quality and quantity of information it receives from management and external sources to ensure that this information allows the oversight body to effectively discharge its oversight responsibilities.

5. Risk management

Criterion: The oversight body has established a risk management policy framework for the organization.

Criterion: The oversight body and its committees effectively overseethe organization’s risk management policies and processes.

Sub-criteria:

  • The oversight body understands the organization’s key risks.
  • The oversight body reviews and challenges management’s plans on how to avoid, control, accept, or transfer key risks to the organization before approving them.
  • The oversight body monitors the organization’s implementation of risk management policies, processes and internal controls to ensure they are working as intended.

6. Performance monitoring

Criteria:

The oversight body has established a Performance Management Framework for the organization.

Performance targets and pertinent indicators are in place to enable the oversight body to properly monitor the organization’s performance.

Criterion: The oversight body is effectively monitoring the organization’s performance in relation to its mandate and stated objectives.

Sub-criteria:

  • The oversight body regularly monitors organizational and management performance and challenges management about the quality and reliability of the available performance information.
  • The oversight body regularly monitors and evaluates the CEO’s performance and takes appropriate action where that performance is judged to be below expectations.

7. Compliance

Criterion: Systems and practices are in place to monitor the compliance of the organization with enabling legislation, regulations, bylaws, and oversight body policies.

Criteria:

The oversight body obtains assurance that enabling legislation, regulations, bylaws, and board policies are being complied with.

The oversight body ensures that the organization’s code of conduct is communicated to all staff, that compliance with its requirements is monitored, and that action is taken when deviations are identified.

8. Taking corrective actions

Criterion: The oversight body has put in place adequate controls to ensure that corrective actions are taken in a timely manner (to address performance or compliance issues, weak risk management or financial management practices, etc.).

Criterion: Evidence exists that, based on the information they receive, oversight body members make decisions, provide direction, and follow up on actions taken in response.

9. External reporting

Criterion: The oversight body has determined which accountability reports it needs to receive, review and approve.

Criteria:

The oversight body and its committees regularly review and approve key accountability reports.

The audit committee provides an adequate challenge and review of financial statements and the associated management discussion and analysis, and of any other financial information and performance information to be released by the organization, before their release.

10. Performance Assessment

Criteria:

The oversight body has adopted a policy that requires it to periodically assess its performance.

A process is in place to periodically assess the performance of the oversight body and its committees in discharging their oversight responsibilities.

Criterion: The performance of the oversight body and its committees in discharging their oversight responsibilities is assessed periodically.

Sub-criteria:

  • The collective performance of the oversight body, its committees, and individual members is self-assessed periodically, and an appropriately transparent mechanism is used in reporting the assessment results.
  • The oversight body complies with the corporation’s values and ethics.
  • The oversight body and its committees hold a sufficient number of meetings each year to fulfill their roles and responsibilities.
  • The oversight body and its committees keep adequate meeting minutes and supporting documentation.
  • The oversight body works well as a team and has effective decision-making processes in place.

11. Government oversight

Criterion: The government has defined and communicated its expectations with regard to the organization’s performance and reporting thereof.

Sub-criteria:

  • Government provides a letter of expectations or similar document annually to the overseen organization that specifies expected performance for the year, including the targets that government will use in evaluating its performance.
  • Government clearly communicates the performance reporting it requires from the overseen organization in order to evaluate its performance.
  • The conditions under which the overseen organization should consult government for direction are clearly documented.

Criterion:

The government exercises adequate oversight of the organization.

Government takes, and follows up on, corrective actions when significant issues in the overseen organization are brought to its attention.

Source: These criteria and sub-criteria have been modified from the CCOLA Governance Study Group’s Crown Agency Governance: Audit Objectives & Criteria and from the Office of the Auditor General of Canada’s Recommended General Criteria & Sub-Criteria (for special examinations of Crown corporations).