Featured Audits

What is this audit about?

The audit examined the emergency preparedness and response activities under the responsibility of Ottawa Public Health (OPH). While the focus was on OPH, the audit was planned and conducted in the context of the Emergency Management Program for Ottawa as a whole.

An emergency management program manages and coordinates prevention, mitigation, preparedness, response, and recovery activities for all hazards in a jurisdiction. The program includes all organizations, agencies, departments, and individuals responsible for emergency management and security for that jurisdiction. Emergency management in Ottawa involves a large number of agencies at all levels of government, as well as numerous non-governmental organizations. While various agencies involved have specific and unique requirements in emergency management, initiatives must be well coordinated and cannot be developed in isolation from the City of Ottawa’s Emergency Plan.

The audit’s overall objective was to assess OPH’s emergency management and response activities. The audit was to assess processes used to develop, monitor, and report on emergency management.

What did the audit find?

As part of the preliminary phase of audit work, the audit office identified eight risks. These risks were related to the work of the City’s Emergency Management Program’s Steering Committee and Advisory Committee, Security & Emergency Management branch, and OPH. The risk areas reviewed were:

• clarity of governance, oversight, and direction
• compliance with legislation
• clarity of roles, responsibilities, and delegations
• identification of emergency situations
• formal emergency management plans
• coordination between OPH, the Emergency Management Program, the Advisory Committee, and Security & Emergency Management branch and external partners
• preparedness of participants for emergency response, including monitoring and evaluation of performance and provision of training
• preparedness of OPH to maintain public health services to residents while responding to and recovering from emergencies that disrupt normal operations

Audit Team

Ken Hughes, Auditor General, City of Ottawa

The audit project was supervised by Sonia Brennan, Deputy Auditor General, with Ted Pender and Dana Mersich from Samson and Associates, and Margaret Purdy of Margaret Purdy Consulting Inc.

The audit findings were elaborated on in each of these risk areas. The audit concluded that the residual risk levels of all areas were low. As a result, the audit report stated that there would be little value in conducting additional audit work related to the OPH Emergency Management Program.

What difference did the audit make?

The audit provided extra assurance to the City Council and management about the emergency preparedness and response for health services. The initial performance of the City of Ottawa during the COVID-19 crisis appears to attest that it was ready to address a pandemic.

What can we learn from this audit?

An accreditation process can provide a valid basis for reliance

In 2016, the City of Ottawa obtained accreditation of its disaster preparedness program (which includes pandemic and health crisis) through the Emergency Management Accreditation Program (EMAP). EMAP, an independent non-profit organization, fosters excellence and accountability in emergency management and homeland security programs by establishing credible standards applied in a peer review accreditation process. It has accredited the emergency management programs of numerous states, counties, municipalities, and major universities in the United States. Ottawa is the only municipality in Canada to have obtained this accreditation.

The accreditation is based on EMAP’s ANSI/EMAP 4-2016 Emergency Management Standard. It is a tool for continuous improvement as part of the voluntary accreditation process for local, state, federal, higher education, and tribal emergency management programs. These programs that apply for EMAP accreditation are evaluated against a set of 64 standards. The accreditation process is as follows.

• The program performs a self-assessment.
• A team of specialists conducts an on-site review to verify information in the program’s application and documentation materials.
• The EMAP program review committee reviews the documentation and conclusions of this team.
• The EMAP program review committee produces a report that includes conclusions about compliance and recommendations on accreditation.

When the City of Ottawa audit team started planning the audit, they became aware of this EMAP process and what it entailed. They started considering if they could rely on it. They turned to the CSAE 3001 and The Institute of Internal Auditors standards (2050-3), which recognize reliance. These standards both state that external organizations could be valid sources of evidence for auditors, providing that they meet certain criteria. The following is a summary of the criteria for relying on the work of third parties (note that this summary does not reflect the precise language of the standards):

• This information is relevant to the purpose of the audit. The information relied on should be directly related to the audit topic, especially in terms of the risks and operational elements of the activities examined.
• The level of competence of the provider of information. Staff conducting the work relied on should have the appropriate level of expertise as attested by their education, professional certifications, and experience. They understand the risks they examine and the nature of the controls implemented to mitigate them.
• The process generating this information is objective. The organization providing the information relied on is impartial and independent and perceived as such. Factors like reputation and overall acknowledgements of the organization in its field of expertise should be considered.
• The process generating this information is rigorous. It should be based on clearly articulated policies and procedures, backed by quality control procedures, and results and conclusions should be based on sufficient and appropriate evidence.

When the audit team became aware that the OPH had sought and obtained the EMAP, it realized that the audit could be redundant given the extent of work done to obtain this accreditation. The team also saw that it could use this accreditation as information that could be relied on. As part of the planning phase, the audit team verified and documented the EMAP’s relevance, competence, objectivity, and rigour, and concluded that it could be relied on.

The reliance on the accreditation process allowed for an efficient audit process

The audit was able to leverage what amounted to a substantial amount of work (interviews, and review and analysis of documentation) that had been done to obtain this accreditation. This allowed the audit to reduce considerably the resources required for the audit examination and reporting phases. It also prevented the “audit fatigue” that could have resulted from an audit that would have replicated, to a large extent, the accreditation process that had been conducted just a year earlier.

Having a subject matter expert on the team can greatly strengthen an audit

The team had access to a former federal deputy minister with 28 years of diverse experience in Canada’s emergency management, national security, and intelligence community. During her public service career, she undertook many significant emergency assignments. This subject matter expert’s experience related to emergency management was a crucial asset for the review and analysis of key aspects of the OPH Emergency Management Program. Her opinions were explicitly referred to throughout the report to support and clarify the audit conclusions.