Introduction

Every year at the Toronto Auditor General’s Office, we prepare an annual report that explains the financial and non-financial benefits of our work. This report is an important tool to:

• communicate our office’s achievements,
• show the value the office delivers, and
• provide confidence to City Council and to the public that our performance audit recommendations are practical and are implemented to the benefit of taxpayers.

Our annual report highlights financial savings, which includes cost reductions, increased revenues, and avoidance of unnecessary costs, as well as non-financial benefits, such as improved internal controls, better customer service, and increased public safety.

Although many, if not all, audit offices report on the non-financial benefits arising from their recommendations, the focus of this article is on communicating financial impacts. It will shed light on why we prepare this annual report, the advantages and challenges of this work, and how we determine and quantify these savings.

Why we report on both financial and non-financial benefits

The Toronto Municipal Code requires our office to report annually on our activities, including savings achieved. Even though we are required to prepare this report, we welcome the opportunity to do it every year.

By quantifying our savings, we are able to express, in part, the office’s value in dollar terms. For example, from 2015 to 2019, the City achieved a return of $11.50 for every dollar invested in our office. As shown in Figure 1, the quantifiable financial benefits that were realized from implementing our audit recommendations in previous years significantly exceeded the costs to operate our office.

About the Authors

The City of Toronto’s independent Auditor General’s Office was created in May 2002, and since then has issued over 15 annual reports highlighting financial and non-financial benefits from its performance audits and investigations. Ina Chan, Assistant Auditor General (left), and Niroshani Movchovitch, Senior Audit Manager (right), are responsible for the annual report.

Ina and Niroshani have also delivered sessions on “The Secret to Identifying Cost Savings” and “Conducting Cost Savings Audits: Identifying Significant Savings Opportunities in Audits” for the Association of Local Government Auditors. They also contributed to CAAF’s April 2019 Discussion Paper “The Impact of Performance Audits: Defining, Measuring and Reporting Impact.”

Contact the authors at:

Ina.Chan@toronto.ca

Niroshani.Movchovitch@toronto.ca

Our annual report is also an important way to educate taxpayers on the non-financial benefits the City has achieved by implementing our recommendations. Non-financial benefits are just as important as financial benefits (Figure 2). Many of our audit reports result in non-financial benefits, and while they can be difficult to measure, they are important to include in our annual report. You cannot easily put a price on planting and maintaining a better tree canopy for the city, providing families with more timely access to subsidized housing, or protecting the City from the loss of sensitive and confidential data.

Our annual report presents our value in an easily digestible format, and we’ve been fortunate to have City Council recognize our value: our budget has grown from $4.7 million in 2015 to $6.6 million in 2019.

In addition to helping communicate our value, particularly when we have requested budget increases, our annual report encourages us to pay more attention to clearly articulating potential savings opportunities when reporting on the results of our audits.

Challenges in reporting financial benefits

There are some challenges when it comes to reporting financial benefits, especially in quantifying savings.

Time and resources

One of the biggest concerns raised by audit offices that do not currently determine the financial impacts of their performance audits is that to do so would take too much time and resources from their audit teams. Yet, most audit offices already have processes in place as part of their standard performance audit approach and follow-up audit procedures. In fact, performance audit standards require auditors to place their results in perspective to give readers a sense of the consequences of the findings. Quantifying these results in terms of dollar value is one way to meet these audit requirements.

Dependence on post-audit action to achieve savings

In order to realize the impacts or benefits of any audit, the auditee needs to implement the recommendations. To help in this regard, we describe the potential financial impacts of our findings directly in our audit reports in order to help explain to City Council and the auditee the importance of promptly implementing recommendations. Fostering open communication about the impact of our findings, why recommendations are important, and the dollars at stake, throughout and after the audit, leads to effective implementation of recommendations and the achievement of financial impacts.

Continuously tracking previous audits

Our office has four full-time staff dedicated to following up on the implementation of prior audit recommendations¹. Many audit offices already have recommendation follow-up processes in place and issue some form of report on the follow-up results. Quantifying realized savings is just taking these existing processes one step further by obtaining financial information at the same time as collecting evidence to show that recommendations have been implemented. Whether you have a dedicated follow-up team that monitors savings achieved or whether you incorporate steps into your existing follow-up audit processes, the end result will be the same.

How it works

It’s important to establish a process to quantify an audit’s impact and to use it consistently. Over time, this makes it easier to gather information about audits that have resulted in realized savings. Our approach identifies one-time benefits, any benefits that will continue into the future², and potential benefits that may arise if an audit recommendation is implemented.

No audit is the same, so the methods to determine savings will vary. But our office uses several techniques to quantify financial benefits and potential savings, including:

• being aware of common audit themes that result in financial benefits,
• stating the potential financial impact of recommendations in audit reports, and
• having procedures to calculate realized savings when following up on audit recommendations.

Being aware of what results in financial benefits

It’s important to note that we do not select our audits based on their potential to have high-dollar savings. Audit projects included in our annual work plan are identified through a City-wide risk prioritization process³, reviews of emerging issues, and analyses of trends in allegations made to our Fraud and Waste Hotline.

Still, every audit should be looked at as an opportunity to identify increased revenues, reduced operating costs, cost avoidance, or new revenue streams. Savings are a natural by-product of any performance audit that looks at efficiency, economy, and effectiveness. When conducting audits, our staff keep the potential for savings on their radar, from audit planning all the way through the examination and reporting phases.

Some common audit themes that tend to result in savings include identifying opportunities to:

• consolidate administrative functions or contracts,
• improve contract management,
• improve warranty management,
• implement alternative service delivery models, and
• better leverage information technology and data.

In planning our audits, staff are mindful of designing procedures (such as data analytics, detailed contract reviews, and benchmarking) that can help identify and quantify any financial benefits. The ability to easily spot potential financial benefits comes with experience, sharing knowledge between colleagues and other performance audit offices, and questioning the status quo.

¹ Currently, there are 698 open recommendations in our follow-up process. Since the inception of our follow-up process in 1999 (which continued when the office became independent in 2002), about 88% of recommendations have been fully implemented.

² For the purposes of quantifying savings that will continue into the future, in general, the Toronto Auditor General’s Office determines the current-year annual savings and projects this value over a five-year horizon. If there are significant changes within the five-year horizon, we would make upward or downward adjustments to previously reported savings.

³ Factors considered in audit selection include complexity of operations; alignment of strategic planning; organizational competence; financial, contractual, and legal exposure; adequacy of controls; technology exposure; and susceptibility to fraud.

Page 1 of 2