Research Highlights

Introduction

Artificial intelligence (AI) is an intriguing emerging technology. Rapidly adopted by the IT industry, it has found myriad applications in our economy and now appears to be ubiquitous. Its capacity to automate processes and extend the reach of the human mind is already causing profound changes in a number of fields, including auditing.

This article explores how AI is revolutionizing the audit profession by expanding the range, efficiency, and extent of audit evidence collection and analysis. It describes the main applications of AI to auditing and discusses the opportunities and challenges that AI presents for public sector auditors.

What is AI?

Artificial intelligence is surrounded by a fair amount of hype. Like all technological trends, it generates considerable enthusiasm and skepticism. Yet AI is far from a passing science-fiction fad and has solid academic roots. The term “AI” was coined in 1955 by a professor of computer science at Dartmouth College, John McCarthy, to describe what was already a vibrant and ambitious multidisciplinary field of research. In his words, the study of AI proceeded “on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”¹ He further speculated that machines would use language, form abstractions and concepts, and solve problems reserved for humans. These prophetic musings sound eerily true to 21st-century readers.

AI’s fate and success have waxed and waned through a number of cycles since the 1950s. AI has, through these ups and downs, been used in a number of ways, from appliances to Internet search engines. In the last few years it seems to be undergoing a breathtaking explosion. This “AI Revolution” has been fuelled by quantum leaps in:

• processing power;
• machine-readable data; and
• the sophistication, quality, and quantity of AI algorithms.

Every day, announcements are made about the use of AI in new areas. For example:

• self-driving cars
• online search engines
• customized online purchases
• product customization by media companies
• government weather forecasts
• virtual assistants on personal devices.

A lot of these AI-driven innovations are now familiar to many consumers and are becoming run-of-the-mill technologies in various industries. But a number of them are still in their infancy and very far from being proven, mainstream technologies.

As seen in Figure 1, AI capabilities have diversified in recent years and can now be applied to various tasks. This diversification has been made possible by exploiting:

• exponentially growing datasets (such as financial statements, client databases, digital images, sound recordings, and text documents);
• innovative algorithms that can make logical inferences and understand human language; and
• “smart” equipment (robotics) that can learn autonomously.

Figure 1 – Types of Artificial Intelligence

Source: Modified from Romanini, F. (2018, March 15). “Is Artificial Intelligence and Machine Learning the Future of Financial Regulation?” Available at: https://www.linkedin.com/pulse/artificial-intelligence-machine-learning-future-franco-romanini-/

Use of AI in Auditing

The audit profession has started to find many ways to integrate AI in its operations. Some authors have provided exhaustive inventories of potential uses of AI by auditors.² In this article, we will focus on three areas that are fuelling the most promising technologies for auditors: image recognition, speech recognition, and machine learning.

Image recognition is possible because AI can process video and digital images mathematically and identify patterns emerging from the data. Bar code scanning and Apple’s face recognition technology on laptops and devices are well-known applications of this technology. In its more advanced form, image recognition can automate routine audit procedures. For example, it can do inventory counts using a video camera attached to drones, identify and locate items, and even report on their condition. Such video analytics can be performed very efficiently, reviewing hours of recordings in a few minutes. Performance auditors can also use this technology. For example, satellite imagery has been used by the European Court of Auditors to facilitate checks on, for instance, crop classification and the agricultural activity taking place on land parcels.³ This capacity to process images is changing the way auditors implement audit procedures, such as observations and inspections.⁴ A detailed overview of this technology, together with a case study of its implementation, can be found here.

Speech recognition, another subfield of AI, aims to leverage streamed or recorded audio such as voice mails, phone calls, conference calls, interviews, management presentations, and recording of management or board of directors’ meetings. Speech recognition software “understands” and analyzes grammatical and semantical language structures, taking into account accents, errors, and background noise. Speech recognition applications are widely used in mobile telephony, video games, and home automation, to mention a few. A promising audit application is the ability to accurately transcribe verbal recordings of interviews and testimonies, thus efficiently generating reliable transcripts of conference calls or interviews.⁵ More importantly, this technology can understand and analyze the language used by the interviewees, speakers, or participants, to infer emotional states, attempts at deception, or uncertainty of answers.⁶ A good overview of the literature on this topic is available here.

Machine learning is an advanced form of AI. It is the engine that can leverage the other subsets of AI mentioned above and it conforms most closely with the popular concept of a machine replacing humans. Machine learning uses algorithmic models to perform complex data analysis on large datasets without specific instructions. It is programmed to iteratively process the data and build mathematical models, in order to identify patterns and make predictions.⁷ In its most advanced version, known as deep learning, AI is operated by neural networks emulating the human brain and can help make decisions and diagnostics.⁸ Applications using this technology are emerging in the audit profession. For example, software that can read complex contracts,⁹ identify key terms, and correlate with transactions could identify patterns and instances of non-compliance. An example of a commercially available audit solution is AI Audit by MindBridge,¹⁰ which assesses risks of fraud, identifies data anomalies, and examines full transactions, looking at the monetary flows between accounts. Also, software like the Auditors Deep Learning Dashboard by SageTea¹¹ and AuditMap by Lemay.ai¹² aim to analyze risks and controls, and generally help the planning and conduct of audits.

Opportunities and Challenges for Auditors

Before AI becomes a staple of the audit toolbox, auditors will be made aware of it through its presence in multiple operations of the organizations they audit. This will demand that audits take AI into account, or even provide assurance on its functioning. Therefore, as organizations deploy AI, auditors will be asked to assess the risks and vulnerabilities of the technologies themselves.¹³

Auditing AI technologies presents an array of challenges for auditors.

• Because AI is relatively new, there are few accepted standards or regulations that apply to it.
• AI does not operate on a set of predetermined code and programmed rules, making it difficult to understand or predict from an audit standpoint.
• There is precious little audit precedence and established methodologies to rely on.
• There are considerable variations in AI technologies (see Figure 1) and their implementations.
• Most organizations outsource their AI solutions, resulting in a dispersion of the corporate memory on their implementation that causes a “black box” effect that can make audit trails tenuous.¹⁴

However, efforts to provide audit frameworks are starting to emerge.¹⁵

To audit and use AI, auditors will need the technical knowledge and tools to assess this technology and its impact on the economy, efficiency, and effectiveness of auditees. Therefore, auditors will have to hone and perfect skills they already have while acquiring new ones.¹⁶ The most relevant skills needed are:

• Understanding the strengths and limitations of AI technology. Having staff with a practical knowledge of “what is under the hood” of AI codes will be a great asset to understand the ramification of AI applications on an auditee’s operations.
  
  
• Understanding and integrating data. Big data is the lifeblood of AI. Therefore, this will magnify the need for the audit skills required to perform data analytics.
  
  
• Performing reasoned analysis. This skill is more valuable than ever. This seems counter-intuitive, because “smart” software can make decisions, but in fact the human capacity to sift intuitively will be highlighted by the gradual automation of mechanical tasks.

Objectivity is paramount for auditors. It is the very foundation of auditing standards and of auditors’ credibility. Therefore, concerns about AI biases are both a risk to auditees and a constraint on the use of AI in audits. At the end of the day, AI is based on code and is bound to reflect and even amplify the biases of coders. There are many examples of AI biases, such as software powered by AI used in Florida to assess inmates’ risks of recidivism.¹⁷ It was found that with this software, black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism. Another example is an algorithm used to score New York City landlords on how well they take care of their properties, which was shown by an audit to be adversely impacted by biases in the inspection reports on which it relied.¹⁸

These biases are magnified by the limitations of the neural network technology underpinning deep learning. Some AI programs produce decisions and actions that are fundamentally unexplained and unexplainable. For example, a research group at Mount Sinai Hospital in New York created a deep learning application named Deep Patient to analyze a vast database of 700,000 individuals. It proved very effective at predicting health issues. For instance, it was remarkably effective at anticipating the onset of schizophrenia. However, when the researchers tried to identify how Deep Patient was piecing data together to reach its conclusions, they could not. This made it impossible to understand the rationale of the diagnostic and therefore the justification for a specific treatment or a change in medication.¹⁹ Interestingly, this phenomenon, frequently called the “black box” effect, has been recognized by the audit profession.²⁰ Efforts are ongoing to try to identify, trace, and document the decision-making process of AI using deep learning. The ability to document the logic behind a conclusion reached by an AI application would be of great value to performance auditors, who must support their conclusions with sufficient and appropriate evidence.

AI shines especially well when undertaking tedious tasks. The audit profession entails a number of “low-hanging fruits” in this respect: contract reviews, inventories, compliance reviews of transactions, and so on. AI can do all that better than humans. It is faster, efficient, and accurate, and can often review 100% of transactions.²¹ Some studies predicted that up to 40% of transaction accounting work would be automated or eliminated by 2020.²² Humans can also be surpassed by AI’s ability to find patterns that would be undetectable otherwise, which is particularly suited to detecting fraud or conducting root-cause analysis. The effort to develop self-driving cars has also led to the development of ethical algorithms that can make decisions based on an ethical framework.²³ This creates the possibility of reviewing minutes of meetings or reports and other corporate documents to identify ethical breaches.

Although the automation of tasks requiring professional judgment is still imperfect, the automation of tedious tasks is underway. This can enhance and extend the role of humans, not eliminate it.²⁴ Humans are still better than computers at telling the story behind the data and at making qualitative judgments that add value for auditees.

Conclusion

The roots of artificial intelligence in computer science go too far and its modern applications are too important for it to be a passing fad. It is already fundamentally changing our society and our economy. Despite its challenges, it will fundamentally change the way audits are conducted. This article barely scratched the surface of this complex technology but hopefully provided useful suggestions of where to dig for more knowledge of AI. Like in many other fields, AI will not replace professionals, but if properly harnessed, it will enhance their capacity to deliver quality products and services.