Introduction

In recent years, auditors have increasingly been using new technologies to uncover hidden relationships among diverse types of information, thus providing more insight and value to their clients and reducing the audit risk of issuing incorrect opinions and conclusions.

One innovative technology that has come to the attention of business leaders and audit professionals is process mining. Process mining allows auditors to visualize and understand an organization’s critical processes (such as procurement processes) as they actually operate, as opposed to how they were designed to function. This makes it a powerful technology for conducting risk, compliance, and performance assessments. By using this tool, performance auditors can gain a better understanding of where process weaknesses and inefficiencies exist. They can formulate targeted recommendations that will help public sector organizations in redesigning and optimizing their business operations.

This article presents an overview of process mining and explains how this technology can be used in planning and conducting performance audits.

Process mining essentials

What is process mining?

Process mining is a relatively new tool that sits between machine learning and data mining on the one hand and process modelling and analysis on the other hand. It is used to discover, monitor, and improve real processes—as they take place every day, not as they were meant to operate by design—by extracting knowledge from event logs readily available in today’s IT systems (van der Aalst, 2016). Event logs record events taking place in the execution of an IT system in order to provide an audit trail that can be used to understand a system’s activity and to diagnose problems. They are essential to understand the activities of complex systems, particularly applications with little user interaction (Wikipedia, n.d.).

In other words, process mining connects data and processes in an entirely new way. It allows auditors to visualize and analyze a vast amount of data, even over 100 million records, either statically or dynamically—such as a flowchart or an animation of the cases flowing through the discovered process model—about the operation of a specific process from its beginning to its end.

Another advantage of process mining is that it makes it possible to examine an entire population of transactions or cases, thus replacing sampling and providing increased audit assurance.

About the Author

Ricardo Rocha works as an auditor at the Court of Accounts of the State of Bahia in Brazil. He has over eight years of experience working in public sector external and internal audit. During those years, he has conducted compliance and performance audits and has examined claims of fraud and public funds misuse.

Ricardo combines a graduate academic background in law and an interest in data science. He has a practical knowledge of data analytics and business intelligence and a solid understanding of the public sector’s regulatory environment. He is currently involved in the development of an automated tool to detect errors or fraud attempts in public procurement procedures.

Contact the author at:

ricardo.franca.rocha@gmail.com

Furthermore, process mining allows users to analyze meta-data, which is data automatically recorded in IT systems for every data entry. Meta-data contains information about the circumstances under which the data was entered, such as who did the input, when, and every change made after the original entry. This additional information allows auditors to reconstruct the history of a transaction and to identify relationships between that transaction and all other transactions in the database that share parameters with it. This information can reveal anomalous patterns that may indicate fraudulent or corrupt activities (Jans et al., 2013).

How to use process mining in auditing

The first step in using process mining is data preparation, which involves the extraction of event data from an audited organization’s information systems, either a central enterprise resource planning (ERP) system or a program-specific system. These event data are called event logs and they are stored in many forms, such as database tables, message logs, emails, and transaction logs, to name a few. However, for process mining to be feasible, a minimal data standard is required. For each event, the event log needs to identify the following:

• the process instance (case ID),
• the activity that was executed (e.g., opening new case, review, or approval); and
• when it was implemented (time-stamp).

Event logs are aggregations of cases in a single file; an example is in Figure 1. In this example, the event log includes the key information (Case ID, Timestamp, Activity) as well as additional data attributes and context information (Medium, Service Line, Urgency). This additional data can serve as useful input for some process mining techniques. (For instance, a social network analysis could be performed if the data included the name of the person executing each activity.)

After obtaining the event log, it is possible to apply process mining techniques. This article describes three process mining techniques that can be used in performance audits: process discovery, conformance checking, and performance analysis. The decision to use one or several of these techniques will be based on the audit’s objective.

Process discovery

The first technique is called process discovery. In process discovery, an algorithm scans the time-stamps in an event log to create sequences of events and then produces a process model without using any other prior information. In other words, the algorithm discovers the actual process and then describes it in a visual format (van der Aalst et al., 2011).

Because process discovery’s output is a description of the actual process and not of the one initially designed by the audited organization, it most likely will include deviations from expectations; that is, activities executed in reverse order, missing or redundant activities, and uncommon or unique activities in some cases. As a result, a complex process model created by process mining may be challenging to read, as shown in Figure 2.

Nonetheless, it is possible to simplify visualization and focus only on the most frequent activities to describe how a process is really executed in an audited organization. Alternatively, it can be more interesting to visualize details only for certain paths or deviations.

The process discovery phase of process mining has significant importance in auditing because it can help auditors to conduct risk assessments and to identify potential material weaknesses in controls (Alrefai, 2019). By giving auditors a comprehensive view of a given process, process discovery can help auditors to better understand the business operations they are interested in and to target their audit plans toward the higher risk areas.

Two other process mining techniques, conformance checking and performance analysis, are better suited to the examination phase of an audit. Deciding which technique to use (either one or both) will depend on the audit objective.

Conformance checking

Conformance checking is used to verify if process activities, as recorded in event logs, conform to the process model designed by the audited organization. The process model used for this purpose can be either discovered or constructed manually. This type of analysis is mostly used for compliance purposes. However, auditors can also apply it to find process inefficiencies, such as bottlenecks or unnecessary steps, or desirable deviations (positive variations of the process not foreseen by the organization), and recommend changes to refine the process. In addition, conformance checking can help auditors to identify non-compliance instances that reveal a risk of fraud or corruption (e.g., non-compliance with segregation of duty rules).

Performance analysis

Through performance analysis, the performance of a process can be evaluated along three different dimensions: time, cost, and quality. Auditors can use key performance indicators (KPIs) for each of these three dimensions and gain insight by analyzing the relevant data in event logs (van der Aalst, 2016).

When evaluating time, auditors can define indicators for the average completion time of a process or for minimal and maximal time taken to process a case. Alternatively, auditors may want to examine the time taken to perform particular activities, such as making an important decision in the process or evaluating the time a client or service user needs to wait before being attended to. For example, in one study based on real hospital data, time-related KPIs were used to measure the time taken to provide treatment and the time patients had to wait to be transferred between departments of the hospital. These KPIs were then compared with the hospital’s predefined targets (Stefanini et al., 2018).

Performance analysis can also provide information that auditors may use to examine the cost of executing a particular activity or the allocation of resources over a given period (assuming cost data is captured in the event log). In the hospital case described above, KPIs for resource utilization (for example, the number of specialist consultations required in a medical case) could also be analyzed to assess process costs.

Finally, the quality dimension of a process or its output can also be the focus of process mining. This is possible when data on customer satisfaction is available (for example, the number of complaints made by a service’s users). Customer satisfaction information provided by service users on websites or social media could be collected in an event log by using an application programming interface, a software intermediary that allows two applications to talk to each other and exchange data.

Summary: Process mining techniques

To summarize, process mining offers an extensive toolkit, elements of which can be used at different stages of a performance audit to align with the audit’s objective. The textbox below presents an overview of process mining techniques and their use in performance audits.

Page 1 of 3