Process mining challenges and limitations

Large external auditing firms have been experimenting with process mining for a few years. Some of them have recently begun to partner with process mining vendors to build platforms on which they can run their auditing activities and produce reports in a more automated way (Gartner, 2020). However, despite its benefits of increasing human capability to analyze data, process mining is not yet widely used for audits. There are human factors and data-related aspects that can explain the slow adoption of process mining in auditing in North America.

Unawareness may be one of the key human factors at play. Process mining emerged late in the United States market compared with other markets (Gartner, 2020). While North America is the largest market for business analysis, process mining technology is still relatively unknown in this market and not as popular as other data analytics technologies. Therefore, vendors, researchers, and users of process mining tools for audit purposes have remained restricted to a few countries (especially the Netherlands).

Another important human factor is the resistance of many auditors to using new techniques and technologies. This resistance can be explained to some extent by common factors that include individual personalities, a lack of understanding of the benefits of new technologies and a lack of required skills to use them, as well as an organizational culture that does not sufficiently emphasize innovation.

To counteract this resistance to change, IT advisers will need to convincingly explain the benefits that can arise from process mining (Fluxicon, 2014). Audit firms and public sector audit institutions also need to be aware of the need to expand their auditors’ skillsets to foster process mining adoption. Training staff and hiring professionals with data analysis and programming skills will be crucial to increasing the impact of process mining in the audit profession.

Regarding data, the main concern is data quality. Process mining possibilities are only as broad as the quality of the data the analysis is based on. If event data is missing or cannot be trusted, then most of the techniques described in this article are less valuable or cannot be used at all. Depending on the ERP system settings, event data may also not be recorded. For example, some ERP systems can be configured to disable logs or backups to increase software performance.

As with data quality, data availability can also limit the use of process mining. For example, opportunities to use process mining will be limited in organizations that rely heavily on manual controls and printed forms instead of IT systems.

Finally, besides the current challenges described above, there are some intrinsic limitations of process mining possibilities. First, process mining on its own cannot answer all the questions related to a specific process. For example, process mining techniques cannot determine if a key decision taken in a case—such as approving a contract or a project—was the correct one. Process mining can establish what activities took place, in what order and when, and who was involved. However, further information and human judgement may be needed to reach conclusions on key assessment questions.

Additionally, to fully understand some of the information provided by process mining on a given process and its variations, auditors might need to push the analysis further by using other means or by acquiring more contextual information. Unique variances in a process may be explained by particular circumstances in a business sector, such as weather conditions or heavy workloads at certain times of the month or the year. Consequently, the auditors’ knowledge about a process and their professional judgement are indispensable for the effective use of process mining.

Conclusion

The examination of public sector processes is a significant part of performance audits of government services and programs. However, in an environment where the amount of processed data and automation is constantly increasing, traditional audit procedures need to be complemented with new types of analysis. In these circumstances, process mining has emerged as a valuable analytical technique that allows auditors to:

• gain a clear view of actual processes at the planning phase of an audit,
• identify deviations from the described process,
• uncover potential weaknesses in controls and instances of fraud, and
• assess efficiency using key performance indicators.

Today, the use of process mining applications for audit purposes is mostly limited to audits of financial statements and internal audits, because major audit firms have developed capabilities in these fields for some time. However, as described in this article, process mining is suitable not only for standard business processes (such as simple procurement processes), but also for more complex and abstract processes, such as those that support the delivery of many important public sector programs and services.

While it is still a relatively new analytical approach, process mining is already used with success in a number of public sector audit institutions. As other audit offices continue their digital transformation, they would do well to consider adopting process mining too.

References

Alrefai, A. (2019). Audit Focused Process Mining: The Evolution of Process Mining and Internal Control. Dissertation, Graduate Program in Management, majoring in accounting information systems. Rutgers, The State University of New Jersey.

European Court of Auditors (ECA) (2019a). Special Report no 14/2019: “Have Your Say!”: Commission’s Public Consultations Engage Citizens, but Fall Short of Outreach Activities.

European Court of Auditors (ECA) (2019b). The Process Mining in Practice: Monitoring and Visualising Business Processes.

Fluxicon (2014). Process Mining Camp 2014: The Benefits of Process Mining in Auditing.

Fluxicon (2018). Process Mining in Practice.

Fluxicon (2020). Process Mining Camp 2020: A Journey Through the Use Cases for Audit.

Gartner (2020). Market Guide for Process Mining.

Jans, M., M. Alles, and M. Vasarhelyi (2013). The Case for Process Mining in Auditing: Sources of Value Added and Areas of Application. International Journal of Accounting Information Systems, Vol. 14, pp. 1–20.

Moggia, G. and Z. Varga (2020). Connecting Data and Processes in Audit: Some Considerations About the Use of Process Mining. ECA Journal, No. 1, pp. 68–73.

Office of the Auditor General of Canada (2017a). Spring Reports – Report 3 – Preventing Corruption in Immigration and Border Services.

Office of the Auditor General of Canada (2017b). Fall Reports – Report 2 – Call Centres: Canada Revenue Agency.

Office of the Auditor General of Canada (2019). Spring Reports – Report 1 – Call Centres.

Stefanini, A., D. Aloini, E. Benevento, R. Dulmin, and V. Mininno (2018). Performance Analysis in Emergency Departments: A Data-Driven Approach. Measuring Business Excellence, Vol. 22, No 2 pp. 130–145.

van der Aalst, W. et al. (August 2011). Process Mining Manifesto. In International Conference on Business Process Management (pp. 169–194). Springer, Berlin, Heidelberg.

van der Aalst, W. (2016). Process Mining: Data Science in Action. Second edition. Springer-Nature

Wikipedia (n.d.) Log File.