While it’s easy to see how a pipeline analogy fits an activity that distributes supplies, this analogy can also be applied to a wide range of results audits, as detailed in Table 1.
Table 1 – Using the Pipeline Analogy in Different Types of Results Audits
|
Audit Focus |
Inputs, Outputs, and Leaks |
|
Projects |
The pipeline in audits of projects should produce completed projects on time, on budget, and meeting identified needs. If not (and the late Auditor General of Canada Michael Ferguson would often say that he had never found one that met all three), where in the pipeline do results fall short and what controls failed? Typical culprits are inadequate needs assessments, poor business cases, failure to identify risks, and insufficient attention to testing. |
|
Support payments |
In audits of support payments, the pipeline should provide all eligible applicants with accurate and timely payments that comply with court orders. If some payments do not meet these criteria, leaks may be caused by decision delays, incorrect eligibility assessments, or inaccurate calculations. |
|
Services |
In audits of services, the pipeline should produce a timely service that meets quality standards. If some or all clients receive unsatisfactory services, where are the main leaks? Leaks may be specific to certain regions or client groups (gender inequality being one possible cause), or across-the-board problems such as delays, incorrect eligibility assessments, or delivery problems. |
|
Economy |
Audits of economy often focus on procurement or human resources compensation, where outputs from the pipeline should meet the required specifications at the least cost. If not, where in the pipeline is due regard for economy lacking? Typical leaks are time delays, overly narrow procurement competitions, and poor definition of the required resources. |
|
Efficiency |
The pipeline in audits of efficiency should produce results efficiently based on predetermined or benchmarked standards. If not, where in the pipeline are the efficiency leaks? For example, benchmarking internally (comparing regions, for example) or externally (comparing similar jurisdictions) can identify how costs per output vary and why some locations are less efficient than others. Key leaks might be antiquated processes, inadequate training or equipment, or overstaffing. |
|
Effectiveness |
Audits of effectiveness involve a pipeline where results meet the program’s objective, with all the intended outcomes achieved without significant unintended negative consequences. If some or all of the program outcomes fall short, where do the problems originate? |
|
Environment |
Environmental audits share characteristics with other types above, as they may be environmental projects, financial support or services, and the audit objectives may cover economy, efficiency and effectiveness. Results may involve meeting international environmental commitments or enhancing sustainability. |
Using the Model for Root Cause Analysis
Many of the examples above also show how this pipeline model can be a simple way of identifying and reporting root causes. That should not be a surprise, because causal relations are just results relations in reverse—for example, if parts shortages result in unavailable equipment, then the cause of unavailable equipment is parts shortages.
Another recent audit (not yet published) used the pipeline model to look for root causes in a hazardous waste audit. In this case, inputs start with national policies; flow through activities that communicate standards and provide training at national, regional, and facility levels; and then flow through implementation at each facility. If everything worked well, all hazardous waste would be treated so that human health and the environment were protected. Laying out the components of the results pipeline and their flow helps auditors to have a good understanding of the processes and controls involved (as required by the applicable Canadian audit standard CSAE 3001), which in turns facilitates the identification of “leaks” and their root cause.
How to Apply the Pipeline Model to an Audit of Results and Root Causes
The standard steps in audit planning, examination, and reporting are listed below, with suggestions on how to apply the pipeline model in bold.
- Gain the required knowledge of the audited activity—always the starting point of good audit planning.
- Identify the most significant aspects of the subject, including the inputs to the audited activity, the key sub-activities, and the main intended results. Try depicting these in a simple pipeline graphic.
- Conduct a risk assessment to identify the results issues with greatest inherent likelihood and impact, considering economy, efficiency, and effectiveness.
- Gather indications of actual results. What shortfalls seem to be occurring? Look at possible root causes. Show on the pipeline where results are likely leaking, and which management controls seem to be failing. This allows you to refine your risk assessment to reflect residual risk (if your organization focuses scoping on residual rather than inherent risk).
- Determine the audit scope based on the risk assessment, along with considerations of auditability and value added. Develop the audit objective and criteria to use.
- Identify evidence collection and analysis strategies that will provide sufficient appropriate evidence on results and the management control weaknesses that cause results shortfalls.
- Conduct the examination by applying the evidence collection and analysis strategy to gather sufficient appropriate evidence to assess each audit criterion.
- Draft the report incorporating the significant findings. Try telling the story using the pipeline analogy—what results are lacking, where were the leaks, and which management controls are at fault?
Conclusion
Every performance audit is complex, especially those that examine results and root causes. Faced with complexity, the auditor’s best strategy is to keep it simple. I’ve found this simple analogy of a pipeline to be a powerful way to scope performance audits, and an effective way to report their findings. I hope these tips help!
Page 2 of 2
DISCLAIMER: The opinions expressed in this article are those of the author and do not necessarily reflect the views of the Foundation.
See more Voices from the Field
- 1
- 2
LinkedIn
Twitter