• Cart
Log in

Log in

home page banner blank


Practice Guide to Auditing Efficiency


Sample Criteria with a Systems Emphasis

Criteria in Table 5 are taken from the general criteria presented in the Practice Guide that can be used as part of a systems audit and are supplemented with criteria specific to the efficiency of the inspection and enforcement function. (Additions to the general criteria in the Practice Guide are shown in Table 5 in italics.) Auditors may also choose to add criteria for the four support functions that contribute to the achievement of efficiency.

An appropriate set of criteria should be selected based on specific audit objectives. For example, if the audit objective is number 1 in Table 3 (normative objective focusing on systems for achieving efficiency), criteria related to all seven management activities in Table 5 would likely be relevant. However, if the audit objective is number 2 in Table 3 (normative objective focusing on systems for measuring and reporting efficiency), criteria relating to a subset of management activities would likely be selected (for example, commitment and tone from the top, IT systems, performance monitoring and reporting, and continuous improvement and innovation). In this case, the remaining management activities (strategic planning, operational planning, and project and operations management) would be less relevant.

Similarly, for objective number 3 in Table 3 (normative objective focusing on implementation of recognized efficiency improvement framework), the criteria would need to relate specifically to the selected framework and good practices for its implementation, and the systems criteria listed in Table 5 would only play a minor role.

Click on any of the following seven management activities to view eamples of criteria:

  1. Commitment and tone from the top
  2. Strategic planning
  3. Operational planning
  4. Project and operations management
  5. IT systems
  6. Performance monitoring and reporting
  7. Continuous improvement and innovation

Table 5 - Examples of Criteria for Audits of an Application Process with a Systems Emphasis

Management Activity

Criteria for Audits with a Systems Emphasis

1. Commitment and tone from the top

Efficiency as a priority. Senior management has emphasized that efficiency is a priority for the application process and that achieving it is a shared responsibility of all managers and staff.

Clear expectations. Senior management has clearly established and communicated expected results in terms of efficiency, complete with targets and indicators for the application process.

Established baselines. The organization has established clear baselines for the cost, quality, and level of service of each of its main application process activities.

Policies. Clear guidance is provided in the operational procedures on the steps that must be taken to process an application or issue a licence and who has decision-making authority.

Risk management. Risk management policies and procedures are fully documented and endorsed by the head of the organization.

 

Renewal period. Efficiency was considered when the time period for periodic reapplication or renewal of licences was established. 

2. Strategic planning

Strategic planning. The organization has a strategic planning process that identifies organizational inefficiencies and prioritizes efficiency improvement initiatives for the application process.

Risk assessment. The organization has assessed the risks and potential consequences of maintaining application processoperations identified as inefficient.

Assessing opportunities. The organization continually identifies and evaluates opportunities to improve efficiency of the application process.

Program design. Key program design features and related administrative processes are commensurate with the scale, nature, complexity, and risks involved in the application process.

Collaboration and partnerships. Effective consultation and a constructive and cooperative relationship between the administrators, program beneficiaries, and other relevant stakeholders contribute to achieving more efficient, effective, and equitable administration of the program.

Risk-smart culture. The organization should have the capacity to assess risks and act on opportunities to innovate, to simplify the application process, and to improve performance.

3. Operational planning

Operational planning. The organization’s systems and practices to allocate financial, human, and material resources to its application process projects and operations are designed to increase operational efficiency.

Service levels.The organization has adopted application process service level standards that are used by operational planners to identify, budget for, and allocate required inputs.

Input costs. The organization identifies and analyzes the input costs for all its majorapplication process services and programs.

Unit costs. The organization calculates the unit cost of delivering its main application process services and tracks how the unit costs change over time.

Cost variation. The organization has a clear understanding of how costs change in response to changing levels of application process activity.

Comparable financial information. The organization continually compiles relevant financial information on the application-processing and licensing functions and produces information that is comparable over time.

Personnel allocation. The organization’s systems and practices to allocate its personnel to its various application processservices or business units are designed to increase efficiency.

Allocation. Human resources are allocated based on priority to the most important application process risk areas.

Qualified personnel. Application process operations are designed and carried out by qualified personnel with clear roles and responsibilities.

Accountability. Roles, responsibilities, authority, and accountability for efficiency matters in the application process are clearly defined, attributed, and communicated. 

4. Project and operations management

Due regard to efficiency. The organization’s project and operations management controls, operational systems, and work processes demonstrate due regard to efficiency.

Operating systems and procedures. The organization’s service delivery operations are designed and carried out using efficient systems, processes, and procedures.

Utilization of production capacity. The organization optimizes the available production capacity, facilities, equipment, and employees to produce targeted volumes of goods and services.

Guidance for applicants. Clear and comprehensive guidance is provided to assist applicants in preparing and submitting quality applications.

Application form design.

  • The application form design directs the applicant to supply required information.
  • The information obtained provides sufficient, reliable evidence to determine whether the applicant satisfies eligibility criteria set out in the program guidelines or legislation.
  • Applications comply with security and privacy requirements.

 

Timely decisions. Decisions on an entity’s (or individual’s) application or licence are made in a timely manner and communicated promptly to the applicant.

 

Appeal process. There are mechanisms to facilitate timely discussion, and where possible resolution, of a disagreement regarding a decision.

5. Information technology (IT) systems

IT systems. The organization periodically assesses opportunities to use IT technologies to improve the efficiency of its activities and services.

Information. The IT system provides appropriate information to decision makers, at the right time and at a reasonable cost to the organization, so that application or licensing decisions can be made efficiently.

 

Documentation policies. Guidance is provided to decision makers on the information that must be compiled, retained, and stored in support of an application or licensing decision.

 

Information retrieval. Guidance is provided on how to prepare and assemble documentation so that, in the event of an enquiry about a decision, information retrieval costs are minimized.

6. Performance monitoring and reporting

Performance monitoring. The organization continually monitors the performance of its main activities and services using reliable indicators of efficiency.

        

Quality and level of service monitoring. The organization continually monitors the quality and level of service achieved for each of the main application process services it delivers.

Benchmarking.

  • The organization regularly benchmarks the main application process services it delivers in order to assess their relative efficiency and identify areas for improvement.
  • The entity uses comparative data from similar application processes to understand the likely range of costs and areas of focus for obtaining greater value for money.

Reporting on efficiency initiatives. The organization periodically reports on progress against its efficiency objectives and initiatives.

  • The reports include relevant, timely, reliable, and complete information on efficiency achievements.
  • The reports include information on the efficiency gains that have been achieved from individual projects and on how these gains have improved the services delivered by the organization.

Reporting efficiency savings. In reporting efficiency savings, the organization:

  • Reports consistently over time, using valid and reliable measures and indicators.
  • Compares current values against baseline data.
  • Explains how efficiency savings are affecting cost, quality, and level of service, to show the full impact of changes.
  • Is transparent about the upfront investments and recurrent costs incurred in delivering efficiency.

7. Continuous improvement and innovation

Improving existing methods of operations. The organization continually assesses the feasibility of streamlining its systems and procedures, optimizing the allocation of its resources, and eliminating duplication and waste.

Innovation. The organization periodically identifies and assesses innovative ideas for improving the efficiency of its key activities and services.

Service delivery alternatives. The organization periodically identifies and assesses the merits of alternative service delivery methods and models that could increase its efficiency.

Efficiency through collaboration. The organization periodically assesses the merit and feasibility of increasing efficiency through new or improved collaborative arrangements (such as pooling of resources, removal of duplication, and use of shared services, particularly with similar organizations within the government or public sector).

Continuous improvement process. The organization has implemented a continuous improvement process to review and improve its service delivery systems and practices.

Source: Many of these criteria were adapted from the Office of the Auditor General of Canada’s Auditing of Efficiency (1995), the Northern Ireland Audit Office’s Improving Public Sector Efficiency: Good Practice Checklist for Public Bodies (2010), the Australian National Audit Office’s Implementing Better Practice Grants Administration – Better Practice Guide (2013), as well as from recent audits of efficiency.