Area Summary: Procurement Processes – Oversight Mechanisms and Expertise

Relevant Audits

Audit Office	Report Title (click on title to access summary)	Publication Date
Victorian Auditor General’s Office	Planning, Delivery, and Benefits Realisation of Major Asset Investment: The Gateway Review Process	May 2013
Australian National Audit Office	The Administration of the Gateway Review Process	February 2012
National Audit Office - UK	Commercial Skills for Complex Government Projects	November 2009

Examples of Audit Objectives

• To examine the effectiveness of the administration of the Gateway Review Process (GRP) by responsible entities. (View report summary)
• To determine the extent to which those Gateway reviews that have been conducted have contributed to improvements in the delivery of major projects undertaken by responsible agencies. (View report summary)
• To examine the current level of commercial skills and experience in Government, and whether departments are being successful in improving them. (View report summary)

Examples of Audit Criteria

• The Gateway Unit has appropriate procedures and guidance in place to effectively manage the administration of Gateway across the Government; (View report summary)
• The application of the thresholds for inclusion in Gateway is sufficient to ensure that all major projects are subject to review as intended; (View report summary)
• Gateway review teams are adequately skilled and reviews are carried out in accordance with relevant guidance; (View report summary)
• Agencies have procedures in place to ensure project team compliance with Gateway requirements; and (View report summary)
• Agencies respond in a timely manner to the findings and recommendations of Gateway reviews. (View report summary)

Examples of Evidence Gathering and Analysis Techniques

File Review

• Review of relevant files, records and information including all Gateway reports issued between January 2007 and December 2012. (View report summary)
• Review of evidence from departments and agencies on responses to Gateway reviews of sampled projects. (View report summary)

Testimonial Evidence

• Interviews with:
  • Officers from relevant departments and agencies.
  • A sample of Gateway review team leaders and team members.
  • Managers of the Gateway Review Process in other Australian and international jurisdictions. (View report summary)
• Techniques used to identify commercial skills gaps in central government, and the barriers to addressing them:
  • A focus group with seven Senior Responsible Owners of complex projects.
  • Semi-structured interviews with 14 senior executives of major private sector contractors, and four consultancies.
  • Discussion forums with the Confederation of British Industry and the Major Projects Association.
  • A survey of all departments’ commercial directors or heads of procurement, followed by a focus group with six of the 16 directors and three in-depth interviews. (View report summary)

Analysis

• Review of empirical data on commercial staff salaries to calculate the difference in salary between different government departments and the public and private sector. (View report summary)

Examples of Findings

• The GRP is a valuable concept capable of assisting better performance in project delivery. The Department provides high-quality materials to guide participation in the GRP and has effective processes in place to select, engage and train Gateway reviewers. (View report summary)
• However, the implementation of the GRP in Victoria reflects a number of missed opportunities. The audit identified 62 projects valued at $4.3 billion that were not included in the GRP between 2005 and 2012. (View report summary)
• The Department cannot demonstrate that over that period it actively and consistently identified projects that may have been candidates for the GRP. This meant the process was largely applied on an opt in decision by agencies and not all high-risk projects were subject to review. As a result, the fundamental objective for the GRP—to improve the management and delivery of significant projects—has not been fully met. (View report summary)
• Further missed opportunities resulted from projects commencing the GRP, only to drop out of the process after completing a few Gates. No single project has completed the full suite of Gateway reviews since its introduction in 2003. The benefits from applying the GRP have not been fully realised. (View report summary)
• The Department has not measured the impact of the GRP on projects. It has not tracked agency action taken on Gateway recommendations, which is a fundamental success indicator for the GRP, and so cannot demonstrate whether the GRP has resulted in any benefit to individual projects. (View report summary)
• The Department has missed opportunities to use the GRP to build public sector project management and review capability. It has not done enough to capture and share lessons learned from Gateway reviews and the participation of public sector staff as Gateway reviewers is very low. (View report summary)                    
• Overall, the Gateway review process has been effectively implemented within the Government. There has been a focus on high risk, high value projects with 46 projects valued at more than $17 billion across 23 agencies examined in the first five years of Gateway’s application. In the first three years, about one in every five reviews identified that there were significant issues that needed to be addressed before the project proceeded further. In the last two years, there have been no reviews that have identified major issues requiring urgent action. (View report summary)
• However, participation in the Gateway review process does not guarantee success in meeting specified project objectives. At least three of the nine projects that have completed the full suite of Gateway reviews were not completed on‐time and on-budget and/or did not deliver the outcomes expected when funding was approved. (View report summary)
• While the processes used to determine which projects are to be subject to Gateway reviews were generally effective, several projects that met the criteria were not subject to Gateway reviews due to the timing of their risk assessments. (View report summary)
• The contribution that Gateway makes to improved project delivery depends, to a significant extent, on agencies promptly progressing the issues raised in reviews. However, it is common for agencies to not fully implement review recommendations in a timely manner. (View report summary)
• To date, there has been significant reliance on private sector reviewers, with targets for participation by public sector staff not being met. In addition, to date, only one review has been led by a member of the public sector, and there has been a high degree of reliance on a small number of private sector participants to lead individual reviews. (View report summary)
• Departments continue to experience a shortage of staff with the necessary commercial skills and experience to successfully deliver complex projects. (View report summary)
• Government departments have attempted to fill this commercial skills gap with interim staff and advisers. Whilst both interims and advisers can make a valuable contribution, particularly those with highly specialist skills, an over reliance on them can lead to: higher project staff costs; departmental staff failing to take proper responsibility for commercial decisions; and a loss of commercial knowledge when the interims or advisers leave. (View report summary)
• Departments have significant weaknesses in a number of the commercial skills critical to the delivery of complex projects. The Office of Government Commerce’s reviews found that commercial skills were generally weak across all 16 central government departments. (View report summary)

Examples of Recommendations

The Department of Treasury and Finance should:

• Systematically validate whether projects should be subject to Gateway review, by verifying that robust project risk assessments are completed for new projects. (View report summary)
• Strengthen Gateway Review Process quality assurance processes. (View report summary)
• Track and report on the impact of the Gateway Review Process on improving the outcomes of completed projects. (View report summary)
• Actively monitor agency action in response to Gateway review recommendations. (View report summary)
• Complete the database for sharing lessons learned from Gateway reviews and build case studies to better demonstrate key lessons. (View report summary)
• To provide assurance that the Gateway review process is being applied to all relevant projects, the Department should periodically examine the outcomes of those projects excluded from Gateway on the basis of their assessed level of inherent risk. (View report summary)
• The Department should examine the merits of conducting annual Gateway reviews for projects where there would otherwise be an extended delay between reviews. (View report summary)
• Departments should:
  • put in place project assurance processes that will identify commercial skills gaps in individual project teams; and
  • produce an analysis of the commercial skills required across their future complex project procurements, and identify the contract management skills that are required to prevent value for money being eroded during the delivery phase of complex projects. (View report summary)