News

The week one participants, who became the week two facilitators.

We made Foundation history on 3 March 2019 at the Rydges Capital Hotel in Canberra, Australia. It was the kickoff to an intensive two-week training program led by the Canadian Audit and Accountability Foundation (CAAF) for members of the Australasian Council of Auditors General (ACAG). The program was the result of an innovative approach to course design and delivery and a new model of collaboration between CAAF and ACAG. Assembled in the room were CAAF President and CEO John Reed and Associates Laurie Rose and Neil Maxwell, together with 15 senior audit practitioners from the legislative audit offices of New Zealand, Papua New Guinea, Fiji, Queensland, Victoria, Western Australia, South Australia, Tasmania, Northern Territory, Australian Capital Territory, and the Australian National Audit Office (federal).

It was a one-of-a-kind offering, tailor-made for ACAG members, and something we had never done before: Delivering our Leading Successful Audits (formerly Advanced Performance Audit) and Evidence Collection and Analysis training courses back-to-back two weeks in a row while at the same time teaching week one participants how to facilitate the courses on their own, in week two, and later on their own turf.

A Year in the Making

The event was about a year in the making and was part of John’s vision to build a stronger relationship between ACAG and the Foundation. John had attended ACAG’s IMPACT (International Meeting of Performance Audit Critical Thinkers) conference in Sydney in March 2018, where he struck up many conversations with auditors general and senior audit officials of the member audit offices. This included a pivotal meeting with Lisa Rauter (Group Executive Director, Performance Audit Services Group, Australian NAO), Mike Scott of the NAO New Zealand, and the members of ACAG’s Heads of Performance Audit (HOPA) group (chaired by Mike). They were interested in the Foundation’s performance audit training and wondered whether we could help meet their needs.

As Lisa recalls, “We wanted to build on our existing training for new staff with a course that would focus on the unique aspects of leading a performance audit, teaching more advanced skills to an experienced audit group. We were seeking to recognize performance audit as a career path and a specialty—similar to financial audit.”

Through our back-and-forth discussions with Mike and Lisa, we ultimately landed on delivery of two of our highly acclaimed training courses, Advanced Performance Audit and Evidence Collection and Analysis. We could have delivered them “as is.” But in the spirit of continuous improvement, and to bring best-in-class training products to our Canadian members as well, we decided to embark on a renewal of both courses. Neil led this part of the process, working closely with Yves Genest, Julien Raynaud, and Marc Bélanger at the CAAF, along with staff at Canadian audit offices, and of course Mike and Lisa.

The biggest changes were to the Advanced Performance Audit course. As Neil recalls, the design process began when John challenged the team with the question, “What exactly would we teach as the secrets to leading a successful audit?” It was an intriguing question to Neil, knowing that John wanted to remodel the Advanced Performance Audit course and rename it Leading Successful Audits.

It was a surprising question, too, Neil recalls. “The Advanced course was only a few years old and was one of the most popular and well-received courses in the CAAF curriculum. So why tinker with success? But once we focused on the challenges of leading a successful audit, we realized we had much more to teach audit leaders, especially newly minted ones.”

Why Tinker with Success?

So rather than tinkering with success, we decided to build on and improve it. We started with a blank sheet and a simple exercise: Identifying all the elements of an audit that constitute “success.” We reviewed competency models, brainstormed among ourselves, talked with Canadian practitioners, and consulted with Mike and Lisa. The list of elements and responsibilities we developed was long and daunting. No wonder it is so hard to lead a successful audit!

Looking at the list, we noticed an interesting grouping. They all fit into one of three broad categories. Some were about the content of the audit (for example, adding value for users), some about the process (for example, meeting standards), and others about people (for example, maintaining a healthy team). These groupings were a new insight for the design team. The image of a three-legged stool became the heart of our course design, reinforcing our belief that to realize a successful audit, the leader must pay attention to all three legs of the stool, keeping the seat balanced, while simultaneously managing:

• people to develop strong relationships,
• process to focus on audit quality, and
• content to ensure rigorous thinking.

Soon after the course renewal process got underway, a second design challenge emerged.

As Mike recalls, “Given the distance between Canada and Australia, we were looking for more than a one-off delivery. We wanted to be able to leverage this knowledge across our offices, in a cost-effective way, as an ongoing collaboration in the ACAG group.”

In discussion with Mike and Lisa, we decided to integrate our train-the-trainer programming into the course design and delivery. The idea was simple: CAAF would train a cadre of performance audit practitioners from week one so they could deliver the courses in week two (with CAAF support), and then deliver them later in their respective jurisdictions. Our goal shifted from delivering training to inspiring a sustainable training capacity among ACAG members. That’s where Laurie came in.

Laurie recalls a mix of emotions when first presented with the project. “Designing a new course is always fun and exciting but the plan to do a train-the-trainer week one with delivery week two was a small leap of faith. Having audit experience is one thing—delivering training is completely different. It would be a challenge to blend two different skill sets, but I was up for it.”

Laurie’s design integrated the train-the-trainer dimension in two ways. First, we modified delivery of each course in week one, moving quickly through content that was familiar to the group and spending more time on less familiar areas. This allowed the group to experience the course as a participant while freeing up time to continuously debrief the why and how of delivery from a train-the-trainer perspective. Second, we designed three intensive train-the-trainer modules—focusing on managing the classroom, participants, and themselves—that were to be delivered outside of the regular classroom hours.

We also realized that not only did we have to teach great facilitation techniques, we had to model them ourselves. “There is a temptation to think that subject matter expertise alone carries the day. It doesn’t. This can be hard for first-time trainers to understand,” explains Laurie. “Making the shift from being the ‘sage on the stage’ to the ‘guide on the side’ is a key part of being an effective facilitator.”

While challenging from a design perspective, integrating train-the-trainer into the courses also yielded many benefits. One of these was unintended. As John recalls, “The content renewal and train-the-trainer integration we were going through was incredibly creative and innovative. As we discussed, reflected, and learned, we realized there were even more teaching techniques that we could design into these and our other courses, even more we could do to foster the learning and engagement of participants, and even more we could do to distinguish the CAAF ‘brand.’ We basically threw everything we knew about facilitation at these courses.”

Maintaining participant engagement is another key facilitation skill, according to Laurie. Variety is key. This means that at any particular part of our courses, participants might be

• working in small groups on assigned exercises;
• discussing in triads how they will apply the previous module with their teams;
• engaged in a debate where each half of the class argues opposite sides of a difficult challenge;
• working in teams to devise, present, and defend an audit scope proposal, based on an actual audit their office plans to undertake;
• conducting a self-diagnostic on how to improve their audit scope proposal; and
• reflecting introspectively in their personal journal or devising an action plan.

Going Live

Facilitators and participants during week two.

Lesson plan in hand, John, Laurie, and Neil headed to Australia to see how it would survive first contact. They were pleased to find a group of experienced auditors waiting for them, many with training experience.

Rona Mellor (Deputy Auditor General, Australian NAO) opened the training session, noting “This is an exciting moment for all of us. The ability to work in partnership with CAAF to adapt their course to a train-the-trainer model provides all audit offices in attendance with an ongoing capability that we plan to continue building, with CAAF’s guidance.”

By the end of week one, participants had their teaching assignments for week two and a full complement of notes and slides to prepare with.

Week two was a blur, but when it was over, the participants provided very positive feedback on the value of the course and the multi-jurisdictional learning environment. We heard feedback like “it was enriching, it helped you identify your own strengths and weaknesses” and “the course was well pitched at new audit managers—though any audit manager would benefit from the course.” Others found both the stool and the “guide on the side” analogies really useful, helping them to focus their attention on what mattered most.

And what did our hosts think at the end? “The CAAF trainers offered excellent insights into performance auditing not available through existing training opportunities in Australia,” says Lisa. “The investment in targeted performance audit training enabled us to recognize the unique skills needed by performance auditors and managers and build on these, learning not only from the CAAF trainers, but also from colleagues in the room from other jurisdictions. This, coupled with Laurie’s expertise in course facilitation, left participants with confidence, new skills, and a new network of people they can work with, to deliver this training to their colleagues.”

Click here for more information on our training courses