Featured Audits

OAG Alberta – Human Services – Systems to Manage the Assured Income for the Severely Handicapped Program – October 2016

Featured Audit

What is this audit about?

The Assured Income for the Severely Handicapped (AISH) program provides almost $1 billion a year in benefits to help about 50,000 Albertans with disabilities support themselves and their families.

The audit by the Office of the Auditor General of Alberta focused on the accessibility of services offered under the AISH program and on the efficiency of the systems to manage the program.

At the time of the audit, the program was run by the Department of Human Services. Following a reorganization by the government in 2017, the AISH program is now managed by the Department of Community and Social Services. It is the second largest program delivered by Community and Social Services, being administered by approximately 330 staff and having annual operating costs of $33 million.

What did the audit find?

The audit findings fell into three areas:

Accessibility:

Access to the AISH program through the existing intake process was complex. The audit found that applicants were facing several accessibility barriers such as an onerous application form, inadequate pre-screening, and an inefficient triaging process. Further, the intake process was not supported by user-friendly guidance. The website was difficult to navigate, and key information was hard to locate.

Eligibility:

The audit found that the Department of Human Services did not have standards to monitor the timelines of the application process. The audit team calculated that, on average, 203 days elapsed between receipt of the initial application form and commencement of the applicant on the program. The audit also noted that AISH staff had to use considerable judgment in their assessment of applications but received inadequate training and guidance on making assessments.

Reporting:

The audit found that the department had inadequate performance measures and processes to monitor and report on the efficiency of the AISH program.

What difference did the audit make?

The report received substantial media coverage, drawing attention to improvements needed to better serve the Albertans who need the program and to ensure the $1 billion in annual expenditures is both well managed and achieving the objectives of the program. As a result, it garnered a lot of attention from AISH recipients, stakeholders, and the general public, and the office received a number of letters from the public in reaction to its report.

The department agreed with the audit findings and accepted the recommendations, and it indicated its intention to improve AISH.

The audit recommended that the Department should:

ensure its application processes are user-friendly;
set service standards for application processing times and regularly monitor its performance against these standards;
improve procedures and guidelines to ensure that staff apply policy in a consistent manner; and
improve its processes to measure, monitor, and report on the efficiency of the AISH program.

In April 2017, the Department of Community and Social Services produced an AISH Action Plan responding to the Auditor General’s report, accepting all three recommendations and outlining the Department’s planned actions to make the program more responsive, accessible, and client-focused. According to the Department, as of March 2018 many of these actions have already been implemented, including the launch of a new AISH website, the introduction of a new appeals form, and a modification to its staff training programs. The Department also asserts it is making efforts to improve processes and service standards, as well as to monitor and report on the efficiency of the program and the appeals process.

OAG Alberta plans to conduct a follow-up audit next year to assess whether the recommendations have been implemented.

What can we learn from this audit?

Working with administrative data requires vigilance on the part of the audit team

Assessing how long it took AISH staff to process applications was a key aspect of the audit. As the audit team attempted to calculate the average application processing time by stage, it discovered that over time some of the dates entered in the system were changing. AISH staff were able to modify key dates in the system, so the team could not rely on those dates.

In order to accurately calculate the average application processing time by stage, the audit team needed to obtain source data that was captured within the system but not on the reports. The audit team tested this data to ensure it was complete, relevant, and reliable. This data was used for all testing throughout the audit.

When something seems off, you have to do a deep dive to find out why

An important factor to determine an applicant’s eligibility for the program was an assessment of “earned livelihood”. An applicant who declared being employed 30 hours a week or more could be denied benefits under the AISH policy.

When the audit team reviewed the Department’s policy supporting the legislation, it noticed a discrepancy in the interpretation of “earned livelihood”. The legislation itself did not specify in the definition of “earned livelihood” the number of hours an applicant could work. Rather, the legislation set a formula to reduce AISH financial assistance based on earned income. There was no mention of hours, and the current funding formula still provided AISH financial assistance when income of 30 hours at Alberta minimum wage was calculated into the formula. Thus, the 30 hours was an arbitrary number applied by the Department.

In testing how the program was managed, the audit team confirmed that AISH beneficiaries (those already receiving benefits under the program) were allowed (and even encouraged) to earn a livelihood working 30 hours a week. Therefore, applicants and beneficiaries of the program were treated differently in this respect.

Relying on internal audit reports could be useful

Internal audit reports prepared by the Department’s Internal Audit group provided important insights into the eligibility process, notably by identifying a number of compliance errors. In order for the OAG team to be satisfied that it could rely on Internal Audit’s work, it had to satisfy itself that the internal audit staff conducting the work were qualified and that their work was adequately documented. The OAG team also tested Internal Audit’s results for accuracy.

Critical findings require an ongoing dialogue with all levels of program management

Performance auditors recognize that avoiding surprising the auditee is important for maintaining trust. This trust ensures that auditors are able to obtain the evidence they need while enabling the goodwill that will facilitate the implementation of audit recommendations. This audit was no different. The team maintained ongoing dialogue with various levels of management to ensure transparency and to allow opportunities for clarification. When misunderstandings arose, the team asked for clarification and ensured that they were talking to the right people, facilitating the flow of information between middle-level management and senior management. Ongoing clearance with Department officials and “writing the report as you go” also constitute good practices to proactively prevent any misunderstandings.

Using internal subject matter experts can be beneficial

The Office of the Auditor General of Alberta has an internal specialist group, the Performance Reporting Group, which the team used for the reporting component of this performance audit. This group leads performance audits in the areas of performance measurement and results analysis reporting and provides support to other audit teams in these areas. By using the specialist group, the rest of the audit team was able to focus on the accessibility and eligibility aspects of the audit.

Well-designed tables, graphs, and non-numeric graphics can improve the clarity and readability of published reports

When information is presented visually, it can be easier for the reader to understand essential information, especially numeric or process-type information. Well-designed graphics can also give a more professional look to the publication. Examples in the AISH audit report are the “Average Application Processing Timeline” and the “Continuous Improvement Cycle” graphics.

Leveraging Foundation tools can be a great help

To develop its approach for the AISH audit and provide background in the report on factors affecting application processes, the OAG Alberta audit team used two Foundation resources: the Practice Guide on Auditing Efficiency and the companion guide Auditing the Efficiency of Application Processes for Government Programs or Licences.

Foundation members can obtain these resources here for free.

Non-members can acquire them here.