Comparing Audit and Evaluation

… a core concept used in Implementation and Delivery and Atlas107

Click for pdf of 2014 handout

Concept description

Although the internal audit and evaluation functions within an organization are similar in some respects there are crucial differences in purpose and methodology.

The “Compare Anything” website Diffen, provides comparative perspectives on its entry for Audit vs. Evaluation (reference below), including the differences in definition:

“An Audit is the evaluation of a person, organization, project, product or primarily for the purpose of determining its validity and authenticity, or to verify adherence to a set of pre-defined processes.

“On the other hand, an evaluation is the determination of merit using a set of standards. Though they are both a kind of assessment, audits are mainly done for financial institutions to make sure they are free from misstatements and errors, whereas evaluations can be done for different spheres such as healthcare, arts, government agencies to judge the efficiency of a working system. Earlier, audits were an important way to assess financial systems only. Now they are also done to assess security risks, environmental and other systems performance as well.”

Comparing internal audit and evaluation in the Government of Canada

The following table sets out the following differences in the way the internal audit and the evaluation functions operate in the Canadian federal government. This table incorporates the refinements to Treasury Board of Canada definition and the methodology that have taken place since the 2014 document in the pdf above as a result of the replacement of the Policy on Evaluation with the Policy on Results.

Attribute Internal Audit Evaluation
Treasury Board of Canada Definition Internal auditing in the Government of Canada is a professional, independent and objective appraisal function that uses a disciplined, evidence-based approach to assess and improve the effectiveness of risk management, control and governance processes. (Policy on Internal Audit, 3.1) In the Government of Canada, evaluation is the systematic and neutral collection and analysis of evidence to judge merit, worth or value. … Evaluations typically focus on programs, policies and priorities and examine questions related to relevance, effectiveness and efficiency. … Evaluations generally employ social science research methods. (Policy Results, Appendix)
Common Meaning (Merriam-Webster Online Dictionary) Audit 1a) a formal examination of an organization’s or individual’s accounts or financial situation 2) a methodical examination and review Synonyms: inspection, check, checkup, examination, going-over, look-see, review, scan, scrutiny, survey, view Evaluation 1) to determine or fix the value of 2) to determine the significance, worth, or condition of usually by careful appraisal and study Synonyms: appraise, assess, estimate, guesstimate, rate, set, valuate, value
Varieties Internal Audit (c.f., External Audit by Auditor General), Management Practice Audit, Recipient Audit, Forensic Audit, Financial Audit, Compliance Audit, Performance Audit Interim (Formative) Evaluation, Impact (Summative) Evaluation
Goals (and tension) 1) Accountability; 2)Improvement 1) Improvement; 2) Accountability
2 Ps and 3 Es Prudence, Probity, Economy Efficiency, Effectiveness
Bad Findings Noncompliance, Irregularity, Misuse, Abuse, Illegality, Fraud Inefficiency, Waste
Potential Consequences Political Embarrassment, Dismissal, Imprisonment Reassignment, Program Change
Universe All Auditable Units (e.g., 50 Program AUs; 30 Internal Services AUs) All Programs (preferably aligned with the Program Auditable Units)
Selecting Entities Risk-based, but goal of covering all audit entities 5-year cycle for all grants and contributions programs of more than $5 million annual expenditure
Enthusiasm of Entities Low to Very Low (with exceptions) Neutral to Low (with exceptions)
Methodology Application of the Internal Auditing Standards for the Government of Canada (IIA Standards plus additional points). See also Directive on Internal Auditing in the Government of Canada. Application of the Directive on Results, particularly the Mandatory Procedures for Evaluation and the Standard on Evaluation.
Methodological Tools Audit Scope, Audit Criteria, Interviews, Documentation Review, Sample Testing Evaluation Scope, Logic Model, Literature Review (including Best Practices), Document and File Review, Key Informant Interviews, Case Studies
Performance Measurement Is a performance measurement system in place? What do the performance measures tell us and how can they be improved?
Report Length 20-40 pages 50-200 pages
Formal Client Deputy Minister Deputy Minister
Committee Audit Committee (external members, including Chair) Evaluation Committee (DM Chair, sometimes with external advisors)
Report Output Audit Recommendations Evaluation Recommendations
Response Management Response and Action Plan Management Response and Action Plan
Publication Audit Report and Management Response Evaluation Report and Management Response
Parliamentary and Media Interest Occasionally intense (e.g., Internal audit of HRDC grants and contributions in 2000) Low
Cost $50,000 to $150,000 $50,000 to $200,000
Number per Year 8-12 8-12
Unit Budget $3-6 million per year; 12-25 FTEs $3-6 million; 12-25 FTEs
Central Agency Office of the Comptroller General (Internal Audit Sector) Treasury Board Secretariat, Centre of Expertise for Evaluation
Academic Preparation Accounting, Public Administration Public Policy, Public Administration, Economics, Sociology, Statistics
Professional Societies Institute of Internal Auditors (IIA), Canada Canadian Evaluation Society
Certifications and Designations IIA Certifications (Certified Internal Auditor, Certified Government Auditing Professional, Certified Financial Services Auditor, Certification in Risk Management Assurance Certification in Control Self-Assessment), Accounting Certifications (CA, CMA, CGA) Credentialed Evaluator (CE)
Occupational Group PA (Program and Administrative Services) EC (Economics and Social Science Services)
External Contractors Accounting Firms Evaluation Boutiques, Accounting Firms

Atlas topic, subject, and course

Controlling Fraud, Waste, and Abuse (core topic) in Implementation and Delivery and Atlas107.

Sources

Diffen.com, Audit vs. Evaluation, at http://www.diffen.com/difference/Audit_vs_Evaluation, accessed 14 October 2017.

Ian Clark (2014), Comparing Audit and Evaluation in the Government of Canada, Handout for a presentation to PADM 5420 (Professor Leslie A. Pal), pdf on the Atlas at http://www.atlas101.ca/pm/wp-content/uploads/2017/10/Comparing_Audit_and_Evaluation_in_the_Government_of_Canada_Ian_Clark_17_Janurary_2014.pdf, accessed 14 October 2017.

Treasury Board of Canada, policies and directives on internal audit, evaluation, and results noted in the links in the table above, accessed 14 October 2017.

Page created by: Ian Clark, last modified 14 October 2017.

Image: Ian Clark (2014), Comparing Audit and Evaluation in the Government of Canada, pdf on the Atlas at http://www.atlas101.ca/pm/wp-content/uploads/2017/10/Comparing_Audit_and_Evaluation_in_the_Government_of_Canada_Ian_Clark_17_Janurary_2014.pdf, accessed 14 October 2017.