Comparing Audit and Evaluation
… a core concept used in Implementation and Delivery and Atlas107
Click for pdf of 2014 handout
Concept description
Although the internal audit and evaluation functions within an organization are similar in some respects there are crucial differences in purpose and methodology.
The “Compare Anything” website Diffen, provides comparative perspectives on its entry for Audit vs. Evaluation (reference below), including the differences in definition:
“An Audit is the evaluation of a person, organization, project, product or primarily for the purpose of determining its validity and authenticity, or to verify adherence to a set of pre-defined processes.
“On the other hand, an evaluation is the determination of merit using a set of standards. Though they are both a kind of assessment, audits are mainly done for financial institutions to make sure they are free from misstatements and errors, whereas evaluations can be done for different spheres such as healthcare, arts, government agencies to judge the efficiency of a working system. Earlier, audits were an important way to assess financial systems only. Now they are also done to assess security risks, environmental and other systems performance as well.”
Comparing internal audit and evaluation in the Government of Canada
The following table sets out the following differences in the way the internal audit and the evaluation functions operate in the Canadian federal government. This table incorporates the refinements to Treasury Board of Canada definition and the methodology that have taken place since the 2014 document in the pdf above as a result of the replacement of the Policy on Evaluation with the Policy on Results.
| Attribute | Internal Audit | Evaluation |
| Treasury Board of Canada Definition | Internal auditing in the Government of Canada is a professional, independent and objective appraisal function that uses a disciplined, evidence-based approach to assess and improve the effectiveness of risk management, control and governance processes. (Policy on Internal Audit, 3.1) | In the Government of Canada, evaluation is the systematic and neutral collection and analysis of evidence to judge merit, worth or value. … Evaluations typically focus on programs, policies and priorities and examine questions related to relevance, effectiveness and efficiency. … Evaluations generally employ social science research methods. (Policy Results, Appendix) |
| Common Meaning (Merriam-Webster Online Dictionary) | Audit 1a) a formal examination of an organization’s or individual’s accounts or financial situation 2) a methodical examination and review Synonyms: inspection, check, checkup, examination, going-over, look-see, review, scan, scrutiny, survey, view | Evaluation 1) to determine or fix the value of 2) to determine the significance, worth, or condition of usually by careful appraisal and study Synonyms: appraise, assess, estimate, guesstimate, rate, set, valuate, value |
| Varieties | Internal Audit (c.f., External Audit by Auditor General), Management Practice Audit, Recipient Audit, Forensic Audit, Financial Audit, Compliance Audit, Performance Audit | Interim (Formative) Evaluation, Impact (Summative) Evaluation |
| Goals (and tension) | 1) Accountability; 2)Improvement | 1) Improvement; 2) Accountability |
| “2 Ps and 3 Es” | Prudence, Probity, Economy | Efficiency, Effectiveness |
| Bad Findings | Noncompliance, Irregularity, Misuse, Abuse, Illegality, Fraud | Inefficiency, Waste |
| Potential Consequences | Political Embarrassment, Dismissal, Imprisonment | Reassignment, Program Change |
| Universe | All Auditable Units (e.g., 50 Program AUs; 30 Internal Services AUs) | All Programs (preferably aligned with the Program Auditable Units) |
| Selecting Entities | Risk-based, but goal of covering all audit entities | 5-year cycle for all grants and contributions programs of more than $5 million annual expenditure |
| Enthusiasm of Entities | Low to Very Low (with exceptions) | Neutral to Low (with exceptions) |
| Methodology | Application of the Internal Auditing Standards for the Government of Canada (IIA Standards plus additional points). See also Directive on Internal Auditing in the Government of Canada. | Application of the Directive on Results, particularly the Mandatory Procedures for Evaluation and the Standard on Evaluation. |
| Methodological Tools | Audit Scope, Audit Criteria, Interviews, Documentation Review, Sample Testing | Evaluation Scope, Logic Model, Literature Review (including Best Practices), Document and File Review, Key Informant Interviews, Case Studies |
| Performance Measurement | Is a performance measurement system in place? | What do the performance measures tell us and how can they be improved? |
| Report Length | 20-40 pages | 50-200 pages |
| Formal Client | Deputy Minister | Deputy Minister |
| Committee | Audit Committee (external members, including Chair) | Evaluation Committee (DM Chair, sometimes with external advisors) |
| Report Output | Audit Recommendations | Evaluation Recommendations |
| Response | Management Response and Action Plan | Management Response and Action Plan |
| Publication | Audit Report and Management Response | Evaluation Report and Management Response |
| Parliamentary and Media Interest | Occasionally intense (e.g., Internal audit of HRDC grants and contributions in 2000) | Low |
| Cost | $50,000 to $150,000 | $50,000 to $200,000 |
| Number per Year | 8-12 | 8-12 |
| Unit Budget | $3-6 million per year; 12-25 FTEs | $3-6 million; 12-25 FTEs |
| Central Agency | Office of the Comptroller General (Internal Audit Sector) | Treasury Board Secretariat, Centre of Expertise for Evaluation |
| Academic Preparation | Accounting, Public Administration | Public Policy, Public Administration, Economics, Sociology, Statistics |
| Professional Societies | Institute of Internal Auditors (IIA), Canada | Canadian Evaluation Society |
| Certifications and Designations | IIA Certifications (Certified Internal Auditor, Certified Government Auditing Professional, Certified Financial Services Auditor, Certification in Risk Management Assurance Certification in Control Self-Assessment), Accounting Certifications (CA, CMA, CGA) | Credentialed Evaluator (CE) |
| Occupational Group | PA (Program and Administrative Services) | EC (Economics and Social Science Services) |
| External Contractors | Accounting Firms | Evaluation Boutiques, Accounting Firms |
Atlas topic, subject, and course
Controlling Fraud, Waste, and Abuse (core topic) in Implementation and Delivery and Atlas107.
Sources
Diffen.com, Audit vs. Evaluation, at http://www.diffen.com/difference/Audit_vs_Evaluation, accessed 14 October 2017.
Ian Clark (2014), Comparing Audit and Evaluation in the Government of Canada, Handout for a presentation to PADM 5420 (Professor Leslie A. Pal), pdf on the Atlas at http://www.atlas101.ca/pm/wp-content/uploads/2017/10/Comparing_Audit_and_Evaluation_in_the_Government_of_Canada_Ian_Clark_17_Janurary_2014.pdf, accessed 14 October 2017.
Treasury Board of Canada, policies and directives on internal audit, evaluation, and results noted in the links in the table above, accessed 14 October 2017.
Page created by: Ian Clark, last modified 14 October 2017.
Image: Ian Clark (2014), Comparing Audit and Evaluation in the Government of Canada, pdf on the Atlas at http://www.atlas101.ca/pm/wp-content/uploads/2017/10/Comparing_Audit_and_Evaluation_in_the_Government_of_Canada_Ian_Clark_17_Janurary_2014.pdf, accessed 14 October 2017.