Cost-Benefit Analysis in Evaluation

… a core concept used in Evaluation and Performance Measurement and Atlas108

Concept description

Leslie Pal (reference below) defines cost-benefit analysis as “evaluation of a program in terms of its total costs compared to its total benefits, expressed in monetary terms” (p. 303).

Pal writes (p. 289):

“The logic of cost–benefit analysis is quite simple. The question is not only whether a policy or program has an impact, but at what cost. Policymakers want to relate costs and benefits in some way. Private businesses do this by measuring profits, though they also monitor other indicators such as productivity. Profit equals income minus expenses, both of which are calculated in monetary terms. As well, business firms measure income (or benefits) from their own perspective, not from the perspective of the consumer using a given product. Government services rarely yield revenue,  and since they are services to the community, what counts as “income” is the benefit from the point of view of the community. The translation of these social benefits into purely monetary terms is sometimes difficult compared to a private firm, which simply bases its calculations on a market price. The challenge for government is to go beyond the bottom line (which is either impossible or inappropriate for public programs), and determine the net social benefit by calculating the difference between total benefits and total costs associated with a program.

Basic steps in cost-benefit analysis

“Thus, the basic steps in cost-benefit analysis are logically quite clear (these are explained in more detail below):

  • decide on the accounting unit (whose costs and benefits are to be calculated);
  • catalogue all costs and all benefits over time;
  • monetize (attach a monetary value) to those costs and benefits;
  • discount those costs and benefits over the period of time that the project or program will be operating; and determine the net social benefit.”
Cost-benefit ratio

Pal writes (p. 290):

It may appear that a cost–benefit ratio greater than 1.00 is self-evidently good, since there is some net gain for an investment of resources. But benefits and costs affect different groups of people, and thus may involve equity considerations. Cost–benefit analysis, however, is not concerned with distributional or equity issues; it relies on a social welfare criterion known as Pareto optimality. This criterion states that a change is worthwhile if at least one person is made better off while no one else is worse off. It is not the same as an increase in total benefits, if that increase depends on someone else’s loss. For example, consider a string of 10 rural homes, nine of which have barely adequate road access and the tenth none at all. A tax levy on all 10 homes to build a new road connecting them could possibly be Pareto optimal, as long as the benefits of improved access for the nine homes equalled the levy. The tenth home would, of course, be much better off. In this example, everyone is at least as well off as before, and one person may be much better off. Few policy decisions are this clear. The limitations of the Pareto criterion led economists to develop another, more flexible one: the Kaldor-Hicks criterion, which identifies potential Pareto improvements as those that, assuming that net gainers could compensate losers, would leave at least one person better off without anyone else worse off. The redistribution is hypothetical, and so, in effect, the Kaldor-Hicks version of Pareto optimality is a criterion of net benefits. Potential Pareto improvements, it is argued, will increase total societal benefits (for both winners and losers) over time.

Practising evaluators normally leave these arcane matters of social welfare functions to academic economists. From a practical  perspective, the tough issues in cost–benefit analysis are the determination and quantification of costs and benefits. Two such general problems are the selection of the accounting unit and the issue of intangibles.

Accounting unit

“The accounting unit problem is about whose costs and benefits are to be measured. Three basic choices are the individual, the government, and society. Consider an employment agency decision on whether to provide counselling services to its clients. One way of assessing costs and benefits is to focus on the individual program participants: costs might include less time for leisure or job search, while benefits might be increased job skills. From the governmental or agency perspective, costs would be the budgetary ones of mounting the program, while benefits might be increased hiring rates and tax revenues as former clients get jobs. The societal perspective is the most comprehensive, weighing costs and benefits for total national income. Benefits, for instance, might include the value to the national economy of the jobs that clients get. The estimates of benefits and costs will differ depending on the accounting unit chosen.

Intangibles

“The problem of intangibles concerns the difficulty of placing monetary values on some costs and benefits. Aesthetic considerations in town planning, the value people place on leisure, the sense of security provided by a tough criminal justice system: All are presumably important in determining costs and benefits, but are difficult to quantify. It is arguable that cost–benefit analyses of public programs tend to estimate costs (program or budget costs) more accurately than benefits, many of which are intangible. Educational policy evaluation, for example, has trouble measuring such benefits as civility and cultural breadth and tends by default to concentrate on job-related benefits. Cost–benefit calculations usually avoid quantifying intangibles, preferring instead to simply mention them as considerations.

Uncertainty

“A related issue is that even if costs and benefits can be monetized, not all of them have an equal chance of occurring. Evaluations thus sometimes attach probabilities to costs and benefits.

Opportunity cost vs. direct cost

“The careful classification and treatment of different types of costs is the first step in competent cost–benefit analysis. One important category is opportunity cost, or the forgone benefits of doing one thing and not another. Assuming scarce resources, doing one thing always means forgoing something else. Cost–benefit analysis, insofar as it tries to facilitate comparisons across alternatives, tries to address the issue of opportunity cost.

External vs. internal costs and benefits

“Other common distinctions are made in determining costs. External versus internal costs and benefits refer to indirect or unintended spillovers from a program. For example, polluting industries sometimes create jobs at home (internal benefit) while generating environmental costs elsewhere (external costs).

Incremental vs. sunk costs

“Incremental versus sunk costs is another important distinction. Sunk costs are those incurred in the past; incremental costs are additional or future costs expended to mount or continue a program.

Total vs. marginal costs

“A final distinction is between total and marginal costs. The total costs of a new counselling program for unwed mothers, for example, would include the proportion of total costs of the agency (e.g., clerical services, furniture, building, heat, light) accounted for by the program. But since these costs would occur anyway, the marginal cost of the program would be the additional resources devoted to it.

Time and discount rate

“Costs and benefits usually do not occur immediately, and each may flow in different streams. In the case of capital projects or pensions, it may take many years before benefits are realized. This lag time raises a problem of measurement, since most people prefer their benefits to come now and their costs to come later. The element of time therefore has to be assessed in estimates of future costs and benefits. The usual procedure is to apply a discount rate to the present value of costs and benefits incurred in a project, to arrive at a measure of net present value (Boardman, Greenberg, Vining, & Weimer, 2011, Chapters 6, 10). The trick is in selecting an appropriate rate. The predominant method assesses the opportunity costs of capital, meaning the rate of return if program sums were invested in the private sector. If the economy were uncluttered by monopolies or by taxation regimes that alter pretax rates of return and restrictive foreign trade, then market rate of interest could be taken as the discount rate. Because rates of return are affected by these institutional features, economists make numerous and contentious adjustments to arrive at estimates of the real discount rate.”

Atlas topic, subject, and course

The Study of Evaluation and Performance Measurement in the Public Sector (core topic) in Evaluation and Performance Measurement and Atlas108 Analytic Methods and Evaluation.

Sources

Leslie Pal (2014), Beyond Policy Analysis – Public Issue Management in Turbulent Times, Fifth Edition, Nelson Education, Toronto. See Beyond Policy Analysis – Book Highlights.

Boardman, A. E., Greenberg, D. H., Vining, A., & Weimer, D. L. (2011). Cost-benefit analysis: Concepts and practice (4th ed.). Boston, MA: Prentice-Hall.

Page created by: Ian Clark, last modified 11 April 2017.

Image: Najera Consulting Group, at http://najeraconsulting.com/tuesdays-tool-of-the-trade-cost-benefit-analysis/, accessed 11 April 2017.