Atlas Framework for Curricular Analysis
The Atlas framework employs a number of content categories to map the curricular content of courses offered in MPP/MPA programs and the advice offered by International Government Organizations (IGOs). Using the methodology described below, entries in five of these categories – courses, topics, concepts, competencies, and advice – are assigned to the 34 subjects in the four domains used in Subjects in public management.
The content categories in the Atlas framework
- Domain: a broad area of public policy and management characterized by the nature of pedagogical preparation or of government practice. There are two general preparation domains: Analysis and Skills, where subjects and topics tend to have a “how to” orientation; and Institutions and Context, where subjects and topics tend to have a “why does” orientation. And there are two specific practice domains: Management Functions, where subjects and topics are associated primarily with organizations responsible for management practices; and Policy Sectors, where subjects and topics are associated primarily with sectoral ministries responsible for policy execution. University teaching tends to be concentrated on the two general preparation domains. Most university programs also offer courses in the specific practice domains, with MPP programs typically having more courses in Policy Sectors subjects than MPA programs, and MPA programs typically having more courses in Management Functions subjects than MPP programs. Advice from IGOs is concentrated on subjects in the two specific practice domains.
- Subject: a collectively exhaustive set of mutually exclusive topics in public policy and management (see below for elaboration of these terms). The names we have selected for subjects are derived from names commonly associated with a single university course and/or a government ministry. The Atlas uses 34 subjects, each assigned to one of the Atlas’s four domains. For example, Quantitative Methods is one of the 7 subjects in the Analysis and Skills domain; Governance and Institutions is one of the 4 subjects in the Institutions and Context domain; Public Financial Management is one of the 7 subjects in the Management Functions domain and International Development is one of the 16 subjects in the Policy Sectors domain. An Atlas subject differs from an “academic subject” (for example, macroeconomics or international relations) which is not intended to be a mutually exclusive category. Whereas one academic subject can share topics with another (for example, competitive devaluation is a topic in both macroeconomics and international relations), an Atlas subject is a unique set of mutually exclusive topics. Atlas subjects are thus mutually exclusive in that they do not overlap with other Atlas subjects in their topic content. As described on the Subjects page, subjects can also be classified as Policy-Oriented or Management-Oriented. Some can also be classified as having high Math-Econ (mathematics and economics) content. These distinctions are used in the MPP/MPA Curricular Types categorization. And some can be classified as Archetypal Public Affairs while some others are Archetypal International Affairs. These distinctions are used in Comparisons with International Affairs Programs.
- Topic: a coherent body of concepts and readings suitable for a single 3-hour class in an MPP or MPA program and about 7 hours of outside-class preparation (see Normed Topic Model). Each topic is assigned exclusively to one subject. For example, Negotiating is a topic assigned to Leadership Skills and Commodity Taxes and Subsidies is a topic assigned to Economic Analysis. The names of many of the Atlas topics are derived from the thematic titles used for individual classes in a course.
- Concept: an abstract idea from a theory or from a generalization of particular instances, or a definition of a term. Concepts are employed in all teaching topics and courses, and in all practice advice. Concepts were initially identified by student research assistants actually taking MPP/MPA courses at six Canadian programs (Carleton SPPA, Dalhousie SPA, Ottawa GSPIA, Queen’s SPS, Saskatchewan-Regina JSGS, and Toronto SPPG) in the period 2008-2011. Sources include class notes as well as published syllabi. The approximately 1,000 concepts and terms in the Atlas collection are arrayed alphabetically in Concepts database. Although concepts are not necessarily exclusive to any particular topic or subject, each has been associated with a single topic for categorization purposes.
- Competency: a technical or professional skill or a personal attribute or underlining characteristic that enables the delivery of a role or job. We have derived a set of MPP/MPA Core Competencies based on our analysis of the content of required courses in well-established programs and the competency standards of the Network of Schools of Public Policy, Affairs and Administration (see NASPAA Competencies) and by UNDESA/IASIA (United Nations Department of Economic and Social Affairs/International Association of Schools and Institutes of Administration through its Task Force on Standards of Excellence for Public Administration Education and Training (see UNDESA/IASIA Standards). These and other competency descriptions are included in the Competencies database.
- Course: a university offering that teaches an aspect of public policy and management. Most courses on the Atlas are at the Master’s level and are one semester (typically 3 hours per week of instruction over 12-14 weeks). Although most courses are not exclusive to a single public policy and management subject, many can be primarily associated with a particular subject. For example, University of Toronto’s PPG1000 Governance and Institutions is primarily associated with the institutions and context subject of Governance and Institutions, and Harvard University’s MLD201 Exercising Leadership – The Politics of Change is primarily associated with the analysis and skills subject of Leadership Skills. Although the Atlas lists virtually all the courses offered by each institution on the institutional course maps on their respective program pages (approximately 3,000 courses in total), we are using a more limited sample of a few dozen courses displayed at Mapping Open Access Syllabi by Public Management Subject to identify the principal teaching topics in the field of public policy and management.
- Program: a program of study leading to a Master of Public Policy, Master of Public Administration, or similar degree offered by a university. In most cases the programs are provided through a specific institution (usually called a School) within the university. The institutions with one or more programs included for analysis on the Atlas are listed in the Programs database. For each of these institutions, there is a program page (see Harvard HKS for an example) that provides key program details in a common format. The courses offered are arrayed on an institutional course map indicating the subject most associated with the course content and from this an instructional distribution profile is calculated (see PEACO Algorithm). These profiles enable comparisons among institutions on matters such as the distribution of instruction among Atlas subject-matter domains or among NASPAA Competencies (see Curriculum Comparison Tables).
- Advice heading: a coherent quantum of practice advice and explanatory material provided to government in a report from an international agency. The names of many of the advice headings are derived from the titles of reports or studies. These reports tend to have considerably more detail than course syllabi and the task of generating advisory topics from them is one of summarizing key points in a concise format. We use a common format that includes Summary Advice, Main Points, and Commentary. For advice drawing on reports from more than one agency, the Summary Advice and Main points from each agency are labelled. Each advisory heading is assigned exclusively to one subject. For example, Achieving and Sustaining Fiscal Consolidation (OECD) is an advice heading assigned to Public Financial Management. All the advice headings are displayed, by subject, in the Advice database.
Curricular attributes for comparing MPP/MPA programs
The Atlas methodology described above generates information on MPP/MPA programs in a common format on a number of general attributes: geographic location, name of degree and specializations, minimum time to completion, marketing approach, academic unit within the university, tuition, enrolment, internships and co-curricular activities. It also provides data on quantifiable attributes of curriculum content available from university calendars, such as number of courses needed to graduate, the number of required and elective courses offered, and the subject (and therefore domain) to which courses are assigned by the Atlas editors.
This information is presented on Programs database (including the course maps on the individual program pages) and the Curriculum Comparison Tables to allow Atlas viewers to make comparisons on a program-to-program among basis among institutions.
Exclusivity and exhaustively
Some of the content categories are defined so that their entries are mutually exclusive; some are defined so that their entries can be collectively exhaustive (see illustration below). In particular:
- topics are intended to be mutually exclusive
- subjects are defined to be collectively exhaustive sets of topics
- subjects are mutually exclusive since they contain different sets of mutually exclusive topics
- domains are composed of collectively exhaustive sets of subjects (and, therefore, of collectively exhaustive sets of mutually exclusive topics)
- domains are mutually exclusive since each comprises a different set of mutually exclusive subjects (and topics)
It is worth noting that domain character deals with two levels of distinctions that we believe are important in characterizing pedagogy in this professional field. At the top level the distinction is whether the primary purpose of teaching the topic is that of general preparation for application to multiple areas of practice or whether it is that of preparing for a specific practice area. For the first, a further distinction is whether the focus is on developing tools and skills for application to practice, or whether it is on improving the understanding of the institutions and context relevant to practice. For the second, a further distinction is whether the practice area is one of the generic management functions or one of the policy sectors. Because the four domains noted above are the product of two levels of distinction (in contrast, for example, to the two-by-two matrix associated with binary distinctions at a single level), it would be theoretically possible to generate more than four domains. For example, the tools/context distinction could be applied within both the management functions domain and the policy sectors domain to create a total of six domains. In order to retain four mutually exclusive domains into which to assign topics and subjects the attribute distinctions have to be applied in a consistent order: the general/specific distinction applied before the tools/context distinction.
As in any classification system, choices have to be made and classification judgements applied where assignments to categories are not immediately obvious. The conventions that guide our classification judgements are outlined below.
Classification judgements are required to deal with the horizontal distinctions between domains (and therefore between subjects). The two general preparation domains (Analysis and Skills, Institutions and Context) are intended to contain subjects that provide students with theoretical constructs and knowledge that will be valuable to understand and address a range of public policy and management issues. The two specific practice domains (Management Functions, Policy Sectors) contain subjects of professional work in public policy and management. This gives rise to the question of whether a topic (such as Cost-benefit Analysis) that prepares students with a general technique that has special applicability to a particular practice area should be classified under the general preparation domain subject (in this case, Analytic Methods or under the specific practice domain subject (in this case, Evaluation and Performance Measurement). Similar questions arise for the classification of certain courses, concepts and study reports. We have adopted the following classification convention: where such classification choices arise for teaching topics, courses or concepts, we place them in the general preparation domain subjects (in this case, Analytic Methods rather than Evaluation and Performance Measurement); where they arise for advisory topics or study reports, we place them in the specific practice domain subjects.
Other classification questions arise from the distinction between the two general preparation domains: Analysis and Skills (“how to”) and Institutions and Context (“why does”). The subject pair in which this question arises most frequently is Policy Analysis and Process (Analysis and Skills domain) and Governance and Institutions (Institutions and Context domain). We have found it helpful to imagine the pedagogical challenge or question being addressed in the subject matter. We have adopted the following classification convention: where the implicit challenge is “how to improve public policy and management” the topic is assigned to a subject in the Analysis and Skills domain; where the implicit question is “why does this occur?” it is assigned to a subject in the Institutions and Context domain.
There are vertical distinctions between topics. Given the way that Atlas topics are defined, topics derived from more specialized (usually elective) courses could be viewed as being situated “below” topics in more general (often required) courses. For example, Persuading is a single topic in the core course outline Atlas109 Leadership and Communication. But Harvard HKS offers a full one-semester elective course, MLD342 Persuasion – The Science and Art of Effective Influence. In the latter course, one of the 12 topics is “The Principles of Persuasion – Authority, Conformity.” In this specialized course Authority and Conformity could be treated as one or even two topics. In the more general required course envisaged by Atlas109, these are treated as Authority and Credibility, one of 27 concepts associated with the core normed topic of Persuading.
As a final point on the vertical distinctions issue, it is worth noting that concepts are not necessarily situated “below” topics. Although it is true any single topic and or advisory heading is likely to draw on several concepts, it is also possible for a single concept (e.g., Accountability) to be the subject matter of several topics. Because topics and concepts are different in the way they are defined and used, there is no taxonomic inconsistency with having a topic and a concept use the same name. In practice, topic names on the Atlas tend to be longer than concept names, in the same way that encyclopedia entries tend to have longer names than dictionary entries.
Classification judgements are required to deal with non-exclusivity, particularly for courses and concepts. As noted above, the only content categories that are intended to be exclusively assigned to subjects are topics and advisory headings. Other content categories are not intended to be exclusive: many courses address a variety of topics, not all of which are assigned to the same Atlas subject. For example, Harvard’s MLD110 Strategic Management for Public Purposes is assigned to Policy Analysis and Process on course maps because more of its teaching topics are in that subject than in others. Similarly, as already noted, many concepts are used in more than one topic and often in more than one subject.
Issues and challenges
The definitions and names of Atlas subjects have evolved as we learned more about the curricular content of MPP and MPA programs around the world and as we learned more about the content of the advice provided by international agencies. For example, at one time we had a subject under the Management Functions domain named “Oversight, Risk Management, and Control” which included topics and concepts related to audit, control and risk management but we found that there were few courses taught in this area and little international agency advice provided. We therefore eliminated this subject and expanded the definition of other subjects to include the topics formerly assigned to it. We assigned the audit-related topics and concepts to Evaluation and Performance Measurement and the control and risk management topics and concepts to Public Financial Management.
Some of the issues and challenges presented by the Atlas framework and classification conventions are noted below.
- Most of the subject matter taught in the macroeconomics subdiscipline of economics is assigned to Macroeconomic Policy (in the Policy Sectors domain) in contrast to the subject matter taught in microeconomics which is assigned to Economic Analysis (in the Analysis and Skills domain). This is because concepts in microeconomics (especially if they are taught with this in mind) provide powerful tools for analyzing a wide variety of public policy and management issues while the application of macroeconomics concepts is mostly confined to fiscal and monetary policy.
- Most of the subject matter taught in the policy-related subdisciplines of political science (policy analysis, policy studies, policy design, etc.) as well as the management-related subdisciplines (public management, strategic management, etc.) are assigned to Policy Analysis and Process or to Implementation and Delivery (and a minority to Analytic Methods, also in the Analysis and Skills domain) even though it could be argued that in many political science departments this subject matter is taught with more of a “why does” than a “how to” orientation. We have done this on the assumption that in MPP and MPA programs, an effort is made to teach these subdisciplines with a “how to” orientation.
- Most of the subject matter taught in courses with titles such as “Qualitative Research Methods” is assigned to Analytic Methods. We have done this because there do not appear to be enough courses teaching this material to warrant a separate subject alongside Quantitative Methods, and because such courses do not generally require the same preparation in mathematics and economics as courses assigned to Quantitative Methods.
- Much of the subject matter that we have assigned to Leadership Skills could equally have been assigned to Policy Analysis and Process or to Implementation and Delivery. Indeed, many courses that deal with policy and management analysis address leadership as one topic among many. We have done this partly to balance subject matter among the subjects in the Analysis and Skills domain and partly out of the conviction that teachable leadership skills are important enough in the preparation of MPP and MPA graduates to warrant a separate subject.
- Some of the subject matter in Ethics, Rights, and Accountability (especially that related to accountability) could equally be assigned to Governance and Institutions. We have done this partly to balance the subjects in the Institutions and Context domain.
- Two subjects, Nonprofit Management and Advocacy, and Local Government Management, differ from other subjects in the Management Functions domain in that they are focused on an institutional sector rather than a function. These subject categories are intended to group material pertaining to these sectors in one place, even though it would be possible to assign some of these topics to subjects in the Analysis and Skills domain or the Institutions and Context domain.
Although classification challenges remain, it is worth remembering that for most purposes the crucial unit of public policy and management content is not the domain or subject, but the topic. The important pedagogical question is not whether the topic is classified in a “how to” or a “why does” subject, but whether and how the topic is taught. To be properly prepared for the professional world, MPP and MPA students need a solid grasp of both the “how to” and “why does” aspects of public policy and management. Similarly, the important advisory question is not how the advice is classified but whether the advisory topic is pertinent and whether the specific advice is useful.
Page created by: Ian Clark, last modified 5 June 2016.
Image: Cropped from Procurement Transformation, CEB, at https://www.cebglobal.com/exbd/procurement-operations/transformation/category-management/index.page?, accessed 10 December 2015.