POLICY EVALUATION AND IMPLEMENTATION
Obiettivi formativi
The first objective of the course is to investigate why some societies, and their respective governments have managed to produce public goods and prosperity and others have not, and what accounts for these differences. Using some fundamental texts from the field of comparative politics, both theoretical and empirical, and canvassing contemporary indexes, the class discusses the different theories, which explain the performance of various political and constitutional regimes and weights the evidence in their favor.
The second objective of this course is to understand how states acquire one or another set of such political institutions with a view to identify theories of change grounded in the political economy of specific historical contexts. We also aim to understand how these individual trajectories converge or diverge in an interdependent world, and what are the consequences for the present and future world order. The empirical evidence is drawn from a variety of cases across all over the world, discussed in the comparative framework of their continent or income group, individually or in pairs (i.e. Russia and Ukraine, Estonia and Finland, Mexico and United States, Italy and France, North and South Korea, Botswana and Zimbabwe).
Risultati di apprendimento attesi
KNOWLEDGE AND UNDERSTANDING
By the end of the course, students will acquire:
- Knowledge of what a public policy is and how policy actors and tools operate under different development and governance contexts
- Knowledge of what a policy cycle is and how it works under different governance contexts
- Knowledge of all specific methods of policy analysis needed for the design, implementation and evaluation, such as problem diagnosis, political economy analysis (stakeholder analysis) and options analysis
APPLYING KNOWLEDGE AND UNDERSTANDING
Students will be able to apply the acquired knowledge in their future professional activities in several ways. In particular, they will be trained in:
- Acquisition of the ability to write a professional policy analysis piece (memorandum) on a policy problem from an area of interest, including problem statement, problem diagnosis, stakeholder analysis and options analysis
- Developing the ability to research and identify relevant indicators for successful implementation and an effective policy evaluation
- Using public policy analysis to develop policy arguments and to communicate them in oral and written form.
MAKING JUDGEMENTS
Due to the emphasis placed on re-elaboration and discussion of the readings, students will be able to:
- Critically analyse issues related to policymaking and contextualize them in light of the knowledge acquired in the course.
- Develop problem-understanding and problem-solving capacity.
- Evaluate specific policies and be able to provide advice on their success
COMMUNICATION SKILLS
During the course students will be asked to participate in the discussions in several ways, primarily through questions, comments on the readings and through positioning themselves in a well-argued way within a debate. Because of this, they will:
- Improve the control of course-related terminology in English but at the same time be able to present complex content in an easily accessible form.
- Become acquainted with engaging in critical discussions in front of a larger audience, communicating questions or comments in a clear way to the rest of the class, thus stimulating a general debate.
- Listen to – and learn from – each other.
LEARNING SKILLS
By the end of the course, students are expected to acquire the following skills:
- Capacity to autonomously engage with indicators to plan a policy evaluation
- Capacity to understand governance contexts and plan an effective implementation based on evidence and context
- Capacity to engage with opposite arguments and to develop one’s own position in a well-structured and clear way based on facts/data.
Contenuti Del Corso
The first part of the course examines the stages of the policy process (problem formulation, agenda-setting, and decision-making) with a focus on the design, implementation and evaluation and their evidence basis. The discussion is placed in different governance contexts so that similar policies can be compared across continents and countries.
The course uses case studies to review the main types of policy problems, their generic solutions, and the indicators to be used to plan an effective implementation, evaluate policy adequacy, and decide on its adjustment or termination.
Testi Di Riferimento
Peters, B. G. & Pierre, J. (Eds.). (2006). Handbook of Public Policy. SAGE. – On Learn
Fischer, F., Miller, G. J. & Sidney, M. S. (Eds.). (2007). Handbook of public policy analysis: theory, politics, and methods. CRC Press. – On Learn
Bardach, E. & Patashnik, E. M. (2005). A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving.
2nd edition. CQ Press. –
Moran, M., Rein, M. & Goodin, R. (Eds.). (2008). The Oxford Handbook of Public Policy. Oxford University Press. –
HM Treasury. (2020). The Magenta Book. HM Treasury guidance on what to consider when designing an evaluation. GOV.UK. https://www.gov.uk/government/publications/the-magenta-book
Caramani, D. (Ed.). (2014). Comparative Politics. 3rd edition. Oxford University Press.
King, G., Keohane, R. O. & Verba, S. (2021). Designing Social Inquiry Scientific Inference in Qualitative Research. New edition. Princeton University. – Available via Perlego
Metodologie Didattiche
Each week, the course comprises a lecture (online) and a seminar (on campus). The lecture presents the main issues related to the content of the week. The seminar is a hands-on policy analysis class, stressing problem statements, indicators, stakeholder analysis and evaluation.
The core readings are related to the lecture, although the seminar has some readings of its own, of a methodological nature to guide students into the analysis process. The seminar relies on the group structure of the class. Students will need to meet in groups at least twice on-site or online to agree on the basics of their short presentations and group assignments. Each group will present three times, so groups should organize in such a way that each individual student speaks at least once.
Active participation in the class discussions and debates is expected and will be part of the final assessment. At Luiss, course attendance is mandatory. Students that cannot attend the course or part of it must provide official reasons (attested by a specific document, such as a medical certificate).
Modalità di verifica dell'apprendimento
The grade for this course will be determined on the basis of 4 components graded equally with 25% of the final grade:
1) Group peer learning exercise 1: Problem statement, presentation in class and submission of ppt.
2) Group peer learning exercise 2: Stakeholder analysis presentation in class and submission of ppt.
3) Group writing exercise: Written problem statement with flagging of indicators which can potentially be used for evaluation – 1200 words including figures, excluding references.
4) Final individual exam: Individual research design of a policy evaluation.
Templates for all presentations will be uploaded prior to the course.
Criteri per l’assegnazione dell’elaborato finale
No criteria
Settimana 1
Week #1 Class design. What is public policy?
How do we ask and answer a comparative research question?
Readings:
Wagner, P. (2007). Public policy, social science, and the state: An historical perspective. In F. Fischer, G. J. Miller, & M. S. Sidney (Eds.), Handbook of public policy analysis: Theory, politics, and methods (pp. 29-43). CRC Press. – On Learn
Skocpol, T., & Amenta, E. (1986). States and social policies. Annual Review of Sociology, 12, 131-157.
https://doi.org/10.1146/annurev.so.12.080186.001023
Werner, J. & Wegrich, K. (2007). Theories of the Policy Cycle. In F. Fischer, G. J. Miller, & M. S. Sidney (Eds.), Handbook of public policy analysis: theory, politics, and methods (Chapter 4, pp. 43-63). CRC Press. – On Learn
Week #1 Class design. What is public policy?
Group work:
Random creation of groups. Case discussion: the impact of war sanctions against Russia.
Background reading:
Medunic, F. (2023, April 18). Damage done: Ways to measure European sanctions’ success against Russia. ECFR. https://ecfr.eu/article/damage-done-ways-to-measure-european-sanctions-success-against-russia/
Settimana 2
Week #2 Policy goals
Where do public policy goals come from?
Readings:
Peters, G., & Pierre, J. (Eds.). (2006). The Handbook of Public Policy. SAGE. Chapters 25 & 26 (pp. 433-461). – On Learn
Campbell, J. L. (1998). Ideas, politics, and public policy. Annual Review of Sociology, 24, 131-157.
https://doi.org/10.1146/annurev.soc.28.110601.141111
Caplan, B. (2007). The myth of the rational voter: Why democracies choose bad policies. Cato Institute Policy Analysis Series, No. 594. Retrieved from http://www.cato.org/pubs/pas/pa594.pdf
Week #2 Where do public policy goals come from?
Group work:
First group meeting to work on the problem statement.
Background reading:
Bardach, E. (2005). What Are Public Policy Problems? Problem Statement. In A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving (2nd ed., pp. 1-60). CQ Press. – On Learn
Settimana 3
Week #3 Tools of government and the art of crafting policy memos
What can and should a government do and not do?
Readings:
Fukuyama, F. (2004). The imperative of state building. Journal of Democracy, 15(2), 17-31. https://doi.org/10.1353/jod.2004.0028 – Available on ResearchGate
Hood, C. (2008). The tools of government in the information age. In M. Moran, M. Rein, & R. E. Goodin (Eds.), The Oxford handbook of public policy (Chapter 22, pp. 551-564). Oxford University Press. – Available via Perlego
Ledbury, M., Choudhury, T., Paddick, S., & Patel, K. (2006). Understanding policy options. Home Office Online Report 06/06,
pp. 4 – 14. Retrieved from
https://www.academia.edu/14759367/Understanding_Policy_Options
Week #3 Tools of government and the art of crafting policy memos
Readings:
Jones, N., & Walsh, C. (2008). Policy Briefs as a Communication Tool for Development Research. Overseas Development Institute (ODI). Retrieved from https://odi.org/en/publications/policy-briefs-as-a-communication-tool-for-development-research/
(n.d.). Decision / Action Memo Examples [PDF file]. Harvard Kennedy School, Shorenstein Center on Media, Politics and Public Policy. Retrieved from https://shorensteincenter.org/wp-content/uploads/2012/07/MEMO-WRITING-HO-ALL-Danziger.pdf
Settimana 4
Week #4 Market failure problems and solutions
What are the main forms of market failure and the government responses?
Readings:
Ledbury, M., Choudhury, T., Paddick, S., & Patel, K. (2006). Understanding policy options. Home Office Online Report 06/06,
pp. , 15-22; 28-53. Retrieved from
https://www.academia.edu/14759367/Understanding_Policy_Options
Zerbe Jr, R. O., & McCurdy, H. E. (1999). The failure of market failure. Journal of Policy Analysis and Management, 18(4), 558-578. – Available on GoogleScholar
Week #4 Market failure problems and solutions
Group work:
Set 1 of problem statement presentation by groups (3-4).
Settimana 5
Week #5 Public goods problems and solutions
What are collective action problems and how to solve them?
Readings:
Hardin, G. (1968). The Tragedy of the Commons. Science, 162(3859), 1243-1248. Retrieved from
https://pages.mtu.edu/~asmayer/rural_sustain/governance/Hardin%201968.pdf
Ostrom, E. (2008). Tragedy of the Commons. In S. N. Durlauf & L. E. Blume (Eds.), The New Palgrave Dictionary of Economics (2nd ed.). Palgrave Macmillan. Retrieved from
https://dlc.dlib.indiana.edu/dlc/handle/10535/5887
Moffatt, M. (Mar 19, 2019). The Logic of Collective Action | Special Interests and Economic Policy. [A summary of The Logic of Collective Action (1971) by M. Olson]. https://www.thoughtco.com/the-logic-of-collective-action-1146238
Bridoux, F., & Stoelhorst, J. W. (2022). Stakeholder governance: Solving the collective action problems in joint value creation. Academy of Management Review, 47(2). https://doi.org/10.5465/amr.2019.0441 – Available via Perlego
Week #5 Public goods problems and solutions
Group work:
Set 2 of problem presentation by groups (4-7).
Settimana 6
Week #6 Government failure problems and solutions
What is government failure and how can such policy problems be dealt with?
Readings:
Buchanan, J. (1988). Market failure and political failure. Cato Journal, 8(1), 1-13. Retrieved from
http://www.cato.org/pubs/journal/cj8n1/cj8n1-1.pdf
Ostrom, V., & Ostrom, E. (1971). Public choice: A different approach to the study of public administration. Public Administration Review, 31(2), 203-216. https://doi.org/10.2307/974676 – Available via Perlego
Week #6 Government failure problems and solutions
Group work:
Set 3 of problem presentation by groups (8-10).
Settimana 7
Week #7 Mapping interests in public policy. Stakeholder analysis
How do we map group interests and reconcile them in public policy?
Readings:
Burstein, P. (2003). The impact of public opinion on public policy: A review and an agenda. Political Research Quarterly, 56(1), 29-40. Available on GoogleScholar
Goodin, R., Rein, M., & Moran, M. (2008). The public and its policies. In M. Moran, R. E. Goodin, & M. Rein (Eds.), The Oxford Handbook Of Public Policy (pp. 3-21). Oxford University Press. – Available via Perlego
Week #7 Mapping interests in public policy. Stakeholder analysis
Group work:
Group stakeholder analysis exercise (4 presentations).
Settimana 8
Week #8 Implementation of public policy
Who and how implements public policy?
Readings:
Winter, S. C. (1986). Implementation. In B. G. Peters & J. Pierre (Eds.), Handbook of Public Administration (pp. 151-167). SAGE. – On Learn
Sabatier, P. A., & Mazmanian, D. A. (1979). The implementation of public policy: A framework of analysis. Policy Studies Journal, 8(4), 538-560. https://doi.org/10.1111/j.1541-0072.1980.tb01266.x – Available via Perlego
Mungiu-Pippidi, A. (2012). The Quality of Government and Public Administration. – Available on ResearchGate
Week #8 Implementation of public policy
Group work:
Group stakeholder analysis exercise (6 presentations).
Settimana 9
Week #9 Evaluation research. Choosing indicators
How are indicators for policy evaluation selected? What makes an effective indicator?
Readings:
Vedung, E. (2000). Evaluation Research. In P. G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 23, pp. 397-406). SAGE. – On Learn
UNAIDS. (2010). Monitoring and Evaluation Fundamentals: An Introduction to Indicators.
https://www.unaids.org/sites/default/files/sub_landing/files/8_2-Intro-to-IndicatorsFMEF.pdf
Centers for Disease Control and Prevention. (n.d.). Step by Step – Evaluating Violence and Injury Prevention Policies. Brief 5: Evaluating Policy Impact.
https://www.cdc.gov/injury/pdfs/policy/Brief%205-a.pdf
Week #9 Evaluation research. Choosing indicators
Readings:
UN. (n.d.). Evaluation for Agenda 2030 | Department of Economic and Social Affairs. https://sdgs.un.org/publications/evaluation-agenda-2030-18064
OECD. (Jan 21, 2022). Policy Responses to Coronavirus (COVID-19).
https://www.oecd.org/coronavirus/policy-responses/first-lessons-from-government-evaluations-of-covid-19-responses-a-synthesis-483507d6/
Settimana 10
Week #10 Evaluation research. Surveys and experimental designs
What role do experiments and surveys play in evaluation policy?
Readings:
Mitchell, D. J. B. (2005). The Use (and Misuse) of Surveys Research in Policy Analysis. In G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 24, pp. 369-381). SAGE. – On Learn
Krosnick, J. A., & Presser, S. (2009). Question and Questionnaire Design. In J. D. Wright & P. V. Marsden (Eds.), Handbook of Survey Research (2nd ed., pp. 263–313). Elsevier. – On Learn
Price, P. C., Jhangiani, R., & Chiang, I-C. A. (2015). Conducting Surveys. Retrieved from
https://opentextbc.ca/researchmethods/chapter/conducting-surveys/
Week #10 Evaluation research. Surveys and experimental designs
Class Exercise (bring laptops):
Questionnaire and survey design development.
Settimana 11
Week #11 Evaluation – cost effectiveness
How do we arrive at short listed options and compare them in order to make a decision?
Readings:
Vining, A. R., & Weimer, D. L. (2011). Efficiency and cost-benefit analysis. In G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 24, pp. 417-432). SAGE. – On Learn
Mungiu-Pippidi, A. (2022). Transparency and corruption: Measuring real transparency by a new index. Regulation & Governance, 16(1), 168-180. Retrived from
https://onlinelibrary.wiley.com/doi/10.1111/rego.12502
Week #11 Evaluation – cost effectiveness
Class Exercise (bring laptops):
Groups will use Open Tender (www.opentender.eu) and the T-index (www.corruptionrisk/transparency).
Settimana 12
Week #12 Qualitative evaluation designs
How can understand not only what the public wants or does not want from government, but also why?
Reading:
Kahan, J. P. (2001). Focus Groups as a Tool for Policy Analysis. Analyses of Social Issues and Public Policy, 1(1), 129-146. http://dx.doi.org/10.1111/1530-2415.00007 – Available via Perlego
Week #12 Qualitative evaluation designs
Class Exercise:
The art of focus group. Live focus group in class (10 students participate, the others assist)