POLICY EVALUATION AND IMPLEMENTATION

POLICY EVALUATION AND IMPLEMENTATION

Tatiana-Alina Pippidi

Obiettivi formativi

The first objective of the course is to investigate why some societies, and their respective governments have managed to produce public goods and prosperity and others have not, and what accounts for these differences. Using some fundamental texts from the field of comparative politics, both theoretical and empirical, and canvassing contemporary indexes, the class discusses the different theories, which explain the performance of various political and constitutional regimes and weights the evidence in their favor. The second objective of this course is to understand how states acquire one or another set of such political institutions with a view to identify theories of change grounded in the political economy of specific historical contexts. We also aim to understand how these individual trajectories converge or diverge in an interdependent world, and what are the consequences for the present and future world order. The empirical evidence is drawn from a variety of cases across all over the world, discussed in the comparative framework of their continent or income group, individually or in pairs (i.e. Russia and Ukraine, Estonia and Finland, Mexico and United States, Italy and France, North and South Korea, Botswana and Zimbabwe).

Risultati di apprendimento attesi

KNOWLEDGE AND UNDERSTANDING By the end of the course, students will acquire: - Knowledge of what a public policy is and how policy actors and tools operate under different development and governance contexts - Knowledge of what a policy cycle is and how it works under different governance contexts - Knowledge of all specific methods of policy analysis needed for the design, implementation and evaluation, such as problem diagnosis, political economy analysis (stakeholder analysis) and options analysis APPLYING KNOWLEDGE AND UNDERSTANDING Students will be able to apply the acquired knowledge in their future professional activities in several ways. In particular, they will be trained in: - Acquisition of the ability to write a professional policy analysis piece (memorandum) on a policy problem from an area of interest, including problem statement, problem diagnosis, stakeholder analysis and options analysis - Developing the ability to research and identify relevant indicators for successful implementation and an effective policy evaluation - Using public policy analysis to develop policy arguments and to communicate them in oral and written form. MAKING JUDGEMENTS Due to the emphasis placed on re-elaboration and discussion of the readings, students will be able to: - Critically analyse issues related to policymaking and contextualize them in light of the knowledge acquired in the course. - Develop problem-understanding and problem-solving capacity. - Evaluate specific policies and be able to provide advice on their success COMMUNICATION SKILLS During the course students will be asked to participate in the discussions in several ways, primarily through questions, comments on the readings and through positioning themselves in a well-argued way within a debate. Because of this, they will: - Improve the control of course-related terminology in English but at the same time be able to present complex content in an easily accessible form. - Become acquainted with engaging in critical discussions in front of a larger audience, communicating questions or comments in a clear way to the rest of the class, thus stimulating a general debate. - Listen to – and learn from – each other. LEARNING SKILLS By the end of the course, students are expected to acquire the following skills: - Capacity to autonomously engage with indicators to plan a policy evaluation - Capacity to understand governance contexts and plan an effective implementation based on evidence and context - Capacity to engage with opposite arguments and to develop one’s own position in a well-structured and clear way based on facts/data.

Contenuti Del Corso

The first part of the course examines the stages of the policy process (problem formulation, agenda-setting, and decision-making) with a focus on the design, implementation and evaluation and their evidence basis. The discussion is placed in different governance contexts so that similar policies can be compared across continents and countries. The course uses case studies to review the main types of policy problems, their generic solutions, and the indicators to be used to plan an effective implementation, evaluate policy adequacy, and decide on its adjustment or termination.

Testi Di Riferimento

Peters, B. G. & Pierre, J. (Eds.). (2006). Handbook of Public Policy. SAGE. – On Learn Fischer, F., Miller, G. J. & Sidney, M. S. (Eds.). (2007). Handbook of public policy analysis: theory, politics, and methods. CRC Press. – On Learn Bardach, E. & Patashnik, E. M. (2005). A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving. 2nd edition. CQ Press. – Moran, M., Rein, M. & Goodin, R. (Eds.). (2008). The Oxford Handbook of Public Policy. Oxford University Press. – HM Treasury. (2020). The Magenta Book. HM Treasury guidance on what to consider when designing an evaluation. GOV.UK. https://www.gov.uk/government/publications/the-magenta-book Caramani, D. (Ed.). (2014). Comparative Politics. 3rd edition. Oxford University Press. King, G., Keohane, R. O. & Verba, S. (2021). Designing Social Inquiry Scientific Inference in Qualitative Research. New edition. Princeton University. – Available via Perlego

Metodologie Didattiche

Each week, the course comprises a lecture (online) and a seminar (on campus). The lecture presents the main issues related to the content of the week. The seminar is a hands-on policy analysis class, stressing problem statements, indicators, stakeholder analysis and evaluation. The core readings are related to the lecture, although the seminar has some readings of its own, of a methodological nature to guide students into the analysis process. The seminar relies on the group structure of the class. Students will need to meet in groups at least twice on-site or online to agree on the basics of their short presentations and group assignments. Each group will present three times, so groups should organize in such a way that each individual student speaks at least once. Active participation in the class discussions and debates is expected and will be part of the final assessment. At Luiss, course attendance is mandatory. Students that cannot attend the course or part of it must provide official reasons (attested by a specific document, such as a medical certificate).

Modalità di verifica dell'apprendimento

The grade for this course will be determined on the basis of 4 components graded equally with 25% of the final grade: 1) Group peer learning exercise 1: Problem statement, presentation in class and submission of ppt. 2) Group peer learning exercise 2: Stakeholder analysis presentation in class and submission of ppt. 3) Group writing exercise: Written problem statement with flagging of indicators which can potentially be used for evaluation – 1200 words including figures, excluding references. 4) Final individual exam: Individual research design of a policy evaluation. Templates for all presentations will be uploaded prior to the course.

Criteri per l’assegnazione dell’elaborato finale

No criteria

Settimana 1

Week #1 Class design. What is public policy? How do we ask and answer a comparative research question? Readings: Wagner, P. (2007). Public policy, social science, and the state: An historical perspective. In F. Fischer, G. J. Miller, & M. S. Sidney (Eds.), Handbook of public policy analysis: Theory, politics, and methods (pp. 29-43). CRC Press. – On Learn Skocpol, T., & Amenta, E. (1986). States and social policies. Annual Review of Sociology, 12, 131-157. https://doi.org/10.1146/annurev.so.12.080186.001023 Werner, J. & Wegrich, K. (2007). Theories of the Policy Cycle. In F. Fischer, G. J. Miller, & M. S. Sidney (Eds.), Handbook of public policy analysis: theory, politics, and methods (Chapter 4, pp. 43-63). CRC Press. – On Learn Week #1 Class design. What is public policy? Group work: Random creation of groups. Case discussion: the impact of war sanctions against Russia. Background reading: Medunic, F. (2023, April 18). Damage done: Ways to measure European sanctions’ success against Russia. ECFR. https://ecfr.eu/article/damage-done-ways-to-measure-european-sanctions-success-against-russia/

Settimana 2

Week #2 Policy goals Where do public policy goals come from? Readings: Peters, G., & Pierre, J. (Eds.). (2006). The Handbook of Public Policy. SAGE. Chapters 25 & 26 (pp. 433-461). – On Learn Campbell, J. L. (1998). Ideas, politics, and public policy. Annual Review of Sociology, 24, 131-157. https://doi.org/10.1146/annurev.soc.28.110601.141111 Caplan, B. (2007). The myth of the rational voter: Why democracies choose bad policies. Cato Institute Policy Analysis Series, No. 594. Retrieved from http://www.cato.org/pubs/pas/pa594.pdf Week #2 Where do public policy goals come from? Group work: First group meeting to work on the problem statement. Background reading: Bardach, E. (2005). What Are Public Policy Problems? Problem Statement. In A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving (2nd ed., pp. 1-60). CQ Press. – On Learn

Settimana 3

Week #3 Tools of government and the art of crafting policy memos What can and should a government do and not do? Readings: Fukuyama, F. (2004). The imperative of state building. Journal of Democracy, 15(2), 17-31. https://doi.org/10.1353/jod.2004.0028 – Available on ResearchGate Hood, C. (2008). The tools of government in the information age. In M. Moran, M. Rein, & R. E. Goodin (Eds.), The Oxford handbook of public policy (Chapter 22, pp. 551-564). Oxford University Press. – Available via Perlego Ledbury, M., Choudhury, T., Paddick, S., & Patel, K. (2006). Understanding policy options. Home Office Online Report 06/06, pp. 4 – 14. Retrieved from https://www.academia.edu/14759367/Understanding_Policy_Options Week #3 Tools of government and the art of crafting policy memos Readings: Jones, N., & Walsh, C. (2008). Policy Briefs as a Communication Tool for Development Research. Overseas Development Institute (ODI). Retrieved from https://odi.org/en/publications/policy-briefs-as-a-communication-tool-for-development-research/ (n.d.). Decision / Action Memo Examples [PDF file]. Harvard Kennedy School, Shorenstein Center on Media, Politics and Public Policy. Retrieved from https://shorensteincenter.org/wp-content/uploads/2012/07/MEMO-WRITING-HO-ALL-Danziger.pdf

Settimana 4

Week #4 Market failure problems and solutions What are the main forms of market failure and the government responses? Readings: Ledbury, M., Choudhury, T., Paddick, S., & Patel, K. (2006). Understanding policy options. Home Office Online Report 06/06, pp. , 15-22; 28-53. Retrieved from https://www.academia.edu/14759367/Understanding_Policy_Options Zerbe Jr, R. O., & McCurdy, H. E. (1999). The failure of market failure. Journal of Policy Analysis and Management, 18(4), 558-578. – Available on GoogleScholar Week #4 Market failure problems and solutions Group work: Set 1 of problem statement presentation by groups (3-4).

Settimana 5

Week #5 Public goods problems and solutions What are collective action problems and how to solve them? Readings: Hardin, G. (1968). The Tragedy of the Commons. Science, 162(3859), 1243-1248. Retrieved from https://pages.mtu.edu/~asmayer/rural_sustain/governance/Hardin%201968.pdf Ostrom, E. (2008). Tragedy of the Commons. In S. N. Durlauf & L. E. Blume (Eds.), The New Palgrave Dictionary of Economics (2nd ed.). Palgrave Macmillan. Retrieved from https://dlc.dlib.indiana.edu/dlc/handle/10535/5887 Moffatt, M. (Mar 19, 2019). The Logic of Collective Action | Special Interests and Economic Policy. [A summary of The Logic of Collective Action (1971) by M. Olson]. https://www.thoughtco.com/the-logic-of-collective-action-1146238 Bridoux, F., & Stoelhorst, J. W. (2022). Stakeholder governance: Solving the collective action problems in joint value creation. Academy of Management Review, 47(2). https://doi.org/10.5465/amr.2019.0441 – Available via Perlego Week #5 Public goods problems and solutions Group work: Set 2 of problem presentation by groups (4-7).

Settimana 6

Week #6 Government failure problems and solutions What is government failure and how can such policy problems be dealt with? Readings: Buchanan, J. (1988). Market failure and political failure. Cato Journal, 8(1), 1-13. Retrieved from http://www.cato.org/pubs/journal/cj8n1/cj8n1-1.pdf Ostrom, V., & Ostrom, E. (1971). Public choice: A different approach to the study of public administration. Public Administration Review, 31(2), 203-216. https://doi.org/10.2307/974676 – Available via Perlego Week #6 Government failure problems and solutions Group work: Set 3 of problem presentation by groups (8-10).

Settimana 7

Week #7 Mapping interests in public policy. Stakeholder analysis How do we map group interests and reconcile them in public policy? Readings: Burstein, P. (2003). The impact of public opinion on public policy: A review and an agenda. Political Research Quarterly, 56(1), 29-40. Available on GoogleScholar Goodin, R., Rein, M., & Moran, M. (2008). The public and its policies. In M. Moran, R. E. Goodin, & M. Rein (Eds.), The Oxford Handbook Of Public Policy (pp. 3-21). Oxford University Press. – Available via Perlego Week #7 Mapping interests in public policy. Stakeholder analysis Group work: Group stakeholder analysis exercise (4 presentations).

Settimana 8

Week #8 Implementation of public policy Who and how implements public policy? Readings: Winter, S. C. (1986). Implementation. In B. G. Peters & J. Pierre (Eds.), Handbook of Public Administration (pp. 151-167). SAGE. – On Learn Sabatier, P. A., & Mazmanian, D. A. (1979). The implementation of public policy: A framework of analysis. Policy Studies Journal, 8(4), 538-560. https://doi.org/10.1111/j.1541-0072.1980.tb01266.x – Available via Perlego Mungiu-Pippidi, A. (2012). The Quality of Government and Public Administration. – Available on ResearchGate Week #8 Implementation of public policy Group work: Group stakeholder analysis exercise (6 presentations).

Settimana 9

Week #9 Evaluation research. Choosing indicators How are indicators for policy evaluation selected? What makes an effective indicator? Readings: Vedung, E. (2000). Evaluation Research. In P. G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 23, pp. 397-406). SAGE. – On Learn UNAIDS. (2010). Monitoring and Evaluation Fundamentals: An Introduction to Indicators. https://www.unaids.org/sites/default/files/sub_landing/files/8_2-Intro-to-IndicatorsFMEF.pdf Centers for Disease Control and Prevention. (n.d.). Step by Step – Evaluating Violence and Injury Prevention Policies. Brief 5: Evaluating Policy Impact. https://www.cdc.gov/injury/pdfs/policy/Brief%205-a.pdf Week #9 Evaluation research. Choosing indicators Readings: UN. (n.d.). Evaluation for Agenda 2030 | Department of Economic and Social Affairs. https://sdgs.un.org/publications/evaluation-agenda-2030-18064 OECD. (Jan 21, 2022). Policy Responses to Coronavirus (COVID-19). https://www.oecd.org/coronavirus/policy-responses/first-lessons-from-government-evaluations-of-covid-19-responses-a-synthesis-483507d6/

Settimana 10

Week #10 Evaluation research. Surveys and experimental designs What role do experiments and surveys play in evaluation policy? Readings: Mitchell, D. J. B. (2005). The Use (and Misuse) of Surveys Research in Policy Analysis. In G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 24, pp. 369-381). SAGE. – On Learn Krosnick, J. A., & Presser, S. (2009). Question and Questionnaire Design. In J. D. Wright & P. V. Marsden (Eds.), Handbook of Survey Research (2nd ed., pp. 263–313). Elsevier. – On Learn Price, P. C., Jhangiani, R., & Chiang, I-C. A. (2015). Conducting Surveys. Retrieved from https://opentextbc.ca/researchmethods/chapter/conducting-surveys/ Week #10 Evaluation research. Surveys and experimental designs Class Exercise (bring laptops): Questionnaire and survey design development.

Settimana 11

Week #11 Evaluation – cost effectiveness How do we arrive at short listed options and compare them in order to make a decision? Readings: Vining, A. R., & Weimer, D. L. (2011). Efficiency and cost-benefit analysis. In G. Peters & J. Pierre (Eds.), Handbook of Public Policy (Chapter 24, pp. 417-432). SAGE. – On Learn Mungiu-Pippidi, A. (2022). Transparency and corruption: Measuring real transparency by a new index. Regulation & Governance, 16(1), 168-180. Retrived from https://onlinelibrary.wiley.com/doi/10.1111/rego.12502 Week #11 Evaluation – cost effectiveness Class Exercise (bring laptops): Groups will use Open Tender (www.opentender.eu) and the T-index (www.corruptionrisk/transparency).

Settimana 12

Week #12 Qualitative evaluation designs How can understand not only what the public wants or does not want from government, but also why? Reading: Kahan, J. P. (2001). Focus Groups as a Tool for Policy Analysis. Analyses of Social Issues and Public Policy, 1(1), 129-146. http://dx.doi.org/10.1111/1530-2415.00007 – Available via Perlego Week #12 Qualitative evaluation designs Class Exercise: The art of focus group. Live focus group in class (10 students participate, the others assist)