LIEPP in the public policy evaluation landscape

© Céline Bansard

Interview with Anne Revillard, director of the Laboratory for the interdisciplinary evaluation of public policies (Laboratoire interdisciplinaire d’évaluation des politiques publiques, or LIEPP) since August 2020.

You are taking over the direction of LIEPP after having worked there for several years as a researcher : what role has LIEPP played in your research trajectory?

I am an associate professor of sociology at Sciences Po, where I have been a member of LIEPP and the Observatoire sociologique du changement (Observatory of Social Change) since 2012. My research trajectory is a good example of the impact that a place like LIEPP can have in terms of the emergence and development of new questions. My arrival at LIEPP has indeed marked a turning point in my research: whereas I had previously worked on public policies (in the areas of women's rights, institutional mediation and then disability) rather from the point of view of their definition and implementation, the questions arising from the field of evaluation have contributed to shift my focus towards the question of the consequences of policies for the people they target - a question I have explored based on the concept of policy reception.

In my latest research, I have studied the ways in which people with physical or visual disabilities experience different disability policy measures. At LIEPP, I have been very involved in leading the doctoral seminar, and then since 2015 in the creation and development, with Morgane Laouénan and Mirna Safi, of a research goup on discrimination. I am now taking over the direction of the laboratory from Bruno Palier, with whom I have worked and learned a lot in the past few months as Director of Development, in preparation for this new position.

The landscape of public policy evaluation has changed a lot in recent years (more and more people are talking about it, there are more and more actors who do evaluation) but the academic sphere still invests rather timidly in this field, few researchers venture into this field.

Indeed, academic actors today play a limited role in the field of evaluation in France. Evaluation remains more centrally a practice of public administrations or consultants in the private sector. This marginality of the academic world is paradoxical, insofar as the evaluation of public policies was originally defined, particularly in the United States, as a practice of applied social sciences, consisting on implementing methods derived from the social sciences to shed light on the meanings and consequences of various public action programmes. And this marginality is harmful, because scientific research has much to contribute to evaluation: not only in terms of methodological rigor, but also thanks to the autonomy that researchers have in defining their research agendas. The engagement of academic actors in evaluation, like other civil society actors or independent authorities, contributes to de-indexing the evaluation agenda from that of public authorities - in other words, it contributes to addressing to public policies questions they are not necessarily willing to ask themselves. This contributes to a diversification of points of view, but also often to a form of knowledge-based counter-power, which is essential in democratic terms. There is therefore an important political challenge, in my view, to give more space to knowledge from the University in the field of evaluation. However, academics are not always ready and willing to occupy this space. Firstly because of a general exhaustion in research circles, but also because of reticence towards an evaluation approach which is often perceived as strictly empirical and/or subordinated to the injunctions of a commissioner. There seems therefore to be a lack of understanding on both sides.

What role can LIEPP play in this context?

In this context, I fundamentally see the role of LIEPP as one of translation, mediation and organisation of a healthy confrontation between different worlds and different traditions. First of all, it is a question of demystifying and explaining the process of evaluating public policies to researchers : evaluation is certainly "applied" science, but this does not mean giving up one's autonomy or one's aspiration to theorisation and publication in "fundamental" science journals. LIEPP members also do theory and publish in very good journals. As far as autonomy is concerned, our deontological charter, co-signed with other university actors in evaluation, ensures the independence of the research carried out. In addition, our calls for projects, which are very open thematically, give researchers the opportunity to define their own evaluation questions in complete autonomy. However, the term "public policy" may also seem restrictive for some scientists, who do not necessarily define their research topics in these terms (they work, for example, on education, health, discrimination, etc.) even though their work has very useful implications for public action. Here again, pedagogical work is necessary.

Reciprocally, it is a matter of working to make room for academic work in the thinking of public actors. This implies an effort of communication, of translation of the questions and results of the research into more usual and easily assimilated terms for the actors of public policies. This is what our Policy briefs and the many events we organise to bring research results into dialogue with public actors contribute to.

LIEPP defines itself as an "interdisciplinary" laboratory for the evaluation of public policies: what about this interdisciplinarity?

It is the third dimension of this dialogue that we operate: it is a question of organising confrontation and complementarity between disciplines and between methods in evaluative research. In view of the nature of the approach and the history of the field, interdisciplinarity is consubstantial with the evaluation of public policies. As a problem-oriented approach, evaluation has historically disregarded disciplinary boundaries and borrowed from a diversity of methods. Nevertheless, the renewed interest in evaluation in recent years in France has been associated, both among public actors and within the academic world, with the promotion of a particular type of method, econometric methods, and in particular controlled experimentation. Without denying the interest of these approaches which several of our researchers practice, we choose, at LIEPP, to complete them and to make them dialogue with other approaches, notably qualitative ones. These allow us to question the categories used in quantitative research, to understand the genesis and the meanings of the programs under study, or to analyse their implementation and reception.

Relevant for evaluation, this choice of interdisciplinarity does not go without raising difficulties on the institutional level, in a scientific world still very organised into disciplinary silos (especially in terms of journals, career logic, etc.). It also raises epistemological challenges (tensions between positivist and constructivist approaches in particular). Identified at the beginning of the LIEPP project, these have been worked on over the years to encourage the development of a more reflective science. In this respect, I pay tribute to the work of my predecessors in the leadership of the LIEPP project, Etienne Wasmer, Cornelia Woll and Bruno Palier, who have organised, brought to life and brought to fruition this interdisciplinary dialogue which is the strength of our laboratory.

What are LIEPP's priority projects for the coming years?

At the dawn of its 10th anniversary (LIEPP was created in 2011), the laboratory is entering a new phase of development. This is marked, on an institutional level, by our redeployment in partnership with the University of Paris (UP). This redeployment is precisely for us an opportunity to reinforce our interdisciplinary approach and to open up new themes. In the early years of LIEPP, our interdisciplinarity mainly brought together sociology, economics and political science. The redeployment in partnership with UP is an opportunity to establish collaborations with researchers from the various UP laboratories in these three disciplines, but also to extend our interdisciplinary thinking to new horizons, including for example health, cognitive sciences, history, psychology, environmental sciences, etc. On a thematic level, our four founding research areas (on educational policies, discrimination, socio-fiscal policies and the evaluation of democracy) have redefined their orientations in preparation for this new phase, and two new research groups are being launched, on environmental policies and on health policies.

The activities of these six research groups will be fed by the projects that will be selected in the framework of two calls for projects that we launched last spring with a view to this redeployment, a general call for projects on the interdisciplinary evaluation of public policies, and another more specific call for projects on the interdisciplinary evaluation of gender policies.

At the same time, we have launched an initiative aimed at doctoral students and young PhDs in a perimeter including Sciences Po and the University of Paris, the young research programme, which aims to encourage exchanges between disciplines and between institutions based on (post-)doctoral research relevant to the evaluation of public policies. It is on the basis of all this new research, and the experience accumulated over the past 10 years, that we hope to demonstrate the value of an interdisciplinary approach to evaluative research. We are also reaching a stage of development where this approach can be more precisely shaped and theorised, based on our practices and in dialogue with the international field of evaluation. This is one of the objectives of the "Methods and Approaches in Evaluation" seminar cycle (METHEVAL).