Assessing Comparability and Explaining Why It is Absent

Katharina Meitinger (Assistant Professor, Utrecht University)

21st May, 11:30am, Online

As part of the Méthodes quantitatives et analyse des données seminar (La soufflerie interdisciplinaire Sciences Po)

There has been a tremendous increase in cross-national data production in social science research in recent decades. Before drawing substantive conclusions based on cross-national survey data, researchers need to verify whether the measures are indeed comparable. If cross-national data are not tested for comparability, researchers risk confusing methodological artifacts as “real” substantive differences across countries. Therefore, quantitative measurement invariance tests are an essential tool for researchers working with cross-national data.

However, researchers often find it particularly challenging to establish the highest level of measurement invariance, that is, exact scalar invariance. When measurement invariance is rejected, it is crucial to understand why this is the case. The recently developed qualitative method of web probing can supplement a quantitative assessment and reveal why measures are incomparable.

This presentation will start with a short introduction of different types of biases that can threaten the comparability of data (e.g., construct bias, item bias). This will be followed by an overview of quantitative measurement invariance tests and how missing exact scalar invariance is addressed in the quantitative approach (e.g., alignment, BSEM). It will introduce the qualitative approach of web probing and present results from an article by Meitinger (2017) that illustrates a mixed-methods approach to assess and explain (in)comparability with the example of patriotism and nationalism and the 2013 International Social Survey Program (ISSP) module on “National Identity.”

In addition, the presentation will also discuss further substantive applications of web probing, such as measures of gender attitudes, health, and national identity (national pride and patriotic feelings).

Registration is mandatory to join the ZOOM meeting (the link will be sent one day before).

MORE INFORMATION

Dr Katharina Meitinger, activities and publications (Utrecht University)