Do Audit Experiments Reflect Applicant Behavior? Cautions for Calculations of Probabilities of Success

LIEPP' Discriminations and category-based policies and OSC are glad to invite you to the seminar: 

Do Audit Experiments Reflect Applicant Behavior? Cautions for Calculations of Probabilities of Success

May 20th, 11h30-13h

Location: Room K008, 1 place Saint Thomas d'Aquin

Manatory registration

Speaker: Michael Vuolo, PhD, is an Associate Professor in the Department of Sociology at The Ohio State University and Editor-in-Chief of Sociological Methodology, the official methods journal of the American Sociological Association. His research interests include law, crime, and deviance; health; employment; substance use; the life course; and statistics and methodology.

Abstract: Audit and correspondence studies have flourished in sociology and related disciplines. By sending actual applications that differ only by particular treatments, this method allows researchers to tap into discrimination by decision-makers such as employers, landlords, and schools that surveys and qualitative interviews are unlikely to reveal, with the strong internal validity of an experiment. However, the applicant is assumed to apply to all available openings for which they are qualified, and the probability of success represents an estimate at the unit level rather than that of the applicant. This presentation uses two studies in progress to consider these two assumptions. First, I present the results of an experiment and qualitative interviews with individuals with criminal records regarding whether they apply for jobs based on inquiries regarding records appearing on job applications. We find that when applications have criminal record questions or warnings of a criminal background check, applicants are less likely to apply for a position and that the reasoning behind such self-selection differs by race and gender. In other words, applicants do not apply to all positions as audits typically assume. Second, I demonstrate via the binomial distribution that unit-level probabilities do not accurately reflect real world applicant chance of success. For most studies of the labor market, they underrepresent the effects of discrimination and do not consider the degree of choice in jobs that applicants of differing treatments attain. For studies of other units such as housing and schools, the unit-level probabilities that note discrimination are of relatively little consequence for real-world applicants. Both studies necessitate a new focus on applicants within audit experiments, with a consideration of both self-selection into applying and applicant-level probabilities of success.