Blending Probability and Nonprobability Samples for Survey Inference

26. September 2018, 13:00 Uhr

GESIS, Mannheim, B2,8 (rechts)

Joseph Sakshaug

Abstract

Scientific surveys based on random probability samples are ubiquitously used in the social sciences to study and describe large populations. They provide a critical source of quantifiable information used by governments and policy-makers to make informed decisions. However, probability-based surveys are increasingly expensive to carry out and declining response rates observed over recent decades have necessitated costly strategies to raise them. Consequently, many survey organizations have shifted away from probability sampling in favor of cheaper non-probability sampling based on volunteer web panels. This practice has provoked significant controversy and skepticism over the representativeness and usefulness of non-probability samples. While probability-based surveys have their own representativeness concerns, comparison studies generally show that they are more representative than non-probability surveys. Hence, the survey research industry is in a situation where probability sampling is the preferred choice from an error perspective, while non-probability sampling is preferred from a cost perspective. Given the advantages of both sampling schemes, it makes sense to devise a strategy to combine them in a way that is beneficial from both perspectives. We examine this notion by evaluating a method of integrating probability and non-probability samples under a Bayesian inferential framework. The method is designed to utilize information from a non-probability sample to inform estimations based on a parallel probability sample. The method is evaluated through a real-data application involving two probability and eight non-probability surveys that fielded the same questionnaire simultaneously. We show that the method reduces the variance and mean-squared error (MSE) of a variety of survey estimates, with only small increases in bias, relative to estimates derived under probability-only sampling. The MSE/variance reductions are most prominent when a small probability sample is supplemented by a larger non-probability sample.

About the Speaker

Joseph Sakshaug is Distinguished Researcher, Head of the Data Collection and Data Integration Unit, and Acting Head of the Statistical Methods Research Department at the IAB. He is also Honorary Full Professor in the School of Social Sciences at the University of Mannheim. Previously, he was Associate Professor in Social Statistics at the University of Manchester, UK, and Assistant Professor of Statistics and Social Science Methodology at the University of Mannheim. He received his MS and PhD in Survey Methodology from the University of Michigan-Ann Arbor and BA in Mathematics from the University of Washington-Seattle. From 2011-2013 he was an Alexander von Humboldt Postdoctoral Research Fellow at the IAB and the Ludwig Maximilian University of Munich. He is also a faculty member in the International Program in Survey and Data Science, and an Adjunct Research Assistant Professor at the Survey Research Center of the Institute for Social Research at the University of Michigan. His research focuses on data quality issues in complex surveys, the integration of multiple data sources, and empirical research methods.