Nonresponse Bias: Three Paradoxes

November 17, 2015, 15:00 h

Mannheim, MZES in A5,6, Room A231

Roger Tourangeau 

Abstract

A number of recent papers suggest that nonresponse bias is not a function of the nonresponse rate for a survey, even though the mathematics of nonresponse is clear—the rate of nonresponse should be related to the level of nonresponse bias.   Thus, one paradox is why nonresponse rates predict nonresponse bias so poorly.   The second paradox involves other study-level statistics, such as the R-Indicator.  Unfortunately, if most of the variance in nonresponse bias is within-survey rather than across surveys, no single number can predict the average level of nonresponse bias for a survey.  Thus, the second paradox is that no survey-level measure is likely to be useful for assessing the overall quality of a data collection effort or managing the field work.  Finally, many researchers have proposed that adaptive designs can be used to improve data quality.  However, given the first two paradoxes (the weak relation between nonresponse bias and nonresponse rates and the absence of a good summary measure of the impact of nonresponse), it is not clear how well researchers can redirect efforts to improve the quality of a survey.  The paper reviews both empirical efforts and simulation studies to evaluate responsive and adaptive designs and finds that, as expected, the gains are quite small or nonexistent.

About the speaker

Roger Tourangeau is a Vice President and Associate Director at Westat.  Before joining Westat in 2011, he was Research Professor at the University of Michigan’s Survey Research Center and the Director of the Joint Program in Survey Methodology at the University of Maryland.  He has been a survey methodologist for more than 30 years.  He is the lead author of a recent book on web survey design (The Science of Web Surveys) with Fred Conrad and Mick Couper, published by Oxford University Press in 2013.  His earlier book (The Psychology of Survey Response, with Lance Rips and Kenneth Rasinski) received the 2006 Book Award from the American Association for Public Opinion Research.  In 2002, he received the Helen Dinerman Award, the highest honor given by the World Association for Public Opinion Research.  He was also elected a Fellow of the American Statistical Association in 1999.  He is also the founding co-editor of the Journal of Survey Statistics and Methodology.