Course 8 - Nonresponse, Week 1: Unit Nonresponse (Course 8.1)

August 13-17

Instructors: Ineke Stoop, Jelke Bethlehem

Course content

Unit nonresponse is a serious concern because it can affect survey results. This course describes two general approaches to deal with the unit nonresponse problem. One is to reduce nonresponse in the fieldwork as much as possible. The other is to correct for the negative aspects as much as possible.

Survey response rates can be increased through a wide range of tactics and strategies, partly related to the fieldwork mode. The course will discuss in detail how to enhance contact rates and how to minimize refusal. It will pay special attention to the risk of unbalanced response rates (i.e. high response rates among one group and low among another) and how to tackle this. The course discusses several indicators for the quality of the survey response. One is the response rate. This is an imperfect indicator. Therefore also another indicator is proposed: the R-indicator.

Notwithstanding all efforts to reduce nonresponse as much as possible, there will always remain an amount of nonresponse. To avoid biased estimates of population characteristics, some adjustment procedure must be carried out. An overview of adjustment techniques will be given. This includes adjustment weighting, use of propensity scores, a follow-up survey among nonrespondents and the Basic Question Approach. For the analysis and correction of nonresponse, auxiliary variables are of the utmost importance. This is stressed throughout the course. Attention is paid to sources of auxiliary variables, including paradata.

The course will focus on nonresponse in a cross-national survey, the European Social Survey. This means that evidence from a wide range of European countries will be available and that the main emphasis is on face-to-face studies. Some attention will be paid to the issue of nonresponse in online panel surveys.

The course will include two types of exercises: computer exercises and practical assignments.  In computer exercises, participants will carry out a nonresponse analysis of a real survey data set. They will do this using the R-language. In addition, participants will have to prepare short presentations on nonresponse issues, based on the literature and if possible their own experience.

Course and learning objectives

The course caters to the needs of several target audiences. After following the course, substantive researchers will be more aware of the possible impact of nonresponse on their survey outcomes, and how to handle this; survey designers will be familiar with strategies to enhance response rates and to minimize nonresponse bias; and survey methodologists will get a concise overview of current developments in the area of nonresponse research.

Course prerequisites

General knowledge of survey research is required, especially on sampling and survey modes. Some basic knowledge of R helps, but is not required. Note: Participants will need to bring a laptop computer in order to perform the practical exercises in this course. Participants should download and install R on their laptops before coming to the summer school. The software is available for free at www.r-project.org

Exam

Please note that for this course, the optional exam will be in form of a classroom exam, taking place on August 25th, 10:00-13:00.

Recommended reading

  1. Bethlehem, J. G., F. Cobben & B. Schouten (2011). Handbook of Nonresponse in Household Surveys. Hoboken, NJ.
  2. Billiet, J., H. Matsuo, K. Beullens & V. Vehovar (2009). Non-Response Bias in Cross-National Surveys. Designs for Detection and Adjustment in the ESS. Institute of Philosophy and Sociology Publishers (Ed.). ASK. Research & Methods, 18 (1): 3-43.
  3. Billiet, J., M. Philippens, R. Fitzgerald & I. Stoop (2007). Estimation of Nonresponse Bias in the European Social Survey. Using Information from Reluctant Respondents. Journal of Official Statistics, 23 (2): 135-162.
  4. Groves, R. M. (2006). Nonresponse Rates and Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70: 646-675.
  5. Koch, A., A. G. Blom, I. Stoop & J. Kappelhof (2009). Data Collection Quality Assurance in Cross-National Surveys. The Example of the ESS. Methoden Daten Analysen. Zeitschrift für Empirische Sozialforschung, 3 (2): 219-247.
  6. Stoop, I. (2005). The Hunt for the Last Respondent. Nonresponse in Sample Surveys. The Hague.
  7. Stoop, I., J. Billiet, A. Koch & R. Fitzgerald (2010). Improving Survey Response. Lessons Learned from the European Social Survey. Chichester.

About the instructors

Dr. Ineke Stoop is head of the Department of Data Services and IT, The Netherlands Institute for Social Research/SCP. She studied psychology at Leiden University and obtained her PhD at Utrecht University for a thesis on survey nonresponse. She is a member of the European Statistical Advisory Committee (ESAC) and of the Central Coordinating Team of the European Social Survey, Scientific Secretary of the International Association of Survey Statisticians, and a Laureate of the 2005 Descartes Prize for Excellence in Scientific Collaborative Research. Her main research interests are comparative social surveys and nonresponse. She has taught courses on comparative surveys and nonresponse as part of the ECPR summer school, as ESS training courses, and at Dutch universities, and recently co-authored a book on nonresponse in the European Social Survey.

Prof. Dr. Jelke Bethlehem is Senior Survey Methodologist in the Methodology Department of Statistics Netherlands. He is also part-time professor in Survey Methodology at the University of Amsterdam. He studied mathematics and statistics, also at the University of Amsterdam. From 1974 to 1978 he worked at the Mathematical Centre in Amsterdam, a research centre for mathematics and computer science. Since 1978 he is involved in research and development at Statistics Netherlands. He obtained his PhD on the treatment of nonresponse in surveys in 1986. Other research topics are adjustment weighting, disclosure control and web surveys. He published two books: “Applied Survey Methods” and “Handbook of Nonresponse in Household Surveys”.

Back