Measurement error, quality of survey questions and the software SQP

June 25, 2013

Diana Zavala Rojas

Abstract

Measurement error in survey research introduces systematic bias to results and analysis. There are several procedures at the design stage of a survey to minimize measurement error and there is also a growing literature suggesting statistical correction.
However, there is a general difficulty in getting an approximation of the size of those errors. The most explored way is to have multiple measures of the same concept, but this is something very expensive and a burden for survey designers and respondents. We have proposed a new alternative by developing a program that predicts the reliability, validity and method effects of survey questions : Survey Quality Prediction (SQP).
This presentation presents the statistical and theoretical bases of the software: the predictions come from a meta-analysis of a large database of multi-trait multi-method (MTMM) experiments analyzed in combination with around 60 measurement properties of each survey items.
The strength of the software to get information of the quality of survey instruments to correct for measurement error will be highlighted, as well as the current limitations that this approach faces.