Measuring political knowledge in web-based surveys: An experimental validation of visual versus verbal instruments

May 05, 2015, 13:45 h
Mannheim, Conference room B2,1

Simon Munzert (Universität Konstanz)

Abstract:

Does the opportunity to deliver visual instead of verbal stimuli of political knowledge to respondents in web-based surveys make a difference in terms of data quality? For instance, does the presentation of visual knowledge items reduce cheating, that is, looking up the answer via the web? And do visual and verbal stimuli capture the same underlying construct? We propose a novel measurement strategy in which the retrieval of political knowledge is almost exclusively based on visual, not verbal stimuli. To assess the quality of the instruments, we administer a question form experiment in an online survey of the German Longitudinal Election Study. Respondents are randomly assigned to one of two question formats—visual or verbal—and are asked to solve a set of eight questions on political leaders and their offices. The instruments are validated based on nonparametric item response theory and analyses of response latency. While there is no clear evidence for cheating behavior under either of the conditions, both instruments form strong knowledge scales. Results from a regression analysis indicate that the scales provide measures of closely related but not identical concepts.

About the Speaker:

Simon Munzert absolvierte von 2005 bis 2011 sein Bachelor- und Master-Studium der Politik- und Verwaltungswissenschaft an der Universität Konstanz. Er ist wissenschaftlicher Mitarbeiter am Lehrstuhl für Umfrageforschung und promoviert zur Messung öffentlicher Meinung in Subpopulationen.  Zu seinen Forschungsinteressen gehören Methoden der Wahlvorhersage, Messung öffentlicher Meinung, Bayesianische Modellierung sowie Web-Scraping-Verfahren.