Cognitive Pretests

Timo Lenzner

Survey Design and Methodology

Fragebogendesign & -evaluation
Mitarbeiter

+49 (621) 1246-227
E-Mail
vCard

Timo Lenzner

Cornelia Neuert

Survey Design and Methodology

Fragebogendesign & -evaluation
Mitarbeiter(in)

+49 (621) 1246-225
E-Mail
vCard

Cornelia Neuert

Wanda Otto

Survey Design and Methodology

Cross-cultural Surveys
Mitarbeiterin

+49 (621) 1246-502
E-Mail
vCard

Wanda Otto

Pretesting is generally considered to be indispensable to survey research for the successful development of a questionnaire, and for the quality of the data obtained by the survey.

The GESIS Pretest Lab supports researchers in optimizing their survey questions prior to data collection with various services. In addition, we provide researchers access to the findings of our cognitive pretests via the GESIS pretest database.

Services

The GESIS Pretest Lab supports researchers in optimizing their survey questions prior to data collection with the following services:

Conducting cognitive pretests (in combination with eye tracking)

In cognitive pretests, we evaluate the comprehensibility of questions, identify problems respondents may have in answering and generate constructive suggestions for eliminating such problems. The pretests work well with all modes of questioning (personal-oral, telephone, written, online).

The additional application of eye tracking offers the possibility to follow and record (live) the eye movements of subjects. Our research findings suggest that this procedure helps to discover additional problems with questions (Neuert & Lenzner, 2015a, 2015b).

How do we conduct cognitive pretests?

  • With small samples of 10 to 30 participants
  • The individual interviews are limited to 60 to maximally 90 minutes
  • Per interview, we are able to test about 50 questions or items with the eye tracker and thereof about 20 questions or items via pre-scripted cognitive pretesting techniques.

Conducting cognitive online pretests (web probing)

Cognitive online pretests are an alternative to pretests conducted in the lab. With the probing questions used it is possible, for example, to determine whether all respondents interpret a question in the same way or whether there are systematic differences in the understanding of a question between different groups of respondents.

How do we conduct cognitive online pretests?

  • With samples of 80 to 120 participants
  • The online questionnaires are designed to take about 20 to 30 minutes to complete
  • Per questionnaire, we are able to test about 10 to 15 questions or items 

Consulting on choosing and independent usage of various pretest procedures

Depending on the chosen mode of survey (personal-oral, telephone, written, online) various pretesting procedures for optimizing questionnaires are available to researchers (for example: cognitive interviewing, web probing, interviewer/respondent debriefing, focus groups, eye tracking, behavior coding). We are happy to consult with researchers in choosing and utilizing the appropriate pretesting procedures for those wanting to test their own survey instruments themselves.

Please note:

Please contact us as soon as possible, so we can consider your pretest in our planing and will block binding time slots.

Costs & Duration

Consultations on pretesting procedures are free of charge. The costs for conducting the pretests are project-specific and based on the number of test persons, number of questions being tested and the pretesting method to be used.

Example of a cognitive pretest (with/without eye tracking):

We normally conduct cognitive pretests with 16 participants. Planning, execution and evaluation of pretests such as this generally takes 8 to 10 weeks, including the written report on the results.

Example of a cognitive online pretest:

Cognitive pretests are usually conducted with 100 to 120 participants and 10 questions to be tested. Planning, execution and evaluation of pretests such as this generally takes 5 to 6 weeks, including the written report on the results.

References

Our varied client list thus far includes a number of externally funded research projects, in addition to the following survey programs, institutes and agencies: