Course 17: Questionnaires for cross-cultural surveys

Instructors: Dorothée Behr, Brita Dorer, Silke Schneider

Date: August 25-29, 2014
Time: 14:00-18:00

About the instructors:

Dr. Dorothée Behr is a senior researcher at GESIS – Leibniz Institute for the Social Sciences, Germany. She holds a degree in translation studies and a doctorate on questionnaire translation. Her current research focuses on assessing item equivalence using qualitative data from web probing, translation vs. adaptation and translation assessment methodology. Previous projects include, for instance, the European Social Survey, the Programme for the International Assessment of Adult Competencies and PISA 2015, where she was responsible for guiding translation activities and/or was involved in instrument design.

Brita Dorer is a researcher at GESIS – Leibniz Institute for the Social Sciences, Germany, specialized in the field of questionnaire translation. She is heading the translation team of the European Social Survey (ESS). Her scientific interests are the evaluation and quality enhancement of questionnaire translation and adaptation, translatability of source questionnaires/advance translations, and intercultural aspects of questionnaire translation. She holds a degree in English, French and Italian translation studies from Johannes-Gutenberg-University Mainz, FTSK Germersheim, where she also worked as a freelance lecturer for English-to-German and French-to-German translation. She has been involved in translating survey questionnaires into German, such as ESS, ISSP, PIAAC and SHARE.

Dr. Silke Schneider is head of the Knowledge Transfer Department at GESIS – Leibniz Institute for the Social Sciences, Germany and engages in research on educational attainment and its measurement in cross-national surveys. Her main interests are in social stratification and social structure as well as survey methodology, especially measurement issues and cross-national comparability of socio-structural variables. She studied for her Master’s degree at the University of Cologne, after which she completed her doctorate at Nuffield College, University of Oxford. She is currently conducting a project on the development of a new instrument for measuring education in computer-assisted surveys.

Short course description:

Cross-cultural surveys are increasingly being set up in numerous disciplines and around the globe. They allow testing for the generality of hypotheses across cultures and systematic research into institutional and other contextual differences. While it is essential to have a sound knowledge of survey methodology in general when setting up these surveys, there are additional methodological aspects to take into account in order to collect cross-culturally comparable data. This course focuses on what it means to develop questionnaires for use in comparative research.

It is organized around the following themes: measurement in comparative surveys, developing questionnaires for cross-cultural surveys, measuring background variables and their harmonization, and best practice in questionnaire translation and translation assessment. We will also touch upon pretesting methods for cross-cultural surveys and assessing comparability of survey data as well as cultural factors impacting on response processes and survey data. Class hours will be a mixture between lectures, short practical exercises and discussions. During the study hours outside class, students will work on their own projects (if applicable) and/or gain hands-on experience in designing source questions, translating and reviewing translations, and harmonizing variables, with support from the instructors.

Here you can find the full syllabus (161 kB) of the course with complete information on the topics, literature, and day-to-day schedule.

Course prerequisites:

  • knowledge of at least one language besides English to be able to benefit from the practice sessions regarding translation;
  • interest in the impact of linguistic, cultural and institutional contexts on cross-cultural survey research;
  • knowledge of general questionnaire design principles is highly desirable; (this could be gained in the course "Questionnaire design" in the preceding week;
  • no specific statistics or software knowledge is needed.

Target group:

Participants will find the course useful if they:

  • are involved in developing a source questionnaire (attitude, personality, behaviour items, background variables) for several cultures and/or countries;
  • want to understand the implications of questionnaire design and translation as well as variable harmonization for the comparability of cross-cultural survey data;
  • organize questionnaire translation and assessment, or translate and review translated questionnaires themselves.

Course and learning objectives:

By the end of the course participants will:

  • be aware of cross-cultural requirements during source questionnaire development to ensure comparability;
  • be able to take informed decisions about how to measure social background ("demographic") variables to ensure comparability through design and harmonization;
  • be familiar with best practice in carrying out questionnaire translation and assessment;
  • be able to better account for the needs of cross-cultural questionnaire design and translation in project proposals.

Organizational structure of the course:

The course is structured around 4 hours of classroom instruction and 2 extra hours of individual and group work, either on student’s own projects or on assignments given by the course instructors. The assignments may include:

  • operationalizing a theoretical concept and developing source questionnaire items suitable for a cross-cultural survey;
  • harmonization exercises with background variables of interest to students (e.g. education, income or marital status), optionally using real survey data (please have a statistics package installed on your computer if you wish to do so);
  • translating and assessing questionnaire items by different methods (e.g. individually vs. in a team);
  • discussing translation and harmonization resources;
  • background reading.

Software and hardware requirements:

Course participants will need to bring a laptop computer for performing the practical exercises for this course.