Sampling, Data Protection, and Quality Standards
Sampling
In the PIAAC Cycle 2 main study (2022/23), approximately 5,000 people were interviewed in each participating country. To this end, each country drew a representative random sample from their PIAAC target population. The target population comprises persons aged 16–65 years residing in private households in the participating country, irrespective of their nationality, residence status, or language skills.
In Germany, the sample was drawn from the population registers, using a two-stage stratified and clustered sampling design. In the first stage, municipalities from all over Germany were randomly selected. The latest version of the Federal Statistical Office's municipal directory served as a sampling frame. In the second stage, persons from the target population were drawn at random from the population register in each selected municipality.
Prior to the main study, a three-month field trial was conducted in 2021.
Data Protection and Ethics
The PIAAC International Project Management and the National Project Management of PIAAC in Germany have committed to comply with statutory data protection regulations and ethical guidelines. All institutions involved in implementing PIAAC in Germany work in accordance with applicable data protection regulations, in particular the EU General Data Protection Regulation (GDPR). At GESIS, compliance with data protection is ensured by the data protection officer. The survey data will be treated as strictly confidential. Personal data will not be made accessible to third parties, and it will be ensured that individual persons are not identifiable in the data.
The study is conducted according to ethical principles; GESIS has obtained a confirmation of ethical approval for the PIAAC study. Interviewers were specially prepared for their tasks in a five-day in-person training course. One aspect of training focused on ensuring that interviewers interact with target persons in a respectful way and respect their rights. Participation in PIAAC is voluntary; during the interview, the respondent may opt not to answer a question at any time. The selected persons will receive a letter, a data protection sheet, and a flyer informing them in detail about the study and their rights.
Quality Standards
The main objective of PIAAC is to produce high-quality cross-nationally comparable data on adult skills. The data are of particular public interest. They are used by scientists from a wide range of disciplines, and the results of the PIAAC study offer key players in policy-making and society a basis for policy interventions and strategy choices.
In order to achieve the objectives and to ensure that the data are of high quality, the survey must be conducted in a comparable and standardized way in all participating countries. To this end, the international PIAAC Consortium has defined extensive and detailed quality standards. The Consortium regularly monitors compliance with and the implementation of these international standards in the participating countries. In Germany, the internationally prescribed quality assurance standards are enhanced by further national measures.
The international PIAAC standards reflect the current state of research on survey methodology, and cover all areas of the survey life cycle:
- Quality assurance and quality control
- Ethics
- Study planning
- Sample design and sample selection
- Survey instruments
- Translation and adaptation of survey instruments
- IT hardware and configuration and integration with national survey management software
- Field management standards
- Interviewer training
- Data collection, fieldwork monitoring, and quality control of fieldwork
- Data management and verification
- Confidentiality and data security, data protection
- Weighting and variance estimation
- Quality indicators (e.g., response rate, sampling bias)
Sample Quality
For the interpretation of the PIAAC results, it is crucial that the respondents are representative of the target population. In order to accurately assess the current situation in the participating countries and to be able to compare the results cross-nationally, it is crucial that as many of the selected persons as possible participate in the survey. According to international standards, a response rate of at least 70% should be achieved. Countries with lower response rates must utilize nonresponse bias analyses to prove that the realized sample is not – or is only marginally – biased.
Quality Assurance Through Technical Advisory Group
Before data publication, the Technical Advisory Group will review each participating country's compliance with the quality assurance standards and determine the usability of its PIAAC data for analysis purposes (data adjudication). Key methodological aspects are the quality of the sample and the sampling procedure, coverage rate and sampling bias, data collection, and the quality of the instruments and the data. The results of this review will determine whether the data of the respective countries will be included in the international dataset and used in the international analyses.