Programme for the International
Assessment of Adult Competencies

Chair:  Anouk Zabal (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract:  PIAAC collects internationally comparable data on key foundation skills as well as a wide range of background information, and it offers rich opportunities for both academic and policy-relevant research. The second cycle of PIAAC continues a tradition of international adult literacy surveys and pursues the dual aims of monitoring change over time as well as introducing necessary adjustments and innovations to reflect societal and technological change. This symposium addresses the challenge of striking the right balance between consistency and change. The first contribution looks at how the theoretical frameworks for literacy and numeracy have been updated and enhanced for the second cycle of PIAAC, and how the cognitive instruments reflect the enriched constructs while maintaining the link to the first cycle. It will consider the reading components and the newly developed numeracy components, both designed to obtain more differentiated information at the lower end of the proficiency scale. Adaptive problem solving (APS) is a key skill in a rapidly changing world, and the assessment of this cognitive domain is an important innovation. The second contribution explores similarities and differences between the domain problem solving in technology-rich environments assessed in the first cycle of PIAAC and APS. It elaborates on the theoretical underpinnings of APS and discusses the challenges of operationalizing this construct for a large-scale assessment.The information collected through the background questionnaire lends PIAAC its analytical power. The third contribution discusses how constructs and measurements from the cycle one background questionnaire have been adapted or transformed to account for societal change while considering the need to maintain trend measurement. It also reviews new constructs that have been included to broaden the scope and strengthen the analytical possibilities. A valid, reliable, and comparable measurement of educational attainment is key for PIAAC. At the same time, the measurement of formal education is one of the most challenging in the context of achieving comparability across cultures and time. The approach followed in the second cycle of PIAAC to tackle these complex issues is presented and reflected upon in the fourth contribution.

Contributions                                                                              

1. Measuring Literacy and Numeracy Skills in Adults – Trends and Innovations

Laura Halderman (Educational Testing Service/ETS, USA) (555 kB)

2. From Problem Solving in Technology-Rich Environments to Adaptive Problem Solving – Concept and Measurement  

Samuel Greiff^* & Juliana Gottschling (University of Luxemburg, Luxemburg)   (6.82 MB)   

3. The Background Questionnaire: Maintaining Trend, Accounting for Societal Change, and Adding Innovative Elements

Tim Huijts (ROA, Maastricht University, The Netherlands) (397 kB)

4. Formal Education in PIAAC Cycle 2: Challenges and Opportunities

Silke Schneider (GESIS - Leibniz-Institute for the Social Sciences, Germany)  (423 kB)

*Presenting Author

Chair:  Simon Wiederhold (Catholic University Eichstaett-Ingolstadt, Germany)

Abstract:  Analysis of the PIAAC data has shown the crucial importance of cognitive skills for individual success. Thus, there is a profound interest in the determinants of cognitive skills. The literature consistently argues that family background plays a key role in the formation of skills, leading to a strong persistence in educational achievement across generations. However, surprisingly little is known about which family characteristics actually matter for skill formation. The four studies combined in this symposium try to dig deeper into the determinants of cognitive skills. Some of the questions addressed in the symposium are: How strong is the correlation of cognitive skills of parents and their children, and what are the mechanisms that give rise to the intergenerational persistence of skills? How important is the transmission of cultural values from parents to children as a determinant of cognitive skills? Finally, can policies substituting for lacking family support be successful in helping disadvantaged children to improve their skills?

Contributions

1. Can Mentoring Alleviate Family Disadvantage in Adolescence? A Field Experiment to Improve Labor-Market Prospects

Sven Resnjanskij¹, Jens Ruhose², Simon Wiederhold³ & Ludger Woessmann^1* (2.83 MB)  

(¹ifo Institute, Germany; ²University of Kiel, Germany; ³Catholic University Eichstaett-Ingolstadt, Germany)

2. Culture and Student Achievement: The Intertwined Roles of Patience and Risk-Taking

Eric A. Hanushek¹*, Lavinia Kinne², Philipp Lergetporer² & Ludger Woessmann²  (431 kB)

(¹Hoover Institution, Stanford University, USA; ²ifo Institute, Germany)

3. Individualism, human capital formation, and labor market success

Katharina Hartinger¹, Sven Resnjanskij², Jens Ruhose³ &  Simon Wiederhold¹*   (281 kB)

(¹Catholic University Eichstaett-Ingolstadt, Germany; ²ifo Institute, Germany; ³University of Kiel, Germany)

4. The intergenerational transmission of cognitive skills: An investigation of the causal impact of families on student outcomes

Eric A. Hanushek¹, Babs Jacobs², Guido Schwerdt^3*, Rolf van der Velden², Stan Vermeulen² & Simon Wiederhold⁴   (1.41 MB)

(¹Hoover Institution, Stanford University, USA; ²ROA, Maastricht University, The Netherlands; ³University of Konstanz, Germany; ⁴Catholic University Eichstaett-Ingolstadt, Germany)

*Presenting Author

Chair:  Julia Gorges (Philipps-University Marburg, Germany)

Abstract: The OECD PIAAC study provides in-depth information regarding adult competencies in a wide range of countries. Hence, some refer to PIAAC as “PISA for adults”. However, PIAAC is more than that. Thanks to its references to research from different disciplines and to different data sources, PIAAC has stimulated much research beyond adult competencies. Using data from the PIAAC background questionnaire, researchers have tested established assumptions on adult learning, investigated participation in education, and used national longitudinal extensions of the PIAAC study and other related data sources. This symposium brings together research beyond competencies from different disciplinary perspectives using a wide range of datasets. Using cross-sectional data from the first PIAAC data collection in 2012, MASSING explores the importance of participation in non-formal training in order to upgrade or to maintain skills among migrants in selected countries. In particular, this contribution will focus on how different policies in different countries might affect training participation and barriers to training. Drawing on data from the 1994–1998 International Adult Literacy Survey, the 2003-2008 Adult Literacy and Life Skills and the 2011-2012 OECD Survey of Adult Skills, MERONI focuses on overeducation and skill mismatch in different age cohorts and different countries. To identify skill mismatch and overeducation, this contribution looks at both skills and educational credentials. The contribution by HESSE uses the German longitudinal extension of PIAAC to investigate the role of motivation to learn and parenthood for participation in non-job-related training. The study provides insights into the neglected concept of adult motivation to learn. To look at PIAAC’s contribution in a broader context, Hernández-Torrano & Courtney conducted a bibliographic analysis of publications using large-scale datasets in educational research.  They describe the role of PIAAC data in current educational research alongside other large-scale assessments such as PISA. Each contribution takes a specific disciplinary perspective: sociology (MASSING), economics (MERONI), psychology (HESSE), and educational research (HERNANDEZ-TORRANO & COURTNEY). In this interdisciplinary symposium, disciplinary priorities and major topics building on PIAAC data will be highlighted and discussed.

Contributions

1. Training Participation of Migrants and Barriers to Training – A Cross-Country Comparison

Natascha Massing (GESIS - Leibniz-Institute for the Social Sciences, Germany)

2. An Age-Period-Cohort Approach to the Incidence and Evolution of Overeducation and Skills Mismatch

Elena Meroni (European Commission, DG Joint Research Centre) (759 kB)

3. How Compatible are Parenthood and Participation in Further Education and Training? Findings from the German PIAAC Longitudinal Study

Franzisca Hesse (Philipps-University Marburg, Germany)

4. Modern International Large-Scale Assessment in Education: An Integrative Review and Mapping of the Literature

Daniel Hernández-Torrano* & Matthew G.R. Courtney (Nazarbayev University, Kazakhstan)

Chair:  Débora B. Maehler & Silke Martin (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract: The symposium addresses content and methodological issues and aims to present research based on PIAAC data linked to administrative data or other large-scale country surveys. Thereby projects and studies of three countries will be presented. Initial research based on PIAAC 2011/2012 linked to register data in Norway will be presented (N = 901). The relationship between skills acquired before the age of 16 and skills acquired later to formation of skills and subsequent NEET status was addressed in the study. The second contribution is based on PIAAC data linked to administrative data from the Institute for Employment Research (IAB) in Germany (N = 2.086). The study investigates the measurement error resulting from the difference between information on earning available in both data sources. The third contribution is based on the PISA Young Adult Follow-Up Study (PISA YAFS) conducted in the US. Study design and results comparing for instance literacy from students in PISA 2012 and in the follow up 2016 using PIAAC instruments, assessed as repeated measure, will be presented (N = 2.320). The study examined the relationship between performance, employment and educational outcomes of young adults at age 19 and their earlier reading and mathematics performance in PISA 2012 at age 15.

Contributions

1. NEET Status and Early versus Later Skills Among Young Adults: Evidence from Linked Register-PIAAC Data (using Nordic PIAAC Data)

Erling Barth¹, Anna-Lena Keute²*, Pål Schøne¹, Kristine von Simson¹ & Kjartan Steffensen²   (393 kB)

(¹Institute for Social Research, Norway; ²Statistics Norway, Norway)  

2. What’s my Wage Again? Comparing Survey and Administrative Data to Validate Earning Measures (using German PIAAC-L Data)

Britta Gauly, Jessica Daikeler*, Tobias Gummer & Beatrice Rammstedt (GESIS - Leibniz-Institute for the Social Sciences, Germany)    (710 kB)

3. Administering Education and Skills Online (ESO) to PISA 2012 Cohort in the United States: Findings from the 2012-2016 PISA Young Adult Follow-up Study (using PISA YAFS)

Saida Mamedova¹*, Maria Stephens¹, Yuqi Liao¹, Josh Sennett¹, Paul Sirma¹ & Samantha Burg²     

(¹American Institutes for Research/AIR, USA; ²National Center for Education Statistics, USA)    (1.15 MB)   

*Presenting Author

Chair:  Martin Ehlert (WZB Berlin Social Science Center, Germany)

Abstract:  Learning during adulthood after initial education and training is a widespread and growing phenomenon in many countries. Especially the increasing pace of technological change and digitalization is likely to further fuel this trend. New technology has the potential to change the labor market as well as many other aspects of life. The result is that skills acquired during initial training no longer suffice to remain employable and to participate in social life. Therefore, skill acquisition later in life becomes more important to ensure that nobody is left behind. Therefore, many politicians and pundits advocate lifelong learning to ensure inclusive growth despite the challenges of digitalization. Yet, it is not clear whether current adult education and training programs and systems are capable to counteract the negative repercussions of digitalization. It is well known that access to adult training and education is unequally distributed. Less-educated and less-skilled people participate less often in training courses and other learning activities. Also, workers that are more likely to lose their jobs because of technological change are less likely to take part in learning opportunities. Thus, adult education and training programs often do not reach those most in need of skill acquisition later in life. At the same time, the benefits of existing learning arrangements for adults are often unclear. Many studies show that participation in further training courses does not lead to higher skills or better jobs. Against this background, this symposium aims to advance the knowledge on lifelong learning and cognitive skills. Based on new empirical results, we will discuss both the determinants of participation in lifelong learning and its effect on the development of cognitive skills. Also, we will discuss the acquisition and development of skills over the life course more generally. A special focus will be on cross-national comparative studies. Using results on participation and skill acquisition from different countries, we aim to assess the influence of institutions such as adult education and training policies. Furthermore, we will especially discuss lifelong learning among the most vulnerable groups such as less-skilled workers.

Contributions

1. Does Learning Help Workers to Keep up with Technological Change? Cross-National Variance in the Effect of Further Training on Problem-Solving Skills.

Martin Ehlert^1*, Marie-Christine Fregin², Didier Fouarge², Mark Levels², Liisa Martma³ & Rolf van der Velden²

(¹WZB Berlin Social Science Center, Germany; ²ROA, Maastricht University, The Netherlands; ³ Tallinn University, Estonia) 

2. Skills and Education as Factors of Participation in Non-Formal Learning

Liisa Martma* & Ellu Saar (Tallinn University, Estonia)   (1.76 MB)

3. Cross-National Variation in the Training Disadvantage of Less-Educated Employees: The Role of Job Allocation versus Skills

Carla Hornberg*, Heike Solga & Jan Paul Heisig (WZB Berlin Social Science Center, Germany) 

*Presenting Author

Chair:  Frank Goldhammer, Ulf Kröhne & Carolin Hahnel 

(DIPF | Leibniz Institute for Research and Information in Education, Germany)

Abstract:  The first round of the PIAAC study 2011–2012 was innovative in many ways. It was not only the first international large-scale assessment to be mainly computer-based, but it was also the first large-scale assessment that systematically collected log data, that is, events, event-related attributes, and timestamps reflecting the test taker’s interactions with the PIAAC assessment system. Log data found their way into the public use file in the form of generic process indicators such as time on task or number of interactions by item. Moreover, the majority of countries participating in PIAAC round one provided their log data that was recorded during the computer-based assessment to make it publicly available to the research community. Together with the result data and the data from the background questionnaire, the PIAAC log data already has inspired intensive research activities (for an overview see Goldhammer et al., 2020).  In this invited symposium, we have brought together four recent contributions dealing with PIAAC process data from different – both substantive and methodological – perspectives. The first presentation by Pokropek and colleagues addresses the question of how process data can be used to explore gender differences in literacy as assessed in PIAAC and PISA. The following contribution by Hahnel and colleagues investigates strategies of information processing in simulated web search environments included in the PIAAC assessment of problem solving in technology-rich environments. The third presentation by He and colleagues focuses on evaluating consistency of adult behavioral patterns across multiple problem solving tasks using PIAAC process data from the US. The final presentation by Maddox is about the PIAAC testing situation at home and considers process data derived from video data, log data etc. to capture sources of variation in the test administration.

Contributions

1. Utilizing Process Data to Examine Gender Differences in Literacy in PISA and PIAAC.

Artur Pokropek^1*, Francesca Borgonovi², & Lale Khorramdel³

(¹Institute of Philosophy and Sociology of the Polish Academy of Sciences, Poland; ²University College London, UK, and OECD, France; ³ Boston College, USA)

2. Identifying Strategies of Information Processing in PIAAC Web Search Environments

Carolin Hahnel^*, Frank Goldhammer, & Ulf Kroehne (DIPF | Leibniz Institute for Research and Information in Education, Germany)   (1.00 MB) 

3. Evaluating Consistency of Adult Problem-Solving Behaviors across Multiple Tasks Using PIAAC Process Data

Qiwei He^1*, Dandan Liao², Hok Kan Ling³, & Hong Jiao⁴  

(¹Educational Testing Service, USA; ²Cambium Assessment, Inc, USA; ³ Queen’s University, Canada; ⁴ University of Maryland, USA) 

4. Testing Situations at Home

Bryan Maddox (University of East Anglia & Assessment Micro-Analytics Ltd, UK)  (718 kB)

*Presenting Author

Chair:  Dorothée Behr (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Contributions                                                                              

1. Effects of response styles on secondary analysis in International large-scale assessments.

Tomasz Żółtak*, Artur Pokropek & Marek Muszyński (Polish Academy of Sciences, Poland) 

2. Is careless responding also a problem in face-to-face mode? Analysis of PIAAC noncognitive data.

Marek Muszyński*, Tomasz Żółtak & Artur Pokropek (Polish Academy of Sciences, Poland)  (325 kB) 

*Presenting Author

Chair:  Simon Wiederhold (Catholic University Eichstaett-Ingolstadt, Germany)

Contributions

1. Labour market returns to adult literacy and numeracy skills: a focus on migrant assimilation over the lifecycle.

Christopher Erwin¹*, Lisa Meehan¹, Marco Paccagnella², Gail Pacheco¹ & Stephen Reder³

(¹Auckland University of Technology, New Zealand; ²OECD, France; ³Portland State University, USA) 

2. The part-time wage gap along the distribution: The role of cognitive and computer skills and job characteristics.

Britta Gauly¹* & Bernd Fitzenberger²

(¹GESIS – Leibniz-Institute for the Social Sciences, German; ²Institute for Employment Research (IAB), Germany)

3. Skills use and their effect on employment income over time: Evidence from the longitudinal and international study of adults.

Alexander El-Hajj* (Statistics Canada, Canada)  

4. Selection corrected wage gaps within occupations in Germany.

Caroline Neuber-Pohl^1* & Britta Gauly²

(¹Federal Institute for Vocational Education and Training (BIBB), Germany; ²GESIS – Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author

Chair:  Jan P. Heisig (Berlin Social Science Center, WZB, Berlin)

Contributions

1. What are the skills of parents of school-age children?

Saida Mamedova* & Emily Pawlowski* (American Institutes for Research, USA) (628 kB) 

2. Knowing teacher´s skills in Slovakia: Evidence for further education of teachers.   

Olga Zelmanová* & Zuzana Wirtz (National Institute for Certified Educational Measurements - NICEM, Slovakia) (1.16 MB) 

3. Inequality in quality: Population heterogeneity in literacy skills around the world.

Claudia Reiter* (University of Vienna, Austria)  

*Presenting Author

Chair:  Beatrice Rammstedt (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Contributions

1. Soft skills and immigrants’ economic outcomes in Europe: Evidence from PIAAC data.

Agnieszka Kanas* & Menno Fenger (Erasmus University Rotterdam, The Netherlands) 

2. To score or not to score? A simulation study on the performance of test scores, plausible values, and SEM in regression with socio-emotional skill or personality scales as predictors.

Nivedita Bhaktha* & Clemens Lechner (GESIS – Leibniz-Institute for the Social Sciences, Germany)

3. Socio-emotional skills and labour market outcomes: New evidence using the Canadian longitudinal PIAAC study.

Ashley Pullman* & Ross Finnie (University of Ottawa, Canada) (485 kB) 

*Presenting Author

Chair:  Mariya Brussevich (International Monetary Fund, USA)

Contributions

1. Automation and human capital investment.

Sidharth Rony * & Arnaud Chevalier (Royal Holloway University of London, UK)

2. Who will bear the brunt of lockdown policies? Evidence from tele-workability measures across countries.

Mariya Brussevich*, Era Dabla-Norris & Salma Khalid (International Monetary Fund, USA) 

3. Using cognitive skills of ordinary citizens to predict preparedness for pandemic: Lessons from PIAAC.

Chong Ho Yu^1*, David Zizhong Xiao² & Jolia Awadallah³

(¹Azusa Pacific University, USAy; ²University of Maryland, USA; ³Alliant International School of Psychology, USA)

*Presenting Author

Chair:  Anouk Zabal (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Contributions                                                                              

1. The impact of text characteristics on gender differences in adults’ literacy skills.

Ai Miyamoto^1*, Britta Gauly² & Anouk Zabal²

(¹University of Freiburg, Germany; ²GESIS – Leibniz-Institute for the Social Sciences, Germany)   (807 kB)

2. Statistical literacy assessment – necessity and framework.

Tanja Ihden* (IMC University of Applied Sciences Krems, Austria) 

3. What makes mathematics difficult for adults? The role of reading components in solving math items.

Nadine Cruz Neri*, Jenny Wagner, & Jan Retelsdorf (University of Hamburg, Germany)

4. PIAAC’s Survey of Adult Skills and low literacy / functional illiteracy.

Aleksandar Bulajić^12*, Réka Vágvölgyi¹, Kirstin Bergström¹ & Thomas Lachmann¹

(¹University of Kaiserslautern, Germany; ²University of Belgrade, Serbia)

*Presenting Author

Chair:  Corinna Kleinert (Leibniz Institute for Educational Trajectories, LIfBi, Bamberg)

Contributions

1. Non-conventional pathways and credential accumulation behaviours in postsecondary education in Canada: Statistical portrait and evaluation of labour market outcomes.

Xavier St-Denis^1*, Yacine Boujija² & Stephen Sartor³

(¹Institut National de la Recherche Scientifique, Canada; ²Université de Montréal, Canada; ³University of Western Ontario, Canada)

2. Getting it right: Identifying literacy and numeracy skill mismatch in OECD countries using the job analysis method.

Sandra Pérez Rodriguez^1*, Rolf van der Velden², Tim Huijts² & Babs Jacobs²

(¹Maastricht University, The Netherlands; ²ROA/ Maastricht University, The Netherlands)

3. Measuring numeracy skills mismatch with PIAAC data. 

Tina Dulam^* & Kees Hoogland (HU University of Applied Sciences Utrecht, The Netherlands)  (765 kB)  

Chair:  Julia Gorges (Philipps-University Marburg, Germany)

Contributions

1. Motivation to learn and multilingualism across the adult life stages in the USA.

Shalini Sahoo^1*, Takashi Yamashita¹, Roberto Millar² & Phyllis Cummins³

(¹University of Maryland, USA; ²The Hilltop Institute, USA; ³Miami University, USA)  (752 kB)  

2. Motivation to learn by age, education, and literacy skills among working-age adults in the USA.

Takashi Yamashita^1*, Thomas Smith², Shalini Sahoo¹ & Phyllis Cummins³

(¹University of Maryland, USA; ²Northern Illinois University, USA; ³Miami University, USA) 

3. Job tasks and cognitive skill accumulation.

Qinyi Liu* (University of International Business and Economics, China)

*Presenting Author