Programme for the International
Assessment of Adult Competencies

March 23th, 2022

GESIS Mannheim

18.30 - 20.00 Informal Get-together

March 24th, 2022

Dorint Hotel Mannheim

9.00 – 9.15 Welcome Address
9.15 – 10.45

3 Parallel Sessions

10.45 - 11.15 Coffee Break
11.15 - 12.45 3 Parallel Sessions
12.45 - 14.00 Lunch
14.00 - 15.00

Keynote Speech

15.00 - 15.30 Coffee Break
15.30 – 17.00 3 Parallel Sessions
19.00 Conference Dinner

March 25th, 2022

Dorint Hotel Mannheim

8.30 - 10.00

3 Parallel Sessions

10.00 - 10.30 Coffee Break
10.30 - 12.00 3 Parallel Sessions
12.00 - 13.15 Lunch
13.15 - 14.15 Keynote Speech
14.15 - 15.45 3 Parallel Sessions
15.45 - 16:30

Closing Remarks

 

Keynote:  Enhancing the Utility of International Large-Scale Assessments (ILSA)

Dr. Irwin Kirsch (Educational Testing Service/ ETS, USA)

Irwin Kirsch is the Ralph Tyler Chair in Large Scale Assessment and Director of the Center for Global Assessment at ETS in Princeton, NJ.  In his role as director of the center he oversees several teams of research scientists, assessment designers and platform developers who are responsible for the development, management and implementation of various large-scale national and international assessments. Over the course of his career, Dr. Kirsch has worked in close collaboration with a number of state, national and international organizations including the World Bank, UNESCO, the International Association for the Evaluation of Educational Achievement (IEA), and the Organization for Economic Co-operation and Development (OECD) where he currently oversees the development and conduct of the two largest international assessments that provide policy makers and key stakeholders with national and international comparative data on literacy and workforce preparedness – PIAAC and PISA. In addition to his assessment work, Dr. Kirsch is a member of the ETS research management team, serves on the board of a non-profit literacy organization, and as a reviewer for several journals.  He has also published numerous research articles and book chapters dealing with issues around designing, developing and interpreting cognitive-based scales and has written a number of policy reports using large-scale assessment data that focus on the growing importance of skills and their connections to life outcomes.

 

 

Keynote:  Why Skills Matter

Prof. Dr. Rolf van der Velden (Maastricht University, The Netherlands)

Rolf van der Velden is Professor at Maastricht University and director of the Research Centre for Education and the Labour Market (ROA). He is research fellow at the Graduate School of Business and Economics at Maastricht University (GSBE) and fellow of the Amsterdam Centre for Learning Analytics (ACLA). Van der Velden supervised several national and international studies on the transition from school to work, such as the international REFLEX project (see “The Flexible Professional in the Knowledge Society” by Allen & Van de Velden, 2011) and the “Higher Education as a Generator of Strategic Competences” project (called HEGESCO). Currently he coordinates the Netherlands Cohort Study on Education (https://nationaalcohortonderzoek.nl/) of the Netherlands Initiative for Education Research (NRO) and is advisor for the Programme for the International Assessment of Adult Competencies (PIAAC). His expertise is on transition from education to work (e.g., Humburg and van der Velden, 2015; Levels, van der Velden and Di Stasio, 2014), knowledge economy and the demand for 21st century skills (e.g., Humburg and van der Velden, 2017), and skills mismatches (e.g., van der Velden and Verhaest, 2017; van der Velden and Bijlsma, 2019; Fregin, Levels, and van der Velden, 2020).

 


PIAAC Cycle 2: Between Trend and Innovation

Chair:  Anouk Zabal (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract:  PIAAC collects internationally comparable data on key foundation skills as well as a wide range of background information, and it offers rich opportunities for both academic and policy-relevant research. The second cycle of PIAAC continues a tradition of international adult literacy surveys and pursues the dual aims of monitoring change over time as well as introducing necessary adjustments and innovations to reflect societal and technological change. This symposium addresses the challenge of striking the right balance between consistency and change. The first contribution looks at how the theoretical frameworks for literacy and numeracy have been updated and enhanced for the second cycle of PIAAC, and how the cognitive instruments reflect the enriched constructs while maintaining the link to the first cycle. It will consider the reading components and the newly developed numeracy components, both designed to obtain more differentiated information at the lower end of the proficiency scale. Adaptive problem solving (APS) is a key skill in a rapidly changing world, and the assessment of this cognitive domain is an important innovation. The second contribution explores similarities and differences between the domain problem solving in technology-rich environments assessed in the first cycle of PIAAC and APS. It elaborates on the theoretical underpinnings of APS and discusses the challenges of operationalizing this construct for a large-scale assessment.The information collected through the background questionnaire lends PIAAC its analytical power. The third contribution discusses how constructs and measurements from the cycle one background questionnaire have been adapted or transformed to account for societal change while considering the need to maintain trend measurement. It also reviews new constructs that have been included to broaden the scope and strengthen the analytical possibilities. A valid, reliable, and comparable measurement of educational attainment is key for PIAAC. At the same time, the measurement of formal education is one of the most challenging in the context of achieving comparability across cultures and time. The approach followed in the second cycle of PIAAC to tackle these complex issues is presented and reflected upon in the fourth contribution.

Contributions Abstracts available here (1.66 MB)                                                                                 

Measuring Literacy and Numeracy Skills in Adults – Trends and Innovations

Laura Halderman (Educational Testing Service/ETS, USA)

From Problem Solving in Technology-Rich Environments to Adaptive Problem Solving – Concept and Measurement

Samuel Greiff* & Juliana Gottschling (University of Luxemburg, Luxemburg)

The Background Questionnaire: Maintaining Trend, Accounting for Societal Change, and Adding Innovative Elements

Tim Huijts (ROA, Maastricht University, The Netherlands)

Formal Education in PIAAC Cycle 2: Challenges and Opportunities

Silke Schneider (GESIS - Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author

 

Determinants of Cognitive Skills

Chair:  Simon Wiederhold (Catholic University Eichstaett-Ingolstadt, Germany)

Abstract:  Analysis of the PIAAC data has shown the crucial importance of cognitive skills for individual success. Thus, there is a profound interest in the determinants of cognitive skills. The literature consistently argues that family background plays a key role in the formation of skills, leading to a strong persistence in educational achievement across generations. However, surprisingly little is known about which family characteristics actually matter for skill formation. The four studies combined in this symposium try to dig deeper into the determinants of cognitive skills. Some of the questions addressed in the symposium are: How strong is the correlation of cognitive skills of parents and their children, and what are the mechanisms that give rise to the intergenerational persistence of skills? How important is the transmission of cultural values from parents to children as a determinant of cognitive skills? Finally, can policies substituting for lacking family support be successful in helping disadvantaged children to improve their skills?

Contributions Abstracts available here (0.91 MB)

The Intergenerational Transmission of Skills

Eric A. Hanushek1, Babs Jacobs2, Guido Schwerdt3*, Rolf van der Velden2 & Simon Wiederhold4

(1Hoover Institution, Stanford University, USA; 2ROA, Maastricht University, The Netherlands; 3University of Konstanz, Germany; 4Catholic University Eichstaett-Ingolstadt, Germany)

Can Mentoring Alleviate Family Disadvantage in Adolescence? A Field Experiment to Improve Labor-Market Prospects

Sven Resnjanskij1, Jens Ruhose2, Simon Wiederhold3 & Ludger Woessmann1*

(1ifo Institute, Germany; 2University of Kiel, Germany; 3Catholic University Eichstaett-Ingolstadt, Germany)

Culture and Student Achievement: The Intertwined Roles of Patience and Risk-Taking

Eric A. Hanushek1*, Lavinia Kinne2, Philipp Lergetporer2 & Ludger Woessmann2

(1Hoover Institution, Stanford University, USA; 2ifo Institute, Germany)

Individualism, Human Capital Formation, and Labor Market Outcomes

Katharina Hartinger1, Sven Resnjanskij2, Jens Ruhose3 &  Simon Wiederhold1*

(1Catholic University Eichstaett-Ingolstadt, Germany; 2ifo Institute, Germany; 3University of Kiel, Germany)

*Presenting Author


Beyond Competencies – Potential of PIAAC for Interdisciplinary Research

Chair:  Julia Gorges (Philipps-University Marburg, Germany)

Abstract: The OECD PIAAC study provides in-depth information regarding adult competencies in a wide range of countries. Hence, some refer to PIAAC as “PISA for adults”. However, PIAAC is more than that. Thanks to its references to research from different disciplines and to different data sources, PIAAC has stimulated much research beyond adult competencies. Using data from the PIAAC background questionnaire, researchers have tested established assumptions on adult learning, investigated participation in education, and used national longitudinal extensions of the PIAAC study and other related data sources. This symposium brings together research beyond competencies from different disciplinary perspectives using a wide range of datasets. Using cross-sectional data from the first PIAAC data collection in 2012, MASSING explores the importance of participation in non-formal training in order to upgrade or to maintain skills among migrants in selected countries. In particular, this contribution will focus on how different policies in different countries might affect training participation and barriers to training. Drawing on data from the 1994–1998 International Adult Literacy Survey, the 2003-2008 Adult Literacy and Life Skills and the 2011-2012 OECD Survey of Adult Skills, MERONI focuses on overeducation and skill mismatch in different age cohorts and different countries. To identify skill mismatch and overeducation, this contribution looks at both skills and educational credentials. The contribution by HESSE uses the German longitudinal extension of PIAAC to investigate the role of motivation to learn and parenthood for participation in non-job-related training. The study provides insights into the neglected concept of adult motivation to learn. Using another longitudinal extension of PIAAC, ROESSGER’s contribution investigated the benefits of earning a bachelor’s degree for adults based on U.S. PIAAC 2012 and 2017 data. To overcome bias introduced to previous studies by comparing individuals who did versus did not earn a bachelor’s degree, these analyses will also consider adults who have been to college but left with versus without earning a degree. Each contribution takes a specific disciplinary perspective: sociology (MASSING), economics (MERONI), psychology (HESSE), and educational research (ROESSGER). In this interdisciplinary symposium, disciplinary priorities and major topics building on PIAAC data will be highlighted and discussed.

Contributions Abstracts available here (1.56 MB)

Training Participation of Migrants and Barriers to Training – A Cross-Country Comparison

Natascha Massing (GESIS - Leibniz-Institute for the Social Sciences, Germany)

An Age-Period-Cohort Approach to the Incidence and Evolution of Overeducation and Skills Mismatch

Elena Meroni (European Commission, DG Joint Research Centre)

How Compatible are Parenthood and Participation in Further Education and Training? Findings from the German PIAAC Longitudinal Study

Franzisca Hesse (Philipps-University Marburg, Germany)

Comparing Five-Year Outcomes Among Adult Degree Completers, Stop-outs, and Non-Attempters in U.S. Colleges and Universities

Kevin M. Roessgen (University of Arkansas, USA)


Linking PIAAC Data to Administrative Data and other Large-Scale Assessments

Chair:  Débora B. Maehler & Silke Martin (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract:  The symposium addresses content and methodological issues and aims to present research based on PIAAC data linked to administrative data or other large-scale country surveys. Thereby projects and studies of four countries will be presented. Initially research based on PIAAC 2011/2012 linked to register data in Norway will be presented (N = 901). The relationship between skills acquired before the age of 16 years versus those acquired later to formation of skills and subsequent NEET status, were addressed in the study. The second contribution is based on the PISA Young Adult Follow-Up Study (PISA YAFS) conducted in the US. Study design and results comparing for instance literacy from students in PISA 2012 and in the follow up 2016 using PIAAC instruments, assessed as repeated measure, will be presented (N = 2.320). The third contribution integrates PIAAC data linking to PISA data as well as to administrative data. Study design and research based on linking PISA 2000, PIAAC 2011/2012 and register data in Denmark will be presented (N =1.210). In this case the PIAAC sample is a sub-sample of the PISA 2000 sample. The contribution focuses on the relationship of cognitive and non-cognitive skills to earnings and employment rate across time.  Finally, the fourth contribution covers the linkage of administrative data with four large-scale assessments in Germany. Following surveys were linked to administrative data of the Institute for Employment Research (IAB): the PIAAC-L (PIAAC Longitudinal; N = 2086), the NEPS (The German National Educational Panel Study, Starting Cohort 6 (SC6); N = 12.621), the SOEP (Socio-Economic Panel; N = 11.500) and the PASS (Panel Study Labour Market and Social Security; N = 30.212). This methodological contribution aims to compare measurement errors for employment related indicators across these surveys.

Contributions Abstracts available here (1.45 MB)

NEET Status and Early versus Later Skills Among Young Adults: Evidence from Linked Register-PIAAC Data (using Nordic PIAAC Data)

Erling Barth1, Anna-Lena Keute2*, Pål Schøne1, Kristine von Simson1 & Kjartan Steffensen2

(1Institute for Social Research, Norway; 2Statistics Norway, Norway)

Administering Education and Skills Online (ESO) to PISA 2012 Cohort in the United States: Findings from the 2012-2016 PISA Young Adult Follow-up Study (using PISA YAFS)

Saida Mamedova1*, Maria Stephens1, Yuqi Liao1, Josh Sennett1, Paul Sirma1 & Samantha Burg2

(1American Institutes for Research/AIR, USA; 2National Center for Education Statistics, USA)

Childhood and Adulthood Skill Acquisition - Importance for Labor Market Outcomes (Danish PISA PIAAC Data)

Karl Fritjof Krassel1* & Kenneth Lykke Sørensen2

(1VIVE/The Danish Center for Social Science Research, Denmark; 2The National Board of Social Services, Denmark)

Validating Measures of Employment Related Information in Surveys Using Linked Administrative Data in Germany (using PIAAC-L/ NEPS/ PASS/ SOEP/ IAB Data)

Manfred Antoni1*, Nadine Bachbauer1, Corina Frodermann1, Knut Wenzig2 & Débora B. Maehler3

(1Institute for Employment Research/ IAB, Germany; 2German Institute for Economic Research/ DIW, Germany; 3GESIS - Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author


Cognitive Skills and Lifelong Learning

Chair:  Martin Ehlert (WZB Berlin Social Science Center, Germany)

Abstract:  Learning during adulthood after initial education and training is a widespread and growing phenomenon in many countries. Especially the increasing pace of technological change and digitalization is likely to further fuel this trend. New technology has the potential to change the labor market as well as many other aspects of life. The result is that skills acquired during initial training no longer suffice to remain employable and to participate in social life. Therefore, skill acquisition later in life becomes more important to ensure that nobody is left behind. Therefore, many politicians and pundits advocate lifelong learning to ensure inclusive growth despite the challenges of digitalization. Yet, it is not clear whether current adult education and training programs and systems are capable to counteract the negative repercussions of digitalization. It is well known that access to adult training and education is unequally distributed. Less-educated and less-skilled people participate less often in training courses and other learning activities. Also, workers that are more likely to lose their jobs because of technological change are less likely to take part in learning opportunities. Thus, adult education and training programs often do not reach those most in need of skill acquisition later in life. At the same time, the benefits of existing learning arrangements for adults are often unclear. Many studies show that participation in further training courses does not lead to higher skills or better jobs. Against this background, this symposium aims to advance the knowledge on lifelong learning and cognitive skills. Based on new empirical results, we will discuss both the determinants of participation in lifelong learning and its effect on the development of cognitive skills. Also, we will discuss the acquisition and development of skills over the life course more generally. A special focus will be on cross-national comparative studies. Using results on participation and skill acquisition from different countries, we aim to assess the influence of institutions such as adult education and training policies. Furthermore, we will especially discuss lifelong learning among the most vulnerable groups such as less-skilled workers.

Contributions Abstracts available here (4.92 MB)

Does Learning Help Workers to Keep up with Technological Change? Cross-National Variance in the Effect of Further Training on Problem-Solving Skills.

Martin Ehlert1*, Marie-Christine Fregin2, Didier Fouarge2, Mark Levels2, Liisa Martma3 & Rolf van der Velden2

(1WZB Berlin Social Science Center, Germany; 2ROA, Maastricht University, The Netherlands; 3 Tallinn University, Estonia)

Skills and Education as Factors of Participation in Non-Formal Learning

Liisa Martma* & Ellu Saar (Tallinn University, Estonia)

Cross-National Variation in the Training Disadvantage of Less-Educated Employees: The Role of Job Allocation versus Skills

Carla Hornberg*, Heike Solga & Jan Paul Heisig (WZB Berlin Social Science Center, Germany)

Patterns and predictors of literacy and numeracy development during adulthood: Longitudinal findings from PIAAC-L and NEPS

Clemens Lechner (GESIS - Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author


What are PIAAC process data used for?

Chair:  Frank Goldhammer, Ulf Kröhne & Carolin Hahnel 

(DIPF | Leibniz Institute for Research and Information in Education, Germany)

Abstract:  The first round of the PIAAC study 2011–2012 was innovative in many ways. It was not only the first international large-scale assessment to be mainly computer-based, but it was also the first large-scale assessment that systematically collected log data, that is, events, event-related attributes, and timestamps reflecting the test taker’s interactions with the PIAAC assessment system. Log data found their way into the public use file in the form of generic process indicators such as time on task or number of interactions by item. Moreover, the majority of countries participating in PIAAC round one provided their log data that was recorded during the computer-based assessment to make it publicly available to the research community. Together with the result data and the data from the background questionnaire, the PIAAC log data already has inspired intensive research activities (for an overview see Goldhammer et al., 2020).  In this invited symposium, we have brought together four recent contributions dealing with PIAAC process data from different – both substantive and methodological – perspectives. The first presentation by Pokropek and colleagues addresses the question of how process data can be used to explore gender differences in literacy as assessed in PIAAC and PISA. The following contribution by Hahnel and colleagues investigates strategies of information processing in simulated web search environments included in the PIAAC assessment of problem solving in technology-rich environments. The third presentation by He and colleagues focuses on evaluating consistency of adult behavioral patterns across multiple problem solving tasks using PIAAC process data from the US. The final presentation by Maddox and Care expands the perspective in that it takes into account not only PIAAC log data, but other sources of (multimodal) process data such as verbal interaction, facial expressions and gesture.

Contributions Abstracts available here (1.67 MB)

Utilizing Process Data to Examine Gender Differences in Literacy in PISA and PIAAC.

Artur Pokropek1*, Francesca Borgonovi2, & Lale Khorramdel3

(1Institute of Philosophy and Sociology of the Polish Academy of Sciences, Poland; 2University College London, UK, and OECD, France; 3 Boston College, USA)

Identifying Strategies of Information Processing in PIAAC Web Search Environments

Carolin Hahnel*, Frank Goldhammer, & Ulf Kroehne (DIPF | Leibniz Institute for Research and Information in Education, Germany)

Evaluating Consistency of Adult Problem-Solving Behaviors across Multiple Tasks Using PIAAC Process Data

Qiwei He1*, Dandan Liao2, Hok Kan Ling3, & Hong Jiao4

(1Educational Testing Service, USA; 2Cambium Assessment, Inc, USA; 3 Queen’s University, Canada; 4 University of Maryland, USA)

Multimodal Process Data in PIAAC

Bryan Maddox1* & Naomi Care2

(1University of East Anglia & CEMO, University of Oslo, Norway; 2Assessment Micro-Analytics Ltd, UK)

*Presenting Author

 

The PIAAC workshops welcome researchers from different disciplines interested to work or already working with PIAAC data. It is expected that the participants have good empirical knowledge and experience in the respective statistical software. The workshop comprises lectures and practical sessions covering the following elements: (a) Theoretical and methodological input from the lecturers (see description of contents above); (b) Opportunity for participants to present their own research or research ideas with PIAAC data (optional); (c) Discussion of the questions outlined in the workshop regarding the data used and methods as well as specific feedback from the lecturers.

There will be no participation fees. The workshop will be limited to a maximum of 15 participants. In cases that workshop cannot take place at GESIS Mannheim, they will be conducted virtually.

Workshop: Analyzing PIAAC data with structural equation modeling in Mplus

Lecturer: Ronny Scherer (Centre for Educational Measurement at the University of Oslo)

Date: March 22-23, 2022

Place: GESIS Mannheim

Content: Structural equation modeling (SEM) represents a statistical approach to disentangle the relationships among latent and/or manifest variables, across groups, over time, and at different analytical levels. The potential of SEM has been recognized in many areas, including educational sciences, sociology, psychology, and business. This workshop provides an introduction to the principles and procedures of basic and more advanced SEM in the context of the international large-scale assessment PIAAC. Specifically, the following topics are covered: (a) Principles of structural equation modeling (model specification, identification, estimation, and evaluation), (b) Measurement models and confirmatory factor analysis, (c) Measurement invariance testing with few and many groups (including multi-group CFA, multilevel CFA, and the alignment method), and (d) Structural regression and indirect effects models (including multi-group and multilevel SEM). Participants can also present their own research or research ideas using PIAAC data and receive feedback on how to improve the analysis (optional).

Data: PIAAC Public Use Files

Software: SPSS and Mplus

Schedule: Preliminary agenda (712 kB)

Recommended reading: Maehler, D. & Rammstedt, B. (2020). Large-scale cognitive assessment: Analyzing PIAAC data. Series: Methodology of Educational Measurement and Assessment (MEMA). Springer: Cham. https://link.springer.com/book/10.1007/978-3-030-47515-4

For any questions please contact the RDC PIAAC (fdz-piaac@gesis.org).