Programme for the International
Assessment of Adult Competencies

October 6th, 2021

GESIS Mannheim

18.30 - 20.00Informal Get-together

October 7th, 2021

Dorint Hotel Mannheim

9.00 – 9.15Welcome Address
9.15 – 10.45

3 Parallel Sessions

10.45 - 11.15Coffee Break
11.15 - 12.453 Parallel Sessions
12.45 - 14.00Lunch
14.00 - 15.00

Keynote Speech

15.00 - 15.30Coffee Break
15.30 – 17.003 Parallel Sessions
19.00Conference Dinner

October 8th, 2021

Dorint Hotel Mannheim

8.30 - 10.00

3 Parallel Sessions

10.00 - 10.30Coffee Break
10.30 - 12.003 Parallel Sessions
12.00 - 13.15Lunch
13.15 - 14.15Keynote Speech
14.15 - 15.453 Parallel Sessions
15.45 - 16:30

Closing Remarks

 

Keynote:  Enhancing the Utility of International Large-Scale Assessments (ILSA)

Dr. Irwin Kirsch (Educational Testing Service/ ETS, USA)

Irwin Kirsch is the Ralph Tyler Chair in Large Scale Assessment and Director of the Center for Global Assessment at ETS in Princeton, NJ.  In his role as director of the center he oversees several teams of research scientists, assessment designers and platform developers who are responsible for the development, management and implementation of various large-scale national and international assessments. Over the course of his career, Dr. Kirsch has worked in close collaboration with a number of state, national and international organizations including the World Bank, UNESCO, the International Association for the Evaluation of Educational Achievement (IEA), and the Organization for Economic Co-operation and Development (OECD) where he currently oversees the development and conduct of the two largest international assessments that provide policy makers and key stakeholders with national and international comparative data on literacy and workforce preparedness – PIAAC and PISA. In addition to his assessment work, Dr. Kirsch is a member of the ETS research management team, serves on the board of a non-profit literacy organization, and as a reviewer for several journals.  He has also published numerous research articles and book chapters dealing with issues around designing, developing and interpreting cognitive-based scales and has written a number of policy reports using large-scale assessment data that focus on the growing importance of skills and their connections to life outcomes.

 

 

Keynote:  Why Skills Matter

Prof. Dr. Rolf van der Velden (Maastricht University, The Netherlands)

Rolf van der Velden is Professor at Maastricht University and director of the Research Centre for Education and the Labour Market (ROA). He is research fellow at the Graduate School of Business and Economics at Maastricht University (GSBE) and fellow of the Amsterdam Centre for Learning Analytics (ACLA). Van der Velden supervised several national and international studies on the transition from school to work, such as the international REFLEX project (see “The Flexible Professional in the Knowledge Society” by Allen & Van de Velden, 2011) and the “Higher Education as a Generator of Strategic Competences” project (called HEGESCO). Currently he coordinates the Netherlands Cohort Study on Education (https://nationaalcohortonderzoek.nl/) of the Netherlands Initiative for Education Research (NRO) and is advisor for the Programme for the International Assessment of Adult Competencies (PIAAC). His expertise is on transition from education to work (e.g., Humburg and van der Velden, 2015; Levels, van der Velden and Di Stasio, 2014), knowledge economy and the demand for 21st century skills (e.g., Humburg and van der Velden, 2017), and skills mismatches (e.g., van der Velden and Verhaest, 2017; van der Velden and Bijlsma, 2019; Fregin, Levels, and van der Velden, 2020).

 


PIAAC Cycle 2: Between Trend and Innovation

Chair:  Anouk Zabal (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract:  PIAAC collects internationally comparable data on key foundation skills as well as a wide range of background information, and it offers rich opportunities for both academic and policy-relevant research. The second cycle of PIAAC continues a tradition of international adult literacy surveys and pursues the dual aims of monitoring change over time as well as introducing necessary adjustments and innovations to reflect societal and technological change. This symposium addresses the challenge of striking the right balance between consistency and change.  The first contribution looks at how the theoretical frameworks for literacy and numeracy have been updated and enhanced for the second cycle of PIAAC, and how the cognitive instruments reflect the enriched constructs while maintaining the link to the first cycle. It will consider the reading components and the newly developed numeracy components, both designed to obtain more differentiated information at the lower end of the proficiency scale. Adaptive problem solving (APS) is a key skill in a rapidly changing world, and the assessment of this cognitive domain is an important innovation. The second contribution explores similarities and differences between the domain problem solving in technology-rich environments assessed in the first cycle of PIAAC and APS. It elaborates on the theoretical underpinnings of APS and discusses the challenges of operationalizing this construct for a large-scale assessment.  The information collected through the background questionnaire lends PIAAC its analytical power. The third contribution discusses how constructs and measurements from the cycle one background questionnaire have been adapted or transformed to account for societal change while considering the need to maintain trend measurement. It also reviews new constructs that have been included to broaden the scope and strengthen the analytical possibilities. A valid, reliable, and comparable measurement of educational attainment is key for PIAAC. At the same time, the measurement of formal education is one of the most challenging in the context of achieving comparability across cultures and time. The approach followed in the second cycle of PIAAC to tackle these complex issues is presented and reflected upon in the fourth contribution.

Contributions:

Measuring Literacy and Numeracy Skills in Adults – Trends and Innovations

Laura Halderman (Educational Testing Service/ETS, USA)

From Problem Solving in Technology-Rich Environments to Adaptive Problem Solving – Concept and Measurement

Samuel Greiff* & Juliana Gottschling (University of Luxemburg, Luxemburg)

The Background Questionnaire: Maintaining Trend, Accounting for Societal Change, and Adding Innovative Elements

Tim Huijts (ROA, Maastricht University, The Netherlands)

Formal Education in PIAAC Cycle 2: Challenges and Opportunities

Silke Schneider (GESIS - Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author


Determinants of Cognitive Skills

Chair:  Simon Wiederhold (Catholic University Eichstaett-Ingolstadt, Germany)

Abstract:  Analysis of the PIAAC data has shown the crucial importance of cognitive skills for individual success. Thus, there is a profound interest in the determinants of cognitive skills. The literature consistently argues that family background plays a key role in the formation of skills, leading to a strong persistence in educational achievement across generations. However, surprisingly little is known about which family characteristics actually matter for skill formation. The four studies combined in this symposium try to dig deeper into the determinants of cognitive skills. Some of the questions addressed in the symposium are: How strong is the correlation of cognitive skills of parents and their children, and what are the mechanisms that give rise to the intergenerational persistence of skills? How important is the transmission of cultural values from parents to children as a determinant of cognitive skills? Finally, can policies substituting for lacking family support be successful in helping disadvantaged children to improve their skills?

Contributions:

The Intergenerational Transmission of Skills

Eric A. Hanushek1, Babs Jacobs2, Guido Schwerdt3*, Rolf van der Velden2 & Simon Wiederhold4

(1Hoover Institution, Stanford University, USA; 2ROA, Maastricht University, The Netherlands; 3University of Konstanz, Germany; 4Catholic University Eichstaett-Ingolstadt, Germany)

Can Mentoring Alleviate Family Disadvantage in Adolescence? A Field Experiment to Improve Labor-Market Prospects

Sven Resnjanskij1, Jens Ruhose2, Simon Wiederhold3 & Ludger Woessmann1*

(1ifo Institute, Germany; 2University of Kiel, Germany; 3Catholic University Eichstaett-Ingolstadt, Germany)

Culture and Student Achievement: The Intertwined Roles of Patience and Risk-Taking

Eric A. Hanushek1*, Lavinia Kinne2, Philipp Lergetporer2 & Ludger Woessmann2

(1Hoover Institution, Stanford University, USA; 2ifo Institute, Germany)

Individualism, Human Capital Formation, and Labor Market Outcomes

Katharina Hartinger1, Sven Resnjanskij2, Jens Ruhose3 &  Simon Wiederhold1*

(1Catholic University Eichstaett-Ingolstadt, Germany; 2ifo Institute, Germany; 3University of Kiel, Germany)

*Presenting Author


Beyond Competencies – Potential of PIAAC for Interdisciplinary Research

Chair:  Julia Gorges (Philipps-University Marburg, Germany)

Abstract:  The OECD PIAAC study provides in-depth information regarding adult   competencies in a wide range of countries. Hence, some refer to PIAAC as “PISA for adults”. However, PIAAC is more than that. Thanks to its references to research from different disciplines and to different data sources, PIAAC has stimulated much research beyond adult competencies. Using data from the PIAAC background questionnaire, researchers have tested established assumptions on adult learning, investigated participation in education, and used national longitudinal extensions of the PIAAC study and other related data sources. This symposium brings together research beyond competencies from different disciplinary perspectives using a wide range of datasets. Using cross-sectional data from the first PIAAC data collection in 2012, MASSING explores the importance of participation in non-formal training in order to upgrade or to maintain skills among migrants in selected countries. In particular, this contribution will focus on how different policies in different countries might affect training participation and barriers to training. Drawing on data from the 1994–1998 International Adult Literacy Survey, the 2003-2008 Adult Literacy and Life Skills and the 2011-2012 OECD Survey of Adult Skills, MERONI focuses on overeducation and skill   mismatch in different age cohorts and different countries. To identify skill mismatch and overeducation, this contribution looks at both skills and educational credentials. The contribution by HESSE uses the German longitudinal extension of PIAAC to investigate the role of motivation to learn and parenthood for participation in non-job-related training. The study provides insights into the neglected concept of adult motivation to learn. Using another longitudinal extension of PIAAC, ROESSGEN’s contribution investigated the benefits of earning a bachelor’s degree for adults based on U.S. PIAAC 2012 and 2017 data. To overcome bias introduced to previous studies by comparing students and non-students, these analyses   will focus on adults who have been to college but left with versus without earning a degree.  Each contribution takes a specific disciplinary perspective: sociology (MASSING), economics (MERONI), psychology (HESSE), and educational research (ROESSGEN). In this interdisciplinary symposium, disciplinary priorities and major topics building on PIAAC data will be highlighted and discussed.

Contributions:

Training Participation of Migrants and Barriers to Training – A Cross-Country Comparison

Natascha Massing (GESIS - Leibniz-Institute for the Social Sciences, Germany)

An Age-Period-Cohort Approach to the Incidence and Evolution of Overeducation and Skills Mismatch

Elena Meroni (European Commission, DG Joint Research Centre)

How Compatible are Parenthood and Participation in Further Education and Training? Findings from the German PIAAC Longitudinal Study

Franzisca Hesse* & Julia Gorges (Philipps-University Marburg, Germany)

The Power of Retaining Late Bloomers: Comparing Five-Year – Outcomes of Bachelor’s Degree Completers and Non-Completers in U.S. Colleges and Universities

Kevin M. Roessgen (University of Arkansas, USA)

*Presenting Author


Linking PIAAC Data to Administrative Data and other Large-Scale Assessments

Chair:  Débora B. Maehler (GESIS - Leibniz-Institute for the Social Sciences, Germany)

Abstract:  The symposium addresses content and methodological issues and aims to present research based on PIAAC data linked to administrative data or other large-scale country surveys. Thereby projects and studies of four countries will be presented. Initially research based on PIAAC 2011/2012 linked to register data in Norway will be presented (N = 901). The relationship between skills acquired before the age of 16 years versus those acquired later to formation of skills and subsequent NEET status, were addressed in the study. The second contribution is based on the PISA Young Adult Follow-Up Study (PISA YAFS) conducted in the US. Study design and results comparing for instance literacy from students in PISA 2012 and in the follow up 2016 using PIAAC instruments, assessed as repeated measure, will be presented (N = 2.320). The third contribution integrates PIAAC data linking to PISA data as well as to administrative data. Study design and research based on linking PISA 2000, PIAAC 2011/2012 and register data in Denmark will be presented (N =1.210). In this case the PIAAC sample is a sub-sample of the PISA 2000 sample. The contribution focuses on the relationship of cognitive and non-cognitive skills to earnings and employment rate across time.  Finally, the fourth contribution covers the linkage of administrative data with four large-scale assessments in Germany. Following surveys were linked to administrative data of the Institute for Employment Research (IAB): the PIAAC-L (PIAAC Longitudinal; N = 2086), the NEPS (The German National Educational Panel Study, Starting Cohort 6 (SC6); N = 12.621), the SOEP (Socio-Economic Panel; N = 11.500) and the PASS (Panel Study Labour Market and Social Security; N = 30.212). This methodological contribution aims to compare measurement errors for employment related indicators across these surveys.

Contributions:

NEET Status and Early versus Later Skills Among Young Adults: Evidence from Linked Register-PIAAC Data (using Nordic PIAAC Data)

Erling Barth1, Anna-Lena Keute2*, Pål Schøne1, Kristine von Simson1 & Kjartan Steffensen2

(1Institute for Social Research, Norway; 2Statistics Norway, Norway)

Administering Education and Skills Online (ESO) to PISA 2012 Cohort in the United States: Findings from the 2012-2016 PISA Young Adult Follow-up Study (using PISA YAFS)

Saida Mamedova1*, Maria Stephens1, Yuqi Liao1, Josh Sennett1, Paul Sirma1 & Samantha Burg2

(1American Institutes for Research/AIR, USA; 2National Center for Education Statistics, USA)

Childhood and Adulthood Skill Acquisition - Importance for Labor Market Outcomes (Danish PISA PIAAC Data)

Karl Fritjof Krassel1* & Kenneth Lykke Sørensen2

(1VIVE/The Danish Center for Social Science Research, Denmark; 2The National Board of Social Services, Denmark)

Validating Measures of Employment Related Information in Surveys Using Linked Administrative Data in Germany (using PIAAC-L/ NEPS/ PASS/ SOEP/ IAB Data)

Manfred Antoni1*, Nadine Bachbauer1, Corina Frodermann1, Knut Wenzig2 & Débora B. Maehler3

(1Institute for Employment Research/ IAB, Germany; 2German Institute for Economic Research/ DIW, Germany; 3GESIS - Leibniz-Institute for the Social Sciences, Germany)

*Presenting Author


Cognitive Skills and Lifelong Learning

Chair:  Martin Ehlert (WZB Berlin Social Science Center, Germany)

Abstract:  Learning during adulthood after initial education and training is a widespread and growing phenomenon in many countries. Especially the increasing pace of technological change and digitalization is likely to further fuel this trend. New technology has the potential to change the labor market as well as many other aspects of life. The result is that skills acquired during initial training no longer suffice to remain employable and to participate in social life. Therefore, skill acquisition later in life becomes more important to ensure that nobody is left behind. Therefore, many politicians and pundits advocate lifelong learning to ensure inclusive growth despite the challenges of digitalization. Yet, it is not clear whether current adult education and training programs and systems are capable to counteract the negative repercussions of digitalization. It is well known that access to adult training and education is unequally distributed. Less-educated and less-skilled people participate less often in training courses and other learning activities. Also, workers that are more likely to lose their jobs because of technological change are less likely to take part in learning opportunities. Thus, adult education and training programs often do not reach those most in need of skill acquisition later in life. At the same time, the benefits of existing learning arrangements for adults are often unclear. Many studies show that participation in further training courses does not lead to higher skills or better jobs. Against this background, this symposium aims to advance the knowledge on lifelong learning and cognitive skills. Based on new empirical results, we will discuss both the determinants of participation in lifelong learning and its effect on the development of cognitive skills. Also, we will discuss the acquisition and development of skills over the life course more generally. A special focus will be on cross-national comparative studies. Using results on participation and skill acquisition from different countries, we aim to assess the influence of institutions such as adult education and training policies. Furthermore, we will especially discuss lifelong learning among the most vulnerable groups such as less-skilled workers.

Contributions:

Does Learning Help Workers to Keep up with Technological Change? Cross-National Variance in the Effect of Further Training on Problem-Solving Skills.

Martin Ehlert1*, Marie-Christine Fregin2, Didier Fouarge2, Mark Levels2, Liisa Martma3 & Rolf van der Velden2

(1WZB Berlin Social Science Center, Germany; 2ROA, Maastricht University, The Netherlands; 3 Tallinn University, Estonia)

Skills and Education as Factors of Participation in Non-Formal Learning

Liisa Martma* & Ellu Saar (Tallinn University, Estonia)

Cross-National Variation in the Training Disadvantage of Less-Educated Employees: The Role of Job Allocation versus Akills

Carla Hornberg*, Heike Solga & Jan Paul Heisig (WZB Berlin Social Science Center, Germany)

*Presenting Author


What are PIAAC process data used for?

Chair:  Frank Goldhammer, Ulf Kröhne & Carolin Hahnel 

(DIPF | Leibniz Institute for Research and Information in Education, Germany)

Abstract:  The first round of the PIAAC study 2011–2012 was innovative in many ways. It was not only the first international large-scale assessment to be mainly computer-based, but it was also the first large-scale assessment that systematically collected log data, that is, events, event-related attributes, and timestamps reflecting the test taker’s interactions with the PIAAC assessment system. Log data found their way into the public use file in the form of generic process indicators such as time on task or number of interactions by item. Moreover, the majority of countries participating in PIAAC round one provided their log data that was recorded during the computer-based assessment to make it publicly available to the research community. Together with the result data and the data from the background questionnaire, the PIAAC log data already has inspired intensive research activities (for an overview see Goldhammer et al., 2020).  In this invited symposium, we have brought together four recent contributions dealing with PIAAC process data from different – both substantive and methodological – perspectives. The first presentation by Pokropek and colleagues addresses the question of how process data can be used to explore gender differences in literacy as assessed in PIAAC and PISA. The following contribution by Hahnel and colleagues investigates strategies of information processing in simulated web search environments included in the PIAAC assessment of problem solving in technology-rich environments. The third presentation by He and colleagues focuses on evaluating consistency of adult behavioral patterns across multiple problem solving tasks using PIAAC process data from the US. The final presentation by Maddox and Care expands the perspective in that it takes into account not only PIAAC log data, but other sources of (multimodal) process data such as verbal interaction, facial expressions and gesture.

Contributions:

Utilizing Process Data to Examine Gender Differences in Literacy in PISA and PIAAC.

Artur Pokropek1*, Francesca Borgonovi2, & Lale Khorramdel3

(1Institute of Philosophy and Sociology of the Polish Academy of Sciences, Poland; 2University College London, UK, and OECD, France; 3 Boston College, USA)

Identifying Strategies of Information Processing in PIAAC Web Search Environments

Carolin Hahnel*, Frank Goldhammer, & Ulf Kroehne (DIPF | Leibniz Institute for Research and Information in Education, Germany,)

Evaluating Consistency of Adult Problem-Solving Behaviors across Multiple Tasks Using PIAAC Process Data

Qiwei He1*, Dandan Liao2, Hok Kan Ling3, & Hong Jiao4

(1Educational Testing Service, USA; 2Cambium Assessment, Inc, USA; 3 Queen’s University, Canada; 4 University of Maryland, USA)

Multimodal Process Data in PIAAC

Bryan Maddox1* & Naomi Care2

(1University of East Anglia & CEMO, University of Oslo, Norway; 2Assessment Micro-Analytics Ltd, UK)

*Presenting Author

 

The pre conference workshops welcome researchers from different disciplines interested to work or already working with PIAAC data (including supplement and longitudinal data on a national level). It is expected that the participants have good empirical knowledge and experience in the respective statistical software. The workshop comprises lectures and practical sessions covering the following elements: (a) Theoretical and methodological input from the lecturers (see description of contents below); (b) Research input from the participants, that is, presenting their own research or research ideas with PIAAC (-L) data; (c) Discussion of the questions outlined in the workshop regarding the data used and methods as well as specific feedback of the lecturers.

Interested researchers can apply with a short expose (max. 1000 words) of their research project/ questions based on the PIAAC data. The application also must include the title of the project, the specific dataset used, as well as name, institute/ university affiliation, and contact information of the applicant. Please send your application with the respective subject (“Application Workshop 2021 Analyzing PIAAC Data with SEM” or “Application Workshop 2021 Analyzing PIAAC Data with Stata”) to the Research Data Center (RDC) PIAAC (fdz-piaac@gesis.org) by March 30, 2021. There will be no participation fees. Each workshop will be limited to a maximum of 15 participants. In cases that workshops cannot take place at GESIS Mannheim, they will be conducted virtually. The confirmation of participation will be sent by April 30, 2021.

Workshop A: Analyzing PIAAC data with structural equation modeling in Mplus

Lecturer: Ronny Scherer (Centre for Educational Measurement at the University of Oslo)

Date: October 5th to October 6th, 2021

Place: GESIS Mannheim

Content: Structural equation modeling (SEM) represents a statistical approach to disentangle the relationships among latent and/or manifest variables, across groups, over time, and at different analytical levels. The potential of SEM has been recognized in many areas, including educational sciences, sociology, psychology, and business. This workshop provides an introduction to the principles and procedures of basic and more advanced SEM in the context of the international large-scale assessment PIAAC. Specifically, the following topics are covered: (a) Principles of structural equation modeling (model specification, identification, estimation, and evaluation), (b) Measurement models and confirmatory factor analysis, (c) Measurement invariance testing with few and many groups (including multi-group CFA, multilevel CFA, and the alignment method), and (d) Structural regression and indirect effects models (including multi-group and multilevel SEM). In a second part of the workshop, participants will present their own research or research ideas using PIAAC data and receive feedback on how to improve the analysis.

Data: PIAAC Public Use Files

Software: SPSS and Mplus

Recommended reading: Maehler, D. & Rammstedt, B. (2020). Large-scale cognitive assessment:Analyzing PIAAC data. Series: Methodology of Educational Measurement and Assessment (MEMA). Springer: Cham. https://link.springer.com/book/10.1007/978-3-030-47515-4

 

Workshop B: Analyzing PIAAC and PIAAC-L data with Stata

Lecturer: Simon Wiederhold (Catholic University of Eichstätt-Ingolstadt) & Guido Schwerdt (University of Konstanz)

Date: October 5th to October 6th, 2021

Place: GESIS Mannheim

Content: The first part of the workshop will lay the conceptual foundation for the analysis of PIAAC and its longitudinal extension, PIAAC-L. In particular, participants will gain insights into how the data can be used to improve knowledge about the determinants and consequences of cognitive skills. Special emphasis will be on the possibilities of estimating causal effects. After this general discussion of the analysis potential of PIAAC and PIAAC-L, workshop participants will develop their methodological skills in analyzing the data in various applied sessions. Participants will acquire knowledge in both cross-sectional and panel data analyses. It is also discussed how regression results can be depicted graphically. In the second part of the workshop, participants will present their own research or research ideas with PIAAC(-L) data and receive detailed feedback on how to improve the analysis.

Data: PIAAC Public Use Files; German PIAAC Scientific Use File (doi: 10.4232/1.12660); PIAAC-L Scientific Use Files (doi: 10.4232/1.12925).

Software: Stata

Recommended reading: Maehler, D. & Rammstedt, B. (2020). Large-scale cognitive assessment:Analyzing PIAAC data. Series: Methodology of Educational Measurement and Assessment (MEMA). Springer: Cham. https://link.springer.com/book/10.1007/978-3-030-47515-4

Note: To participate in this workshop signing a PIAAC-L data use agreement  is mandatory, in order to comply with the data protection regulations and to enable work with the data during the workshop.

 

For any questions please contact the RDC PIAAC (fdz-piaac@gesis.org).