Better and more information on factors influencing learning across Southeast Asia
The Southeast Asia Primary Learning Metrics (SEA-PLM) is a new regional assessment developed specifically for the countries represented under the Association of Southeast Asian Nations (ASEAN) and the Southeast Asian Ministers of Education Organization (SEAMEO). The novelty of SEA-PLM is such that it has only recently completed all field trials (i) in the seven ASEAN member states (Brunei Darussalam, Cambodia, Lao PDR, Malaysia, Myanmar, Philippines and Vietnam)signed up for SEA-PLM, and preparations are under way for the first main survey scheduled for 2019. One of the important aspects of SEA-PLM is its regional focus and contextualisation. This goes for both the relevance and appropriateness of the educational test items, but equally and just as important, for the four background questionnaires targeted the student, parent/caretaker, teacher and school principal.
SEA-PLM targets learners at the end of primary cycle (grade 5) and combines assessment of foundational skills (reading, writing and mathematics) with selected domains of 21st century skills (global citizenship). The combination of domains aims to embrace the increased demand for formal education to include the development of generic skills as well as traditional academic subjects. To ensure these domains truly capture and reflect the education policies of each engaged Ministry of Education (MoE), the curricula of all ASEAN member states (from grade 3-6) have been analysed with the technical support of the Australian Council for Educational Research (ACER). In close consultation with national and regional experts, and led by the SEAMEO Secretariat and UNICEF EAPRO, the SEA-PLM Assessment Framework was developed to identify ‘what should be measured and in what way’. Recognising the variations between countries, rather than being curriculum-based, SEA-PLM is curriculum-referenced and represents structures, conceptual underpinnings and overarching orientations across all ASEAN countries.
SEA-PLM aims to measure both curricular and cross-curricular knowledge, skills and understanding that are likely to allow school-aged students to progress successfully through school and ultimately to play a constructive and fulfilling roles as citizens in society. It adopts broad definitions for the domains of mathematics, reading, writing and global citizenship that are consistent with curriculum specifications but that allow for a focus on the extent to which students in a Southeast Asian context are able to make effective use of their knowledge in a variety of relevant contexts. To convey this broadness and the parallels in the way that these four domains are conceived, the domains are referred to as reading literacy, mathematical literacy, writing literacy and global citizenship. This concept of literacy within the SEA-PLM Assessment Framework is also what closely connects the learner domain frameworks with the contextual framework, recognising that the learning of individual students is set in overlapping contexts of school and out-of-school learning.
The SEA-PLM contextual framework provides a classification of factors according to the multi-level structure inherent to the process of student learning, divided into the context of the individual, the context of home and immediate out-of school environment, the context of school and classroom as well as the context of the wider community. In addition, the status of contextual factors explores the ways in which learning takes place (antecedents, e.g. socio-economic status) and the factors that directly influence learning (processes, e.g. students’ learning environment at home). This contextual information is captured through four questionnaires, where variables related to the context of schools and classrooms are collected through a school and a teacher questionnaire. Variables related to information on antecedents of the individual student and the home environment as well as about some process-related variables (for example, learning activities) are collected through a student and a parent questionnaire.
The questionnaires were composed in close consultation with relevant country experts from all participating countries. The first drafts of the student, school and parent questionnaires were dispatched to respective country MoE representatives to assess and ensure if the questionnaires appropriately captured their educational context; if the material could be deemed appropriate for their national context; and if they had any suggestions for additions and amendments.
The global citizenship domain also uses the questionnaires (ii) as it was deemed as the most appropriate instrument to measure its domain, however, the questionnaires function as test instruments rather than contextual instruments. This means that the global citizenship domain measures competencies, in a comparable manner to how mathematics or reading competencies are measured, and therefore, is categorised and measured as a learning domain for which learning outcomes can be analysed. Given the limited time available for the SEA-PLM global citizenship assessment and its status of being a relatively new and contested domain, it was decided that, for the first round of the main survey, the assessment of global citizenship should focus primarily on capturing attitudes and values (and to a limited extent, behaviours and skills). The global citizenship instruments are merged with the student and teacher questionnaires respectively, essentially combining a contextual instrument with a test instrument.
As SEA-PLM is planned to be conducted over time, the comprehensive contextual data collected as part of the assessment will provide evidence to guide effective and purposeful improvements in each participating country’s education system and will in time allow nuanced interpretation of the impact of educational reforms. Specifically, by analysing the relationship between contextual factors and student learning outcomes, SEA-PLM can inform teacher improvement agendas; provide a reference point for the development of teacher standards; provide a helpful framework for curriculum reform initiatives; or provide a means to assess the effectiveness of financial reform initiatives to improve access and equity to high quality education. In short, the comprehensive contextual questionnaires applied as part of the SEA-PLM will place student achievement results in a context that is specifically relevant for the countries of the ASEAN region and provide unparalleled opportunities for improved understanding of students learning outcomes across and within ASEAN member states.
- The field trial aims to test the protocols and appropriateness of the survey instruments and is administered among minimum 13,000 students, across 7 countries, tested in 9 languages. This amounts to data collected for over 1 million items using approximately 30,000 booklets.
- Items were constructed using a Likert-style item response format which will require respondents to indicate the best response for them. Students are advised that there are no right or wrong answers in this instrument. For more details, please consult the Southeast Asia Primary Learning Metrics (SEA-PLM) Global Citizenship Domain Assessment Framework, February, 2017.
- Southeast Asia Primary Learning Metrics (SEA-PLM)
- Australian Council for Educational Research (ACER)
- Southeast Asia Primary Learning Metrics (SEA-PLM) "Assessment Framework”. Revised Draft Document incorporating feedback from the Domain Technical Review Teams
- Southeast Asia Primary Learning Metrics (SEA-PLM) Global Citizenship Domain Assessment Framework
Written by: Camilla Woeldike, UNICEF East Asia and Pacific Regional Office (EAPRO).
Editors: Jacqueline Cheng & Jeaniene Spink, ACER and Ethel Valenzuela, SEAMEO Secretariat
For more information, please contact: Camilla Woeldike [cwoeldike(at)unicef.org] or Dr Ethel Agnes P Valenzuela [ethel(at)seameo.org]
Photo credit: ©Shutterstock/Paulaphoto; ©UNICEF/Woeldike;