Giving countries the support they need: Ensuring educational quality through assessment

The Network on Education Quality Monitoring in the Asia-Pacific (NEQMAP) turns five – assessing the network’s contributions to national capacity development and SDG4 in the past, present and future

A recent NEQMAP workshop in Penang, Malaysia, focused on three dimensions: Conceptualization of contextual factors, methodological issues of contextual data and data analysis, and multi-level modelling to analyze factors related to learning outcomes. Using this opportunity, the NEQMAP Secretariat wanted to hear from country participants on their views on how UNESCO Bangkok, through NEQMAP, can contribute to their progress towards SDG4.

As a veteran NEQMAP member, Bhutan has participated in many of the platform’s initiatives. We spoke with Ms Kinley Dema, an Education Monitoring Officer from the Bhutan Council for School Examinations and Assessment (BCSEA), to understand Bhutan’s views on NEQMAP’s contributions. Conversely, as a newcomer, the Kyrgyz Republic can provide insights on how NEQMAP can support new participants moving forward. Ms Dinara Dautova, a Methodologist in the National Testing Center (NTC) of the Ministry of Education and Sciences of the Kyrgyz Republic, offered that complementary view.

How useful did you find the workshop? What were some of the important takeaways that you can apply to your home country?

Ms Dema: I found the workshop extremely beneficial, particularly the presentations and discussions on types of educational models related to contextual factors, contextual data in light of different international large-scale assessments (examining similarities and differences), and conceptualizing and operationalizing a construct and analysis of test items using Content Validity Ratio (CVR). It helped in appreciating the importance and use of data collected from background questionnaires designed to identify factors affecting student learning.

I found the group discussions very insightful as they offered opportunities for the member countries to debate issues related to education assessments, share experiences and exchange best practices. Discussions also offered solutions to common problems encountered in education assessment and advice on tackling challenges. It was invaluable to have met and shared experiences with participants from 20 countries.

 

I found the group discussions very insightful as it offered opportunities for the member countries to debate on issues related to education assessments, share experiences and exchange best practices. 

 

The knowledge and skills learned at this workshop come at the right time, as they can be incorporated into our preparation for the next National Education Assessment (NEA) in 2021 conducted by BCSEA. The new concepts and methodologies learned can help to refine the NEA with test items using standard tools such as CVR, and improved contextual questionnaires being designed.

Ms Dautova: I found the workshop very insightful as I was exposed to new content and methodologies, especially with regard to assessment software and statistical concepts. In particular, learning the practical skill of using the Hierarchical Linear Model (HLM) software was eye-opening as SPSS is the main statistical tool in my line of work. By utilizing new statistical software in our data analyses of Kyrgyz Republic’s assessments and programmes, it will help to provide a greater depth and breadth to our findings and our policy recommendations moving forward.

The modality of the workshop was in my opinion both interactive and effective. Through the implementation of practical exercises interspersed with panel and group discussions, participants including myself were able to understand, apply and internalize the knowledge gained at the workshop. This will ultimately aid me in effectively disseminating this knowledge and skills to my colleagues in my home country.

The facilitators were very knowledgeable on their areas of expertise and yet very open to questions and competently explained key concepts. Their adept facilitation of teamwork among the participants is impressive.

Bhutan has been a long-time member of NEQMAP. How has NEQMAP and its past capacity development workshops contributed to your country’s capacity-building efforts in monitoring educational quality through student assessment?

Ms Dema: Since BCSEA joined the network as an institutional member, it has participated in seven capacity development workshops of NEQMAP. Our participation in NEQMAP workshops has helped us in refining our monitoring of the quality of education as well as affected tangible policy outcomes.

Firstly, our NEA became more effective and successful after BCSEA joined NEQMAP. We could draft an NEA framework and successfully assess the national language (Dzongkha literacy) in 2013. The policymakers and stakeholders were cognizant of the findings and made necessary policy interventions to improve the quality of Dzongkha literacy.

Secondly, the capacity development workshops enhanced our technical skills and guided us in the development of competency-based items and contextual questionnaires, sampling procedures, data analysis and report writing. In particular, it aided the development of year-end test instruments for grades 3, 4, 10 and 12, and of assessment instruments in various subjects for different grade levels (Teacher’s Reference for Competency Based Assessment). These processes and methodologies helped us in getting a realistic picture of our education quality, promoting both vital knowledge sharing with stakeholders as well as effecting policy changes. In the last two years, BCSEA studied existing year-end assessments for grades 3 and 5, which resulted in substantial findings that propelled policymakers and schools at the grassroots level to intervene for the improvement of education quality. Schools could also share best practices with each other based on the data analysis conducted by BCSEA.

Lastly, the study on School-Based Assessment conducted in 2016 under the aegis of NEQMAP was another milestone. We unveiled the different methods of assessments practised in different schools across the country and how the Ministry of Education, Royal Education Council and BCSEA differed in their descriptions of formative assessments. The outcome revealed the state of education among the sample schools, which was indicative of the divergence in practices leading to underachievement of a common learning outcome.

As part of the Education 2030 Incheon Declaration, countries committed to ‘By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes’ as stated by SDG4.1. How have NEQMAP’s regional activities helped Bhutan in its progress towards achieving SDG4, particularly Target 4.1? Are there other areas that UNESCO Bangkok, via NEQMAP, can further develop to support countries in the Asia-Pacific region in making SDG 4.1 a reality?

Ms Dema: Since planned development was implemented in Bhutan and modern schooling introduced in the 1960s, education has been provided free of cost. However, no research was done in the early days to assess whether the quality of education was comparable to other education systems in the region, primarily due to a lack of capacity and know-how. With the technical assistance provided by NEQMAP over the last few years, however, we can now measure the standard and quality of our education at both the primary and secondary levels better. This monitoring and evaluation helps us in refining our education policies as well as teaching and learning initiatives to achieve SDG4.1.

Some of the procedural changes and initiatives that we have implemented to help us monitor and achieve equitable and quality primary and secondary education include referring to the learning outcomes at each grade while framing test items for the year-end assessment papers. We apply the skills learned from NEQMAP workshops to construct test items and draft teacher reference guidebooks in various subjects. We focus more on competency items so that students become competent and skillful in their lives. We even share our experiences, knowledge, and resource materials with teachers during the test development workshop and evaluation camp. At every opportunity that BCSEA gets to interface with the teachers, these materials and information are imparted as professional development resources and we urge them to share it with their colleagues.

Looking to the future, NEQMAP can support countries like Bhutan in the Asia-Pacific region to make SDG4.1 a reality by firstly, continuing to organize technical workshops to strengthen national capacities in the area of learning assessments, including the construction of test items (competency-based questions), development of contextual questionnaires, data analysis and report writing. Secondly, training could be provided on how to align prescribed curricula with education assessments, and lastly, a study could be conducted to investigate whether primary and secondary school curricula are suited to target students’ age group.

How do you think NEQMAP’s ongoing regional activities can contribute to Kyrgyz Republic’s capacity-building efforts in monitoring educational quality through student assessment, as well as in its progress towards achieving SDG 4, particularly target 4.1?

Ms Dautova: The Kyrgyz Republic has made great gains in its progress towards achieving SDG Target 4.1. In accordance with the country’s Constitution (2010) and the Law on Education (2003), all citizens are guaranteed free basic general and secondary general education. A large part of the national budget expenditure is directed specifically to developing and improving school education. Annually, more schools and kindergartens are being established to ensure a stable growth in the provision of preschool and school education, and there is increased funding in the school nutrition programme year on year.

 

 

We need technical and staffing support especially in terms of strengthening our capacity in the field of statistics and assessments, and we believe that NEQMAP is the right platform to help us grow. 

 

Nevertheless, there is a need to monitor the quality of the education provided so that future policy interventions can be effected for the betterment of the education system. Consequently, there is a capacity gap in the technical capabilities of the staff to competently assess if the existing educational mechanisms are both equitable and leading to relevant and effective learning outcomes.

Therefore, we believe that participating in NEQMAP’s regional initiatives in the diverse fields of research, capacity development and knowledge sharing can help to develop and upgrade the professional capabilities of my colleagues and myself in monitoring and evaluating the equitability and efficacy of our education system. Indeed, being able to work with the best experts is rare in the context of our professional work. Therefore, NEQMAP’s capacity development workshops at the international level with highly qualified facilitators is a unique source of knowledge of educational evaluation, which my colleagues and I can gain a lot from.

Though Kyrgyz Republic’s involvement in NEQMAP initiatives is relatively young, we have been very impressed thus far and are very grateful to have been given this opportunity to be a part of the platform’s vibrant and diverse international exchange on best practices in learning assessment. We see a lot of synergies moving forward in helping our country in its progress to achieving SDG4.1, and the director of NTC, Mrs Shamshidinova BS is keen for the NTC to become an institutional member of the network. At the end of the day, we need technical and staffing support especially in terms of strengthening our capacity in the field of statistics and assessments, and we believe that NEQMAP is the right platform to help us grow and achieve our SDG targets.

Interviews by Farah Nadine Seth
Main photo credit: Shutterstock