It is proposed that moderation would include a combination of school-managed assessment and external assessment processes that would differ to some extent from current strategies.
The moderation of school-managed assessment would be strengthened and new processes for monitoring introduced. School-managed assessment would not be statistically adjusted by external assessment; however, the Council would intervene in speсific cases where it was evident that standards were not being applied consistently.
Post-Compulsory Education Review Position Paper 43
Structures and processes underpinned by strategies similar to those in the current system would include the steps listed below.
Step 1: Monitoring of teacher ratings would involve processes to strengthen teacher judgments about the application of standards.
Step 2: Verification of ratings of achievement levels for students would involve endorsement of final ratings by the course of study assessment and moderation panel.
Step 3: Quality assurance processes would involve a range of strategies to evaluate structures and processes for the application of standards.
To implement these processes for each course of study the Curriculum Council would have:
• a course of study assessment and moderation panel to oversee the school-managed and external assessment;
• school-based moderators; and
• Curriculum Council moderators.
Sее Figure 3 (p 48, 49) for more information about these roles.
Step 1: Monitoring of teacher judgments
To build a common understanding of standards on the scales of achievement and their application in a range of contexts, there would be processes for:
• accreditation of schemes of assessment;
• assessment seminars;
• consensus meetings; and
• school visits by Cuniculuni Council moderators.
Accreditation of schemes of assessment: the school-based moderator for a course of study would oversee the development of schemes of assessment in a school or group of schools. A scheme of assessment for each class group enrolled in a course of study, outlining the types of assessments and contexts to be used to measure achievement of the course of study outcomes, would need to be endorsed by the Curriculum Council moderator in consultation with the school-based moderator.
A scheme of assessment would play an important role in the process of validating teacher ratings of student achievement of outcomes as represented in the selected samples of work for a course of study. The samples would need to be consistent with the scheme of assessment.
Examples of schemes of assessment for the full range of students in a course of study would be provided in support materials.
44 Post-Compulsory Education Review Position Paper
Assessment seminars: district seminars would be replaced by assessment seminars. These seminars would adopt a professional development approach and would specifically support teachers
beginning to teach a course of study for the first time and those teachers who had been identified through monito
ri
ng processes as requiring extra support and guidance in the application of standards.Consensus moderation: there would be annual consensus meetings for each course of study to provide opportunities for teachers to build a common understanding of the scale of achievement for each course of study outcome.
A teacher representative from a school would be required to bring to consensus meetings a range of samples3 for particular students from across all class groups enrolled in a course of study. The whole collection of samples would need to reflect the range of levels on the scales of achievement.
Teachers would initially be involved in common marking exercises using unmarked samples.
There would be discussion and justification for ratings. When there was a common understanding of ratings for the levels on the scales of achievement, the teachers would work through the samples they brought to the meetings.
included with each student's collection would be evidence of assessment conditions
(time, group/individual, etc.) and an explanation by the teacher outlining how, and in what particular context, on-balance judgments were made in relation to each of the outcomes.
In small groups they would cross-mark each other's work discussing and justifying ratings. This approach would provide clarification for teachers and assist them in the consistent application of standards.
Curriculum Council moderator school visits: there would be regular visits to schools to support teachers in their understanding of the scales of achievement and to provide advice and recommendations about assessment.
Following consensus meetings Council moderators would follow up on teacher ratings of student work samples (school-selected and random) where there were indications that standards were not being applied consistently. They would advise school-based moderators about necessary
adjustments and make recommendations for professional development strategies for some teachers:
r
for example, attendance at assessment seminars and additional school visits.
in addition, the course of study assessment moderation panel could request a proportion of the samples from consensus meetings for auditing and validation purposes.
з The samples for each student should represent a collection of their work across a range of assessment contexts.
Post-Compulsory Education Review Position Paper 45
Step 2: Verification of levels of student achievement
Verification of achievement levels for students would involve processes in which final
school-managed ratings of achievement levels for students would be considered by the Council moderator and endorsed by the course of study assessment and moderation panel. Advice would be provided about teachers' proposals for awarding levels of achievement.
Preliminary data on achievement levels for students would be collected from each class group , toward the end of the first year in which the students were enrolled in a course of study. The data would be used both for monitoring purposes as part of the consensus moderation process and for certification of the achievement of those students who withdrew from a course of study.,
The final verification process would build on the ongoing process of consensus moderation, whereby a representative range of students' collections of samples at each level on the scale of achievement would be checked for comparability by:
• school-based moderators (a wide range within the school/schools);
• teachers at consensus meetings (a range of teacher-selected and random samples);
• Council moderators (consensus meeting samples and further deep sampling of those that might require adjustment); and
• the course of study assessment and moderation panel (a random sample or Council-selected sample).
In consultation with the Council moderator the course and assessment moderation panel would adjust final teacher ratings in situations in which standards in the scales of achievement were not being applied consistently.
Step 3: Quality assurance
The current statistical moderation process would be replaced by a statistical process designed to monitor the effectiveness of the strengthened consensus moderation processes and to identify groups for future moderation.
The Council believes that there would no longer be a need to adjust teachers' ratings to match those for the external assessment because the scales of achievement for the course of study outcomes would provide the external measure for moderating school-managed and external assessment. The levels of achievement for course of study outcomes would already be on common scales of comparable difficulty. .
46 Post-Compulsory Education Review Position Paper
As part of the rigorous quality assurance process, statistical analysis would continue to be used to compare achievement levels for students for external assessment and school-managed assessment;
however, no end-of-year adjustment would be carried out. The results from external assessments would be used to construct statistical models for investigating any systematic bias in
school-managed assessment. It would be expected that the results from school-managed and external assessment would be closely correlated because they are both assessments of the same course of study outcomes.
1f there were a significant difference in the two sets of ratings for a cohort of students in a school, there would be intensive follow-up moderation processes and school visits in the following year.
The teacher/s involved would be provided with additional support by the school-based moderator, the Council moderator and through assessment seminars.
There would also be an intervention process by which the course of study assessment and moderation panel would review the school-managed and external assessment to determine the adjustments to be made if a pattern of atypical ratings persisted.
An expert advisory group would be established to work with the Curriculum Council secretariat to identify the statistical processes most suitable for comparing ratings for school-managed and external assessment using a common scale with explicit descriptions of the standards for each level.
These statistical processes would also need to provide the mechanism for the finer discrimination required for devising a Tertiary Entrance Rank.
Recommendation 23
That current consensus moderation structures and processes be strengthened to:
• reflect an outcomes approach and the introduction of scales of achievement;
• provide more specific moderation strategies for developing a common understanding of course of study outcomes; and
• include more school-based moderators, assessment seminars and course of study assessment and moderation panels.
Recommendation 24
That statistical moderation be replaced by rigorous quality-assurance processes, with the Curriculum Council to intervene where standards are not being applied consistently.
Post-Compulsory Education Review Position Paper 47
There would be a course of study assessment and