SECTION 3: OVERVIEW OF THE RESEARCH FINDINGS
3.3 SUMMARY OF KEY FINDINGS IN RELATION TO THE DART STUDENT ASSESSMENT
3.3.3 Data Gathered through DART
The following information was generated about student progress through the DART:
•= An indication of each student's CSF level for Reading and Writing based on the DART Descriptive reports - a set of indicators describing the literacy skills typically displayed by students at this level of achievement (In general, the indicators below their reported achievement would be easy for them to do, while those above their position would be either difficult, or not achievable at this stage). A sample is attached as Appendix 1.
•= Identification of individual student shifts between CSF levels.
•= Analysis of individual student progress in the reading and writing strands in CSF levels for Reading and Writing between March and November 2000, based on raw scores.
•= An average of the shifts within CSF levels for each school (for each year level where multiple levels were assessed).
•= An average of the shifts within CSF levels across all schools and an indication of the percentage of students who had gone up, down or remained the same.
Research findings on the progress of students in the Reading and Writing strands between the pre and post assessment phases needs to be read and interpreted, however, in light of the following important qualifying information.
•= The time frames between the pre and post assessments were closer than is ideal which minimised the time in between for development of aspects of literacy.
•= In many schools the selected focus for their research did not necessarily match the specific skills and abilities assessed by the DART strands, which limited the validity of DART in assessing student achievement in those areas.
•= There may have been inconsistency of moderation across schools.
•= Assessment tasks were not always clear, which led to confusion for both teachers in their assessment and moderation and for students in their interpretation of the task.
•= The post assessment materials in Reading were found to be (as acknowledged by ACER) considerably more challenging than those of the pre assessment phase in both the literacy demands and in the volume of the material to be completed - although this was calibrated accordingly to show comparative shifts from the pre-assessment, the degree of frustration for students, created through the more sophisticated tasks and instructions, could be considered to impact on the results.
Broad Trends
The CSF levels for all students were calculated through The DART assessment. To refine this further, each CSF level was divided into low (L) medium (M)and high (H). Overall
results of the research students across the 12 case study schools, when analysed within year levels, indicated the following range of levels in both March and November.
READING
Year level Student Numbers Range in March (Pre) Range in November (Post)
Year 5 28 CSF 3(L) – CSF 5(L) CSF 3(L) – CSF 5(H)
Year 6 58 CSF 2(M) – CSF 5(L) CSF 3(M) – CSF 5(L)
Year 7 188 CSF 1 – CSF 6 CSF 2(H) – CSF 5(L)
Year 8 11 CSF 3(M) – CSF 4(H) CSF 3(H) – CSF 5(L)
Table 1: Range of Student CSF levels, by year level, in the Pre and Post Assessment - Reading WRITING (content)
Year level Student Numbers Range in March (Pre) Range in November (Post)
Year 5 28 CSF 2(H) – CSF 5(H) CSF 2(H) – CSF 4(H)
Year 6 58 CSF 2(U) – CSF 5(M) CSF 2(U) – CSF 5(M)
Year 7 184 CSF 2(M) – CSF 4(M) CSF 1 – CSF 5(H)
Year 8 9 CSF 2(H) – CSF 4(H) CSF 3(H) – CSF 5(M)
Table 2: Range of Student CSF levels, by year level, in the Pre and Post Assessment – Writing (content)
WRITING (language)
Year level Student Numbers Range in March (Pre) Range in November (Post)
Year 5 28 CSF 2(L) – CSF 4(L) CSF 2(L) – CSF 5(M)
Year 6 58 CSF 2(H) – CSF 4(M) CSF 2(M) – CSF 5(H)
Year 7 184 CSF 2(H) – CSF 4(L) CSF 1 – CSF 5(M)
Year 8 9 CSF 2(L) – CSF 4(H) CSF 3(M) – CSF 5(H)
Table 3: Range of Student CSF levels, by year level, in the Pre and Post Assessment – Writing (language) Although the number of students at some year levels was low, when the range of CSF levels were summarised for each school, it was very clear that these schools were dealing with a hugely diverse range of student literacy abilities. All schools, for example, covered a range of 3 or 4 CSF Levels within one year level. This highlights the enormous challenges facing schools in meeting the literacy needs of all of their students.
Both Reading and the two aspects of Writing (content and language) showed considerable improvement across the whole cohort when comparing the average level of student achievement in March and in November. This is indicated in the graph below, where the percentage of students who moved up or down, or who stayed the same (stay), were displayed. These shifts included the shifts within the CSF levels (high , medium and low).
For example a student may have moved from low level 4 to high level 4, a significant shift within one CSF level.
Graph 1: DART Results - Total Cohort – Average Shifts in CSF levels between March and November 2000
In the majority of schools, the students at the lower end of the CSF levels demonstrated a greater shift than those at the higher end. One school for example, had 34% of its students at the upper end of Level 2 for Writing (content), but by November all of these students had moved to at least the upper end of Level 3.
Although there were no significant trends shown in the differences in shifts between high ESL and low ESL schools, there were differences in the average results of boys and girls across the full cohort, as shown in the graphs below. The exception to this was one school that focused their research on using technology for literacy. In this school the progress made by the boys was, on average, greater than that made by the girls.
Graph 2: Average Shifts in CSF levels between March and November 2000 Comparison of Girls and Boys
GIRLS BOYS
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Reading Writing -
Content
Writing - Language
Percentage of Students
Up Down Stay
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Reading Writing -
Content
Writing - Language
Percentage of Students
Reading Writing - Content
Writing - Language
The information gained by the schools through the DART could be used in a variety of ways.
Some schools used their data to identify general areas for improvement.
As a result of implementing DART it was decided that we needed to address the students’
reading skills.
Others focused on strategies to address more specific aspects which were highlighted by the indicator descriptions within each level, eg the need to ‘select a main idea from competing information’ (higher end of Reading Level 4).
More specific comments on the data at an individual school level have been included in some of the case studies.
Up Down
Same