Comision Simce Anexo2 Forster

8/8/2019 Comision Simce Anexo2 Forster

1/87

ANEXOS

LA EXPERIEN CIA INTERNACIONALEN SISTEMASDE MEDICIN :

ESTUDIODE CASOS

MARGARET FO RSTERG ILBERT A. VALVERDE

Comisin para el Desarrollo y Uso del Sistemade Medicin de la Calidad de la Educacin

DICIEMBRE 2003


2/87

La Experiencia Internaciona l en Sistemas de Med ici n:

Estudio de Casos

Co misin pa ra el Desarrol lo y Uso del Sistema d e M edic i n

de la C a l idad de la Educac in

ISBN 9 5 6 -2 9 2 -0 8 1 -X

M inisterio de Educacin, Repblica d e Chile

A la me d a 1 3 7 1 , Sa n tia g o

www.mineduc .c l

D ic iembre 2003


3/87

PRESENTACI N

La Comisin para el Desarrollo y Uso del Sistema de Medicin de la Calidad de la Educacin

recibi entre sus encargos recopilar insumos que permitan conocer los modelos de sistemas

de medicin que se estn utilizando en diferentes pases, y cules son los beneficios y proble-

mas que se asocian a dichos modelos. Ello, con el fin de rescatar aquellos aspectos que se

consideren adecuados para ser incorporados de acuerdo a la realidad de nuestro pas.

Nace as la iniciativa de contar con dos consultores internacionales expertos en el rea de

medicin a nivel de sistemas educativos. Ellos seran los encargados de entregar informacin

para que las recomendaciones de la Comisin tuvieran en cuenta los inconvenientes y solu-

ciones que se han encontrado en otros pases del mundo.

Para cumplir con esta tarea, la Comisin cont con el apoyo de Margaret Forster investiga-

dora del Consejo Australiano para la Investigacin Educativa (Australian Council for

Educational Research, ACER) y de Gilbert Valverde investigador de Albany, Universidad

Estatal de Nueva York.

El trabajo realizado por estos expertos fue dado a conocer en el Seminario Internacional sobre

Sistemas de Medicin y Uso de Resultados1, en el que participaron como expositores. Los

principales planteamientos presentados en dicho seminario fueron completados y

profundizados en documentos elaborados especialmente para la Comisin. Estos documen-

tos son Assessment systems: two case studies, de Margaret Forster y La poltica en evaluaciny currculo ante el desafo de la calidad, de Gilbert Valverde.

A continuacin, se presentan dichos documentos, ellos constituyen los anexos que acompa-

an al documentoEvaluacin de Aprendizajes para una Educacin de Calidad, elaborado por

la Comisin.


4/87


5/87

NDICE

PRESENTACI N 05

LA POLTICA EN EVALUACIN Y CURRCULO ANTEELDESAFODELACALIDAD 09

G. VALVERDE

Introduccin 11

TENDENCIASMUNDIALESEN LAO PERACIO N ALIZACIN DECRITERIOSDECALIDAD 13

Fijando criterios para la evaluacin de la calidad 15

ESTUDIO DECASOS 19

Pases Bajos 21

Una prueba longitudinal del aprendizaje para monitorear 2 2

el impacto de polticas

Un modelo mixto y voluntario de pruebas de egreso en 2 3

educacin primaria

Un modelo mixto y obligatorio de pruebas de egreso de 2 3

educacin secundaria

Estado actual de la discusin 2 4

Dos casos en los Estados Unidos 26Nueva York 29

Perfil general del sistema de evaluacin actual 2 9

Sistema actual de pruebas de egreso 3 1

Sistema de responsabilizacin 3 2

Tennessee 34

Perfil general del sistema de evaluacin actual 3 4

El sistema de evaluacin del valor agregado 3 5

Algunas lecciones ilustradas por los casos 37

Obras Consultadas 39

ASSESSMENTSYSTEM S: TWO CASESTUDIES 45

M. FO RSTER

Executive summary 47

Background 51

Introduction 52


6/87

CASE STUDY 1: W ESTERN AUSTRALIA (WALN A & MSE) 55

Contextual features 57

Geographic, demographic, economic, po li ti ca l and religious context 57

Responsibility for schooling in Australia 5 8

Provision of schooling in Australia 5 9

Funding for schooling in Australia 6 0

C ommon and agreed national goals for schooling in Australia 6 5

The Provision of schooling in W estern Australia 6 6

Student intake 6 7

Funding 6 8

Systemwide assessment 69

Different programs for different purposes 6 9

W ALN A 7 0

M SE 7 3

Educational impact: positives 75

System level monitoring 7 5

School level: use of results and acceptance of program 7 7

Educational impact: concerns 88

CASE STUDY 2: ENGLAND (N ATIONAL CURRICULUM ASSESSMEN T) 93

Contextual features 95

Geographic, demographic, economic, po li ti ca l and religious context 95

Responsibility for schooling in England 9 5

The provision of schooling in England 9 6

Funding for schooling in England 9 6

Agreed goals for schooling in England 9 8

Changed processes of school management 9 9

Target setting 1 00

Student intake 10 0

Funding 10 5

Systemwide assessment 107

Educational impact: positives 109

System level monitoring of standards and initiatives 1 09

School level use of results 1 09

W hat evidence is there that these strategies have improved student learning? 1 15

Educational impact: concerns 117

Final reflections 121

References 122

Useful websites 12 3


7/87

ASSESSMEN TSYSTEMS:TW OCASESTUDIES

MARGARET FO RSTERAUSTRALIAN COUNCILFOR EDUCATIONAL RESEARCH

THISPAPERHA SBEEN PREPAREDBYTHE AUSTRALIAN COUNCILFOR EDUCATIONAL RESEARCH (ACER)FORTHE CHILEAN M IN ISTRYO F EDUCATION TO INFORMDISCUSSION W ITHTHE N ATIONAL

COMMITTEEFORTHEDEVELO PMENTAN DBETTERUSEO FTHE N ATIONAL ASSESSMEN T SYSTEM .


8/87


9/87

4 7A SSESSMENT SYSTEM S: TW O C A S E STUDIES

EXECUTIVE SUMMARY

The management of an education system is a complex and expensive operation, and effective

management demands dependable information on educational outputs. Systemwide (state

or national) assessment programs provide this information through the systematic and regu-

lar measurement of student learning. These programs are designed primarily to investigate

and monitor the health of the system and to improve student learning by providing

information to stakeholders at different levels of the system: policy makers, schools, teachers,

parents, students and the general community.

Of increasing interest internationally is the role that systemwide assessment programs can

play as agents of reform and accountabilityto provide both direction and motivation toschools, teachers, parents and students. In some countries, student achievement data collected

through state assessment programs are used as a measure of schools contributions to student

learning. In some, schools are rewarded or punished on the basis of students results. In

others, public comparisons of schools achievements in the form of league tables are made.

Using assessment programs to drive change directly brings some challenges. First, there can

be unintended negative consequences of assessment driven reform. Second, using assessment

programs as both the instrument of change and as the measure of change makes procedures

open to scrutiny from two perspectives and takes them into the arena of public debate both

about their effectiveness and about their reliability and validity as measures.

The two case studies explored hereWestern Australias state Literacy and Numeracy Assessment

(WALNA) and Monitoring Standards in Education (MSE), programs; and Englands National

Curriculum Assessmentprovide two different approaches to the collection and use of student

achievement data, in the context of reform and accountability demands.

Western Australia has two parallel assessment programs. The WALNA, a full cohort (census) program

administered annually to students in years 3, 5 and 7, provides information on students knowledge,

skills and understandings in numeracy, reading, spelling and writing. The purpose of WALNA is to

provide parents and caregivers with information about the performance of their children in relation

to nationally agreed benchmarks, and in relation to that of other Year 3, 5 or 7 students across

Western Australia; teachers with information on the performance of individual students and classgroups, and on the level of their students literacy and numeracy development in relation to the

performance of other Year 3, 5 or 7 students; and schools with summary information of their overall

performance.

The MSE program is administered annually to a representative sample of students in each of

Years 3, 7 and 10 in one or two of the eight curriculum learning areas. (Aspects of learning areas


10/87

4 8 A SSESSMENT SYSTEMS: TW O C A S E STUDIES

that are assessed through the WALNA testing program are not assessed through the MSE

program.) The MSE assessments are part of the Departments public accountability procedures

and the results provide a basis for initiating developments that will further improve the outcomesof education in government schools. The purpose of the MSE program is to provide information

to stakeholders about the progress students make in their learning; assist the Department and

schools to monitor and report student performance; and assist the Department and schools to

plan for improvement in student performance.

England has a single systemwide assessment program. The National Curriculum Assessment, a

full cohort (census) program, is administered annually to students at the end of keys stages 1, 2

and 3; that is, at ages seven, 11 and 14. The tests provide information on students knowledge,

skills and understandings in mathematics, reading, writing and spelling and except at key stage 1

in science. The purpose of the tests is to give teachers information about their students attainment,

for use in planning teaching and learning; to provide information to the Government, to localeducation authorities (LEAs) and to the public in general about standards in education nationally

and locally; and to encourage school accountability.

There are many similarities between the Western Australian and the English assessment systems,

in particular the intention to address a number of purposes for assessment and to provide

information of different kinds to a range of stakeholders. Both systems also express a key

intention to impact directly on the work of the classroom teacher.

Both systems:

assess aspects of English (reading, writing, and spelling), and aspects of mathematics at

key stages of schooling; address explicit learning outcomes in the development of their tests;

monitor performance against expectations detailed as described levels of achievement; and

require teachers and schools to use systemwide achievement data in their planning.

The reform and accountability demands addressed directly by the assessment systems are

different, however: The stakes are significantly higher in England than in Western Australia.

The level of expectation is higher in England. When the standards were established in England,

63 per cent of students were above the standard, compared with 85 per cent in Australia.

Target setting has been introduced in England as a management and motivational device.Schools are required to set targets for individual students stating what they are to learn next.

They must also set targets for the school as a whole. One of these targets has to be in terms of

the proportion of students who will achieve level 4 in the National Curriculum tests in

English.


11/87


The test results are high stakes for politicians and their opponents as well as schools, teachers

and students. In 1997 the Government set an overall target that 80 per cent of 11-year old

children should achieve level 4 or higher in English by the year 2002 (an increase of 17 percent over five years). The politician responsible, the Secretary of State for Education and

Employment promised to resign if the target was not met.

The underlying model for the national assessment system is one of competition between

schools with the publication of unadjusted results (league tables). The model assumes that

pressure from parents will assist the government to raise standards.

In other words the central difference between the Western Australian and English systems is

the degree to which the programs are driven by direct accountability pressures. In Western

Australia, although schools need to use the systemwide assessment data in their planning, the

stakes are not high and there are no public school-by-school comparisons. Rather, thegovernment has moved quietly forward to influence teacher practice by sending the message

that all areas of the curriculum are valued, that assessment materials will be made available as

a model for teachers, and that support will be provided to assist schools to use their data more

effectively. The use of two parallel and complementary assessment programs has also assisted

this process. And although this is partly historical (MSE was in existence before the National

agreement to undertake full cohort testing), it is still a strategy worth contemplating.

The difference in reform and accountability pressures also has impacted on the debates

surrounding the assessment systems. In England debate has focused on the validity of

the tests for formative assessment purposes, the distortion of the curriculum and the

misleading use of league tables leading to calls for the publication of value-added measures.This is understandable given the pressure of national targets, and the public nature of

comparisons. Western Australia has so far avoided a heated debate about curriculum

fidelity and is instead exploring issues of equity, and ways to better support teachers in

their classroom practice.

Looking at the changes in school practice in the last couple of years in both countries one

might ask whether the debates are ones now held by primarily by academics and researchers,

rather than by teachers and the community more generally. There is some evidence that in

both countries teachers and schools have accepted the assessment programs and are now

getting on with the job of using data more effectively.

Looking forward and continuing to compare the two systems one might reasonably ask in a

few years time whether the accountability pressures have made enough of a difference to be

worth the initial upheaval. Or whether a less demanding set of expectations can achieve the

same goal of improving teacher practice and student learning outcomes.


12/87


And taking the big view, as noted by Whetton et al (2000), the experience in England illustrates

that the setting and maintenance of standards is a social and societal process. Assessment is a

social activity that we can understand only by taking account of the social, cultural, economicand political contexts in which it operates. While the setting of standards, for example, relies

on empirical information, the information must be interpreted, and the integrity and expertness

of those making the judgements must be accepted publicly and politically (Gipps, 1999).


13/87


BACKGROUND

The Chilean Ministry of Education has appointed a National Committee to revise the National

Assessment System so that it is better aligned with the purpose of improving education. This

National Committee is providing advice to the Minister of Education in particular about

ways to improve the communication of the assessment systems results to different stakeholders.

To achieve its objectives, the Committee is gathering information about assessment models

used worldwide and considering the benefits and problems of each model with a view to

improving the Chilean assessment system.

This paper contributes to the Committees deliberations by providing two detailed case studies:

Englands National Curriculum Assessment, and the Western Australias (WA) state Monitoringin Education (MSE), and Literacy and Numeracy Assessment (WALNA) programs. The

paper provides a complete description of these assessment systems, including the social and

educational contexts in which they operate, and evaluates the contribution that each makes

to the education system. In addition detailed bibliographic references and possible sources of

information about the systems and tests are provided.

The paper draws on information from websites, research papers, public documents, discussions,

and previous research undertaken in preparation for the publication of ACERs Policy makers

guide to systemwide assessment programs (Forster, 2001) and Principals guide to the use of

systemwide assessment data (in final draft form).


14/87


IN TRODUCTION

The management of an education system is a complex and expensive operation, and effective

management demands dependable information on educational outputs. Systemwide (state

or national) assessment programs provide this information through the systematic and regu-

lar measurement of student learning. These programs are designed to investigate and moni-

tor the health of the system and to improve student learning by providing information to

stakeholders at different levels of the system:

policy makers with information to monitor standards over time, to monitor the impact

of particular programs, and to make decisions about resource allocation; and

schools (principals, councils) and teachers with information about whole school, class

and individual pupil performance that they can use to make decisions about resourceallocation and to support learning in the classroom.

Full-cohort (census) programs also provide:

parents with information about their childs progress to assist them to make decisions

about the best ways to support their child; and

students with information about their progress to assist them to take an active role in

their own learning.

Of increasing interest internationally is the role that systemwide assessment programs can

play as agents of reform and accountabilityto provide both direction and motivation to

schools, teachers, parents and students. In some countries, student achievement data collectedthrough state assessment programs are used as a measure of schools contributions to student

learning. In some, schools are rewarded or punished on the basis of students results. In

others, public comparisons of schools achievements in the form of league tables are made.

The theoretical appeal of this intention is easy to understand. The influence of assessment

programs in motivating educational change is well known. When results matter, (when the

stakes are high) assessment programs influence what is taught by reinforcing aspects of the

curriculum; and they engage schools, teachers, and the community in educational debate.

The political appeal is understandable also:

Compared with reforms such as targeting instructional time, professional development forteachers, and reducing class sizes, state assessment programs are relatively inexpensive. The

assessments also can be mandated (unlike changes in classroom practice), can be rapidly

implemented, and have a public visibility (Linn, 1995).

However, using assessment programs to change the education system more generally brings

some challenges. First, there can be unintended negative consequences of assessment driven


15/87


reform. For example, the impact of the minimum competency testing movement in the United

States is well known (Marion & Sheinker,1999). Second, using assessment programs as both

the instrument of change and as the measure of change makes procedures open to scrutiny fromtwo perspectives and takes them into the arena of public debate both about their effectiveness

and also their reliability and validity as measures (Whetton et al, 2000, p.5).

This paper explores two systemwide assessment systems: Western Australias (WA) state

Monitoring in Education (MSE), and Literacy and Numeracy Assessment (WALNA)

programs; and Englands National Curriculum Assessment. Despite their different political,

social and educational contexts, there are many similarities in the Western Australian and the

English assessment systems, in particular the intention to address a number of purposes for

assessment and to provide information of different kinds to a range of stakeholders. Both

systems also express a key intention to impact directly on the work of the classroom teacher.

The reform and accountability demands addressed directly by the assessment systems aredifferent, however with the stakes significantly higher in England than in Western Australia.


16/87


17/87

CASE STUDY 1

W ESTERN AUSTRALIA(W ALNA & MSE)


18/87


19/87


CONTEXTUALFEATURES1

Geographic, demographic, economic, politicaland religious context of Australian schooling

Australia is a large island continent located close to south east Asia and the South Pacific. The

land mass totals 7.7 million square kilometres and much of the interior is extremely arid. The

vast majority of people live in the major cities on the coasts.

The total population in 1996 was approximately 18,311,000, with an annual population

growth rate of 1.2 per cent. Around 80 per cent of the population live on narrow coastal

strips covering less than four per cent of the total land area. As a result, Australia has a high

level of urbanisation.Aboriginal people and Torres Strait Islander people, the indigenous

population, account for around 1.6 per cent of the total population and are spread throughout

urban and rural areas. There is considerable diversity of cultures within both Aboriginal and

Torres Strait Islander communities and particular efforts are made to provide for their special

educational needs and to assist in cultural maintenance.

In recent years, Australia has had one of the highest levels of immigration of any developed

country. By 1996, for example, approximately 24 per cent of the population had been born

overseas. Since the early 1970s, as the population has grown, the proportion of the population

of school age has decreased.The International Labour Organisation unemployment rate for

Australia (persons aged 15 years and over) for 1999 was 7.2 per cent (7.3 per cent for men;

7.1 per cent for women). Australia is secular, having a wide variety of religious affiliations,

including Christians (mainly Catholics and Anglicans), Buddhists, Muslims, Hindus and

Jews. The Catholic church is the largest provider of school education, other than the State.

The Australian political system includes an elected federal government and elected governments

of the six States (New South Wales, Victoria, Queensland, Western Australia, South Austra-

lia, Tasmania) and two self-governing Territories (Northern Territory and the Australian Ca-

pital Territory). Voting is compulsory.

1 This section of the paper draws directly from two texts: the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and Youth Affairs

(MCEETYA) website http://www.curriculum.edu.au and the Archive of the International Review of Curriculum and Assessment Frameworks Project (INCA) http://www.inca.org.uk/


20/87


Responsibil ity for schooling in Australia

The Constitution of Australia allocates responsibility for school education to State and Territory

governments, all of whom provide and manage government schools as well as supporting

non-government schools. Government schools operate under the direct responsibility of the

relevant State or Territory Minister, while non-government schools are established and operate

under conditions determined by State or Territory government registration authorities. Many

non-government schools have some religious affiliation, most commonly with the Catholic

Church; 19.9 per cent of all students and 63.6 per cent of non-government students were

enrolled in Catholic schools in 2001.

The Ministers of Education in the States and Territories have responsibility for primary and

secondary schooling, student enrolment policies, curriculum content, course accreditation

and certification procedures, and student assessment. Each of the State/Territory Governments

is responsible for determining teacher qualifications, establishing and paying teachers salaries,

recruiting and appointing teachers in government schools and supplying buildings, equipment

and materials.

In cooperation with the States and Territories, the Federal government through the

Commonwealth Department of Education, Science and Training (DEST) plays a significant

role in addressing equity and quality issues through general recurrent, capital and specific

purpose programs. In addition, it has specific responsibilities for Aboriginal people and

migrants and is responsible for international relations in education.

The key national body is the Ministerial Council for Education, Employment, Training and

Youth Affairs (MCEETYA) which comprises the Commonwealth, State, Territory and New Zealand

Ministers with responsibility for the portfolios of education, employment, training and youth

affairs, with Papua New Guinea having observer status. MCEETYA meets on a biannual basis to

discuss issues of mutual interest and to coordinate collaborative policies and approaches and generally

to facilitate the exchange of information on education in Australia and overseas.MCEETYA is

supported by taskforces that are convened as needed for particular tasks. These taskforces have

prescribed timeframes and reporting arrangements and are reviewed annually.

MCEETYA has established a range of consultative mechanisms at Federal and State levels to

obtain the views of the non-government school sector and, whenever possible, extends membership

of its working parties and committees to the major national bodies representing the non-government

employing authorities: the National Catholic Education Commission and the National Council

of Independent Schools Associations. Regular consultation also takes place with the principal

national bodies representing parents teachers and the business sector. Two national research and

development companies also facilitate cooperative initiatives in schooling: the Australian Council

for Educational Research (ACER), an independent research body, and the Curriculum Corporation

which is wholly funded by Commonwealth and State Ministers of Education.


21/87


Provision of schooling in Australia

Schooling in Australia is compulsory for children aged 6 to 15 (16 in the state of Tasmania)

years of age. However, the majority of children start school before they are six and remain at

school beyond the age of 15. In most States, children start full-time schooling at around the

age of five, when they enrol in a kindergarten or preparatory year. Commonly, the majority

of these students will have already had some part-time school or preschool experience. After

the preparatory year, primary education lasts for either six or seven years, and secondary

schooling for five or six years, depending on the State. Students usually commence secondary

school at about age 12.

In 2001, there were approximately 1.9 million primary school students and 1.36 million

secondary school students in Australian schools. Approximately 72.4 per cent of primary

students were enrolled in government schools, and 63.7 per cent of secondary students in

government schools. Most government schools are coeducational, but a significant number

of non-government schools are single-sex schools.

Some features of the structure of Australian schooling in 2001 were as follows:

There were 9596 schools in Australia.

There were 3.268 million full-time students in Australian schools.

The proportion of students enrolled in non-government schools continued to rise in

all States and Territories. In 2001, 31.2 per cent of students were enrolled in non-

government schools compared to 30.8 per cent in 2000.

There were 115,465 Indigenous students enrolled in Australian schools in 2001. This

represented 3.53 per cent of the total school population and was a significant increase

from 2.13 per cent in 1990.

In 2001, 49.9 per cent of Australias 15-19-year-olds were full-time students at school.

The largest year cohort of primary school students was in Year 1 (138,576 students)

and the largest year cohort of secondary students was in Year 8 (131,507 students).

There were 221,927 teaching staff (in full-time equivalents) employed in Australian

schools, which resulted in an average of 14.8 students per teacher in government schools,

15.9 per teacher in Catholic schools and 12.8 per teacher in independent schools.

Females comprised 78.7 per cent of the teaching staff in primary schools and 54.8 per cent

in secondary schools, compared with 78.3 per cent and 54.4 per cent respectively in 2000.


22/87


Funding for schooling in Australia

Within each State and Territory, ministers, departments, statutory authorities and individual

schools (particularly in the case of non-government schools) variously determine policies and

practices and provide resources for infrastructure in areas such as curriculum, course

accreditation, student assessment and certification, resource allocation and utilisation, and

teacher employment and professional development.

Through DEST, the Commonwealth provides funding to both government and non-government

school authorities to support agreed priorities and strategies. The overall result is that government

schools receive the majority of their government funding from State and Territory governments

and less from the Commonwealth, while non-government schools receive the majority of their

government funding from the Commonwealth, and less from the relevant State or Territory.

Funding is provided also from fees, charges and levies paid by students and their parents or

caregivers and private donations and income, including that derived from fundraising.

Figure 1 Public funding for school educationSource: 2000 National Report on Schooling in Australia Amounts in AUD


23/87


24/87


In the financial year 1999-2000 the total per capita expenditure on government schools was

$AUD 6 358 ($5,687 for primary students; $7,416 for secondary students.)

Non-government schools derive their income from fees and donations, and State and

Commonwealth government grants. Per capita expenditure on non-government schools in

the financial year 1999-2000 is shown below in Table 1.

Source: Derived from data provided by Commonwealth DETYA , 2000 National Report on Schooling in Australia

Of the $4.78 billion of grants for schools spent under the States Grants (Primary and Secondary

Education Assistance) Act 1996 and the Indigenous Education (Supplementary Assistance) Act

1989, the government sector received $1.77 billion and the non-government sector $3 billion.

A further $10 million was spent on joint national programs benefiting both sectors.

Table 2 below shows how funding from the States Grants (Primary and Secondary Education

Assistance) Act and the Indigenous Education Supplementary Assistance Act was distributed

between government and non-government authorities, by State.


25/87


26/87


Note: Expenditure with respect to a certain program year may continue in relation to that year in future years. Source: 2000National Report on Schooling in Australia, Commonwealth DETYA (2000 program year cash expenditure as at 30 June 2000)


27/87


Common and agreed national goals for schooling in Australia

In 1989 the Education ministers from the States and Territories and the national

(Commonwealth) Government published a set of Common and Agreed (National) Goals

for Schooling in Australia. Known as the Hobart Declaration, these goals provided a structure

for cooperation between schools, States, Territories and the Commonwealth, named eight

Key Learning Areas (the arts; English; health and physical education (HPE); languages other

than English (LOTE); mathematics; science; studies of society and environment (SOSE);

and technology) and described ten national goals for schooling.

In 1999 the State, Territory and Commonwealth Ministers for Education endorsed a revised

set of National Goals, the Adelaide Declaration2 See and affirmed their commitment to

national reporting on comparable educational outcomes in literacy; numeracy; student

participation, retention and completion; vocational education and training in schools; science;

and information technology. (The Ministers also noted the need to develop performance

indicators for civics and citizenship education, and enterprise education.)

The literacy and numeracy goal is described as follow: students should have attained the

skills of numeracy and English literacy; such that, every student should be numerate, able to

read, write, spell and communicate at an appropriate level. To achieve this goal Ministers

agreed to support:

assessment of all students by their teachers as early as possible in the first years of

schooling;

early intervention strategies for those students identified as having difficulty;

the development of agreed benchmarks for years 3, 5, 7 and 9, against which all

student achievement in these years could be measured;

the measurement of students progress against these benchmarks using rigorous State-

based assessment procedures;

progress towards national reporting on student achievement against the benchmarks,

with reporting commencing in 1999 within the framework of the annual National

Report on Schooling in Australia; and

professional development for teachers to support the key elements of the plan.

By 2000 performance standards for literacy and numeracy at years 3, 5 and 7 were developed

and the Ministers agreed that benchmarking for years 9 or 10 be postponed pending findings

from the OECD Program for International Student Assessment (PISA) project. The

benchmarks represent the minimum acceptable standard of numeracy without which a student

will have difficulty making sufficient progress at school. They were developed with reference

to levels of achievement as demonstrated in national surveys and State assessment programs

through an extensive consultation process with stakeholders and with experts in the areas of

numeracy and educational measurement and trial testing in classrooms in all States and

Territories. Existing State-based programs are used for assessment against the national

2 See http://www.detya.gov.au/schools/adelaide/index.htm


28/87


benchmarks and a nationally agreed procedure to equate State and Territory tests ensures that

reporting of student achievement data is comparable by States and Territories.

Professional development for teachers is an integral part of the National Literacy and Numeracy

Plan, as it is recognised that the classroom teacher is the major determinant of the literacy

learning of students. During 2000, there was considerable professional development related to

the assessment and intervention programs described above. In particular, teachers were assisted

to interpret data from assessment programs and devise programs based on their findings.

Many teachers were involved in professional development programs associated with the

introduction of particular intervention programs. As many schools had instituted literacy

teams, a number of the professional development initiatives were directed towards the team

leaders. A train-the-trainer approach was common in such programs.

Another professional development strategy was the establishment of literacy networks among

teachers. The increasing use of modern communication technology contributed significantly

to the extent and effectiveness of such strategies. Many authorities also provided opportunities

for teachers to engage in postgraduate, accredited study in literacy teaching and learning.

The provision of schooling in Western Australia

In Western Australia under the provisions of the School Education Act 1999, every person

living permanently in Western Australia and aged between six and 15 years must be enrolledat a government school or a non-government school, or receive approved home tuition.

Government schools provide schooling during the period of compulsory attendance to any

person who wishes to enrol. In addition, access to optional schooling is available for four-

year-old and five-year-old children and persons aged over 15.

The Education Department is the largest employer in Western Australia employing the

equivalent of 25 000 fulltime staff and accounting for a quarter of the State budget. The

Department is responsible for the education of 258 000 students at 771 schools across the

state from metropolitan Perth (the capital) to rural and remote centres. Over 1300 students

receive approved home education.

In the Perth metropolitan area and major rural and remote centres, government primary

schools provide sessional kindergarten, optional full-time pre-primary and full-time primary

education (Kindergarten to Year 7 or Primary to Year 7), and high, middle and senior high

schools and senior campuses full-time secondary (7-12, 8-10, 8-12 or 11-12) education. A

small number of early childhood education centres and junior primary schools offer Kinder-

garten to Year 2, or Kindergarten to Year 3 education respectively.


29/87


Outside the major rural and remote centres, smaller primary schools, district high schools

and remote community schools are provided. Kindergarten to Year 10 (and, in some cases,

Kindergarten to Year 12) schooling is delivered by district high schools supported bytelematics learning systems which integrate the use of personal computers, interactive

television, the Internet, fax and telephone. The remote community schools deliver Kinder-

garten to Year 10 education with the support of communications technology to Aboriginal

children, for many of whom English is a second language or second dialect. A number of

campuses deliver specialised services to male youths experiencing social or behavioural

difficulties. Education support schools and centres cater for students with disabilities for

whom specialised school environments are most appropriate. Students with mild to severe

intellectual disabilities attending rural or remote schools are supported by the provision of

equipment and learning materials and, for their teachers, professional development and

assistance with the design of individualised education programs. Delivery of schooling to

geographically-isolated students involves the Schools of Isolated and Distance Education(SIDE); small Kindergarten to Year 7 and Kindergarten to Year 10 schools in remote areas;

and a combination of face-to-face, interactive and traditional distance education teaching

and learning.

Student intake

Government schools in Western Australia cannot exclude students from enrolment within

their intake area. The School Education Act 1999 1/Jan/2001 states under section 78

Enrolment of children of compulsory school age at local-intake school.

A child of compulsory school age is entitled to be enrolled at a local-intake school if his

or her usual place of residence is in the intake area for that school; and an appropriate

educational programme is available for the child at that school.

A child of compulsory school age whose usual place of residence is not in the intake

area for a local-intake school is entitled to be enrolled at that school if there is available

for the child at that school an appropriate educational programme and classroom

accommodation, and the enrolment would conform with any other criteria prescribed

by the regulations for the purposes of the subsection of the Act.

Schools must give priority to students who are within their intake area. If there are spaces left,

they can take students from outside the school intake boundaries. Schools can also apply to

have special programs recognised and funded by the State (for example, a program for

gifted and talented students). Schools delivering these programs can set aside a number of

places that can be filled from within or outside of the school intake boundaries.


30/87


31/87


SYSTEMW IDE ASSESSMENT3

Different programs different purposes

The Western Australian Department of Education conducts two systemwide assessment

programs:

the Western Australian Literacy and Numeracy Assessment (WALNA) program; and

the Monitoring Standards in Education (MSE) program.

The WALNA is a full cohort (census) program administered annually to students in years 3,

5 and 7. The assessments address explicit curriculum learning outcomes in numeracy, reading,

spelling and writing.

The purpose of WALNA is to provide:

parents and caregivers with information about the performance of their children in

relation to nationally agreed benchmarks in numeracy, reading, spelling and writing,

and in relation to that of other Year 3, 5 or 7 students across Western Australia;

teachers with information, not only on the performance of individual students, but also on

that of class groups, and on the level of their students literacy and numeracy development

in relation to the performance of other Year 3, 5 or 7 students. This information can be used

in conjunction with their own class records in future teaching and learning programs;

schools with a report showing summary information of their overall school performance.

This data is useful additional information in the schools identification of students needs.

Through the, schools are also provided with assistance in interpreting their results for

school improvement and accountability purposes.

The MSE program is administered annually to a representative sample of students in

each of Years 3, 7 and 10 are tested in one or two of the eight learning areas. Aspects of

learning areas that are assessed through the WALNA testing program are not assessed

through the Random Sample assessment program. The assessments are part of the

Departments public accountability procedures and the results provide a basis for initiating

developments that will further improve the outcomes of education in government schools.

The purpose of the MSE program is to: provide information to stakeholders about the progress students make in their learning;

assist the Department and schools to monitor and report student performance4; and

assist the Department and schools to plan for improvement in student performance.

3 This section of the paper draws information primarily from three sources the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm information provided on the Western Australian Department of Education websites for the two system-level assessment programshttp://www.eddept.wa.edu.au/

walna/ and http://www.eddept.wa.edu.au/mse/ A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western

Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003).4 The Department also produces School Assessment Materials for the key learning areas. These materials assist schools to fulfill their accountability requirements

as well as provide diagnostic information to teachers on the strengths and weaknesses in individual student learning. This information can help teachers plan forstudent progress through the curriculum. Some of these documents are available in PDF on the Department website: http://www.eddept.wa.edu.au/mse/materials.html


32/87


WALNA

The WALNA full cohort (census) assessment program commenced in 1998 with the assessment

of Year 3 students in literacy (defined as reading, writing and spelling) and expanded in 1999

to include Year 5 students and the assessment of numeracy. In 2001 the program was further

extended to include Year 7 students. In 2002 almost all students in Government schools in

Year 3, Year 5 and Year 7 (approximately 61,500 students) undertook the WALNA tests in

literacy and numeracy5. In addition, around 22 000 students from Catholic and independent

schools now also participate in the testing program. The annual budget for WALNA is $AUD

1.3 million. Around 40% of the budget is spent in the hand-marking operation of the writing,

the open ended reading and numeracy questions and the hand marking of spelling.

The WALNA program was developed in response to the National Literacy and Numeracy

Plan. The tests, which address both outcomes in the WA Outcome and Standards Framework

(1998)6 and the National Benchmarks, are administered annually in August (the school year

runs from February to December).

The timeline for the 2002 WALNA program for all WA schools involved government and

private is provided in Table 3.

(Source: Murdoch University and Estill & Associates, 2003).

5 Parents/caregivers can exercise their right to withdraw their child or children from the testing program. This can be carried out through the completion of anexemption form, available from schools prior to testing. Exemptions are also granted, at the school level, on the grounds of disability or impairment. Decisions toexempt students are made by the principal, after discussion with the classroom teacher and school-based specialists, and with the signed agreement of the parents/caregivers. Some English as Second Language (ESL) students in mainstream classes, as well as those in Intensive Language Centres, may also be granted exemptions.In 2002, less than 1% of exemptions were lodged on the grounds of temporary or permanent disability, and 0.2% of children were withdrawn from the testingprogram by their parents.

6 All Department of Education schools are required to plan and structure their learning and teaching programs using this Curriculum Framework.


33/87


Extensive consultation with classroom teachers is a feature of the development of the assessment

materials and the tests are written to cater for the diverse range of students in Western Australian

schools to ensure that there is no systematic bias associated with such factors as gender,culture or geographic location. The numeracy and reading components of the test take between

30 and 60 minutes, and the writing component takes 65 minutes (though children only

spend up to 35 minutes writing). The spelling component takes about 20 minutes. In most

cases, the regular teacher conducts the assessment tasks as part of the normal school program.

Samples of the WALNA literacy and numeracy tests can be found on the WALNA website

at:http://www.eddept.wa.edu.au/walna/pdfs/sample%20magazine.pdf and

http://www.eddept.wa.edu.au/walna/pdfs/sample%20test.pdf

A team of experienced teachers completes all marking and to ensure reliability, markers undergo

extensive training. To further enhance reliability, all marking is conducted at a central location.Markers work in groups under the direction of a team leader, and reliability scripts are administered

at each session. Each group leader monitors the marking, and provides advice as necessary.

The tests are designed to:

provide an indication of how students are performing in relation to the National

Benchmarks,

measure student performance in relation to the Outcomes and Standards Framework and

provide comparative information on student performance in relation to State perfor-

mance.

Data are analysed using the Rasch Measurement Model. This model allows the abilities ofstudents and the difficulty of the assessment tasks they have completed in an area of learning

to be brought onto a single scale. Each scale provides two pieces of information: a continuum

of skills and understandings in the area of learning, based on the assessment items, arranged

in order of increasing difficulty, and the distribution of students in relation to the tasks.

This method of scaling allows inferences to be made about the probability of the success of

any student on all individual items. Each item can also be mapped to the levels of achievement

described in the WA standards framework (the Student Outcome Statements) making it

possible to identify areas of the sale that relate to each level. In turn, this makes it possible to

draw inferences about the level of achievement of groups of students. (Appendix 2 provides

an explanation of this process.)


34/87


35/87


MSE

The MSE program began in 1990. Each year a random sample of approximately ten percent of

students in each of years 3, 7 and 10 in government schools participate in the program8. Schools

are notified of their involvement in the weeks prior to the assessments. Schools decide the days

and times to administer the assessments during the specified week (week 8 of term 3).

Assessments occur in all key learning areas, as well as in the social outcomes of schooling, and

address the significant learning outcomes (knowledge, understandings, skills and processes)

that students are expected to develop as they move through the compulsory years of schooling.

These learning outcomes are described in the Outcome and Standards Framework. In most

cases, all sequenced strands of a learning area are assessed. For example, in the 1997 Random

Sample assessment of Science, Year 3, 7 and 10 students were assessed in Life and Living,

Energy and Change, Natural and Processed Materials, Earth and Beyond and Investigating

Scientifically. Some learning areas, such as Mathematics and English, are assessed more regularly

than others so that trends in performance to be closely monitored. For example, aspects of

the English learning area have been tested in 1992, 1995, 1997, 1999 and 2001.

The annual budget for the Random Sample program is almost $AUD 1 million. This budget

covers the development or administration of three learning areas (two being assessed and at

least one under development), and the production and on-going provision of school release

to schools.

The MSE program uses rigorous procedures to train markers to make consistent judgements

of student performances. Markers are trained to use standardised marking guides, in

conjunction with student work samples, to judge student work.

As with the WALNA program, MSE data are analysed using the Rasch Measurement Model

and single vertical scales from Year 3 through to Year 10 provide the basis for reporting.

Student performance is reported using the eight levels of achievement described in the Student

Outcome Statements. Aggregated information on the performance of students in Years 3, 7

and 10 is reported. Within these year groups, information on boys, girls, Aboriginal and

Torres Strait Islander (ATSI) students and Language Backgrounds Other Than English

(LBOTE) students is also reported. For each of these groups, the mean performance and the

range of performance across the state are described in terms of the levels of the Outcomes

Framework.

MSE produces public reports for the Western Australian government education system . The

reports present information descriptively, in table form and through graphs. In most reports,

samples of students work from the assessments are included, to exemplify level achievement.

Examples of MSE reports are included in Appendix 2.

8 Year 3 marks the end of the early childhood phase of schooling, Year 7 the end of the middle childhood phase and Year 10 the end of the compulsory years ofschooling, and the end of the early adolescent phase.


36/87


MSE releases a selection of the random samples materials free of charge to WA Government

schools. The intention is that the materials will provide models of good assessment tasks, assist

teachers to set their own assessment criteria, and develop teachers capacity to make judgementsabout students work using the Student Outcome Statements. Depending on the nature of the

learning area, the released materials consist of stimulus materials (print/audio/video), student

test booklets, teacher administration instructions, marking keys with moderation instructions,

guidelines for analysing results, and performance profiles or progress maps.

At the classroom level, teachers use the results to:

diagnose individual students strengths and weaknesses in terms of actual skills and

understandings that the student does or does not demonstrate on the assessment;

determine the strengths and weaknesses of the class as a whole;

determine the strengths and weaknesses in the teaching of aspects of the curriculum;

obtain an indication of level achievement in the substrands described in the CurriculumFramework

At the whole school level, school administrators use the results to:

infer level achievement in the substrands described in the Curriculum Framework;

make comparisons with the state mean;

make comparisons with the state range;

track year cohorts longitudinally;

track groups of students over time;

report to the school community;

inform school development plans; and

provide continuity for students between schools (primary/primary and primary/high)

School administrators and teachers are reminded that the MSE assessments do not cover the

entire curriculum and that the results provide only one source of information. They are also

reminded that the administration instructions need to be followed carefully in order to maximise

standardisation and that the materials cannot be used to compare teachers and schools.


37/87


EDUCATIONALIMPACT: POSITIVES9

System level monitoring

Since 1990, MSE data have facilitated the monitoring of student achievement at a system level.

Now, the combined data from WALNA and MSE allow for the monitoring against an expected

standard. Data from WALNA have also provided a check on the random sample findings.

The WA Department reports two major findings in relation to the standards:

1. Some minor fluctuations are evident in the data from year to year and in the different

strands but the overall trend emerging from the data is one of slight improvement.

2. The percentages of students achieving the standards in reading, writing and spelling

vary from 1998 to 2002. Variations of this sort appear in all large scale testing programs,

and the apparent improvements or declines should be interpreted cautiously. The

observed trend of slight improvement over the years of testing is encouraging.

The Department also reports the following findings in relation to previous MSE findings:

1. Girls continue to do better than boys in literacy at all years. The difference in perfor-

mance between boys and girls increases from Year 3 to Year 5 and from Year 5 to Year 7

and is most evident in writing.

2. Girls and boys numeracy results are comparable.

3. Aboriginal students literacy performance has shown a trend of improvement since

1998. While Aboriginal students performance is still below that of the total population,

there are indications that their rate of improvement in reading and writing is slightly

greater than that of the general population.

9 This section of the paper draws information and direct text primarily from the following sources the Western Australian Education Department Annual Report 2001-2202 http://www.eddept.wa.edu.au/AnnualReport/index.htm information provided on the Western Australian Department of Education website http://www.eddept.wa.edu.au/ A paper evaluating the 2002 system level assessment program prepared for School and System Performance Directorate, Department of Education, Western

Australia by Murdoch University and Estill and Associates (Murdoch University, Estill and Associates, 2003). information provided electronically by Helen Wildy, Associate Professor of Educational Leadership, School of Education, Murdoch University an article written by Helen Wildy reviewing the work of the Data Club (Wildy, 2003) the Western Australian School Accountability Framework, 2002 http://www.eddept.wa.edu.au/regframe/index.cfm discussions with Jocelyn Cook, Manager Educational Measurement, Standards and Accountability, Education Department Western Australia


38/87


The following three tables show the percentage of students reaching the benchmark standards

at Year 3, Year 5 and Year 7.


39/87


School level: use of results and acceptance of the program

At the school level, evidence of the impact of WALNA data is confirmed by the work of the Data

Club, by an independent review, and by the requirements of the school accountability framework

The Data Club

The Data Club assists schools to use their WALNA information through an annual half-day

workshop program that the principal and one other person with curriculum responsibilityfrom each participating school attend. The Club began in 2000 as a pilot project with two

hundred volunteer schools and grew over the following three years until in 2003, 508 schools

(of a total of 612 government schools) were participating10.

In advance of the workshops WALNA data are re-analysed to provide participants with box

and whisker representations of school level performance on each of the four WALNA strands:

Reading, Writing, Spelling and Numeracy. Comparisons are made over time, as the data are

accumulated, and regression analyses are used to show expected growth or value added in

school performance in a given strand over time. Standardised residuals are used to indicate

the relative amount of change.

Workshop participants are introduced first to the box and whisker method of representing

data as well as to regression analysis, standardized residuals, confidence limits and examples

of valid and invalid inference. In the second part of the workshop participants are given

envelopes containing their processed data and are supported as they work through tasks

designed to explore their data.

10 The project began as a partnership between Edith Cowan University and the Western Australian Department of Education and Training, supported by theCommonwealth Department of Education, Science and Training to develop schools capacity to make performance judgements in relation to the NationalLiteracy and Numeracy Plan. From 2003 the project was retained, managed and funded entirely within the WA Department.


40/87


41/87


Independent review of WALNA

In 1999, in response to criticism of the WALNA test, an evaluation survey was conducted by

the Education Department. The survey report indicated that teachers criticisms of the length

and difficulty of the test were not supported by parents who regarded the test information as

an appropriate mechanism to help raise literacy standards. Four years later there was anecdotal

evidence that the WALNA program had become more accepted by teachers as the longitudinal

benefits to parents, teachers and schools become apparent. In light of this anecdotal evidence,

the Department of Education commissioned an independent evaluation study of the 2002

WALNA (Murdoch University & Estill and Associates, 2003).

The study addressed two main questions: What are the attitudes of Year 3, 5 & 7 teachers

and parents to WALNA five years after its commencement, and how have they changed since1999? How is the information from WALNA used by schools?

To answer the first question, the evaluation re-administered a revised form of the teacher and

parent questionnaires from the 1999 WALA evaluation to allow comparison with the data

from that year. The second question was addressed by developing a new questionnaire for

principals, and by supplementing the teacher questionnaire with additional items.

In all, 458 principals, 1458 teachers and 1414 parents responded to the survey. Principals responses

were analysed as a total group and according to school type and size. Teachers responses were

analysed as a total group and according to school type and size, year level taught, and the number

of years teaching experience. Parents responses were analysed as a total group and according toschool type and year level of their child.

Overall, principals reported that they made considerable use of the WALNA information, found

it useful for teachers and parents, and had adopted it into their schools performance measurement

and planning activities. However, principals in Remote Community Schools and Education

Support Schools did not have the same positive perceptions of the WALNA information.

Generally, teachers had positive perceptions about the WALNA program. The majority

understood the purposes of the program, felt they received sufficient information and support

to administer the tests and analyse the results, and were not overly concerned that the tests

were too stressful or too long for most students. The exception to this is teachers in RemoteCommunity Schools and Education Support Centres, who had consistently less positive

perceptions. Compared to teachers perceptions in the 1999 survey, the current survey shows

a general improvement in teacher perceptions.


42/87


The perceptions of parents and caregivers of children who undertook the 2003 WALNA tests

were generally positive and had improved since the 1999 survey in nearly all areas.

Nearly all the parents agreed that the purpose of the test was to provide them with information

about the achievements of their children (90%) and to assist schools to provide better literacy

and numeracy teaching for all students (95%). The large majority also agreed that it provided

information about student weaknesses (81%). The majority (60%) thought that the test was

used to compare different educational systems; however, they disagreed (57%) that the purpose

of the test was to judge the quality of the teachers. They were evenly divided over whether a

purpose of the test was to judge the quality of schools. Nearly all parents (96%) agreed that the

report gave them information about the achievement of their child in relation to the benchmark

and that the report was clear and understandable (89%). About half the respondents (52%)

agreed that the school had provided additional information which helped them understand the

report. The large majority agreed that the report confirmed what their childs regular schoolreports were saying (74%) but thought that the report gave them additional information (73%).

The large majority of parents/caregivers did not believe that their children were unusually stressed

in the lead-up to the test (87%) or when completing the test (90%). Neither did they think that

the test was too difficult (80%) or too long (83%) for them. Rather they believed that their

child enjoyed participating in the test (74%) and were pleased with their test results (82%).

Only a few (12%) thought that their child was upset with their result. While they were divided

as to whether the test results had encouraged their child to work harder (54% agreed with this

statement), almost all (95%) disagreed that it had had a discouraging effect on them.

Both in 1999 and 2003 parents/caregivers and teachers generally agreed that the purpose ofthe test was to provide parents with information about their children, and highlight student

weaknesses.


43/87


Role of the test in assisting schools toprovide better literacy teaching for all students

The 1999 report pointed out that nearly all parents/caregivers (92%) believed that the purpose

of the test is to help provide better literacy teaching and that the information from the test

program will help to raise literacy standards across the State. In contrast, a small majority of

teachers (58%) felt the results would be useful in this way. This difference was reduced

dramatically in 2003: as shown below, 73% of teachers valued the test information more as a

source of diagnostic information about their students, and used it to identify students at risk

and who required extension.

QUESTION: ASSISTSCHOOLSTOPROVIDEBETTERLITERACYTEACHINGFORALLSTUDENTS


44/87


The quality of the information provided

In 1999, the majority of parents believed that the reports confirmed regular school reports

but nonetheless supplied them with additional information. On the contrary, teachers surveyed

in 1999 agreed that the results were in line with school-based tests but only a minority of

teachers (37%) agreed that they provided valuable diagnostic information about their students.

This pattern has changed over the intervening period to the point where the majority of

teachers (62%) now value the test information, using it to identify students at risk or requiring

extension. A greater percentage of parents also believe the tests provide useful additional

information (62% to 73%).

QUESTION: THEREPORTGAVEMEADDITIONALINFORMATIONNO TAVAILABLEINTHEREGULARSCHOOLREPORT

QUESTION: THETESTRESULTSPROVIDEDMEWITHVALUABLEDIAGNOSTICINFORMATIONABOUTMYSTUDENTS


45/87


Reaction of students to the test

In 1999, considerably more teachers than parents/caregivers were of the opinion that students

were stressed before and during the test. In 2002, the proportion of teachers with this opinion

was reduced, though still considerably greater than that of parents/caregivers.

QUESTION: INTHELEAD-UPTOTHETESTMYCHILDWASUNUSUALLYSTRESSEDABOUTPARTICIPATIN GINTHETEST

QUESTION: INTHELEAD-UPTOTHETESTSOMEOFMYSTUDENTSWEREUNUSUALLYSTRESSEDABOUTPARTICIPATIN GINTHETEST


46/87


Confidentiality of results

In 1999, a large majority of teachers (70%) were concerned about the confidentiality of results,

a view not shared by many parents (23%). This concern was shown to be considerably reduced

in 2003, dramatically for teachers (decline to 38%) but also for parents (decline to 14%).

Overall, in the areas where there were differences between teachers and parents in 1999, the

differences have been dramatically reduced and both groups have more positive perceptions.

Compared to 1998, teachers in particular, have become more accepting of WALNA. Parents

and caregivers were generally positive and had more positive views in all areas than werereported in the 1999 survey. With familiarity over the years and the help of the Data Club,

the WALNA data is accepted as a tool which can be used diversely, for example, by individual

teachers for judging students or by the schools for assisting with curriculum planning.

Feedback from principals, teachers and parents from Remote Community Schools and

Education Support Centres indicates they view the WALNA tests are not as appropriate for

QUESTION: I AMCONCERNEDTHATMYCHILDSRESULTSWILLNO TBEKEPTCONFIDENTIAL

QUESTION: I HAVECONCERNSABOUTTHECONFIDENTIALITYOFTESTRESULTS


47/87


students in these schools as they are for students in mainstream schools. This is because they

see the tests as more stressful for the students and more disruptive to their learning.

Nevertheless, there is some support for the view that despite this, WALNA can still play apositive role in measuring progress in improving student outcomes over the years.

The 2002 survey results demonstrate that the WALNA is now accepted by the three key stakeholder

groups as part of the normal cycle of reporting on student numeracy and literacy outcomes.

The School Accountability Framework

The School Accountability Framework sets out the requirements for all schools in relation to

accountability and review.

All schools are required to produce, in partnership with their school community, a school

plan setting out their objectives, priorities, major initiatives and evaluation measures. They

are required also to assess their performance in terms of standards of student achievement

and the effectiveness of the school; and make available to the public and to the District

Director a School Report that describes the schools performance. Central to this activity is

the use of achievement data collected through the systemwide assessment programs. School

staff are accountable to the principal and school principals are accountable to the District

Director for the performance of the school.

The Framework is built on five fundamental commitments: high standards; effectiveness;

quality teaching; quality information for parents about the standards being achieved by their

schools; information for the wider community about the standards being achieved across the

government schools system. The accountability policy is based on five key concepts.

1. Accountability involves responding to information about performance.

That is, accepting responsibility for the achievement of outcomes and taking appropriate

action to improve performance. Key accountability questions for schools include:

What are we trying to achieve?

How well are we achieving it?

How can we improve?


48/87


2. Accountability is linked to outcomes.

The core purpose of schools is to enable the success of each and every student. Schools

therefore must be able to demonstrate to themselves and to the communities they servethat what they do makes a difference to student learning. Since schools do not have

control over all the variables that impact upon student achievement, they cannot be

held accountable for it in an absolute sense. However, because the school can make a

difference to the level of achievement of the students, schools are accountable for

maximising the difference they make in terms of student outcomes.

3. Demonstrating accountability is a professional responsibility of school staff.

Schools need to take responsibility for demonstrating their effectiveness to others. That

is, accountability is located within the professional responsibility of the school staff, by

creating an expectation that staff demonstrate the quality of their work as part of the

expression of their professionalism.

4. Rigorous self-assessment is at the heart of effective accountability.

While schools and systems vary in the approach they take to school accountability, the

success of any approach ultimately depends on how well the school reviews its own per-

formance. Effective internal review involves the school gathering information about the

levels of student achievement, analysing and judging the adequacy of those levels,

identifying strengths and weaknesses in its overall performance, and assessing which aspects

of the schools operation should be changed to generate improved performance.

5. Schools are accountable to their communities and to the Director-General.

The Framework acknowledges the two-way accountability of schools. Schools listenand respond to their local communitys needs and provide information about the schools

performance. They also respond to systemic priorities and requirements and provide

an assurance that every government school is meeting accountability requirements.

The planning model used by a school should reflect the needs of the school. That is, so

long as the plan enables the school to respond to the communitys needs, accommodate

Departmental initiatives, and demonstrate its accountability, schools have flexibility in

terms of the planning model they adopt.

The Framework makes the role of school self-assessment clear and the changed context of

teacher assessment explicit. In the past teachers have not used a common set of outcomes on

which to base their judgements of student performance. Establishing the comparability ofteachers judgements against the Outcomes and Standards Framework requires a great deal of

moderation activity within and across schools. Teachers will need to have opportunities to

discuss and share examples of what constitutes a particular level of achievement by a student.

Many schools already use Monitoring Standards in Education (MSE) test materials as part of

a process of moderating teachers judgements about levels of student achievement. The process

of fine-tuning teachers judgements will also be assisted by annual literacy and numeracy


49/87


benchmark testing for year 3, year 5 and year 7 students. As well as monitoring the perfor-

mance of students across the school and at particular year levels, school staff also need to

know about the performance of particular groups of students: perhaps the girls are doingbetter than the boys or the performance of the Aboriginal students is different from that of

the other students.

Analysing achievement data will assist schools to target their improvement efforts at the

points where improvement is most needed. Having gathered some information about student

achievement levels, a key question in any self-assessment is whether that level of performance

is up to expectations. Reaching an answer to this question involves comparing the schools

performance with a standard of some kind. The standard might be a State average; it might

be based on other schools with similar student populations: it might be a national benchmark;

or it might be the schools own performance in previous years. Schools will be assisted in this

process and will be provided with information about the performance of like schools onnational literacy and numeracy benchmark testing.

Having reviewed their performance in terms of student achievement, school staff also need to

assess whether the school is operating as effectively as it could so that the maximum value is

added to the students learning. They are also required to produce a School Report that gives

parents and members of local communities a clear sense of the standards being achieved and

the schools effectiveness in reaching the objectives set out in their school plans. Parents need

to have information reported to them that shows the progress being made in the agreed

priority areas of the school plan. Any information that could identify individual students

must not be published in School Reports. In general, information about school performance

should be communicated in such a way as to ensure that the maximum amount of informationis placed in the public domain while avoiding simplistic league tables based on narrow measures

of school performance. The kinds of information that are to be included in the report are

detailed in Appendix 4.

As a final accountability measure, reviews of schools by District Directors provide assurance

to all stakeholders that schools have conducted rigorous and comprehensive self-assessments,

reported the required outcomes in their School Reports and responded with appropriate

improvement plans. This validation enables those with an interest in a school to have confidence

in the quality of the schools review and reporting processes. It also provides schools with

valuable feedback about their performance and directions for improvement.


50/87


EDUCATIONALIMPACT: CONCERNS12

A number of issues relating to the identification, measurement and presentation of data have

emerged in Australia. Below is a summary of the issues that are currently under consideration.

Ethical principles underpinning reporting

The performance information that is collected and published has the potential to affect greatly

the public perception of school education as well as to influence the decision-making for

government and non-government schools. It is essential, therefore, that the presentation of this

information is underpinned by a clear set of ethical principles. During 2000, this matter was

considered by one of national working groups, which proposed that the fundamental ethical

principle should be truth in reporting. Such a principle requires that the information reported

should be accurate and comprehensive, reporting should not be selective or partial, and all

relevant information should be reported. This principle should also be followed when data are

presented in graphical and tabular forms. Two other supporting ethical principles were proposed:

the communitys right to information; and

the avoidance of unwarranted harm to members of the community.

The first of these is based on the assumption that the public has the right to access information

that has been collected by government for public purposes. However, this principle has to be

considered in the light of the second, which acknowledges that, because of the potential for

harm to occur to individuals and institutions, there can be no absolute right of publication.

Ensuring privacy

Most jurisdictions have in place legislation covering privacy and/or freedom of information. Typically,

this legislation refers to matters such as the collection of personal information, solicitation of

information, its storage and security as well as access to the information. The Australian NationalReport operates within these provisions and also acknowledges that some jurisdictions have agreements

with particular organisations and sections of the community that need to be respected. The over-

riding protocol being observed is that no information should be reported that allows the identification

of individual students or schools. Where information is provided about individual schools, teachers

or students (for example, in some case studies) appropriate approvals are sought prior to publication.

12 This section of the paper draws directly from Appendix 4 the 2000 National Report on Schooling in Australia which can be found on the Ministerial Council on Education, Employment, Training and

Youth Affairs (MCEETYA) website http://www.curriculum.edu.au


51/87


Selection and reporting of performance measures

The Ministerial Council on Education, Employment, Training and Youth Affairs has defined

key performance measures as: A set of measures, limited in number and strategic in orientation,

that provides nationally comparable data on aspects of performance critical to the monitoring

of progress against the National Goals for Schooling in the Twenty-first Century.

Initial efforts to develop such data have focused on the more measurable aspects of students

performance. However, it is expected that performance measures will eventually be developed

to cover the broad range of educational outcomes, even though this might be conceptually and

operationally more difficult in some areas. Some general issues surrounding the reporting of

performance measures include the following:

Time-series data are preferable to single-year data and are included where possible with

descriptive comments regarding trends.

Caution should be taken whenever data involves comparisons between jurisdictions,

sectors or institutions, as extraneous factors can influence the data and lead to readers

drawing false conclusions.

Comparative data of this kind should draw the attention of the reader to all relevant factors.

Data will also be provided, where possible, for students according to gender,

socioeconomic status and geographic location.

International studies may provide good information about the performance of Australian

students and when used, they need to be accompanied by appropriate contextual information.

Data quality

It is intended that only data that are deemed to be of high quality will be published. This

requires clear data specifications which, while not always being published, must be available

from the organisations responsible for data collection. It is important that estimates of statistical

uncertainty are published with the performance measures and not in a separate appendix.

The usefulness of data will be enhanced by the publication of contextual information that

enables the reader to take a considered and critical view of the measure. Similarly, when

information is necessary to explain the data, caveats should be included and be placed as near

as possible to the data.


52/87


Census (full cohort) versus sample data

The latest edition of the National Report on Schooling In Australia contains data derived from

whole-of-cohort census measurement as well as data obtained from samples. Both kinds of

measurement can provide useful information for nationally comparable reporting. Whole-cohort

testing is likely to be both more expensive and intrusive than sample testing and these factors may

make sample testing the preferred option. It is also recognised that sample testing may make

possible the use of richer assessment tasks than are likely to be possible in census testing. However,

sample size must be sufficient to allow reporting by jurisdictions and by sub-groups. Stratified

samples are usually constructed to ensure adequate numbers of students in some sub-groups.

A similar situation exists when using survey data. Census data can be disaggregated to report

by jurisdictions and by sub-groups but a

Comision Simce Anexo2 Forster

Documents

Transcript of Comision Simce Anexo2 Forster