Preliminary Development and Trial Testing of School

Preliminary Development and Trial Testing of School
Curriculum Evaluation Criteria for Taiwan
By
Ging-Ying Lin and Ying-Shu Liu
Abstract
The aim of this study was to produce a school evaluation handbook that can be applied to elementary and
junior high schools in Taiwan. Criteria for the handbook were developed based on data from questionnaires,
interviews, focus group discussions, and open forums, assisted by analysis of the relevant literature and
legislation. The criteria developed follow the curriculum guidelines provided by the Ministry of Education, and
provide ample space for schools to highlight their special characteristics while they adhere to national policy.
The handbook has been trial tested in 43 schools nationwide.
The contents of the evaluation handbook included: evaluation objectives, target for evaluation, organization
of participants, procedures, and descriptions of the evaluation items and their criteria. The evaluation framework
is summarized as follows:
1. Curriculum planning: (1) establishing appropriate school based curriculum objectives, (2) developing concrete
school curriculum plans that can be implemented, (3) selecting and editing appropriate instructional materials.
2. Curriculum implementation: (1) implementing teaching plans according to timetable, (2) organizing teaching
teams, and expressing teacher professionalism, (3) adopting multiple forms of teaching and learning
assessments, and conducting remedial instruction or instructional improvements based on the results of
assessment.
3. Examining outcomes: (1) understanding instructional outcomes, (2) examining learning outcomes from all
students, (3) making use of curriculum evaluation results.
4. Professional development: (1) planning and providing teacher professional development programs, (2)
providing multiple models for professional development, and actual use in classroom instruction.
5. Administrative assistance and provision of resources: (1) school curriculum team leaders demonstrate
professional knowledge and leadership skills, and can support and engage in curriculum development, (2)
establishing curriculum development committee that has clear tasks, proper division of labor, and smooth
operation, (3) providing the necessary administrative support for curriculum development, and efficient use of
resources, (4) establishing an information management system that can integrate resources, and create a
mechanism for sharing resources.
Research Background and Objectives
Prior to the curriculum reforms in the 1990’s, all schools adhered to the national curriculum without any
room for individuality. Curriculum evaluation was also done by groups outside of the schools. At this time,
schools are encouraged to develop their own curriculum under the guidelines provided by the government. In
addition, they should establish mechanisms within the schools to evaluate the work of the school curriculum
development board. In 2002, the Ministry of Education commissioned the National Academy for Educational
Research to develop a set of curriculum evaluation criteria for use at the national, local, and school levels. This
report describes the progress of our research team at the level of school curriculum evaluation. The study has the
following objectives:
1. To develop a set of evaluation criteria for school based curriculum evaluation.
2. To promote curriculum development and school accountability.
3. To encourage administrators to reflect on school based curriculum and
4. To guide professional development.
Perspectives and Theoretical Framework
1. Mandate: The government’s policy towards curriculum evaluation was not well defined in the past. The
Taiwan Ministry of Education (1998) for the first time stated in its new curriculum directives that curriculum
evaluation should be shared among the national government, the local government, and schools. School
curriculum evaluation should be developed by curriculum development committees within individual schools.
Since that time, guided by curricular evaluation policies and school curricular development, academics,
elementary and junior high schools and the Education Bureaus of local governments have begun
progressively to develop criteria for school curricular evaluation. Their work had been assisted by research
in Taiwan, such as the work of Chen & Kuo (2001) and Chang (2002) .
2. Theory: The functional values of evaluation include decision making, improvement, accountability,
professionalism, and certification (Nevo, 1995). The purpose of evaluation is not so much to prove school
achievements as to improve school curriculum development (Cronbach, 1982; Stufflebeam, 1971, 1983).
Researchers in Taiwan (Chen & Kuo, 2001; Chang, 2002) have suggested the following kinds of criteria for
school curriculum evaluation:

Preparation and design: including school curriculum planning and process, school background
analysis, curriculum mission and goals

Implementation: including the implementation of the curriculum, teaching practices, and professional
development.

Curriculum assessment: including student outcomes, teaching reflections, and curriculum evaluation.

Curriculum organization and operation: including the members, operation, and organization of the
curriculum design team.

Administrative support: including curriculum leadership, interpersonal interaction, administrative
support, sharing resources from parents, computerized management.
3. Empowerment: In the past, curricular evaluation has often meant a great amount of paperwork for schools
without immediately obvious effects. It is the opinion of this study that schools are the agents of curriculum
evaluation. Helping schools build up their evaluation framework would allow to school participants to
engage in professional dialogue and reflection in an empowering process (Short & Greer, 1997).
Method
1. Review of research literature, government policies, and directives.
2. Open-ended questionnaires based on opinions from curriculum experts and local school supervisors.
3. In depth interviews with experts and experienced school administrators.
4. Discussions with focus groups such as teams from schools with recognized excellence.
5. Regular meetings with a panel of experts, who formed a support and oversight body for our consultation.
6. Regular meetings with the larger research team, of which we are a subsidiary project. We discussed each
other’s research progress, what we had in common, and divided our responsibilities at the various levels of
curriculum evaluation.
7. Trial testing of the draft handbook in 43 elementary and junior high schools nationwide.
Data Sources
1. A draft framework for curriculum evaluation was constructed to match the needs of the Taiwan educational
system, based on data from government documents, and the following instruments and procedures:

Open ended questionnaires: including questions on administrative push for curriculum development,
administrative support, the establishment and operations of the curriculum committee, and teacher
professional development. The questionnaires were checked for validity by experts, and filled out
by eligible school supervisors and principals from 14 cities and counties.

In-depth interviews: conducted with several experienced school principals and those currently
involved in developing curriculum evaluation criteria for their own schools.
At this point, the framework for our evaluation criteria included

Curriculum development: deciding on objectives, their interpretation, transformation, and
implementation.

Curriculum framework:
1. What learning domains, what kinds of flexibility, what kinds of learning, their progress, and their
integration.
2. Why such a curriculum: the theoretical background
3. How would it be implemented

Instruction and instructional materials

Outcomes assessment

Use of resources

Support systems
2. Revision of the draft framework was assisted by discussions with several focus groups. We met with school
practitioners from 6 middle schools and 4 elementary schools. They went through the draft framework
word by word, and agreed on the following points:

Evaluation items, criteria, and main points should be concrete and clear.

Those conducting the evaluation should be fair and professional.

There should be a clear statement of how the evaluation results will be used to avoid repercussions
from teachers. Evaluation should begin with self-evaluation, and then outside evaluation after a
period of time.

Special activities that are not possible for most schools should be used to give schools additional
points.

Evaluation has both quantitative aspects and qualitative descriptions, for which there should be
comparative criteria.
3. The structure of the handbook for school curriculum evaluation was finalized based on meetings and
discussions with a panel of experts, experienced educators and administrators, as well as input from local
school representatives in open forums. The forums were held in north, central, and southern Taiwan, with
17 middle schools and 29 elementary schools participating. Aside from changing the content and wording,
they agreed on the following points:

Criteria should be few, the main points should be clear and concrete, and schools can add in their
special characteristics.

Attend to the needs of small schools, hold additional forums for small schools.

For self-evaluation, the ability of teachers needs to be considered. They might be given professional
training or workshops to avoid their misinterpretation of the criteria and procedures, thus voiding the
results of the evaluation.

Simplify the chart, and allow schools to add their own criteria.
4. The draft of the handbook was prepared and underwent numerous revisions after discussions with the panel
of experts and experienced educators. They made the following recommendations:

The Ministry of Education or the local Bureaus of Education should integrate the many forms of
evaluation that are being carried out.

External evaluation should be conducted after self-evaluation. During this time, there should be
good channels for communication to discuss problems during evaluation, and to further revise the
evaluation handbook based on the changes in objectives and criteria used in the individual schools.

There should be meetings detailing the operations of the self or external evaluation before it is
conducted.

The quantitative aspects are not very meaningful and informational for self-evaluation. However,
they create a starting point for discussion and dialogue among teachers. They should still be retained
in the evaluation.
5. The draft was revised again after trial testing for one and a half months in 43 elementary and junior high
school around the country, producing the preliminary handbook, which is the focus of this report.
Results
The contents of the evaluation handbook included: evaluation objectives, target for evaluation, organization
of participants, procedures, and descriptions of the evaluation items and their criteria. The evaluation framework
is summarized in the following chart:
Evaluation Categories Categories of Criteria
Curriculum Planning
1. Establishing appropriate school based curriculum
objectives
2. Developing concrete school curriculum plans that can be
implemented.
3. Selecting and editing appropriate instructional materials.
Curriculum Implementation
1. Implementing teaching plans according to timetable.
2. Organizing teaching teams, and expressing teacher
professionalism.
3. Adopting multiple forms of teaching and learning
assessments, and conducting remedial instruction or
instructional improvements based on the results of
assessment.
Examining Outcomes
1. Understanding instructional outcomes.
2. Examining learning outcomes from all students.
3. Making use of curriculum evaluation results.
Professional Development
1. Planning and providing teacher professional development
programs.
2. Providing multiple models for professional development,
and actual use in classroom instruction.
Administrative Assistance and
Provision of Resources
1. School curriculum team leaders demonstrate professional
knowledge and leadership skills, and can support and
engage in curriculum development.
2. Establishing curriculum development committee that has
clear tasks, proper division of labor, and smooth
operation.
3. Providing the necessary administrative support for
curriculum development, and efficient use of resources.
4. Establishing an information management system that can
integrate resources, and create a mechanism for sharing
resources.
Educational Importance of the Study
1. This study is part of a larger project which aims to use the school as a reference point to establish curriculum
evaluation criteria and mechanisms at the levels of the national government, the local government, and the
school. This is the first time Taiwan has undergone such a large project in curriculum evaluation.
2. The handbook was tested on as many as 43 schools, most of them voluntarily. This indicates that the
interactive process in this study has turned the fear and rejection of curriculum evaluation by the schools into
attitudes of understanding and acceptance.
3. The criteria developed in this study follows that curriculum standards provided by the Ministry of Education,
but provides ample space for individual schools to highlight their special characteristics while they adhere to
national policy.
4. This research constructed a model for developing a set of criteria for evaluating curriculum that may be a
reference for future evaluation research in Taiwan.
Reference
Ministry of Education (1998). Nine-Year Curriculum Guidelines. Taipei: Ministry of Education.
Chang, J. Y. (2002). School-based curriculum reform. Taipei: Guan-Shue.
Chen, M. R. and Guo, C. Y. A Study of school-based curriculum evaluation. Project report to the National
Science Council. (Project No.: NSC 89-2413-H-081B-003) Unpublished.
Cronbach, L. J. (1963).Course improvement through evaluation. In A. A. Bellack & H. M. Kilebard (Eds.),
Curriculum and evaluation (pp.319-333). Berkeley: McCutchan.
Nevo, D. (1995). School-based evaluation. Great Britain: Galliard Ltd.
Stufflebeam, D. L.(1971). The Relevance of the CIPP evaluation model for educational accountability. Journal of
Research and Development in Education, 5(1), 19-25. (ERIC Document Reproduction Service No.
EJ048749).
Stufflebeam, D. L. (1983). The CIPP model for program evaluation. In G. F. Madaus, M. S. Scriven, D. L.
Stufflebeam (Eds.). Evaluation models (pp.117-142). MA: Kluwer Nijhoff publishing.
Short, P. M. & Greer, J. T. (1997). Leadership in empowered school: Themes from innovative efforts. NJ: Merrill.