To enhance understanding of the evaluation process, it is useful to distinguish between the terms assessment and evaluation. These terms are often used interchangeably, thereby causing some confusion over their meanings. Assessment is a preliminary phase in the evaluation process. In this phase, various techniques are used to gather information. Evaluation is the weighing of assessment information against some standard in order to make a judgement or decision (i.e., an evaluation). This may then lead to other decisions and actions.
Evaluations may focus on the effectiveness of school programs (i.e., program evaluation), the effectiveness of the curriculum (i.e., curriculum evaluation), and the progress in student learning (i.e., student evaluation).
Program evaluation is a systematic process of gathering and analyzing information about some aspect of a school program in order to make a decision or to communicate to others involved in the decision-making process. Program evaluation can be conducted at two levels: relatively informally at the classroom level or more formally at the classroom, school, or school division levels.
At the classroom level, program evaluation is used to determine whether the program being presented to the students is meeting both their needs and the objectives prescribed by the province. Program evaluation is not necessarily conducted at the end of the program, but is an ongoing process. For example, if particular lessons appear to be poorly received by students or if they do not seem to demonstrate the intended learnings from a unit of study, the problem should be investigated and changes made.
By evaluating their programs at the classroom level, teachers become reflective practitioners. The information gathered through program evaluation assists teachers in program planning and in making decisions for improvement.
Most program evaluations at the classroom level are relatively informal but they should be done systematically. Such evaluations should include identification of the area of concern, collection and analysis of information, and judgement or decision making.
Formal program evaluation projects use a step-by-step problem-solving approach to identify the purpose of the evaluation, draft a proposal, collect and analyze information, and report the evaluation results. The initiative to conduct a formal program evaluation may originate from an individual teacher, a group of teachers, the principal, a staff committee, an entire staff, or a school division central office. Evaluations are usually done by a team, so that a variety of background knowledge, experience, and skills is available and the work can be shared. Formal program evaluations should be undertaken regularly to ensure programs are current.
To support formal school-based program evaluation activities, Saskatchewan Education has developed the Saskatchewan School-Based Program Evaluation Resource Book to be used in conjunction with an inservice package. Further information on these support services is available from the Assessment and Evaluation Unit, Saskatchewan Education.
There is a need to know whether new curricula are being effectively implemented and whether they are meeting the needs of students. At the provincial level, curriculum evaluation involves making judgements about the effectiveness of provincially authorized curricula.
Curriculum evaluation involves the gathering of information (i.e., the assessment phase) and the making of judgements or decisions based on the information collected (i.e., the evaluation phase) in order to determine how well the curriculum is performing. The principal reason for curriculum evaluation is to plan improvements to the curriculum. Such improvements might involve changes to the curriculum document and/or the provision of resources or inservice to teachers.
It is intended that curriculum evaluation be a shared, collaborative effort involving all of the major education partners in the province. Teachers will be involved in instrument development, validation, field testing, scoring, and data interpretation.
In the assessment phase, information is gathered from students, teachers, and administrators. The information obtained from educators indicates the degree to which the curriculum is being implemented, as well as the strengths and weaknesses of the curriculum. The information from students indicates how well they are achieving the intended learning outcomes and provides indications about their attitudes toward the curriculum.
All provincial curricula are included within the scope of curriculum evaluation. Evaluations are conducted during the implementation phase for new curricula and regularly on a rotating basis thereafter. Curriculum evaluation is described in greater detail in the document Curriculum Evaluation in Saskatchewan.
The main purposes of evaluation are to assist students in their learning and to improve instruction. Teachers make judgements about student progress based on information gathered through a variety of assessment techniques. This information assists teachers in planning or adapting instructional programs which, in turn, helps students learn more effectively. Evaluations are also used for reporting progress to students and their parents, and for making decisions related to such things as student promotion.
Saskatchewan’s Core Curriculum requires that changes be made in the ways young adults are taught and evaluated. Formerly, evaluation of student learning focused on factual content only and student progress was assessed with traditional techniques such as paper and pencil tests.
However, to evaluate learning in areas such as critical and creative thinking, independent learning, and personal and social values and skills, non-traditional strategies are required. More often than before, teachers will rely on techniques such as observation, conferencing, oral assignments, and process assessment in order to gather information about student performance or progress.
Although the responsibility to establish student evaluation and reporting procedures resides with the school principal and the teaching staff, the classroom teacher has the daily responsibility for student evaluation. The teacher is at the forefront in determining student progress by using sound evaluative practices that include careful planning, appropriate assessment techniques and, most importantly, sound professional judgement.
There are three main types of student evaluation: formative, summative, and diagnostic. Assessment techniques are used to gather information for each type of evaluation. Teachers conduct all three types of evaluation during the course of the school year/semester.
Formative evaluation is an ongoing classroom process that keeps students and educators informed of students’ progress toward program learning objectives. The main purpose of formative evaluation is to improve instruction and student learning. It provides teachers with valuable information upon which instructional modifications can be made. This type of evaluation helps teachers understand the degree to which students are learning the course material and the extent to which their knowledge, understanding, skills, and attitudes are developing. Students are provided direction for future learning and are encouraged to take responsibility for their own progress.
Summative evaluation occurs most often at the end of a unit. The teacher uses summative evaluation to determine what has been learned over a period of time, to summarize student progress, and to report to students, parents and educators on progress relative to curriculum objectives.
Seldom are evaluations strictly formative or strictly summative. For example, summative evaluation can be used formatively to assist teachers in making decisions about changes to instructional strategies or other aspects of students’ learning programs. Similarly, formative evaluation may be used to assist teachers in making summative judgements about student progress. It is important that teachers make clear to students the purpose of assessments and whether they will later be used summatively.
Diagnostic evaluation usually occurs at the beginning of the school year or before a new unit. It identifies students who lack prerequisite knowledge, understanding or skills, so that remedial help can be arranged. It also serves to identify gifted learners to ensure they are being sufficiently challenged. Diagnostic testing also identifies student interests. Diagnostic evaluation provides information essential to teachers in designing appropriate programs for all students.
Although student evaluation is not strictly sequential, it can be viewed as a cyclical process made up of four phases: preparation, assessment, evaluation, and reflection. This process involves the teacher as a decision maker throughout all four phases.
During the preparation phase, decisions are made that identify what is to be evaluated, the type of evaluation to be used (i.e., formative, summative or diagnostic), the criteria against which student learning outcomes will be judged, and the most appropriate assessment techniques with which to gather information on student progress. The teacher’s decisions in this phase form the basis for the remaining phases.
During the assessment phase, the teacher identifies information gathering strategies, constructs or selects instruments, administers them to the students, and collects the information on student learning progress. The teacher continues to make decisions in this phase. Important considerations include the identification and elimination of bias (e.g., gender and culture bias) from the assessment techniques and instruments, and determining where, when, and how assessments will be conducted.
During the evaluation phase, the teacher interprets the assessment information and makes judgements about student progress. Based on the judgements or evaluation, teachers make decisions about student learning programs and report on progress to students, parents, and appropriate school personnel.
The reflection phase allows the teacher to consider the extent to which the previous phases in the evaluation process have been successful. Specifically, the teacher evaluates the utility and appropriateness of the assessment techniques used. Such reflection assists the teacher in making decisions concerning improvements or modifications to subsequent teaching and evaluation.
Recognizing the importance of evaluation as an integral part of the curriculum, Saskatchewan Education has developed five general guiding principles to provide a framework to assist teachers in planning for student evaluation:
The following article is reproduced with permission of the National Catholic Educational Association, from the following publication: DiGiacomo, James J. Teaching Religion in a Catholic Secondary School, Ch. 8 entitled "Assessment". NCEA Keynote Series, Ó 1989 National Catholic Educational Association, Washington, D.C.
Whatever one’s educational philosophy, whatever the teaching style, however bright or slow or inquisitive or apathetic our classes, sooner or later we come to what some students consider the bottom line: we must give marks. In our minds, this is probably one of the least important aspects of our work. What really matters is how our students have grown, what they learned, and how it will affect their lives. To many students, the mark is important and besides, this is a school, and report cards have to be filled out. So, it’s time to turn in our grades. How will we assess our students’ achievements?
Before offering some suggestions on how to evaluate performance and assign grades, we should note that for many religion teachers, this presents little or no difficulty, while for others it is a real problem. The latter are uncomfortable with the very notion of assigning marks for religious activity. Let us look at these two groups in some detail. Teachers may find themselves in one of them, or maybe a part of each. Theoretician Graham Rossiter describes them:
Some religion teachers seek to establish religion as a subject in the curriculum with a status similar to that of other subjects. An emphasis on content, study skills, written work, assignments and assessment suggests that the pedagogy in this approach should be similar to that of other subjects. Such an approach ... concentrates on communicating knowledge and understanding of religion, while at the same time not neglecting the affective dimension and not disregarding the importance of other aspects of religious education outside the formal curriculum (liturgy, retreats, etc.).
By way of contrast, other religion teachers oppose this approach, considering that religion should not be presented as an academic subject. According to this way of thinking, pupils should perceive religion very differently from other subjects: there should be emphasis on discussion and sharing of faith insights with no written work and assessment. This approach highlights the religion period as a more personal pastoral alternative to the classwork in academic subjects where competition and examination orientation are sometimes believed to be problematic.
It is easy to see why the first group would have no great difficulty assigning marks in religion, while the second group has problems. Clearly, the argument goes beyond the issue of marks to a more philosophical level. Splits like the one described can be found not only among the members of a department, but also even in individual teachers. Where teachers find themselves will affect not only the way they make up grades, but also the way they approach religion teaching itself.
Rossiter himself lines up with the first group. He criticizes his opponents for trying to do catechesis in an inappropriate setting. Catechesis, properly understood, presupposes a group of believers, effectively evangelized, who are willing to develop deeper understanding of and participation in the Christian faith tradition. The students in our classrooms can no longer be thought of as such a homogeneous group. "What might be a healthy sharing and commending of faith insights in a voluntary group setting could be perceived as presuming too much or applying moral pressure if attempted in the compulsory classroom." He recommends disassociating religious education from catechesis, in order to achieve clarity of purpose, set realistic goals, and avoid courting frustration and resentment.
Not everyone wants to go as far as Rossiter. Some feel that he limits religion teachers to teaching about religion and thus prevents them from reaching the student as a whole person. Whichever side we take, we must deal with the issue on the level of assessment and grading. On what shall we mark the students?
For those teachers who line up with the second group and prefer a more pastoral orientation, which stresses the devotional and downplays the educational, their efforts to make the subject more pastoral may send out some unintended messages. The students may conclude that religion is somehow less of an educational experience than the study of other subjects, or that it is a totally subjective enterprise, not worthy of serious intellectual attention. Not only will teachers be in trouble with students if they give any low marks, but they will be hard pressed to justify the decision, not only to students but also to themselves.
Teachers, who resist choosing between these two views of religious instruction and see a bit of themselves in both of them, probably move back and forth between the educational and the devotional, the academic and the pastoral, the objective and the subjective. Perhaps this works, but it then also is wise to separate out these various elements in their teaching, to see which elements lend themselves to formal grading and which do not.
Grading Elements
In addressing this question, a group of religious educators, in a recently published work, distinguish four different aspects of learning: 1) knowledge of material; 2) critical thinking and interaction with the material; 3) individual acceptance of the material as meaningful; and 4) actual incorporation into one’s personal life. Religion teachers strive to achieve all four outcomes, and to the extent that they happen or fail to happen, perceive their efforts as successful or failing. Which of these lend themselves to evaluation and grading?
Once teachers have clarified for themselves what it is that they are marking and how they arrive at the grades, they should strive not only to be fair, but also to be perceived as fair. Assigning marks is often a potential source of misunderstanding and resentment. To minimize this danger, students should be informed, clearly and explicitly, how they will be graded. This is even more important in courses which stress process than in those that stress content, since the norms for the grading process may not be well defined in the students’ minds. They must also be reassured that their grades are not a function of their belief or disbelief, or of their agreement or disagreement with the teacher on controverted questions. For this reason, teachers should be careful in marking exercises - essays or homework papers - which ask for opinions rather than retention or explanation of information. Nothing must be permitted which would damage the atmosphere of honesty and trust that the teacher has built up in the classroom.
Evaluating Performance
In this chapter, we have been using words like grasp and mastery somewhat loosely. When evaluating students’ performance, it is well to distinguish different degrees of learning. There is passive retention, whereby the student can recognize the right answer (e.g. in a multiple choice test) or follow the line of reasoning in a teacher’s presentation or understand a piece of writing. Then there is active retention, in which the student can recall what is demanded. This active grasp, which is a superior degree of learning, is demonstrated by the ability to express oneself, to verbalize both by the spoken and the written word. These degrees of skill are important when we remember that the goal of Catholic education is to turn out graduates who are not only well-informed and discriminating, but also articulate and capable of communication and leadership. Religion teachers probably cannot expect this of all students, but they should certainly give recognition and encouragement to those who seem to be on the way to achieving it.
Finally, assessment need not be a one-way street. Just as teachers help students by evaluating their work and pointing out their achievements and shortcomings, so also students can help teachers by offering constructive criticism of their work. Administrators can supply specially constructed forms which enable students, anonymously, to offer positive as well as negative observations. These evaluation instruments touch on such items as command of subject, organization, clarity of presentation, ability to sustain interest, fairness, and relationships with students. Some teachers find this process threatening, and indeed, it is sometimes humbling. But, it can also be encouraging and even surprising. There is no better way to find out how one is really doing in the classroom. It is highly recommended to all teachers, from novices to veterans. It’s never too early or too late to learn.
Summary