Royce Sadler: Conversations about the Learning Record
On a sabbatical tour from his university post at Griffith University in Queensland, Australia, Royce Sadler stopped to visit the Center for Language in Learning in California. It was Professor Sadler's 1987 article on the theoretical basis for "standards-referenced" assessment in the Oxford Review of Education which informed the development of the Learning Record's system of student assessment. While at the Center, from the perspective of his extensive experience with Queensland assessment, he responded to questions frequently asked about the Learning Record. The questions and his responses during these talks are given below.
- What exactly is the Learning
First, it is much more than a special type of report card, although the Learning Record is indeed part of a system of assessing and reporting student achievement. It achieves these ends in ways that recognize the different ways children learn, and the different ways in which their learning is expressed. When a school opts into using the Learning Record, it becomes part of a system that produces both detailed and summary information about how students in regular and special classes are performing.
- What is the basic philosophy
behind the Learning Record?
Teachers, by virtue of their close association with their students, develop an extensive knowledge of how their students are achieving. That knowledge is rich and highly contextualized: the teacher knows the history of a student's growth in achievement, sees many things the student does and produces, and progressively builds up a large body of information. This information forms the basis of judgments about how much the student has learned, and the form that learning has taken. The Record provides a systematic way of tapping into that information, and of analyzing and reporting it.
- Won't teachers' judgments
be colored by their knowledge of students as persons?
The Learning Record is part of a system that draws upon the cumulative experience of teachers who collaboratively agree to keep themselves professionally calibrated, so that Records from one school or classroom are comparable with Records from other schools and classrooms.
- Hasn't it been shown
through research that teacher judgments and grades are unreliable?
Some research does indeed show that teacher judgments about the quality of student work is unreliable. Unreliability means that the same teacher may grade student work differently on different occasions, or that different teachers may grade the same work differently. But by and large these studies have been carried out by simply asking teachers to re-grade or cross-grade student work without the benefit of professionally designed systems that could lead to improvement. The Record aims to provide systematic and well-grounded support for teachers in making their judgments.
- This all sounds very subjective.
Surely we want objective assessments, particularly when students'
futures are at stake!
It is true that teacher judgments are subjective. But that does not necessarily make them random or worthless. In fact, if teacher judgments were hopelessly woolly or unreliable, the profession of teaching might as well be abolished. In any case, most professional assessments (not just those in education), and most of the decisions we make in daily life, are subjective in the sense that they are made by people. This does not, in itself, make them dubious or hollow. Subjective judgments cover the full spectrum, from those that are mostly matters of taste or simple preference (such as liking strawberry ice-cream), to those that are more serious in their consequences but are based on weak evidence or guesswork (or even wishful thinking), to those that are soundly based on clear evidence. To question judgments simply because they are subjective would lead to a rejection of most of the decisions made in everyday life. So the real issue is not whether judgments are subjective or objective, but how consistent those judgments are, how meaningfully they can be interpreted, and whether good consequences follow.
- It seems to me that students
need to be tested, so that they all complete the same tasks, the
marks are worked out, and then compared with norms. Numbers are objective,
and we know what they stand for.
All assessment of student achievement (whether the tests are teacher-made or nation-wide) involves subjective judgments. Test items are constructed by people, protested on people, and refined, rejected or retained by people. The criteria on which these decisions are based are debated among experts, and decisions made about which criteria to use and how to use them. Every worthwhile testing program is the result of deliberation, and alternative choices could always have been made. With standardized testing, all of the subjective judgment goes on behind the scenes, but it is there (and important) nevertheless. The final stages of testing programs involve administration of the test, scoring, and comparison of individual scores with state or national norms. These phases may perhaps seem to be more overtly objective or scientific. The scores have an 'objective' meaning primarily in the sense that experts agree on what the correct answer is, and that the resulting scores are numbers that can be processed statistically. But the judgments that give rise to the numbers, as is indeed the prior decision to measure performance using these techniques, are subjective decisions. So the debate is not between subjectivity and objectivity as such, but how credible, how consistent, and how meaningful the assessments are.
- Are you saying then that
standardized testing is not concerned about being credible, consistent
Not at all. Standardized testing is interested in these goals, but approaches the problem from one particular angle. It puts the subjectivity in at the front end, then the formal processing procedures take over. The Learning Record, on the other hand, tackles the "quality-of-assessment" problem from a quite different direction. It aims to exploit the rich, specific and contextualized knowledge teachers develop about their students, and make that the basis for a system of recording and reporting achievement. The Learning Record keeps teacher professional judgments right in there until the end. But it continues to place a high premium on credibility, consistency and meaningfulness, and of course on high-quality evidence for the judgments.
- So apart from the techniques,
the two are basically the same?
The Learning Record aims to establish a standards-based framework to enable student achievement to be assessed and reported. It is fundamentally different in that the standards are not numerical cutoffs from a testing program, but have an entirely different, but equally rigorous, formulation. The standards are related not to test scores, but directly to what students have done. The standards themselves are decided upon, field tested, and refined with the help of assessment experts. They take the form of specific descriptions of behaviors and performance together with concrete examples (called exemplars) to make the meaning of the descriptions quite clear. A particular student's performance is observed (in context) and compared with the relevant set of standards. The teacher's function is to match the student's performance with the standards definition that fits best. The various standards are given numerical labels, and constitute a scale to allow for communication between teachers, and among teachers, students and parents. The performances of students on the scales can be aggregated to provide system-wide performance summaries. So you see, the focus is primarily on how individual student performance can be identified directly and linked to a system of standards, using the teacher's judgments.
- But these judgments could
be based on different evidence for different students. Isn't that
open to question?
The Learning Record is essentially about obtaining evidence about student performance before a judgment is made about the level of performance. The evidence can take a variety of forms, but conclusions are not reached on evidence that is irrelevant, skimpy or of poor quality. The main differences between the Learning Record and standardized testing are that the Record draws on a much more extensive data base than a single testing episode, and that the evidence is generated mostly, but not necessarily exclusively, through normal classroom and learner activity. This not only increases the validity of the evidence, but also ensures that crucial decisions are not made on the basis of a single test event. In other words, the Learning Record is open to multiple sources of evidence, and these are weighed up together. It follows that different students may produce evidence in different ways, but all of it reflects the learning that has taken place.
- Could the Learning Record
be used alongside other forms of testing?
Indeed it could. And it will produce results that are soundly based in calibrated teacher judgments.
- What exactly do you mean
by "calibrated" judgments?
Teachers operating in their classrooms by themselves constantly make judgments about how their students are progressing. That is part of what it means to be a teacher. But when teachers participate in the Learning Record (as an assessment system), they agree to use prescribed procedures that provide a basis for comparable judgments to be made between different teachers, in different schools, and at different times. The aim is to ensure consistent interpretation and application of the standards. By keeping themselves knowledgeable about the standards that have been defined, and through using them in accordance with the set procedures to arrive at judgments that are consistent with the standards (and hence with the judgments other teachers would make, given the same evidence), teachers are said to be "calibrated."
- Tell me again, how are
these "standards" defined?
The main features are verbal descriptions of the qualities required of work at the different levels. This is complemented by concrete examples of work, to illustrate the descriptions. These two, the descriptors and the exemplars, are key elements of the learning scales for different fields of learning. The exemplars show, and the descriptions tell. They work together. But that is not all. Teachers working within the system participate in moderation meetings with other teachers, so that their own ability to make judgments is kept consistent with other teachers, and with the standards as set out in the scales. A spin-off on the system is that students themselves have access to the scale points, and both teachers and parents have available a direct way of seeing exactly what the standards mean. Having access to the standards provides an open system that is both helpful to students and closely aligned with what goes on in class. In fact, the close correspondence between what is assessed and normal classroom activity is an important strength of the Learning Record system.
- With "external" forms
of testing, the students and the teacher are pitted against a common
enemy, the test. The Learning Record places the teacher in the position
of being both coach and judge. Won't this adversely affect the teaching
The design of the Learning Record is such that the standards do have an external formulation, and they are accessible to both the teacher and the learner. The student can actually see examples of what the standards imply, and can work with the teacher towards improving personal performance. If you like, both teacher and learner are "assessed" against the standards, and remain partners. It's not unlike athletic training, where goals (and expectations) are set and the aim is to attain them. But in athletics, of course, the goals can often be set down simply as times or distances. Educational outcomes are considerably more complicated, which is why the standards themselves can't be left to chance (or worse, not determined at all). They need careful thought, trial and analysis. Just collecting evidence is not in itself enough. The evidence must be sifted, understood, and related to a scale of performance. And of course how other students perform at the time is of secondary interest.
- How do teachers find the
system? Don't they feel their reputation as professionals is undermined?
Quite the reverse. Most teachers using the system welcome the collaboration with other teachers, and want their judgments to be consistent with those of other competent professionals. They see this as part of their professional responsibility as a teacher, just as the public expects other professions to make sound, consistent judgments Teachers using the Record are not wedded to the idea that their own, (perhaps idiosyncratic,) judgments are automatically infallible, even if they are very experienced. And of course the Learning Record still leaves teachers with a great deal of professional autonomy as to what goes on in their classrooms.
- How do students react
to this? Don't they expect the teacher to tell them how they are
A central premise of the Learning Record is that students, as part of their learning, need to develop the ability to make reasoned judgments about the quality of their own work. This is for two reasons. The first is that students need to learn how to manage the quality of their own work while they are actually producing it. Otherwise they have to depend on the teacher to tell them how they are going, and have limited opportunity to learn otherwise. Even young children are capable of making judgments, and part of the teacher's responsibility is to help them refine their judgments. Second, learners ultimately need to take responsibility for their own learning, and learning how to do this starts in school. It can only come about by giving learners true experience in making judgments, preferably about work of the same type they are attempting to produce themselves. The Learning Record provides a creative and supportive environment for this to occur. The evidence to date shows two things: students like to become involved in collaborative assessment with the teacher, and the doing of it makes a big positive difference to learning itself and to learners' attitudes to learning.
Royce Sadler is a Professor of Education at Griffith University, Brisbane, Australia. He holds advanced degrees in mathematics and education. Originally a high school teacher, he has taught mathematics and computer programming at two institutes of technology, and courses in assessment and testing at two Australian universities. In 1985, he worked for the Queensland Board of Secondary School Studies as Head of its Assessment Unit, and helped to provide theoretical foundations for school-based assessment, with particular focus on defining and communicating achievement standards for purposes very similar to those used by the Learning Record. He has published widely on assessment issues, and has a particular interest in making assessment effective in promoting learning.