Engendering assessment literacy:
Tim Newfields (Toyo University Faculty of Economics)
Keywords: assessment literacy, teacher development, assessment models, teacher training
- 22 -This paper presents some preliminary findings about how a small group of high school English teachers in Japan feel about testing and the sort of assessment practices they employ. I should begin with some provisos. First, this is a qualitative case study with merely seven informants, so the extent that any of the findings can be generalized to the population of about 30,000 high school English teachers in Japan (MEXT, 2006) is questionable. This study merely offers a limited sketch of the views of a handful of Japanese secondary school foreign language instructors – it is by no means a complete picture. However, just as a likeness of a face can emerge with a few lines, perhaps some of the comments made by these informants will begin to reveal a larger picture.
- 23 -Some Acknowledged Biases
|". . . most so-called language tests are only scratching the surface when it comes to measuring language ability."|
|Humanistic testing paradigm||Mechanistic testing paradigm|
|Often willing to consider complex questions
with more than one "correct" answer.
|Generally only one single correct answer
and preference for simple "right-or-wrong" formats.
|Preference for holistic information and multiple task types to enhance student interest.||Preference for isolated information which is ideally context-free and with just a few task-types.|
|Human factors and emotional responses
are deemed a relevant part of learning.
|Human factors and emotional responses
are considered extraneous variables that are irrelevant.
|Generally adaptive and variable.||Generally fixed and non-adaptive.|
|Social collaboration and team-play is a valued skill.||Usually only individual performance counts –
"collaboration" is often regarded as cheating.
|"Effort" is often considered significant.||Effort is irrelevant – only performance counts.|
|Feedback is often informal and verbose.||Feedback generally consists of formal grades or scores.|
|Validity through qualitative consensus.||Validity through quantitative statistical procedures and perceptions of beneficial washback and social impact
of a test is seldom measured.
- 24 -
|". . . some teachers make valiant attempts at humanistic teaching/testing despite the mechanistic paradigms they are compelled to work in."|
- 25 -
|1||M||10+ yrs.||English||Ph.D. candidate||High-rank public HS in Tokyo|
|2||M||20 yrs.||Japanese||MA degree||High-rank private HS in Tokyo|
|3||F||12 yrs.||English||BA & some grad course||Mid-rank public HS in central Japan|
|4||M||25+ yrs.||English||BA & 3 MA degrees||Mid-rank public HS in central Japan|
|5||F||13 yrs.||English||MA degree||Mid-rank private HS near Tokyo|
|6||F||11 yrs.||Japanese||BA degree||Low-rank public HS in Tokyo|
|7||F||24 yrs.||Japanese||BA degree||Low-rank private HS in central Japan|
- 26 -
- 27 -"and they will learn complex things very quickly." She then pointed out how different the picture is in schools that students who do not do well on the admission tests go to. "Kids feel discouraged and discipline is often a problem there." She then stressed a truism many teachers have heard in various ways: what works well in one context with one group of students might not work well in a different context with others.
|"In a way quite like British Common Law, historical precedent seems to carry more weight than abstract theory."|
- 30 -(6) Test anxiety among students
- 29 -
- 30 -In this regard, Informant #5 suggested that Japanese high school teachers are subject to two competing pressures: (1) a need to conform and test/teach in ways that bureaucrats want them to, and (2) a desire test/teach according to their core beliefs. Some teachers appear to align their core beliefs to the system effectively and experience minimal conflict. The comment by Informant #4 that teachers sometimes need to be like "stealth warriors" seems to be recognized by many teachers in Japan. Older teachers in particular seem to know it is fruitless to outwardly fight the system or exhibit too much overt insubordination, so seek subtle ways to work within the system and still bring some of their core beliefs into practice.
|"Assessment literacy is not something one can simply acquire from reading one or two books - it is probably a lifelong process involving an ongoing and evolving dialog between empirical research and theory."|
- 31 -There are dozens, if not hundreds, of minor academic journals for teachers in Japan and associations for English teachers. Publications such as Eigo Kyouiku [English Education] and Shin-Eigo Kyouiku [The New English Classroom] attract a devoted number of readers. The teachers who write up their classroom research for those journals or make presentations about it are often seem become the most assessment literate. If we accept a constructionist theory of learning (Brown, et al., 1989) with respect to assessment literacy, this makes sense. Assessment literacy is not something one can simply acquire from reading one or two books – it is probably a lifelong process involving an ongoing and evolving dialog between empirical research and theory. Actively writing about classroom research often encourages or forces a writer to become more assessment literate. Whereas prestigious journals can afford to flatly reject articles with poor statistics or sloppy designs, with some smaller journals a mentoring process is more likely to take place. Though standards vary widely from journal to journal, one way some teachers learn about testing is through a process of mentoring when editorial advisor work over their manuscripts. This conjecture certainly requires further corroborating research.
- 32 -
- 33 -
I am grateful to Kristie Sage for her comments on this article.
However, the responsibility of its short-comings is entirely mine.
- 37 -
- 38 -Newfields, T. (2002). Challenging the notion of face validity. Shiken: JALT Testing & Evaluation SIG Newsletter, 6 (3) 19. Retrieved on July 28, 2007 http://jalt.org/test/new_2.htm
- 39 -