| Categorical Index |
Chronological Index |
Subject Index |
Title Index |
www.tnewfields.info/Articles/reviewFLTD.htm
[ -3- ]
Thom Hudson & J. D. Brown (Eds.) (2001)
Honolulu: University of Hawai'i, Second Language Teaching & Curriculum Center.
Pp. xi + 230
ISBN 0-8248-2351-6 (paper)
US $25.00
Eight research studies highlighting alternative approaches to language assessment are offered in this text. Systemically illustrating the entire test development process, these studies underscore some of the difficulties in designing and implementing tests. One strength of the book is that it offers a close-up view of how some non-standard tests were developed. Another is that it provides enough detail to carefully reflect on the design problems inherent in each test study. If you are looking for perfect model studies yielding clearly significant results, this may not be the book for you. However, if you are interested in innovative studies of alternative forms of language assessment, this book has much to offer.
| "if you are interested in innovative studies of alternative forms of language assessment, this book has much to offer." |
1. Evaluating Nonverbal Behavior
After mentioning some of the ways nonverbal behavior impacts on communication, Nicholas Jungheim described a study assessing the correlation between oral proficiency in English and nonverbal ability. Nonverbal ability, in the context of this study, was operationalized as the extent that the head movements, hand gestures, and gaze shifts of Japanese EFL students differed from a native EFL teacher population. Linguistic proficiency was measured by the TOEFL and oral English ability by the ETS SPEAK. The results suggest that nonverbal ability is a separate construct from language proficiency as measured by TOEFL and SPEAK: the correlations between the 3-point nonverbal ability scale used in this study and the language test scores were too low to be meaningful.
2. Pragmatic Picture Response Tests
Highlighting the problems with traditional discourse completion tasks, Sayoko Yamashita next described how a picture response test was developed to explore ways that apology strategies differ between (1) native English speakers speaking English (NSE), (2) native Japanese speaking Japanese (NSJ), (3) Americans speaking Japanese (JSL), and (4) Japanese speaking English (EFL). Yamashita found expressions of dismay were used by NSEs less frequently than the other groups. The NSE group also made offers of repair when apologizing more frequently than the other groups. Since perceptions of politeness partly depend on which apology act strategies are employed, these findings were meaningful. Yamashita concluded by mentioning ways picture response tests could be used to assess speaking, writing, listening, or cross-cultural proficiency skills.
[ -1- ]
3. Assessing Cross-cultural Pragmatic Performance
After noting some of the limitations of self-assessment tests in general, Thom Hudson detailed how a three-phase pragmatic performance assessment was developed. In the first phase, 25 Japanese ESL students rated how they thought they would respond to four roleplays using a 5-point scale of appropriacy. In the next phase, three native speakers rated their actual performance by the same scales. In the final phase, students viewed their performance on video and rated themselves again. Surprisingly, the three rating methods showed little variance. Hudson suggested the test population may have been too advanced and questioned whether self-assessment was the most appropriate way to evaluate pragmatic proficiency. The need for more research on both holistic and analytic self-assessment scales was emphasized at the end of this article.
4. Measuring L2 Vocabulary
After mentioning the pros and cons of the leading vocabulary assessment methods, Yoshimitsu Kudo outlined a pilot study to gauge the vocabulary size of Japanese EFLstudents. A twenty-minute test consisting of 18 definition-matching style questions for five different vocabulary levels was administered. To reduce the task load and enhance speed, a translation format was selected. Kudo obtained a low reliability for the pilot test, suspecting that English neologisms may have affected his data. Kudo then revised the test by filtering out English loan words and the reliability coefficients rose for four of the five tests. Kudo concluded by cautioning vocabulary test developers to consider the impact of foreign loan words. English loan words now permeate many languages.
5. Assessing Collocational Knowledge
After providing a theoretical discussion of what collocations are, and how they differ from idioms and formulaic speech, William Bonk described a study to ascertain the relation between collocational knowledge and general language proficiency. He administered a 50-item collocation test and abridged TOEFL to 98 ESL students at the University of Hawaii. Bonk found .73 correlation between collocational knowledge and general language proficiency and emphasizing the need for further collocational knowledge research.
6. Revising Cloze Tests
Next J. D. Brown, Amy Yamashiro, and Ethel Ogane described three approaches to cloze test development. To see how each method differed they conducted a two-phase research study. In the first phase, cloze tests with different deletion patterns were administered to 193 Japanese university students. In the second phase, two revised versions of this test were created: one with an exact-answer scoring system and the other with a broader acceptable-answer scoring system. The researchers found the acceptable-answer scoring system superior in most respects to the exact-answer system, and concluded with some practical advice for cloze test design.
7. Task-based EAP Assessment
Highlighting some of the ongoing controversies regarding performance assessment, John Norris then detailed how an EAP test at the University of Hawaii was developed. Three diverse representatives of the university community were recruited to identify criteria for an EAP performance test. An analysis of their notes revealed significant variation in ideas concerning benchmark performance criteria and in overall detail. The need for team members to be able to effectively negotiate the final test criteria was underscored. Norris concluded by listing ways to help criteria identification teams work more effectively, and mentioned some topics for further research in performance assessment.
[ -2- ]
8. Criteria-referenced Tests
In the final chapter J. D. Brown outlined how he developed a set of criterion-reference testing materials for a popular textbook series. After creating lots of test items and piloting them, those items that seemed problematic were revised or weeded out. A semester later different revised forms of the test were administered. Items that did not discriminate well or lacked validity or reliability were again discarded. The final test was sensitive to the material covered in the text series, valid for the populations tested, and fairly dependable. Despite this, Brown cautioned against basing high-stakes educational decisions on any single test score.
This volume will appeal to two groups of people: those with some understanding of research design principles who wish to critically reflect on how those principles can be carried out in actual studies, and those with an interest and non-standard types of language assessment.
Tim Newfields
Nanzan Jr. College
| Categorical Index |
Chronological Index |
Subject Index |
Title Index |
[ -3- ]