SHIKEN: JALT Testing & Evaluation SIG Newsletter Vol. 5 No. 3. Autumn 2001. (p. 7) [ISSN 1881-5537]
Book Review
Assessing language for specific purposes
by Dan Douglas (2000)
ISBN: 0521585430, hard: 0521584957
Cambridge University Press (311 pages)
This text offers an overview of what language for specific purposes
(LSP) tests are, how they are developed, how they are used, and how they are changing.
The text begins by defining LSP testing as a special form of
communicative language testing. LSP tests such as the Japanese Language
Test for Tour Guides differ from more general language tests such as the
TOEFL in two basic ways.
First, the tasks in LSP tests reflect specific language use more than
general tests do. Second, non-linguistic background knowledge plays a
more significant role in LSP tests.
Countering claims by Widdowson, Davies, and others that LSP tests are
not theoretically justified, the bulk of the first four chapters of
this book offer a detailed theoretical rationale for LSP testing. Many
psycholinguistic and sociolinguistic concepts are expounded. In
particular, the relation between strategic competence, various domains
of knowledge, internal discourse, and external context are considered in depth.
Chapter 5, arguably the heart of the book, investigates how to develop
test tasks indicative of specialized target language use. It provides
detailed guidelines for constructing LSP tests based on a model of
test development expounded by Bachman and Palmer (1996). Frequent
problems in constructing LSP tests are narrated as well. One frequent
problem is that many LSP test developers do not have a detailed
knowledge of the specialized subject they are writing a test for. To
compensate for their lack of background knowledge, the opinions of
expert informants are often indispensible. Other ways of analyzing
languauage use in a specific domain by using context-based research
(Douglas and Selinker, 1994), and grounded ethnography (Frankel and
Beckman, 1982) are considered. In both cases, considerable time is
needed to do an adequate task of assessing how language is used in
a specific target situation and translating that knowledge into test items.
Another problem in some LSP tests is that raters lack expertise in the
specialized domain of knowledge being evaluated. Depending on the
nature of the response examinees are expected to give and the testing
environment, this can be potentially a serious problem.
Considering the entire test development process, Douglas adds, ". . .
the most difficult aspect of producing test specifications is making
the leap from the analysis of the target language use tasks to the
specifications of test tasks" (p. 113). The author notes there are no
idiot-proof guidelines when it comes to creating test items, but the
modified framework of Bachman and Palmer's (1996) model for test
development discussed in this book makes the task easier.
Throughout the book, Douglas reiterates the need to apply the
characteristics of good testing practise which, among other things,
involve thorough piloting as well as considering a number of validity
factors. "Validation is not a once-and-for-all event," the author
states, "but rather a dynamic process in which many different types of
evidence are gathered and presented in much the same way as a mosaic
is constructed. . . " (p. 258). By using a variety of formal stastical
measures as well as a qualitative feedback from a wide variety of
sources, we can begin to get a better idea of what a test is actually doing.
Chapters 6 and 7 of the book examine several specific LSP tests.
Characteristics of exams such as the Occupational
English Test, IELTS, UETESOL are
analyzed according to a framework established by Wesche (1983) in order
to see how the principles for test developement highlighted earlier in
the book are carried out in actual practise. Douglas gauges the
strengths and weaknesses of several LSP tests, noting how LSP tests
often have a wide variety of specificity, broad range of input and
response types, but a narrow range of assessment criteria. In most LSP
tests, assessment criteria have fairly narrow linguistic focus. Douglas
concedes, ". . . the development of communicative, specific purpose
assessment criteria is emerging as one of the most vexing and
problematic aspects of LSP testing. . . " (p. 245).
Assessing Language for Specific Purposes concludes with an overview of
some still unresolved issues in LSP testing. One issue concerns the
nature of input itself. Douglas expresses concern about obtaining
genuine input in testing, remarking that:
the very nature of the LSP testing enterprise means that there will
always and inevitably be a reduction in the dynamic interplay between
the test taker and the characteristics of the test task, because a
test is, by definition, a controlled and contrived environment. (p. 278)
Another issue of concern is the extent to which test tasks can engage
communicative abilities. Too often, test tasks involve minimal
negotiation for meaning or creation of discourse. Today LSP test tasks
frequently have a high degree of situational authenticity (reflecting
actual target language use characteristics), but a low degree of
interactional authenticity (which fail to engage examinees in
communicatively purposeful activities.) One reason this is often the
case is that more and more people are relying on computers to correct
tests, and computer scoring typically has a conservative format such
as multiple choice. Many test development issues ultimately boil down
to issues of time and money and it is often cheaper to have computers
correct tests than highly trained raters.
Noting the tendency of many institutions to favor general language
tests over LSP tests and the ongoing criticisms of LSP tests
themselves as being atheoretical, unnecessary, or unreliable, Douglas
concludes this work by underscoring the intrinsic value of LSP tests
and the need to ascertain some key psycholinguistic issues more
clearly. "If we want to know how well individuals can use language in
specific contexts of use," he states "we will require a measure that
takes into account both their language knowledge and their background knowledge . . ." (p. 282).
This volume is of particular value to test developers and LSP teachers.
Since this book attempts to grapple with so many diverse and often
theoretical issues, it is a daunting task to read from cover-to-cover.
However, its comprehensive index and well-researched annotations make
this work a valuable reference for all those interested in LSP
testing.
Reviewed by Tim Newfields
Works Cited
Bachman, L. & Palmer, A. (1996). Language testing in practice. Oxford University Press.
Davies, A. (1984). ESL expectations in examining: the problem of English as a foreign language and English as a
mother tongue. Language Testing. 1, (1) 82 - 98. doi:10.1177/026553228400100105
Douglas, D. & Selinker L. (1994). Research methodology in context-based second language research.
In E. Tarone, S. Gass, & E. Cohen (eds.), Methodologies for eliciting and analyzing language in context.
Northvale, NJ: Erlbaum, pp. 119 - 131.
Frankel, R. & Beckman, H. (1982). IMPACT: an interaction based method for preserving and analyzing clinical
transactions. In L. Pettigrew (ed.), Explorations in provider and patient transactions. Memphis, TN: Humana.
Wesche, M. (1983). Communicative testing in a second language. Modern Language Journal, 67 (1), 41- 55. doi: 10.1111/j.1540-4781.1983.tb02499.x
Widdowson, H. (2001). Communicative language testing: The art of the possible. In C. Elder, A. Brown, et al. (eds.),
Experimenting with uncertainty: Essays in honour of Alan Davies. (pp. 12-21). Cambridge University Press.
