Japan Language Testing Association Newsletter. No. 10. 31 Aug. 2001 (p. 3, 4)
Special Lecture Report:
Language for Specific Purpose Testing: The State of the Art
by Dan Douglas (Iowa State University)
After outlining the key points in his recent work
Assessing Languages
for Specific Purposes, Douglas mentioned how the notion of language
testing for specific purposes merits further consideration. He
indicated that it is becoming increasingly difficult to distinguish
language for specific purpose (LSP) tests from general language (GL)
tests. LSP tests, Douglas contends, should be regarded as one type of
communicative test. Nearly all major LSP tests today, he pointed out,
are stringently developed according to Bachman and Palmer's (1996)
criteria for communicative language testing which, among other things,
states that all tests should reflect target language use. For this
reason, Douglas suggested that language tests are best described in
terms of a continuum of specificity, with tests such as the TOEFL® at
one end of the spectrum, and the Air Traffic Contoller's Proficiency
Exam at the other. The key difference between general and LSP tests
is not in the construction process, but the nature of the construct
being measured. "Looking at the way tests are constructed is no longer
a viable way to identify what is a LSP test or a more general test,"
Douglas remarked.
Discussing the relationship between general language knowledge and
specific purpose background knowledge, Douglas asserted that
"background knowledge, far from being a factor leading to construct
irrelevant variance, may be an essential aspect of the construct of
specific language purpose ability." He echoed Fulcher (1999) in
suggesting that some aspects of contextual background knowledge should
be regarded as part of an underlying test construct, rather than a source
of test error.Effective LSP tests . . . integrate elements of linguistic knowledge with specific background knowledge relevant to a particular field.
Pointing out how difficult it is to separate background knowledge
from general language knowledge, Douglas advocated specifying
specific background knowledge in test constructs. At the same
time, he cautioned against attempting to assess some types of
non-linguistic abilities beyond the capacity of the test
developers. It would be inappropriate, for example, for most LSP test
developers to attempt to measure a person's engineering expertise. Effective LSP
tests, Douglas maintains, integrate elements of linguistic knowledge
with specific background knowledge relevant to a particular field. To
achieve this, it is often necessary to discover what people in
a given field consider to be valid performance criteria. Mentioning
how tests often fail to measure critical skills which score users
are seeking to discern, Douglas commented, "The ways we judge performance
in specific language tests need to be considered more closely."
Clapham, Davies, Fulcher, and Widdowson have
recommended that linguistic knowledge should be
a main focus of communicative tests. Widdowson (2001), in
particular, has argued that we should not even try to measure
communicative language ability, since it is a dynamic concept defying
measurement. Douglas points out that language researchers in the
coming decades will need to explore how Widdowson's assertion that
tests should focus on linguistic knowledge. Their answer to
that issue will, in part, influence LSP test
development. Douglas believes that the concept of general language
knowledge itself should be questioned. He conceives of
language as a variable capacity rather than a monolithic
construct.
In addition, Douglas mentioned how the distinction between dialect and
register proposed by Davies (1984) may not always be valid. Davies
contends that dialects are acquired as native languages, while
registers may be acquired for specific purposes at any time, then
disregarded when not called for. Douglas, however, believes that both
dialects and registers are learned and discarded as part of social
behavior. In Douglas' view, one of the main challenges in the years
ahead is to define what is meant by 'specific purpose language
testing'. Currently LSP has been criticized by Davies and others as
being too atheoretical and undefined. Douglas counters by
asserting that, "LSP testing is in fact justified on theoretical
grounds . . . but . . . often not carried out for pragmatic reasons
– it is too expensive or impractical to produce the variety of
field-specific tests that theory would justify." Because language
performance varies with context and since academic or vocational
contexts often call for very specific types of language use, Douglas
regards LSP testing as theoretically justified.
After briefly reviewing current testing research, Douglas remarked how
language tests in general are moving away from discrete point
multiple-choice tasks towards more integrative, communicative tasks.
He voiced concern over the tendency in the field to prefer GL tests
over LSP test. He suggested that LSP tests are often superior to
general language tests in that they tend to contain more situationally
relevant assessment criteria.
In his presentation, Douglas also expressed concern about the lack of
pragmatic reality in many test simulations. Acknowledging that realism
is hard to simulate, he felt test developers must strive to make their
tests as realistic as possible.
Douglas concluded by noting the vibrant activities being conducted in
the field of LSP testing. He mentioned about 30 ongoing research
projects, then underscored the need for further studies on
generalizability, the influence of information technology on test
performance, assessment criteria, backwash, and what actually makes
texts field-specific.
- reported by Tim Newfields
Davies, A. (1984). ESL expectations in examining: the problem of English as a Foreign Language and English as a Mother Tongue. Language Testing, 1, 82 - 98.
Fulcher, G. (1999). Assessment in English for Academic Purposes: Putting Content Validity in Its Place. Applied Linguistics, 20 (2) 221 - 236.
Widdowson, H. (2001). Communicative Language Testing: The Art of the Possible. In C. Elder, A. Brown, et al. (Eds.) Experimenting with Uncertainty: Essays in Honour of Alan Davies. (pp. 12-21). Cambridge: Cambridge University Press.
