Oral Proficiency Testing: One Approach for College EFL Classes (cont'd.)

The Real Time Oral Assessment Inventory –
An Oral Evaluation Procedure for Large Size Classes (cont'd.)

In recent years I have been attempting to develop a procedure to assess the oral proficiency of large classes in a way that will allow students to promptly respond to a range of questions. The procedure of evaluation I currently use is called the "Real Time Oral Assessment Inventory", herein referred to the RTOI. The title is appropriate in three respects: (1) students must answer all questions in a period of three seconds - a time frame that begins to approximate real conversations, (2) the feedback to all answers is immediate, and (3) the test is oral in focus.

Each of the seven steps of this testing procedure will be briefly described.
"Although this activity has value in terms of reducing test anxiety and providing real time oral/aural feedback, there is a need to evaluate its reliability. "

Step 1: Pre-Activity

One week before administering the RTOI, I ask students to select twenty questions from their notes in preparation for the actual test. Although I could easily compile all exam questions, I believe it is more valuable for students to do so because they will review previous material while preparing questions. Moreover, having the students select the test items rather than an instructor seems to reduce student anxiety.

Step 2: Pairwork and Teacher Check

Before administering the RTOI, I give students a chance to quiz each other in pairs for ten to fifteen minutes. During that time, I also correct student materials while moving around the room so that when the actual RTOI is administered, the student test questions are grammatically correct and contextually appropriate.

Step 3: Dividing the Class

The RTOI can be administered effectively to 20-24 students at a time. For classes of less than 12 students, it may be more viable to conduct oral interviews individually, in small clusters, or in pairs. For classes of 25-48 students, I recommend dividing the class into two groups. For classes of 49-74 students, I recommend dividing the class into three groups. Classes with more than 75 students should be divided into four groups. Typically, I will have one group take the test in a neigboring room while the remaining group(s) do an activity unrelated to the test in the main classroom. It is important to have the test group and non-test group in different locations to reduce data skewing. The RTOI should be administered in a location which is wide and open: students actually move forward during the testing process and the distance they traverse from the starting line determines their grade. A space of about six by twelve meters is needed to administer this test to twenty students.

Although the RTOI can be conducted in an empty classroom, having it in a fresh, outdoor environment may be better. By having the RTOI in a non-classroom environment, the gamelike nature of this testing process is underscored.

Step 4: Starting Point and Explanation

The baseline in the RTOI is represented by the positions in which students stand. First, I ask students in the test group to stand in a line as in Figure 1.

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17

Teacher

Figure 1. The distribution of students at the beginning of the RTOI

After this, I explain the test procedure. For sophomore university students, this sort of explanation is offered –
We are going to do a simple test activity. After you hear each of the following questions, if you know the answer raise your hand. You will have up to three seconds to answer. Your classmates will then help you to decide how correct your answer is. If your answer is correct, please take a step forward. If your answer is almost correct, take a half step forward. If you cannot answer within three seconds, stay where you are. There is no penalty for mistakes.

Step 5: Demonstration

Although some students will understand the previous explanation, only after going through several sample questions does the RTOI procedure become evident to everyone in the class. The procedure I use in offering model exam questions appears in Figure 2 below -

(1) QUESTION ASKED (2A) (2B) (2C) (2D) NO ANSWER WRONG NEARLY CORRECT WITHIN ANSWER CORRECT ANSWER 3 SECONDS ANSWER (3A) (3B) TEACHER GIVES CORRECT TEACHER ASKS Ss. FOR CONFIRMATION ANSWER & Ss. REPEAT & PROVIDES ANSWER AS NEEDED (4A) (4B) STUDENT REMAINS STUDENT MOVES STUDENT MOVES IN SAME FORWARD FORWARD POSITION HALF A STEP A FULL STEP (5) NEXT QUESTION ASKED
Figure 2. The RTOI question/response procedure

Step 6: Student Practise

After demonstrating how the RTOI procedure works, I invite one student to step forward and take my place. This student is now the "teacher" and asks two or three questions to the audience. If the student is able to ask two or three questions at natural speed about points which are relevant to what we have studied, s/he is entitled to take one step forward. Students who are unable to do this can read from their prepared questions. However, they will progress only half a step.

Suppose a student asked a question such as, "What's the date?"

If no one raised their hands within three seconds, the student asking the question would offer an answer, have the students repeat, then move to a different question.

If two, three, or even four students raised their hands simultaneously after the question was asked, they would decide by "stick-stone-scissors" and the victor would get the first stab at the question and the student asking the question would be judge the answer. If the student asking the question was unable to assess whether the answer was correct, I would ask the class as a whole to evaluate the answer. If the class could not do this, I would intervene by assessing the answer.

If five or more students raised their hands at the same time after a question was asked, it is obvious the question is too easy. I would then direct the "teacher" to ask another question.

After the first student volunteer asks two or three questions, other students are given chances to volunteer in this questioning process. Although students can move forward one step by asking questions in this game, there is no limit to the number of steps they can progress by answering questions.

Step 7: Grading

As the game progresses, the physical distribution of the students changes. Students who are willing to raise their hands and know the material move forward quickly. Those who are reluctant to raise their hands or cannot answer the questionstend to lag behind.

The goal of this test is to help students speak promptly in a way that is both grammatically correct an communicatively appropriate. Responses which are communicatively correct, though grammatically non-standard are rewarded half a point. Responses which are may represent grammatically correct sentences, but are completely inappropriate as replies are not rewarded.

By the time that all students have had an equal chance to ask questions as "teachers", the testing is complete and the grading process can begin. The distribution of a typical group at the close of the RTOI procedure appears in Figure 3.


S9 S5 S13 S3 S7 S11 S14 S4 S12 S10 S2 S15 S8 S16 S1 S6

Figure 3. A typical distribution of students at the close of the RTOI and one viable ranking scenario.

When the question/answer phase of the RTOI is complete, I ask students to look and their positions and consider what grades are appropriate. A point I emphasize in my testing as well as teaching is that the distinction between "student" and "teacher" roles is not rigid - at times I ask students to become "teachers" and at times I become a "student". Although I could easily take the initiative and assign an "A" to all those who have progressed to zone IV, a "B" to those in zone III, a "C" to those who have inched to zone II, and a "D" to those who are stuck in zone I, sometimes I engage in a process of negotiated grading with students. I regard grading as more than a matter of mathematical formulas. In the process of determining grades, I look for gaps in the distribution curve which suggest natural demarcation points, but also consider the impact that the grading process it is likely to have on students. Although there is a clear-cut demarcation between zones III and IV, the division between other zones is less distinct. There is a good chance that I would offer S10 the option of doing extra assignments to move into zone IV. I might even offer S9 the option of moving into zone II by completing some extra assignments. By challenging students to achieve more – and providing clear incentives for them to do so – I feel their learning experience is enhanced.

In this process of oral assessment, I am less interested in where students are than where they could be. Part of the pedagogical value of this test is in the way it can be used to offer challenges to students to apply themselves and progress further. Quite literally, students also know where they stand in relation to others in the class. If this test were viewed solely as a diagnostic measure, any sort of negotiation with students about their final outcomes could be interpreted as skewing the data unfairly. However, I consider the pedagogic role of this test more important than its diagnostic functions and therefore engage in a process of negotiated grading with students who are near the border of a higher zone, though not those who are firmly located in their zone. The educational and evaluative roles of some tests sometimes conflict. I feel the RTOI represents one way of fulfilling the need assign grades while also answering a need to design learning opportunities.
". . .this activity should be merely one component of an overall classroom evaluation progress: one of several instruments to capture aspects of student performance rather than the sole instrument for student evaluation."

Summary

I have also outlined one method for oral proficiency assessment. Although this activity has value in terms of reducing test anxiety and providing real time oral/aural feedback, there is a need to evaluate its reliability. This may not a test of English "ability" per se, but rather of the ability of students to respond to selected questions in front of a large audience in a way that the evaluator(s) think(s) is communicatively and grammatically appropriate/ Without question, this test favors students who are outgoing and willing to take risks. There is a likelihood that this test also favors those who are adept at the Japanese form of quick decision making, jan-ken-pon. Hence this activity should be merely one component of an overall classroom evaluation progress: one of several instruments to capture aspects of student performance rather than the sole instrument for student evaluation.

One thing which has been evident to me while teaching English in Japan is the anxiety many students have about testing. It would not be an understatement that most college students have "test phobia." Teachers are also likely to have mixed feelings about the testing process. Testing takes away from valuable class time and exams are often a burden to administer. However, in the RTOI students decide on the questions to be asked, collectively decide if the answers are correct, and even have a voice in the final grading. In this sense, the RTOI fulfills the criteria for "humanistic testing" proposed by Smith (1992) and is preeminently a student-centered assessment tool. For both teachers and students in large classrooms, the RTOI may be a one way to reduce the burden associated with oral proficiency testing.

References

Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.

Heltsley A. A., & Swanson, M. A. (1993). Conversational oral testing: A non-task based approach. Tokai University Foreign Language Education Center Journal. Vol. 13.

Laing, J. (1991, Nov 16). Testing as a form of weather prediction. Presentation to the Shizuoka Chapter of the Japan Association of Language Teachers. Shizuoka, Japan.

Smith, G. (1993, Oct 23). Testing oral ability: The hidden benefits. Presentation at the 18th Annual International Conference on Language Teaching and Learning sponsored by the Japan Association of Language Teachers. Tokyo International University, Kawagoe, Japan.