56 Вопрос
Reasons for testing students
At various stages during their learning, students may need or want to be tested on their ability in the English language. If they arrive at a school and need to be put in a class at an appropriate level, they may do a placement test. This often takes the form of a number of discrete (indirect) items (see below), coupled with an oral interview and perhaps a longer piece of writing. The purpose of the test is to find out not only what students know, but also what they don't know. As a result, they can be placed in an appropriate class.
At various stages during a term or semester, we may give students progress tests. These have the function of seeing how students are getting on with the lessons, and how well they have assimilated what they have been taught over the last week, two weeks or a month.
At the end of a term, semester or year, we may want to do a final achievement test (sometimes called an exit test) to see how well students have learnt everything. Their results on this test may determine what class they are placed in next year (in some schools, failing students have to repeat a year), or may be entered into some kind of school-leaving certificate. Typically, achievement tests include a variety of test types and measure the students' abilities in all four skills, as well as their knowledge of grammar and vocabulary.
Many students enter for public examinations such as those offered by the University of Cambridge ESOL, Pitman or Trinity College in the UK, and in the US, the University of Michigan and TOEFL and TOEIe. These proficiency tests are designed to show what level a student has reached at anyone time, and are used by employers and universities, for example, who want a reliable measure of a student's language abilities.
So far in this chapter we have been talking about testing in terms of 'one-off' events, usually taking place at the end of a period of time (except for placement tests). These 'sudden death' events (where ability is measured at a particular point in time) are very different from continuous assessment, where the students' progress is measured as it is happening, and where the measure of a student's achievement is the work done all through the learning period and not just at the end. One form of continuous assessment is the language portfolio, where students collect examples of their work over time, so that these pieces of work can all be taken into account when an evaluation is made of their language progress and achievement. Such portfolios (called dossiers in this case) are part of the CEF (Common European Framework), which also asks language learners to complete language
passports (showing their language abilities in all the languages they speak) and language biographies (describing their experiences and progress).
There are other forms of continuous assessment, too, which allow us to keep an eye on how well our students are doing. Such continuous recording may involve, among other things, keeping a record of who speaks in lessons and how often they do it, how compliant students are with homework tasks and how well they do them, and also how well they interact with their classmates.
Some students seem to be well suited to taking progress and achievement tests as the main way of having their language abilities measured. Others do less well in such circumstances and are better able to show their abilities in continuous assessment environments. The best solution is probably a judicious blend of both.
Good tests
Good tests are those that do the job they are designed to do and which convince the people taking and marking them that they work. Good tests also have a positive rather than a negative effect on both students and teachers.
A good test is valid. This means that it does what it says it will. In other words, if we say that a certain test is a good measure of a student's reading ability, then we need to be able to show that this is the case. There is another kind of validity, too, in that when students and teachers see the test, they should think it looks like the real thing - that it has face validity. As they sit in front of their test paper or in front of the screen, the students need to have confidence that this test will work (even if they are nervous about their own abilities). However reliable the test is (see below) face validity demands that the students think it is reliable and valid.
A good test should have marking reliability. Not only should it be fairly easy to mark, but anyone marking it should come up with the same result as someone else. However, since different people can (and do) mark differently, there will always be the danger that where tests involve anything other than computer-scorable questions, different results will be given by different markers. For this reason, a test should be designed to minimise the effect of individual marking styles.
When designing tests, one of the things we have to take into account is the practicality of the test. We need to work out how long it will take both to sit the test and also to mark it. The test will be worthless if it is so long that no one has the time to do it. In the same way, we have to think of the physical constraints of the test situation. Some speaking tests, especially for international exams, ask not only for an examiner but also for an interlocutor (someone who participates in a conversation with a student). But this is clearly not practical for teachers working on their own.
Tests have a marked washback/backwash effect, whether they are public exams or institution-designed progress or achievement tests. The washback effect occurs when teachers see the form of the test their students are going to have to take and then, as a result, start teaching for the test. For example, they concentrate on teaching the techniques for answering certain types of question rather than thinking in terms of what language students need to learn in general. This is completely understandable since teachers want as many of their students as possible to pass the test. Indeed, teachers would be careless if they did not introduce their students to the kinds of test item they are likely to encounter in the exam. But this does not mean that teachers should allow such test preparation to dominate
their lessons and deflect from their main teaching aims arid procedures.
The washback effect has a negative effect on teaching if the test fails to mirror our teaching because then we will be tempted to make our teaching fit the test, rather than the other way round. Many modern public examinations have improved greatly from their more traditional versions, so that they often do reflect contemporary teaching practice. As a result, the washback effect does not have the baleful influence on teaching which we have been discussing.
When we design our own progress and achievement tests, we need to try to ensure that we are not asking students to do things which are completely different from the activities they have taken part in during our lessons. That would clearly be unfair.
Finally, we need to remember that tests have a powerful effect on student motivation.
Firstly, students often work a lot harder than normal when there is a test or examination in sight. Secondly, they can be greatly encouraged by success in tests, or, conversely, demotivated by doing badly. For this reason, we may want to try to discourage students from taking public examinations that they are clearly going to fail, and when designing our own progress and achievement tests, we may want to consider the needs of all our students, not just the ones who are doing well. This does not mean writing easy tests, but it does suggest that when writing progress tests, especially, we do not want to design the test so that students fail unnecessarily - and are consequently demotivated by the experience.
Test types
When designing tests, we can either write discrete items, or ask students to become involved in more integrative language use. Discrete-item testing means only testing one thing at a time (e.g. testing a verb tense or a word), whereas integrative testing means asking students to use a variety of language and skills to complete a task successfully. A further distinction needs to be made between direct and indirect test items. A direct test item is one that asks students to do something with language (e.g. write a letter, read and reply to a newspaper article or take part in a conversation). Direct test items are almost always integrative. Indirect test items are those which test the students' knowledge of language rather than getting them to use it. Indirect test items might focus on, say, word collocations (see page 75) or the correct use of modal verbs (see page 69). Direct test items have more to do with activation, whereas indirect items are more closely related to study- that is the construction of language.
Indirect test items
There are many different ways of testing the students' knowledge of language construction. We will look at three of the most common.
Multiple choice
Multiple-choice questions are those where students are given alternatives to choose from, as in the following example:
Sometimes students are instructed to choose the 'correct' answer (because only one answer is possible), as in the example above. But sometimes, instead, they can be told to choose the 'best' answer (because, although more than one answer is possible, one stands out as the most appropriate), e.g.
Circle the best answer.
Police are worried about the level of crime.
a juvenile b childish c young d infant
Multiple-choice questions have the great advantage of being easy to mark. Answer sheets can be read by computer, or can be marked by putting a transparency over the answer sheet which shows the circled correct letters. Markers do not have to worry, then, about the language in the questions; it is simply a matter of checking the correct letters for each question.
One problem with multiple-choice questions lies in the choice of distractors, that is the three incorrect (or inappropriate) answers. For while it may not be difficult to write one obvious distractor (e.g. answer a 'to get' in the first example above), because that is a mistake that students commonly make, it becomes less easy to come up with three items which will all sort out those students who know how this piece of language works from the ones who don't. In other words, there is a danger that we will either distract too many students (even those who should get the question right) or too few (in which case the question has not done its job of differentiating students).
Multiple-choice questions can be used to test reading and listening comprehension (we can also use true/false questions for this: students circle 'T' or 'F' next to statements concerning material they have just read or listened to).
The washback effect of multiple-choice questions leads some people to find them unattractive, since training students to be good at multiple-choice questions may not help them to become better language learners. And there is a limit to how much we can test with this kind of indirect item. Nevertheless, multiple-choice questions are very attractive in terms of scorer reliability.
Fill-in and doze
This extremely common form of indirect testing involves the examinee writing a word in a gap in a sentence or paragraph, e.g.
Yesterday I went a __ the cinema b __ my friend Clare. I enjoyed the film c __ she did not.
Gap-fill (or fill-in) items like this are fairly easy to write, though it is often difficult to leave a gap where only one item is possible. In such cases, we will need to be aware of what different answers we can accept. They also make marking a little more complex, though we
can design answer sheets where students only have to write the required word against different letters, e.g.
A variation on fill-ins and gap-fills is the doze procedure, where gaps are put into a text at regular intervals (say every sixth word). As a result, without the test writer having to think about it too much, students are forced to produce a wide range of different words based on everything from collocation to verb formation, etc, as in the following example.
All around the world, students a __ all ages are learning to b __ English, but their reasons for c__ to study English can differ d __. Some students, of course, only
e __ English because it is on f curriculum at primary or secondary g __ ,
but for others, studying the h reflects some kind of a i __ .
The random selection of gaps (every sixth word) is difficult to use in all circumstances. Sometimes the sixth word will be impossible to guess - or will give rise to far too many alternatives (e.g. gaps c and d above). Most test designers use a form of modified doze to counteract this situation, trying to adhere to some kind of random distribution (e.g. making every sixth word into a blank), but using their common sense to ensure that students have a chance of filling in the gaps successfully - and thus demonstrating their knowledge of English.
Transformation
In transformation items students are asked to change the form of words and phrases to show their knowledge of syntax and word grammar. In the following test type they are given a sentence and then asked to produce an equivalent sentence using a given word:
Rewrite the sentence so that it means the same. Use the word in bold Could I borrow five pounds, please?
lend
In order to complete the item successfully, the students not only have to know the meaning of borrow and lend, but also how to use them in grammatical constructions.
A variation of this technique is designed to focus more exactly on word grammar. Here, students have to complete lines in a text using the correct form of a given word, e.g.
It was a terryfying performance. terrify
The acrobats showed no fear even though absolute
their feats of shocked the crowd into stunned silence. dare
These kinds of transformations work very well as a test of the students' underlying knowledge of grammar and vocabulary. However, the items are quite difficult to construct.
There are many other kinds of indirect test item. We can ask students to put jumbled words in order, to make correct sentences and questions. We can ask them to identify and correct mistakes or match the beginnings and ends of sentences. Our choice of test item will depend on which, if any, of these techniques we have used in our teaching since it will always be unfair to give students test items unlike anything they have seen before.
Direct test items
In direct test items, we ask students to use language to do something, instead of just testing their knowledge of how the language itself works. We might ask our students to write instructions for a simple task (such as using a vending machine or assembling a shelving system) or to give an oral mini-presentation.
There is no real limit to the kinds of tasks we might ask students to perform. The following list gives some possibilities:
Reading and listening
Some reading and writing test items look a bit like indirect items (e.g. when students are given multiple-choice questions about a particular word in a text, for example, or have to answer T IF questions about a particular sentence). But at other times we might ask students to choose the best summary of what they have heard or read. We might ask them to put a set of pictures in order as they read or listen to a story, or complete a phone message form (for a listening task) or fill out a summary form (for a reading task).
Many reading and listening tests are a blend of direct and indirect testing. We can ask students direct language - or text-focused - questions as well as testing their global understanding.
Writing
Direct tests of writing might include getting students to write leaflets based on information supplied in an accompanying text, or having them write compositions, such as narrative and discursive essays. We can ask students to write 'transactional letters' (that is letters replying to an advertisement, or something they have read in the paper, etc). In transactional writing we expect students to include and refer to information they are given.
Speaking
We can interview students, or we can put them in pairs and ask them to perform a number of tasks. These might include having them discuss the similarities and differences between two pictures (see information-gap activities on page 129); they might discuss how to furnish a room, or talk about any other topic we select for them. We can ask them to roleplay certain situations (see page 125), such as buying a ticket or asking for information in a shop, or we might ask them to talk about a picture we show them.
When designing direct test items for our students, we need to remember two crucial facts. The first is that, as with indirect tests, direct tests should have items which look like the kind of tasks students have been practising in their lessons. In other words, there is no point in giving students tasks which, because they are unfamiliar, confuse them. The result of this will be that students cannot demonstrate properly how well they can use the language, and this will make the test worthless.
Direct test items are much more difficult to mark than indirect items. This is because our response to a piece of writing or speaking will almost certainly be very subjective -
unless we do something to modify this subjectivity. We will now go on to look at how this can be done.