LANGUAGE IN INDIA

The Effect of Text Authenticity on
the Performance of Iranian EFL Students in a C-Test

Amir Biglar Beigi
Karen Kow Yip Cheng
Masood Solaymani

As part of growing efforts to understand factors affecting c-test this study aims to investigate the effect of text authenticity on the performance of Iranian EFL students in a C-Test. The C-Test is an integrative testing instrument that measures overall language competence, very much like the cloze test. In this study the rule of two has been applied: the second half of every second word has been deleted, beginning with the second word of the second sentence; the first and last sentences are left intact Katona & Dornyei 1993, p.35).

The research involves 60 college students in their third year, majoring in English Literature were randomly selected applying multi-stage sampling. Since the present study intended to investigate the role of two different formats, i.e. authentic and inauthentic texts (text translated from Persian into English), two different tailored C-Tests were made to measure and compare the performances of the participants. Two C-Tests, one with Authentic Text and the other, with Inauthentic Text were administered to this homogenized group comprising 30 subjects.

Introduction

The present study was conducted to investigate the effect of text authenticity on the performance of Iranian EFL college students in a C-Test. The C-Test is an integrative testing instrument that measures overall language competence, very much like the cloze test. In this study the rule of two has been applied: the second half of every second word has been deleted, beginning with the second word of the second sentence; the first and last sentences are left intact Katona &Dornyei 1993, p.35). 60 college students in their third year, majoring in English Literature were randomly selected applying multi-stage sampling in which first a national sample of college districts in Iran stratified by major was obtained. Later on, within selected districts, Tehran colleges were selected using simple random sample technique and within colleges in Tehran, Ershad-Damavand College was selected. Further, within English Literature classes of this college, a simple random sample of students was carried out (Trochim, 2006).

To ensure the homogeneity of the subjects in this study an IELTS exam was carried out and 30 students having scored one standard above and below the mean were selected for the purposes of this study. Since the present study intended to investigate the role of two different formats, i.e. authentic and inauthentic texts (text translated from Persian into English), two different tailored C-Tests were made to measure and compare the performances of the participants. Two C-Tests, one with Authentic Text and the other, with Inauthentic Text were administered to this homogenized group comprising 30 subjects.

Data were analyzed through matched t-test formula due to having one group of students with two different tasks and hoping to compare their performance on the tasks which are inauthentic and authentic C-Tests. The findings indicated a significant relationship between text authenticity and the performances of the participants in the C-Tests.

Founded on the results of this study, one of the most crucial aspects of test method in C-Tests can be regarded as text authenticity (native/target language presentation).This aspect of test method has been recognized by Bachman (1990) as a crucial aspect of test method and the present study provides evidence for such an effect caused by the nature of language presentation on C-Tests.

The C-Test

The C-Test is a teacher friendly way of testing language proficiency. In the 1980s this measuring tool for placement testing was introduced (Katona &Dornyei 1993). The C-test was developed in Duisburg by Christine Klein-Braley and Ulrich Raatz. Although it is known, it is not widely applied. This is due to the fact that, to some extent, the quality of the test depends on the input of the texts. (Baten & Leuven, 2004)

The C-test is an integrative testing instrument that measures overall language competence, very much like the cloze test. It consists of four to six short, preferably authentic texts in the target language, to which "the rule of two" has been applied. The rule of two is where the second half of every second word is deleted, beginning with the second word of the second sentence; the first and last sentences are left intact. If a word has an odd number of letters, the "bigger" part is omitted, e.g., proud becomes pr-. One-letter words, such as I, are ignored in the counting. The students' task is to restore the missing parts. In a typical C-Test there are 100 gaps, that is, missing parts. Only entirely correct restorations are accepted (Klein-Braley, 1994 & 1997). An example of such a C-test is shown below:

"One cool autumn evening, Bob L., a young professional, returned home from a trip to the supermarket to find his computer gone. Gone! All so- of cr- thoughts ra- through h- mind: H- it be- stolen? H- it be- kidnapped? H- searched h- house f- a cl- until h- noticed a sm- piece o- printout pa- stuck un- a maga- on h- refrigerator do-. His he- sank a- he re- this sim- message: can't continue, file closed, bye." ( Katona &Dornyei 1993, p. 35)

Selection of Texts: The Nature of Input

The C-Test comprises four to six short authentic texts, each complete as a sense unit in itself. Since C-Tests consist of different short texts, the selection of these texts (short passages) or input is of the great importance and has been the subject of various investigations in order to make these tests more appropriate and less biased (McBeath, 1990).

Bachman (1990) proposed the test method facets in the book entitled "Fundamental Considerations in Language Testing" The test method facets are the variables affecting the overall performance of the testee and should be taken into account while constructing a test and interpreting the results of the test (Kobayashi, 2002). Bachman (1990) classifies test method facets into five categories: 1) testing environment; 2) test rubrics; 3) the nature of the input; 4) the nature of the expected response; and 5) the interaction between the input and the response.

According to Bachman, these factors can affect test performance; it is important for testers to be aware of their influences and, if possible, minimize them. This study focuses on the third of these facets. 'The nature of the input', that is, the materials presented to test takers. This was chosen as the main variable of the study because reading materials constitute a very important factor in reading comprehension tests. Background knowledge, for example, is a well-researched area (see for example: Alderson and Urquhart 1983, 1985a, 1985b; Bernhardt 1991; Carrell and Eisterhold 1983; Clapham 1996; Johnson 1981, 1982; Mohammed & Swales 1984; Salager-Meyer 1991; Steffensen and Joag-Dev 1984; Steffensen , Joag-Dev & Anderson, 1979; Ulijn and Strother 1990).

Of the various factors, the choice of texts with a focus on authentic versus inauthentic was chosen for this investigation. This decision was based on an extensive literature survey of previous studies on text characteristics, naturalness of the text and readability (e.g. a series of studies by Beck and his colleagues, e.g. 1982, 1984, 1989, 1991, 1995; Britton, Van Dusen, Gulgoz, & Glynn, 1989; Davison and Kantor 1982; Duffy and Kabance 1982; Duffy et al., 1989; Graves et al., 1988 & 1991; Klare 1985; Olsen and Johnson 1989; Reder and Anderson 1980; Urquhart, 1984).

Followed by Bachman's model (1990 & 1995), a large number of studies such as Kobayashi (2002a, 2004, 2005) have endorsed the method facets in reading comprehension tests and cloze tests. Most of the studies manipulated "the nature of the input" and "the nature of the expected response" while the former refers to the materials presented to test takers i.e. authentic/inauthentic, text structure ,etc. the latter suggests the type of answers you expect from the testee, i.e. open-ended questions, cloze tests, etc.

For instance, Kobayashi (2004, 2005) maintains that there is a systematic relationship between the students' test performance and the effects of and the 'nature of the expected response', while the nature of input in testing refers to 'test organization' and 'test method characteristics' , Kobayashi (2002a, p.197) hypothesized that different 'text types' in reading comprehension tests can impinge on the test organization and thereupon the test takers' performance. Likewise, Harrison and Salinger (1998) alongside Koberl and Sigott's (1994) restate the significant role of text type to gauge reading comprehension ability of learners through regular reading tests.

From another perspective, there are a large number of empirical studies dealing with text authenticity and translation. Studies such as Sigott (1995), Gellerstam (1996), Laviosa-Braithwaite (1997) and Mauranen (2000) which carry in substance the same theme suggest that there authentic and translated texts relative drastically differ in number of lexical, grammatical and even pragmatic elements. For instance, Mauranen's (2000) findings on the relative under representation of discourse markers with highly TL-specific pragmatics can also be taken to support the general idea of inauthentic texts deficiency or as Sigott (1995, p.47) stated, "translation creates different texts". Furthermore, these differences can denote Widdowson's (1983) notion of 'Authenticity Stimulus' in. Widdowson comments on the notion of "authenticity" as follows:

"An authentic stimulus in the form of attested instances of language does not guarantee an authentic response in the form of appropriate language activity .... we should retain the term 'authenticity' to refer to activity (i.e. process) and use the term 'genuine' to refer to attested instances of language (i.e. product)" (ib., p.30)

Widdowson (1983) highlights the fact that the very notion of authenticity is open to debate. An authentic stimulus as he points out may well lead to an inauthentic response. In other words, the term 'authentic' is a term which is ambiguous and may lead to further ambiguity.

Widdowson, therefore, recommends that the term be used to refer to attested instances of language produced by native speakers for a normal communicative purpose. In this sense it refers to naturalistic textual data. But the term can be used, quite legitimately, to refer to the communicative activity of the language user, to the engagement of interpretative procedures for making sense, even if these procedures are operating on and with textual data which are not authentic in the first place.

As pointed out earlier an authentic stimulus in the form of attested instances of language does not guarantee an authentic response in the form of appropriate language activity. At this juncture It should be emphasized that in this study 'authentic' texts were chosen based on Widdowson's interpretation mentioned earlier in this section.

The Present Study

The present study sets out to find whether in c-tests converting text type from authentic to inauthentic produce any effect on testees' performance. This investigation scaffolds on Bachman's (1995) contention concerning test method characteristics (facets). It is by Bachman contended that factors affecting the test-taker's performance are multifaceted and thence crucially important for test designers to be attentive to the possible effects of these factors and give utmost attention to minimize these effect to a feasible extent and thus elicit the actual performance of the testee.

It is noteworthy that c-tests were, initially, developed to overcome the perceived weaknesses of cloze test. The C-test was developed as a modification of the cloze test which is a widely used language proficiency testing instrument. The cloze test is more favored due to the ease of constructing the test coupled with its high reliability and validity. Unlike the C-test, the cloze test consists of a longer text of which every fifth to tenth complete word is left out (Klein-Braley and Raatz, 1984).

This is only the beginning part of the article. PLEASE CLICK HERE TO READ THE ARTICLE IN PRINTER-FRIENDLY VERSION.

Levels of Politeness in Malaysian Parliamentary Discourse | Grammatical Influence of Telugu on Hyderabad Tamil | Separated by a Common Language - Asian Students Writing in English | Modality, Mood and Modal Auxiliaries: A Critique | The Impact of Translation Method On Word Meaning And Fill-In-the-Blank Tests Procedures on Short-Term and Long-Term Retention of Vocabulary Items | Proverbs in Tamil and Telugu | Chandra Lekha in He who Rides a Tiger by Bhabani Bhattacharya | A Literary Study of the Parables in the Gospels of the Bible | Can Hurdles be Overcome by Learners of ESL in Learning to Speak English? | A Strategy-based Scheme for Promoting Vocabulary Retention among Language Learners | The Effect of Text Authenticity on the Performance of Iranian EFL Students in a C-Test | On Interrogating Language and Cognition | Towards Education Reforms - Decolonizing English Studies in India | Girish Karnad's Yayati - A Tale of Malcontent All Around | HOME PAGE of September 2009 Issue | HOME PAGE | CONTACT EDITOR