Face-to-face and Computer-mediated Peer Review in EFL Writing

Mei-ching Ho
The Arizona State University
Sandra J. Savignon
The Pennsylvania State University

Abstract:
This paper examines the use of face-to-face peer review (FFPR) and computer-mediated peer review (CMPR) in an Asian English-as-a-foreign-language (EFL) academic writing context. The participants were 33 English majors from a university of science and technology in Taiwan, a new type of school offering 2-year associate degree programs in foreign language studies. Our study contributes to the research on foreign-language-writing collaboration for Chinese learners in two important ways. First, many investigations of FFPR have looked at Chinese learners either in English as a second language (ESL) settings or at 4-year universities. Few have considered Chinese learners at 2-year colleges in EFL contexts. Second, there has been very little documentation of CMPR using annotation features in common word processing software in either ESL or EFL settings (Honeycutt, 2001). This study investigates the attitudes of 2-year college students in Taiwan toward the use of FFPR and CMPR in composition classes. Pedagogical implications are also drawn.

INTRODUCTION

Writing instruction in English as a second language (ESL) has seen considerable change following the influence of the process approach in the 1980s (Leki, 1992; Raimes, 1991; Reid, 1993; Zamel, 1987). This approach emphasizes writing as a process in which prewriting, multiple drafting, and revising are considered important in helping learners develop their skills. During multiple drafting, peer review is often used to afford learners experience in expression, interpretation, and negotiation of meaning. Such engagement in authentic communicative events offers an optimal opportunity for language learning (Savignon, 1983, 1997). Teachers typically ask learners to bring drafts of their writing to class where they then work in groups of two to four to read and comment on one another's writing. This exchange of feedback helps writers to clarify and give form to their meaning in subsequent drafts. The goal of peer review is to foster an atmosphere of reciprocal teaching between learners. The theoretical basis for the use of peer review in the development of writing skills, both in L1 and second language (L2) development, can be traced to (a) the notion of reciprocal teaching/scaffolding (Donato, 1994; Nystrand, 1986; Spear, 1987) and (b) the communicative approach to language teaching (CLT) (Elbow, 1981; Savignon, 1983, 1997, 2002). Second language writing pedagogy has felt the influence of both perspectives (Grabe & Kaplan, 1996; Reid, 1993). Communication, spoken and written alike, is the central focus of CLT and involves "a continuous process of expression, interpretation, and negotiation of meaning." (Savignon, 1997, p. 14; 2002, p. 1). The nature of peer review activities highlights this process. The basic insight of reciprocal teaching/scaffolding in the development of writing skills is that learners learn from one another by giving and receiving advice on the content, organization, and language use of their writing.

Computer-mediated peer review (CMPR) is a recent innovation in writing instruction. Along with the increasing availability of networked computers, computer-mediated communication (CMC) has gained importance in language teaching, especially in the teaching of composition. This study takes into consideration therefore both FFPR and CMPR.

LITERATURE REVIEW

Peer Review in Language Learning

Numerous reports on the use of peer review in both L1 and ESL settings have explored aspects of peer review activities. These include learners' reactions and negotiation patterns, teachers' roles in peer review training, the effects of peer response on learners' writing, and learners' attitudes and affective benefits (Berg, 1999; Mangelsdorf, 1992; Mendonca & Johnson, 1994; Nelson & Murphy, 1993; Paulus, 1999; Villamil & Guerrero, 1996; Zhang, 1995). However, the findings are generally inconclusive. While some studies report the positive effects of peer review (Berg, 1999; Mendonca & Johnson, 1994), others show contrary findings (Carson & Nelson, 1996, 1998; Zhang, 1995).

Mendonca and Johnson (1994) investigated peer negotiation in an ESL writing class at a major university in the northern United States. To examine the negotiation patterns of graduate student learners of English working in pairs, they analyzed audio-taped peer review sessions and learners' written drafts. In addition, posttask interviews were conducted to solicit learners' perceptions toward the usefulness of the peer review dyads. Five types of peer review negotiations were identified: asking questions, giving explanations, making restatements, offering suggestions, and correcting grammar. The analysis showed that during peer review learners focused on both local and global issues of their writing and that after negotiation they appeared to have a better understanding of its strengths and weaknesses. More important, learners developed audience awareness through peer review activities. The authors concluded that the learners in this study found peer review to be beneficial. In addition, peer review was found to "enhance students' communicative power by encouraging learners to express and negotiate their ideas" (1994, pp. 765-766).

Berg (1999) examined the effect of peer responses from ESL learners who received peer review training on revision types and writing quality. The results showed that revised drafts from trained peer response groups contained more macrolevel changes such as changes in the content and meanings. Learners who had received training achieved higher scores on their second drafts than did those who did not. These findings suggest that peer review used with guidance can help ESL learners to improve their writing. It may well be the case that individualism is advantageous when learners doing peer review are expected to state their own ideas and opinions. If so, to the extent that a group of learners may be said to share identifiable attributes, their different cultural backgrounds may need to be taken into account in judging reactions to peer review activities and feedback. In a subsequent study, Carson and Nelson (1998) looked specifically at cross-cultural issues in peer group interaction. Participants included three speakers of Chinese and eight speakers of Spanish from an advanced ESL writing class at a large US urban university. They found that both Spanish and Chinese learners valued the teacher's comments over those of their peers and preferred feedback "that identified problems in their drafts" (p.113). However, learners from different cultural backgrounds were seen to have very different types of interactions as well as views on peer review activities. During peer review sessions, Chinese ESL learners "frequently refrained from speaking because of their reluctance to criticize their peers, disagree with their peers, and claim authority as readers" (p. 127). While the Spanish learners' interactions were "task oriented," the Chinese focused on "maintain[ing] group harmony" (p.127). In other words, Carson and Nelson found the Spanish learners to be more active in pointing out the problems in their peers' writing for revision. The Chinese learners on the other hand tended not to criticize others' work and not to disagree with them. Accordingly, they concluded that peer response groups in a classroom in which the majority of learners are from what they considered to be "collectivist" cultures may not be as effective as those in a setting where learners are from cultures that value individualism.

Computer-assisted Language Learning

The trend toward computer-assisted language learning (CALL) and computer-mediated communication (CMC) in language learning have influenced the teaching of writing in both L1 and L2 settings. More and more writing teachers have started conducting classes in networked computer labs or incorporating writing activities that engage learners with the use of computers (Warschauer, 1996). Computer-mediated peer review (CMPR), for instance, is one of the activities used to enhance learners' L2 writing. With networked computers, learners can do peer reviews online anywhere at anytime. Learners exchange drafts and feedback via email and communicate with one another through the use of interactive software programs. Not only can CMPR offer flexibility for both teachers and learners, it may also reduce psychological pressure on learners who do not like to give feedback in face-to-face situations. Depending on the kind of software used, CMPR could be used synchronously during regular class time or asynchronously at the convenience of the learners (see DiGiovanni & Nagaswami, 2001; Savignon & Roithmeier, 2004).

To date, several studies have been conducted to examine CMPR in L1 and ESL writing classes in which researchers assert the advantages of CMPR. (Brown, Nielson, & Sullivan, 1998; Dean, 2001; DiGiovanni & Nagaswami, 2001; Skinner & Austin, 1999). Among them is an increase in learner motivation due to the new medium of expression. Colomb and Simutis (1996), Kelm (1992), Kern (1995), and Warschauer (1996) have all reported that the use of computers increases learners' motivation in second or foreign language writing.

Although research shows CMPR may have positive effects on learner motivation, it is worth noting that CMC is very different from face-to-face communication and could generate different types of interactions which might affect the effectiveness and quality of feedback. As noted above, CMPR may be conducted synchronously (for example, chat, ICQ, MUDs, etc.) or asynchronously (email, listserv discussion list, etc.), depending on the software used. The medium used for communication could affect language and language use in terms of style, coherence, etiquette, message length, and other features (Herring, 2002a).

RESEARCH QUESTIONS

This study examines the use of face-to-face and asynchronous computer-mediated peer review in academic writing classes at a university of science and technology (UST) in Taiwan offering 2-year associate degree programs in intensive language study.² In recent years, the number of learners enrolled in this new type of school has increased dramatically. The goal of the study reported here was to investigate the perceptions and attitudes of learners in this particular setting toward FFPR and CMPR and to determine whether CMPR as used in this study might be a good supplement to FFPR in writing classes. The specific research questions are the following:

1. How do learners in a UST react to face-to-face and computer-mediated peer review?

1.1 Do learners in a UST feel they benefit from doing face-to-face peer review?

1.2 Do learners in a UST have generally favorable attitudes toward computer-mediated peer review?

2. What are the learners' perceptions concerning the annotation features of CMPR used in this study?

METHOD

Setting and Participants

The participants were 37 English majors, 12 males and 25 females, at a national university of science and technology in Taiwan who comprised two intact classes in academic writing taught by two different teachers in the spring 2002. Learners who enroll in this kind of program have heterogeneous English learning backgrounds, but, to be admitted to the program, all must pass a nationally standardized entrance exam which evaluates their English proficiency. Therefore, although learners' English learning backgrounds may differ, the learners in this study had attained a similar level of language study. While most traditional English departments at universities in Taiwan focus on English literature, the English department in this particular setting emphasizes the teaching of English for practical use. The primary objective of the program is to prepare learners for working in an environment in which English is the primary means of communication. There are three major tracks in the program: business and technical English, English interpretation and translation, and English language teaching. Advanced Writing I, II, III, and IV are required courses for students in all three tracks. Each level requires one semester to complete and has different themes in different tracks. The department incorporated peer review activities as one of the assignments for all levels of the writing courses.

Of the total of 37 participants, 18 were from a senior writing class (IV) taught by an English native speaker, and 19 were from a junior writing class (II) taught by a nonnative speaker of English with a Master's degree in TESOL from an American university. Although the focus of instruction for the two classes differed somewhat, both teachers used peer review to enhance learners' writing skills. In the senior writing class, students learned academic writing skills, including paraphrasing, quoting, and synthesizing outside sources. The students were required to complete two research papers during the semester. The teacher in the junior class structured the class based on instruction in different writing modes or genres (e.g., comparison and contrast, definition, classification, and argumentation). The students in this class were required to write short essays on assigned or self-selected topics.

Procedure

The researchers first sought approval for doing this study from the chair of the English department at the UST. They then informed instructors of junior or senior academic writing classes via email of the purpose of this research. Two instructors in the department were willing to help by seeking volunteers in their classes. Neither instructor had ever used computer-mediated peer review in their instruction and were interested in trying it for a semester. During regular class hours, the teachers told the EFL learners of the purpose of the study, requesting voluntary participation. All of the learners in both classes agreed to participate and were assured that their expression of personal preferences for any type of peer review mode in the survey would in no way affect their grades.

Face-to-face peer review

Since both instructors had had learners do face-to-face peer review (FFPR) before the time this study was conducted, the researchers interviewed both instructors via email to better understand the overall context and how FFPR sessions had been conducted. In both classes, learners were required to write three drafts for each writing assignment. Most often, teachers asked learners to bring in their first draft and paired learners for the exchange and review of each other's drafts. In reviewing the first two drafts, learners were asked to focus on global features of their peer's text (e.g., content, organization, and coherence). Afterwards, the learners themselves checked the local features of their own text before submitting a final draft. In the junior class, learners selected their own partner. However, the teacher encouraged learners to work with a different person for each peer review session during the semester. Learners in the senior class, on the other hand, were assigned to a partner. According to the instructor, learners' ability level was the most important criterion for selecting a partner for each learner.

Computer-mediated peer review

Since none of the learners had had prior experience with computer-mediated peer review (CMPR), researchers cooperated with instructors to set up the activity. In the senior class, most regular class hours were used for one-on-one teacher-learner conferences in the instructor's office; learners did not typically work together in class. Therefore, the instructor suggested having learners do CMPR outside of the class as an assignment. Since CMPR in this study involved only email and simple annotation features in common word-processing programs, learners could easily complete their tasks asynchronously at home or in the computer lab at school. To guide them initially, the instructor asked learners to email their drafts as an attachment and provide their feedback directly on their peer's computer draft using "Track Changes" in Word as shown in the examples below. Learners then emailed the draft with feedback to the peer. In the junior-level class, the teacher agreed to adopt CMPR during regular class sessions, scheduling them in a networked computer lab. The procedures of CMPR were the same as those used in the senior class. Both classes adopted CMPR during the second half of the semester. During CMPR, the learners read the drafts and typed the feedback on the computer. There was no exchange of printouts or paper-based feedback.

Data Collection and Analysis

To investigate the EFL learners' perceptions of both FFPR and CMPR, a questionnaire was developed that included three parts: a biographical section, 30 items using a 5-point Likert scale (5 indicating strongly agree and 1 strongly disagree), and 5 open-ended questions. The biographical section asked for learners' previous major and English learning experience. The 30 Likert-scale items were designed to measure learners' attitudes toward both peer review modes as well as specific features of CMPR, including "Track Changes," and "reading drafts and offering feedback on the computer."

The two EFL instructors distributed the questionnaires to the participants together with an informed consent form. All 37 participants remained anonymous. Since both instructors allowed learners to fill out the questionnaire during the regular class hours or teacher-student writing conferences, there was a 100% return rate. However, four questionnaires had incomplete responses and were excluded from data analysis.

RESULTS

The reliability coefficients for the 30 Likert-scale items are presented in Table 1. According to Sax (1989), a reliability coefficient of more than .6 is required for a self-designed text or survey. The CMPR and features (including Track Changes, font color, and spelling and grammar checks) of CMPR scales achieved alphas of .69 and .71, respectively. The alpha for the FFPR scale was a less than satisfactory .50. The FFPR was the scale with the fewest number of items. The coefficient could perhaps have been improved if the number of items in this scale had been increased (Henning, 1987).

Learners' Perceptions of Peer Review

The descriptive statistics in Table 2 show learners had favorable attitudes toward face-to-face peer review (M = 3.95, SD = 0.41, t = 13.29, p < 0.01). Items 1, 9, and 25 had a mean score higher than 4, indicating a high agreement among learners. More than 82% of learners agreed that peer review helped them improve their writing and should be used in writing classes. In addition, as many as 92% of the learners agreed that they benefited from reviewing their peer's writing. This last finding may reflect what Grabe and Kaplan (1996), Reid (1993), and Spear (1987) have argued, namely, that peer review can boost learners' confidence as a second language writer because they can see other learners also make mistakes and struggle with the writing.

Learners' Reaction to Features of CMPR

The features of CMPR examined in this study include: exchanging drafts via email, Track Changes, font colors, spelling and grammar checks, the nature of typed feedback on the drafts, and reading drafts on the computer. Thirteen items were designed to investigate what features learners found helpful when doing CMPR and what features they found difficult to use.

Face-to-face peer review

The major advantage learners associated with FFPR was that they were able to talk with peers during the review session, which enabled them to seek clarification and negotiate meanings to avoid misunderstanding. Most learners found that this not only eased the peer review process but that it also made the peer review more effective because many found speaking more efficient than writing. One learner wrote, "I can discuss my problems with my peers in detail. Through this face-to-face process, I always get useful suggestions from my peer." In addition, some learners mentioned that when encountering complicated ideas or disagreement, they liked to discuss in their native language. A senior learner wrote, "My partners and I can directly point out each other's weak points in our writings. Oral feedback is more efficient to me than written feedback. When I felt it's difficult to talk in English. We preferred to speak Chinese … ."

Computer-mediated peer review

Learners reported two major advantages of CMPR. First, CMPR offered more flexibility than FFPR. Many indicated that since they and their partner did not need to be logged on to the computer at the same time, they could read and comment when convenient and at their own pace. Learners were able to take time to reflect on their ideas and rehearse responses to their partner. The use of "Track Changes" also made the reviewing process easier. Learners could edit and revise their feedback easily, and most learners found typing more convenient than handwriting. One learner wrote: "I think the benefits are as follows: it's easy to change any points I think not appropriate. It's easy to add whatever I want to express. It's clear to see my point to my peer's paper." It seems that being able to edit feedback on the computer was a real benefit for learners since typed feedback could be changed anytime before being forwarded to the author.

DISCUSSION

Responses to the Likert-scale items suggested that although learners accepted both peer review modes, they had more favorable attitudes toward FFPR than CMPR. Although learners' attitudes were more favorable toward FFPR than toward CMPR, many found it somewhat stressful to review others' work face to face. They were afraid that in pointing out their peer's problems they might hurt the other's feelings or even damage their friendship. Some said that they would not critique and identify problems directly in order to maintain a friendly relationship. These findings corroborate those of Carson and Nelson (1996, 1998) that Chinese learners often avoid discussing problems and disagreeing with peers in order to maintain harmony. However, more research is needed to know whether this is a uniquely Chinese trait or whether learners from Western cultures behave in the same way. The second author of this research report has gathered substantial anecdotal evidence of peer pressure to maintain harmony and consensus among 18- to 20-year-old university students in the US, whether in full class, small group, or one-on-one discussion of each other's writing. In addition, it should be noted that proper training is important before having learners do peer review. Teachers should explain the purpose of peer review and let learners know that their responsibility is to offer honest feedback in an effort to help their peers. In this way, learners may feel less pressure while pointing out problems in their peers' writing. The fact that most learners in this study found FFPR helpful suggests that a concern for harmony did not prevent the activity from being productive. Most learners had positive attitudes toward peer review in general and acknowledged its value in helping them to revise subsequent drafts. Many suggested that they would like to do CMPR first and then have face-to-face discussion with a peer. By doing so, they could read and comment on each other's draft at their own pace and also have a chance to ask questions and clarify ideas. They saw a combination of the two modes as a way to make peer review activities more efficient and effective. The present study yields encouraging results. It shows peer review to be a highly communicative language activity that can be used successfully in an EFL context in Taiwan. The learners in this study not only had favorable attitudes toward theexperience, they found peer review helped them with their writing. These findings do not support the suggestion that peer review may not work well for learners with a Chinese cultural background (Carson & Nelson 1996; 1998). Moreover, research findings related to learners in ESL contexts might not be an accurate representation of peer review in EFL contexts. Although the learners in this study reported experiencing pressure in pointing out problems in one another's writing, the advantages offered by peer review activity in general seemed to outweigh the drawbacks. Peer review appears to be a means of increasing learner communication in writing classes, offering ideas and strategies for revision, and promoting a sense of discourse community (Grabe & Kaplan, 1996; Nelson & Murphy, 1992; Spear, 1987). Most important, peer review may reduce learners' apprehension of writing in a foreign language and increase their confidence as EFL writers.