Computers & Education 53 (2009) 799–808 Contents lists available at ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu Investigating readers’ mental maps of references in an online system Yu-Fen Yang a,*, Wing-Kwong Wong b, Hui-Chin Yeh a a b Graduate School of Applied Foreign Languages, National Yunlin University of Science and Technology, 123 University Road Section 3, Douliu, Yunlin, Taiwan, ROC Department of Electronic Engineering, National Yunlin University of Science and Technology, 123 University Road Section 3, Douliu, Yunlin, Taiwan, ROC a r t i c l e i n f o Article history: Received 16 December 2006 Received in revised form 24 April 2009 Accepted 27 April 2009 Keywords: Evaluation of CAL systems Reading strategy Mental map Reference Interactive learning environments a b s t r a c t Referential identiﬁcation and resolution are considered the keys to help readers grasp the main idea of a text and solve lexical ambiguities. The goal of this study is to design a computer system for helping college students who learn English as a Foreign Language (EFL) develop mental maps of referential identiﬁcation and resolution in reading. Four modules, Natural Language Processing (NLP), User Interface, Recording, and Feedback Tool, are implemented in the system. Results of this study showed that the more-proﬁcient EFL readers were able to identify and resolve most of the references to form a coherent mental map from different parts of a text. The less-proﬁcient readers commonly resolved references by relying on grammatical rules instead of semantic contextual clues. They often referred references to incorrect objects. To overcome the difﬁculties in ﬁguring out the relationship between two words, the less-proﬁcient readers usually asked for more feedbacks. As students progressed in reading, they requested fewer feedbacks in the online system. Some recommendations for future studies are discussed. Ó 2009 Elsevier Ltd. All rights reserved. 1. Introduction Successful reading comprehension depends on whether a reader is able to integrate and interpret textual information appropriately (Grabe & Stoller, 2002; Huang, Chern, & Lin, 2009). To achieve this, a reader should have the ability to identify and resolve cohesive ties for they are claimed to best serve as the contextual clues for connecting information presented in a text (Al-Jarf, 2001; Halliday & Hasan, 1976; Yuill & Oakhill, 1988). Five cohesive ties are considered to help a reader integrate textual meaning; they are reference, substitution, ellipsis, conjunction, and lexical cohesion (Halliday & Hasan, 1976; Pritchard & Nasr, 2004). Among these ﬁve cohesive ties, referential identiﬁcation and resolution are the keys to successful reading comprehension (Chung, 2000; Oakhill & Yuill, 1986). Nunan (1993) states that it is a prerequisite for a reader to identify the referential relationship among sentences in comprehending a text. The reader can further resolve the referential relationship based on his reading proﬁciency and linguistic competence (Al-Jarf, 2001). Eventually, he can construct a well-structured memory representation of the text. The mental representation allows the reader to have a deeper understanding of the text and enables him to answer questions about the text, to recall, or to summarize it (Potelle & Rouet, 2003). According to Chen and Dai’s study (2003), most Taiwanese EFL college students fail to identify pronouns and relative pronouns in texts. Huang’s study (1993) also indicates that most EFL college students have difﬁculties in identifying cohesive ties for they lack instruction and practice in recognizing and resolving cohesive ties in their past learning experiences. It is clear that most EFL students in Taiwan lack skills in referential identiﬁcation and resolution necessary for reading comprehension. They are more likely to encounter difﬁculties and failures in the process of reading. The difﬁculties that EFL students encounter in processing cohesive ties have not been closely studied yet since traditional experiments fail to assess the complicated reading processes of referential ties (Al-Jarf, 2001; Chen, 2001). The major methods used to document students’ reading processes are mainly naturalistic observation, interviews, or think-aloud protocols (Schacter, Herl, Chung, Dennis, & O’Neil, 1999). Researchers in traditional experiments most often implement think-aloud task or recall to understand a reader’s comprehension by asking him to self report his reading process simultaneously or retrospectively. However, these methods are usually time consuming and labor intensive. The reader might be poor at verbally expressing his own reading process. As a result, the verbal report may lead to a shallow investigation. * Corresponding author. Tel.: +886 5 534 2601x3136; fax: +886 5 5312058. E-mail address: [email protected] (Y.-F. Yang). 0360-1315/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2009.04.016 800 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 Online learning systems are proposed to better investigate students’ learning process recently (Demetriadis, Papadopoulos, Stamelos, & Fischer, 2008; Ding, 2009; Martindale, Pearson, Curda, & Pilcher, 2005) since they can support higher-order learning, teach problem-solving skills to students who are struggling with learning difﬁculties, and have a positive impact on learning outcomes (Martindale et al., 2005; Mercier & Frederiksen, 2008). However, very few computer systems have been designed to examine the roles of referential identiﬁcation and resolution in reading. How students encounter difﬁculties in resolving references, how they read and reread the sentences to select and reselect references, and how individual students are supported by feedback in a system are seldom investigated. Particularly, how to manage the process and the product of referential identiﬁcation and resolution in reading has become a big challenge for both EFL students and teachers due to an average class size of more than 40 in Taiwan. This study aims to design a computer system for helping EFL college students develop mental maps of referential identiﬁcation and resolution in reading. Referential resolution, in this study, is deﬁned as a reading strategy applied by a reader to accurately interpret references in texts. It occurs when the reader identiﬁes persons and objects in different parts of a text pointing to the same entity (Walsh & JohnsonLaird, 2004). Mental maps of references are represented as nodes and connected by lines to indicate how a reader ﬁgures out the relationships among references. Taking the following sentences as examples, ‘‘Sigmund Freud was a doctor of psychology in Vienna, Austria at the end of the nineteenth century. He treated many patients with nervous problems through his cure talk.” ‘‘He” in the second sentence refers to ‘‘Sigmund Freud” in the ﬁrst sentence. That is, referential resolution can be used to establish the relationships among text elements that have the same meaning. Based on our research purpose, three research questions are addressed: (1) How is the system developed to support students in referential identiﬁcation and resolution? (2) How do students present their mental maps of referential identiﬁcation and resolution in the system? (3) How do students progress in their referential identiﬁcation and resolutions in the system? 2. System development In the system, students are required to follow two steps in reading a text: (1) to identify references generated by the system, (2) to establish the relationships among the references to form mental maps. Key functions are provided to offer students apprenticeship in a reading environment; they are system-guided instruction, warm-up practice, referential interface, and feedback. System-guided instruction informs students the goal of the system and prepares them for the following practices. Warm-up practice provides students initial attempts to develop their skills in referential identiﬁcation and resolution. The practice is also a guide for shaping and supporting students’ referential strategy development. Feedback serves as a scaffold to help students guess from textual clues and reﬂect on their incorrect answers of referential identiﬁcation and resolution in order to make correct choices. The system built for this study includes four modules, natural language processing (NLP), user interface, recording, and feedback. Fig. 1 shows the system architecture of this study. The teacher designs the course, selects texts which students have to read, and types in the texts to the NLP module through the teacher interface (Fig. 1). The NLP module picks the referential devices from each text and segments the text into sentences. The selected referential devices and the split sentences are then saved in the database. The recording module traces students’ reading process and behavior while they are constructing mental maps in referential identiﬁcation and resolution. These traced data are then studied by the teacher to identify the difﬁculties students encounter and the performances among different reading proﬁciency groups. The feedback module compares students’ initial maps with that of an expert while students are constructing their mental maps. It then provides three candidate references for each device that needs correction to students whenever they encounter difﬁculties ﬁguring out the relationship between two words. 2.1. NLP module In the system, a reader is asked to ﬁnd out three types of references, personal, demonstrative, and locative pronouns since they appear more frequently in texts (e.g., Fortanet, 2004; Kennison, 2003). Personal references refer to individuals or objects by specifying their functions or roles in the speech situation (Halliday & Hasan, 1976), such as ‘‘I,” ‘‘me,” and ‘‘you.” Demonstrative references act as forms of verbal location, such as ‘‘this,” ‘‘these,” and ‘‘that.” The speaker ﬁgures out the reference by means of location on a scale of proximity. Locative Natural Language Processing Module User Interface Teacher Text POS Tagger Learner Chunker Referring Interface Referential Devices Referential Device Selector Sentences Sentence Detector Database Recording Module Feedback Module Expert’s Map Fig. 1. System architecture for referential identiﬁcation and resolution. Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 801 Sigmund Freud was a doctor …at the end of the nineteenth century. 1) He treated many patients with nervous problems through his "talk cure." 2) For this type of treatment, …about anything that was bothering them. 3) While treating his patients, …person's actions in her or his present life. 4) Freud called the place where past … were hidden the unconscious mind. 5) Images from the unconscious … dreams or through the person's actions. 6) Freud wrote a book about …unconscious mind and dreaming in 1899. 7) The title of the book was The Interpretation of Dreams. Fig. 2. Results of sentence detection. Sigmund/NNP Freud/NNP was/VBD a/DT doctor/NN of/IN psychology/NN in/IN Vienna,/NNP Austria/NNP at/IN the/DT end/NN of/IN the/DT nineteenth/JJ century./NN He/PRP treated/VBD many/JJ patients/NNS with/IN nervous/JJ problems/NNS through/IN his/PRP$ "/'' talk/NN cure."/NN Fig. 3. The result of tagging two sentences with POS tags. references are used to indicate location, such as ‘‘here” and ‘‘there.” In addition to these three types of references, deﬁnite noun phrases are also included. The system provides tools to help the reader to reach the goal of referential identiﬁcation and resolution. These tools include POS (Part-of-speech) tagging, phrase chunking, referential device selection, and sentence detection. Most of the tools are provided by the open source system OpenNLP (http://opennlp.sourceforge.net). The system ﬁrst tries to pick out all referential devices in a text automatically. In doing this, the system needs to do POS tagging of each word in the text. With the help of the POS tags, the system chunks the text into noun phrases, verb phrases, prepositional phrases, and adjectival phrases, etc. Not all noun phrases serve as referential devices, so the system must pick out the real referential devices from all noun phrases. When the reader tries to pick out the references of a referential device, the system helps the reader to focus on a sentence by highlighting the sentence at which the reader clicks the mouse. Sentence detection is automatically done by the system. 2.1.1. Sentence detection When reading a text in the system, a reader can click the mouse at a word in the text in order to highlight the sentence he is reading. The system does this by detecting the sentence boundary surrounding the word. Moreover, the system records the sentence selection as one of the reader’s actions in using the system. A sentence usually ends with the punctuation of period, question mark, or exclamation mark. However, the occurrence of a period does not guarantee the end of a sentence. For example, ‘‘Mr.”, ‘‘Ms.”, and ‘‘76.5%” each contains a period that does not indicate the end of a sentence. Our system uses the sentence detector of OpenNLP. The result of sentence detection in the text ‘‘Sigmund Freud” is shown in Fig. 2. 2.1.2. Part-of-speech tagging When given a text, the system automatically picks out a list of referential devices in the text so that the reader can try to identify these devices. The ﬁrst step of automatic selection of referential devices is to tag the words of the text with POS tags. That is, the part-of-speech tag of each word must be determined. The result of tagging the ﬁrst two sentences in the Sigmund Freud text is shown in Fig. 3. The POS tags used by the system are given in Table 1. 2.1.3. Phrase chunking After POS tagging, the system uses grammar rules to do syntactic analysis of each sentence. Some common grammar rules are shown as follows. NP ? modiﬁer + baseNP baseNP ? modiﬁer + noun modiﬁer ? article | number | present particle | past particle | adjective | noun Table 1 Some POS tags used in the system. NNP VBD DT NN IN JJ PRP NNS PRP$ Proper noun Verb Determiner Single noun Preposition Adjective Pronoun Plural noun Possessive pronoun 802 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 VP ? verb + preposition PP ? basePP + baseNP basePP ? preposition After syntactic analysis, each sentence is segmented into a number of phrases, including noun phrases, verb phrases, prepositional phrases, etc. The result of chunking the ﬁrst sentence in the Freud text is shown in Fig. 4. 2.1.4. Referential device detection After chunking a text, the system picks out the noun phrases (Fig. 5) from the text and checks whether they are referential devices or not (Fig. 5). The true referential devices in the ﬁrst two sentences of the text are [NP He] and [PRP his]. The remaining NPs serve as the referents of the referential devices. Therefore, the problem of referential device detection is to screen out those noun phrases that do not have any referential function. The noun phrases are divided into two groups: those without referents and those with referents. Thirty four of the noun phrases have the pattern [NP X1] [PP of] [NP X2], e.g., ‘‘the end of the nineteenth century”. Among the 34 noun phrases, only three are referential devices. Therefore, the pattern [NP X1] [PP of] [NP X2] is used to ﬁlter out the noun phrases (e.g., ‘‘the end of the nineteenth century”) that are unlikely to be referential devices. A simple algorithm for detecting referential devices is developed (Fig. 6). In one case, the chunking result of a noun phrase is ignored, e.g., [NP his talk cure], and the possessive pronoun is considered a true referential device, [NP his]. A complete list of the result of selecting referential devices in the text is given in Fig. 7. 2.2. User interface With the teacher interface, the teacher manages course data, provides texts which students should read, and observes students’ reading process and behavior data. With the student interface, a student draws a map indicating the relationships among references. For the online practice of referential identiﬁcation and resolution, a referring interface, a recording module, and a feedback module are implemented (Fig. 8). They will be described in detail as follows. [NP Sigmund Freud] [VP was] [NP a doctor] [PP of] [NP psychology] [PP in] [NP Vienna, Austria] [PP at] [NP the end] [PP of] [NP the nineteenth century]. [NP He] [VP treated] [NP many patients] [PP with] [NP nervous problems] [PP through] [NP his "talk cure"] Fig. 4. The result of chunking two sentences. [NP Sigmund Freud] [NP the nineteenth century] [NP a doctor] [NP psychology] [NP He] [NP many patients] [NP Vienna, Austria] [NP the end] [NP nervous problems] [NP his "talk cure"] Fig. 5. The noun phrases from Fig. 4 that are candidates of referential devices. Begin All noun phrases are candidates of referential devices. For each noun phrase, If the phrase is a definite noun phrase (which begins with “the”, “this”, etc.) and the phrase does not match the pattern [NP X1] [PP of] [NP X2], then the noun phrase is considered a referential device. End Fig. 6. Algorithm of referential device detection. He his this his him them his he there she or he these the person's her or his the place the unconscious mind the unconscious mind the person's his the unconscious mind Fig. 7. Referential devices automatically detected in the text. Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 803 Fig. 8. The interface and modules in referential resolution practice. A. Toolbar. This includes a number of graphic tools. Connection tool can connect referential devices. Feedback tool compares students’ initial graph and the expert’s. It then informs students their incorrect answers and provides them with three candidate references for each device that needs correction. Other tools managing the canvas are cut, copy, paste, erase, group, ungroup, zoom in, zoom out, undo, and redo. B. Text ﬁeld. This area is used to show the text. Students can select a word or a sentence as a text element and then drag into the canvas directly when they comprehend the relations among the references. The sentence will be highlighted when students select a sentence. C. Referential device list. All referential devices are listed in this area. Students have to understand what these referential devices refer to, then drag and drop them to the canvas. When a referential device is selected, the referential device will also be highlighted in the text ﬁeld. D. Canvas. Students add links to indicate the relationships among references on this canvas. They can add, erase, drag and drop elements on the canvas. Fig. 9 is an example of a student’s mental map. E. Feedback frame. This will inform students the correctness of their answers in the referring practice. In addition, the feedback will provide three options when the student has difﬁculties ﬁnding out the links between two words. 2.3. Recording module The system uses a recording module to trace students’ reading process. From the data, the teacher can observe and identify the difﬁculties the students encounter and the difference in performance among various reading proﬁciency groups. The ﬁndings will be helpful for the teacher to modify his/her instruction according to students’ strengths and weaknesses. The module uses some predicates to record Fig. 9. An example of a student’s mental map. 804 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 Table 2 Some of the recording predicates. Predicates Description Read a sentence: [T] Add a cell: [X] Erase a cell: [X] Referring: [X][Y] Erase referring: [X][Y] Get feedback count: [X] Get feedback of: [X] Select a sentence T which the student is reading in the text ﬁeld Add an element X in the canvas Erase an element X in the canvas Add an meaningful relation between X and Y Erase a relation between X and Y Request the feedback for X times Request the feedback for the reference X students’ behavior data. Table 2 shows some of the recording predicates. For example, the recording module traces a student’s reading process in Fig. 10. 2.4. Feedback module After students have constructed their initial graphs, they can request a feedback. The feedback module compares the initial graph with the expert’s. The results of comparison will inform students which references are incorrect and offer them three possible referents for each incorrect device. A student’s answer is compared to the expert’s referential map. First, the module transforms the expert’s map and the student’s map into predicates. Second, the module ﬁnds all references from the student’s predicates and then compares them with the expert’s. If the student’s references do not match the expert’s, they are incorrect. Then the module will provide one correct reference and two distractors as clues back to the student. For example, Table 3 compares a student’s map with the correct one and ﬁnds out that the referential device A is incorrect in the student’s map. The feedback module will then offer the correct reference B and two distractors D and E back to the student. 3. Method 3.1. Participants A total of 90 junior and senior college students were recruited from two reading classes in a technological university in central Taiwan. Their language proﬁciency levels were identiﬁed by their reading scores in a simulated online exam Testing of English for International Communication (TOEIC) with a reliability of 0.87. The full score in the online exam was 200. The frequency distribution of all the participants’ score was used to divide the participants into three groups of readers. It was found that the highest frequencies fell in two score intervals, 81–90 (8 students) and 131–140 (8 students). These two intervals were used to 2: Read a sentence: [Sigmund Freud … of the nineteenth century.] 3: Read a sentence: [He (3) treated … his (4) "talk cure."] 4: Add a cell: [Sigmund Freud] 5: Add a cell: [He (3)] 6: Add a cell: [his (4)] 7: Add a connection: [He (3)] connects to [Sigmund Freud] …... 38: Read a sentence: [Freud wrote a book … dreaming in 1899.] 39: Add a cell: [his (22)] 40: Add a connection: [his (22)] connects to [Sigmund Freud] Fig. 10. A student’s reading process. Table 3 An illustration of comparing the student’s graph with the correct one. Expert Map Student Predicates Add a cell: [A] Add a cell: [B] Referring: [A][B] Map Predicates Add a cell: [A] Add a cell: [C] Referring: [A][C] 805 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 Frequency Distrbution of Readers' Scores in the Simulated Online TOEIC Exam Frequency 10 8 6 4 2 0 31-40 41-50 51-60 61-70 71-80 81-90 91-100 101-110 111-120 121-130 131-140 141-150 151-160 161-170 171-180 181-190 191-200 more-proficient readers average readers less-proficient readers Fig. 11. The frequency distribution of the participants’ score. identify a less proﬁcient reader group (31–90) and a more proﬁcient reader group (131–200). Participants with scores in the range 91–130 were excluded from the study. The frequency distribution of the participants recruited in the current study is shown in Fig. 11. Thus, 38 more-proﬁcient readers and 37 less-proﬁcient readers were identiﬁed in this study. The mean score of the more-proﬁcient readers was 157.89 with a standard deviation of 18.37. The mean score of the less-proﬁcient readers was 57.84 with a standard deviation of 14.70. 3.2. Material The online practice of referential identiﬁcation and resolution used three texts to examine the participants’ reading comprehension. The three texts were selected from College Reading Workshop (Malarcher, 2005) based on the following four criteria: abundance of references for reading practice, similar length, similar readibility level, texts written for EFL college students. The three texts were presented in sequence in the textbook and they are text 1—Freud and the Meaning of Dreams (number of words: 708; number of referring phrases: 38), text 2—The Tragedy of Echo and Narcissus (number of words: 692; number of referring phrases: 62), and text 3—Commerce through the Internet (number of words: 651; number of referring phrases: 21). Table 4 shows the readability of the three texts. According to the Flesch reading ease test (Farr, Jenkins, & Paterson, 1951), higher scores indicate material that is easier to read and lower scores indicate harder-to-read passages. Texts with scores of 90–100 are understandable to native speakers of English at the average 11-year old students and texts with scores of 60–70 are understandable to 13- to 15-year old students. Passages with results of 0–30 are understandable to college graduates (Flesch, 1948; Kincaid, Fishburne, Rogers, & Chissom, 1975). Among the three texts, text 1 has the most words, 708. Text 2 has the most sentences, 46, and paragraphs, 8. Text 3 is the most difﬁcult with Flesch–Kincaid grade level at 11.5. 3.3. Procedures of data collection The present study was conducted from April 24th, 2006 to June 10th, 2006. The 90 college students were asked to do the online practice of referential identiﬁcation and resolution. The online system recorded the participants’ reading behavior and performance. For each text, the participants were required to do referring practice by selecting referential words and drawing the relationships among these words on the canvas. The online system grades the participants’ referring practice by giving one point to each correct connection between two referential words. 3.4. Procedures of data analysis The participants’ raw data were categorized into two types: reading process and reading product. Reading process refers to the participants’ construction of their mental maps and the records of their reading behavior. Reading product includes the participants’ scores of referring and frequencies of feedbacks. In referring practice, three results were differentiated: correct, incorrect, and missed. These were further analyzed with the Statistical Package for Social Science (SPSS) version 12.0. Table 4 The readability of text 1–3. Number of words Number of sentences Number of paragraphs Flesch reading ease Flesch–Kincaid grade level Text 1 Text 2 Text 3 708 35 7 53.6 10.6 692 46 8 66 7.6 651 35 6 44.5 11.5 806 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 Table 5 The mean scores and standard deviation of referring practice. Text 1 Referring Feedback Text 2 Text 3 Mean SD Mean SD Mean SD 20.71/38 (54.5%) 17.90 8.91 17.31 43.42/62 (70%) 13.61 16.27 13.02 14.47/21 (69%) 7.67 6.11 7.93 4. Result The 90 participants’ mean scores and standard deviations of referring practice and feedback frequency were shown in Table 5. The total points of referring practice in the text 1 are 38; that of text 2 are 62; and that of text 3 are 21. Students made progress in referential resolution from 54.5% in text 1 to 69% in text 3. They also asked for fewer feedbacks as they progressed in reading as the mean of feedback frequency decreases from 17.90 in text 1 to 7.67 in text 3. A t-test was also conducted to examine whether there were signiﬁcant differences between the more-proﬁcient and the less-proﬁcient readers in practice of referential identiﬁcation and resolution and frequency of feedbacks. The result of t-test showed that there was a signiﬁcant difference (t = 5.29, p < .05) between the more- and less-proﬁcient readers. This result was similar to that of feedback frequency. That is, the more- and less-proﬁcient readers’ mean scores in the referring practice and frequency of feedbacks were signiﬁcantly different. Regarding the differences between the more- and less-proﬁcient readers in drawing mental maps of referential resolution, some examples were found. The more proﬁcient readers were able to integrate personal references in different parts of the text to form a coherent network (graph (a) of Fig. 12). In contrast, the less-proﬁcient readers missed many personal pronouns which refer to ‘‘Sigmund Freud.” Although some of the less-proﬁcient readers tried hard to complete the referring practice, many of their answers were still incorrect. For instance, graph (b) of Fig. 12 shows one of the less-proﬁcient readers referred plural pronouns to a single subject. Some of the less-proﬁcient readers referred a pronoun to an incorrect word (graph (c) of Fig. 12). Pearson product-moment correlation coefﬁcient was also conducted to ﬁnd out the correlation among the frequency of feedbacks, errors, and missed rate in the practice of referential identiﬁcation and resolution. Table 6 presents the correlation between the number of errors and feedback frequency and that between the number of missed references and feedback frequency. a his (92) he (3) his (93) his (88) his (4) his (6) him (7) Sigmund Freud his (86) his (73) his (9) his (26) his (22) he (10) b these (65) them (53) the impression (50) they (54) them (67) c their (33) that (29) Fig. 12. Students’ mental maps of referential resolution. Table 6 The correlation between the number of errors and feedback frequency and that between missed referents and feedback frequency. Readers Text 1 Error/feedback Moreproﬁcient readers Less-proﬁcient readers -.43 .19 Text 2 Missed/feedback .38 .81 Error/feedback .87 .14 Text 3 Missed/feedback .76 .62 Error/feedback .50 .36 Missed/feedback .86 .82 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 807 Table 6 shows that the correlation between the number of errors and feedback frequency is negative for the more-proﬁcient readers. That is, when they asked for more feedbacks, they made fewer errors. This is also true for the relationship between the number of missed references and feedback frequency. In contrast, the correlation between the number of errors and feedback frequency is positive for the less-proﬁcient readers. They did not actively ask for feedback and did not select the correct answer when the three options were provided as feedback. However, the correlation between the number of missed references and feedback frequency is negative for the less-proﬁcient readers. That is, the more feedbacks they asked for, the fewer references they missed. 5. Discussion From the results of this study, some ﬁndings are explored. First, the less-proﬁcient readers had some common difﬁculties in referential identiﬁcation and resolution. They misinterpreted the relationship between two words as attribution instead of equivalence in the practice of referential identiﬁcation and resolution. For example, ‘‘The title of the book was The Interpretation of Dreams.” Because of the phrase ‘‘of the book,” ‘‘the title” becomes deﬁnite and it refers to ‘‘The Interpretation of Dreams.” The relationship between ‘‘the title” and ‘‘of the book” is attribution rather than equivalence. The less-proﬁcient readers also misperceived a reference when it syntactically played the role as dummy subject or relative pronoun. For instance, ‘‘A few researchers, however, said that dreams had a more useful function.” ‘‘That” is a relative pronouns to connect the following clause rather than serving as a cohesive tie to refer to other text elements. Another example is ‘‘It may be true that almost every company has a website, but many of these sites are for information rather than for commerce.” ‘‘It” here is a dummy subject used to substitute ‘‘almost every company has a website, but many of these sites are for information rather than for commerce” for the sake of a common sentence structure. It is known that changes of the syntactic roles call for different methods for interpreting the cohesive ties. Second, the online system in several ways scaffolds students in referential identiﬁcation and resolution. The system picks out and highlights all referential devices in the text automatically. This would reduce students’ cognitive overload to look for all possible references in a text. The system can also automatically trace and record students’ reading behavior and process with the trace result illustrating students’ performances in referential identiﬁcation and resolution. That is, students’ difﬁculties in different kinds of references, personal, demonstrate, and locative, can be examined. Based on this examination, the reading teacher can design follow-up lesson plans and classroom activities to assist the struggling students in overcoming their difﬁculties and compensating their weaknesses. In other words, the trace result serves as a guide for the reading teacher to design and plan subsequent remedial courses. Moreover, in overcoming the difﬁculties in referential identiﬁcation and resolution, the feedback with candidate references provided by the system serves as a scaffold. However, this scaffold does not appear automatically in the system. Only when the students requested scaffolding did the system provide it to help students overcome the difﬁculties. It was found that when a student asked for more feedbacks, his incorrect and missed references decreased. Finally, it is suggested that the system is applicable to students from other backgrounds. The system could be employed as a tool for teachers to develop students’ reading strategies in terms of referential identiﬁcation and resolution. It can also assist teachers to evaluate students’ mental maps based on the criteria of referential resolution. Different from other systems which compare students’ mental maps with those of experts but do not reveal the criteria for the comparison, our system shows students’ mental maps composed by referential resolution so that the criteria of comparing experts’ mental maps with those of students become standardized and possible. Regarding suggestions for improvements, some new help functions could be added. 1. After the participants had completed the practice of referential identiﬁcation and resolution, the system should not only provide the results of grading but also the expert’s mental maps. This will help the participants compare their initial mental maps with those of the experts. 2. The percentage of similarities between students’ and the expert’s mental maps should also be provided, so that the participants’ mental maps in the practice of referential identiﬁcation and resolution can be further analyzed. 3. Readers’ self-assessment could be taken into consideration for the next implementation of the system. The participants could be given more freedom in terms of the type and the number of texts they intend to read based on their individual needs. A reader’s portfolio could also be adopted in the system that allows each reader to set up his reading plan for a whole semester. 4. In the next version of the system, more guided instruction on referential devices and examples will be provided. Furthermore, although the system used an audio tutorial to assist students in a trial section, a tutorial video will be developed to show the readers how to draw a mental map. Acknowledgements This study was supported by National Science Council in the Republic of China, Taiwan (NSC 96-2411-H-224-014 & NSC 95-2520-S-224001-MY3). These research grants made the continuation of this study possible. References Al-Jarf, R. S. (2001). Processing of cohesive ties by EFL Arab College Students. Foreign Language Annals, 32(2), 141–151. Chen, I. (2001). A constructivist computer-assisted language learning environment for Pennsylvania State University. DAI, 62, No. 12A, p. 4031. Chen, S. Z., & Dai, H. T. (2003). An analysis toward Taiwanese students’ cohesive ties in reading comprehension. In Proceedings of the 20th international english teaching and learning (pp. 235–258). Taiwan. Chung, S. L. J. (2000). Signals and reading comprehension – Theory and practice. System, 28, 247–259. Demetriadis, S. N., Papadopoulos, P. M., Stamelos, I. G., & Fischer, F. (2008). The effect of scaffolding students’ context-generating cognitive activity in technology-enhanced case-based learning. Computers and Education, 51(2), 939–954. Ding, N. (2009). Visualizing the sequential process of knowledge elaboration in computer-supported collaborative problem solving. Computers and Education, 52(2), 509–519. Farr, J. N., Jenkins, J. J., & Paterson, D. G. (1951). Simpliﬁcation of Flesch reading ease formula. Journal of Applied Psychology, 35(5), 333–337. Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233. 808 Y.-F. Yang et al. / Computers & Education 53 (2009) 799–808 Fortanet, I. (2004). The use of ‘‘we” in university lectures: reference and function. English for Speciﬁc Purposes, 23(1), 45–66. Grabe, W., & Stoller, F. L. (2002). Teaching and researching reading. Malaysia: Pearson Education. Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman. Huang, C. T. (1993). Topic-prominence and morphosyntactic parameter in Chinese. MI: University Microﬁlm Inc. Huang, H. C., Chern, C. L., & Lin, C. C. (2009). EFL learners’ use of online reading strategies and comprehension of texts: An exploratory study. Computers and Education, 52(1), 13–26. Kennison, S. M. (2003). Comprehending the pronouns her, him, and his: Implications for theories of referential processing. Journal of Memory and Language, 49, 335–352. Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Deviation of new readability formulas for Navy enlisted personnel, Research Branch Report 8–75, Millington, TN: Naval Technical Training, US Naval Air Station, Memphis, TN. Malarcher, C. (2005). College reading workshop (2nd ed.). Compass Publishing Inc. Martindale, T., Pearson, C., Curda, L. K., & Pilcher, J. (2005). Effects of an online instructional application on reading and mathematics standardized test scores. Journal of Research on Technology in Education, 37(4), 349–360. Mercier, J., & Frederiksen, C. (2008). The structure of the help-seeking process in collaboratively using a computer coach in problem-based learning. Computers and Education, 51(1), 17–33. Nunan, D. (1993). Challenges in EFL classrooms. TESOL Matters, August/September, 7. Oakhill, J., & Yuill, N. (1986). Pronoun resolution in skilled and less skilled comprehenders: Effects of memory load and inferential complexity. Language and Speech, 29, 25–36. Potelle, H., & Rouet, J.-F. (2003). Effects of content representation and readers’ prior knowledge on the comprehension of hypertext. International Journal of Human Computer Studies, 58(3), 327–345. Pritchard, R. M. O., & Nasr, A. (2004). Improving reading performance among Egyptian Engineering Students. Journal of English for Speciﬁc Purposes, 23(4), 425–445. Schacter, J., Herl, H. E., Chung, G. K. W. K., Dennis, R. A., & O’Neil, H. F. Jr., (1999). Computer-based performance assessments: A solution to the narrow measurement and reporting of problem-solving. Computer in Human Behaviors, 403, 418. Walsh, C. R., & Johnson-Laird, P. N. (2004). Co-reference and reasoning. Memory and Cognition, 32(1), 96–106. Yuill, N., & Oakhill, J. (1988). Effects of inference awareness training on poor reading comprehension. Journal of Applied Cognitive Psychology, 2, 33–45.
© Copyright 2018