system: [SYSTEM]

user: 3. SpeechRater®实证研究 3.1. 研究背景 新东方教育科技集团于2018年开始与ETS进行合作，国外考试推广管理中心引入了SpeechRater®系统作为学员线上练习托福口语的工具，提供全真模拟练习平台，以及官方反馈报告，留下了大量的学员数据。为了从测试数据中了解不同分数段学生口语作答中存在的问题，找到同一水平学生问题的共性，需要参考一些数据指标，并对学生的口语作答进行文本分析。希望能够通过分析和探索研究反哺教学，找出阻碍学生口语分数提高的因素，并从教学角度给出学习建议。 3.2. 研究材料和数据 托福口语题目分为独立口语和综合口语两种类型。独立口语题目贴近学生生活实际，考查内容和范围涉及学校、家庭、社会、生活、教育等多个方面，需要学生在作答时结合个人经历给出观点并进行阐述，学生的表达会更多展示出自己所积累的英语口语知识和能力。综合口语题目多以学术场景为主，并将听力内容和阅读内容与口语考查相融合，更为综合的考查学生理解、分析及总结等口语能力。由于综合口语题目存在听力或阅读材料，考生在作答内容中需要参考和结合，而非完全自我输出，用该类题目下学生的作答去分析存在局限性，故没有选择Task 4等其他综合口语题目。又因ETS 2019年对托福考试进行了改革，口语删除掉了独立口语题Task 1。因此，本次研究材料选择了托福TPO口语中的独立口语题目Task 2。 独立口语题Task 2需要学生从两个或三个题目所陈述的观点中选择一个自己最倾向或同意的观点，然后给出原因、细节、事例去支撑提到的观点。学生有15秒的准备时间，作答时间为45秒。截止2019年底，后台共收录学生作答数据714，923条，我们对各套TPO口语任务中学生的平均分进行了统计，发现没有显著差异(详见图3)，进而选择了在题目内容上较好把握的TPO42 Task 2作为材料进行研究，该套试题的平均分为2.98分。其题目为： Some people prefer living in a big city. Other people prefer living in the countryside， away from urban areas. Which do you think is better? Explain why， using specific details in your explanation. 需要学生进行二选一，是更倾向于住在大城市还是乡村，选择其中一个观点后并用细节和事例来阐述。 图3. 各套TPO口语作答任务的平均分 本研究共收集到4417条学生在TPO42 Task 2的分数(除去0分和TD 2 等无效数据)，根据其分布及学生以往在托福考试中存在的瓶颈分，按比例抽取了80份学生作答录音，并邀请8位有3年及以上托福口语教学经验的老师参与，对学生录音进行精听和文本分析。 3.3. 研究分析及结果 3.3.1. 总样本数据及抽取样本数据描述性统计和分数分布情况分析 为了了解学生在TPO口语练习中的分数情况，以便抽取具有代表性的样本进行分析，研究对后台收录的714923条数据进行了描述性统计分析。其中，为保证分数的有效行，删除了所有0分及TD的分数数据。表1和图4分别为学生(N = 714，923) TPO口语练习的描述性统计量和分数分布直方图。 Table 1 The descriptive statistic data of students’ TPO speaking tasks (N = 714，923 样本量 有效 714923 缺失 0 平均值 2.98 中位数 3.00 众数 3.00 标准差 0.357 方差 0.127 偏度值 −0.279 峰度值 0.274 全距 4.00 最小值 1.00 最大值 4.00 百分位数 25 2.76 50 3.00 75 3.23 表1. 学生(N = 714，923)TPO口语练习描述性统计量 图4. 学生TPO口语练习分数直方图 考虑到本次研究重点分析不同等级分数段学生在TPO42 Task 2口语任务上的表现，因此从后台提取了4417条学生分数数据，同样为保证分数的有效性，删除了所有0分及TD的分数数据。表2和图5为学生(N = 4417) TPO42 Task 2口语练习的描述性统计量和分数分布直方图。 从表1和图4及表2和图5可以看出，学生在TPO口语练习分数、TPO42 Task 2口语练习分数中的平均分分别为2.98和2.95，中位数分别为3.00和2.96，众数分别为3.00和2.51，标准差分别为0.357和0.353，除了众数整体上差别不大，并且从分数分布的直方图来看，均近乎呈正态分布，说明抽取样本的代表性较好。 Table 2 The descriptive statistic data of students’ speaking task (TPO42， Task 2) (N = 4417 样本量 有效 4417 缺失 0 平均值 2.95 中位数 2.96 众数 2.51 标准差 0.353 方差 0.125 偏度值 −0.225 峰度值 0.135 全距 4.00 最小值 1.34 最大值 4.00 表2. 学生(N = 4417) TPO42 Task 2口语练习描述性统计量 图5. 学生TPO42 Task 2口语练习分数直方图 3.3.2. 文本分析数据的整体情况 为了更深入的了解不同等级分数段学生使用SpeechRater®的作答表现，研究按照分数等级区间抽取80份学生录音数据进行文本分析。根据ETS每年发布的官方托福数据报告，近10年大陆考生的托福口语成绩基本在19分 [ 5 ]，按照托福评分标准进行分数转化，对应的是2.50分。在多年的教学实践中发现，中国大陆学生口语24分为瓶颈分，对应的是3.16分。所以，研究增加了这部分样本数据的数量。最终抽取用于文本分析的数据分布和数量如表3： Table 3 The text analysis data of students’ speaking task (TPO42， Task 2) (N = 80 学生等级分数 数量 1~2 8 1.6~1.7 2 1.7~1.8 2 1.8~1.9 2 1.9~2.0 2 2~3 40 2.1~2.3 10 2.3~2.5 10 2.6~2.8 10 2.8~3.0 10 3~4 32 3.0~3.2 20 3.3~3.5 3 3.6~3.8 3 3.9~4.0 6 表3. 学生(N = 80) TPO42 Task 2口语练习文本分析数据量 由于SpeechRater®在口语评分中涉及到语音识别并转文字的过程，我们无法从ETS方面拿到文本，于是对市面上各种语音转写和识别系统进行了比较和分析，最终选择了行业内被认可的科大讯飞语音转写系统 3 进行文本转写，转写后对其内容结合学生音频进行了校对，确认内容基本无误后，将文本通过The Compleat Lexical Tutor 对其词汇使用情况进行检测和标注，最后结合SpeechRater®提供的词汇和语法方面的数据由8位有丰富口语教学经验的老师参考，并进行文本分析，文本从词汇和语法角度切入，找到不同等级学生在此方面的表现和存在的问题。 3.3.3. 文本分析结果 1) 不同等级分数段学生Lexical Tutor用词统计 “新通用高频词表”NGSL (New General Service List)和“新学术词汇词表”NAWL (New Academic Word List)是被学术界广为认可的词汇学习词表，是从庞大的剑桥英语语料库中筛选出的高频和通用词汇，其中NGSL Level 1到Level 3使用频率由高到低。Lexical Tutor可对文本所包含的单词使用情况进行检测和统计，经过对不同等级分数段学生用词进行统计，得到下表4： Table 4 The statistics using by Lexical Tutor of different level students’ vocabulary usages (N = 80 平均百分比 NGSL_1 NGSL_2 NGSL_3 NAWL 1~2分 88.03% 3.04% 0.00% 0.23% 2~3分 88.74% 4.38% 1.29% 0.20% 3~4分 89.81% 4.16% 1.06% 0.41% 表4. 不同等级学生(N = 80) Lexical Tutor用词统计 由表4可知，三个不同分数段的学生使用最多的是NGSL_1中的词汇，分别占到了88.03%、88.74%和89.81%，可见对于口语表达而言，基础、常用的词汇足以学生用来组织语句，表达观点。NGSL_2和NGSL_3中三个阶段学生的使用情况分别是3.04%、4.38%、4.16%和0%，1.29%，1.06%，但是经过对文本内容的分析发现，2~3分学生在NGSL_2和NGSL_3的词汇使用中，由于同一单词在回答中重复的次数过多，使得用词占比偏高，所以实际上是达不到4.38%和1.29%的。如下方文本展示： 可以看到在学生的作答中，urban作为题干所给出的内容，一共重复出现了4次，同时libraries也重复出现了3次，使得NGSL_2词汇的使用频率占比达到了10%，实际上学生的用词并没有达到多样性的标准。相反，在3~4分学生的作答中，可以看出高分段的学生能更好掌握词汇，同一单词使用的重复率相对较低，如下方文本展示： 高分段学生对NAWL词汇的使用为0.41%，要高于其他两个分数段的0.23%和0.20%。同时，笔者发现1~2分段学生在使用学术词汇时会出现错用的情况，如发音错误导致识别成学术词汇，进而导致NAWL词汇使用占比虚高。 2) 不同等级分数段学生词汇和语法错误统计及描述 在对80份学生录音文本进行分析后，笔者对不同分数段学生的词汇和语法错误及问题进行了整理，发现1~2分学生存在的问题主要有以下几大类(表5)： Table 5 Vocabulary and grammar errors in students’ responses (scored 1~2 错误类型统计 频率 文本内容描述 发音不准造成的识别错误 15.5% how he can [hang out] with them every thick tie [sick time] because [besides] 定冠词、限定词错误/丢失 13.8% I prefer to live in [a] big city; living in [the] big city is better. 介词丢失/错误 6.90% construction from [for] people in thousands of years. save more time in [on] the way 名词使用错误 13.8% 1) if you live in a big city， you can get a lot of opportunity [opportunities] 2) secondly， people can have a lot of friend [friends] 3) There are also many， many， uh， entertainment [entertainments/entertainment options] in the big city. 4) I prefer to live in a big city because living in the urban [urban area] 5) One of the reason [reasons] is that living in big city is convenient for us to buy necessity [necessities] 动词/形容词使用错误 20.7% 1) Besides I think living [in a big] city， um， we can have more friends because live [living，动词使用错误] in [the-冠词丢失] city. 2) won’t [can’t-情态动词误用] see them 3) and having [have-动词错误] meals 4) high [good] air quality 5) have more [many-形容词错误] people 代词使用错误 3.5% 1) First. You [I-代词使用错误] can expand my friend circle. because there [it-代词错误] is a quiet [place-名词缺失] 用词不当 1.7% living in the urban is a terminal [ultimate] aim 主谓一致 5.2% 1) There is [are] many things we can do. 2) a big city have [has] 成分残缺，句子不完整 6.9% One of the reason [reasons-用词不当] is that living in big city can [help us-句子成分残缺不全-缺宾语] obtain success …… 句式结构错误 12.1% 1) there has been [there are-句式结构错误] a few [few-用词不当] activities 2) a big city have the advanced medical level [客观存在应用there be句式there is an advanced medical level in a big city] 表5. 1~2分学生作答中(8份)词汇和语法存在问题归类 上述错误类型，在8份学生录音中共发现错误58处，其中在动词/形容词误用，名词使用错误(主要是单复数问题)，丢失/错用定冠词，句式结构等方面问题突出。同时，介词、主谓一致、句子成分残缺等也都是易常出错的问题。如初学者常会受到汉语母语的影响出现介词的遗漏，汉语中“敲门”之间并不需要加介词，但英文中需要说knock at/on the door；或者介词的搭配问题，on the way而不是 in the way。还有，会常常在名词单复数上犯错，如One of the reasons，a lot of friends等等应该用复数的地方用了单数；动词/形容词的使用也常出现该用动词原形的时候用了动词ing形式；and并列结构前后的动词形式不一致；主语为复数many things，谓语没有用are却用了is；句式上对于简单固定结构的There be句型掌握不佳等等。对于这个分数阶段的学生，还有一个问题是学生在发音上的掌握较弱，会由于发音不准问题造成语音识别的错误，从而导致句子内容等出现错误，影响理解。 在SpeechRater®词汇复杂度和准确性、词汇多样性及语法准确性这三个维度上，学生表现所处的百分位数区间基本在1~30，即其表现仅超过1%~30%的学生。仅有个别特例在某一项上会超过70%的学生，但从整体来看，能力表现偏低。主要的问题在于词汇、语法基础弱，语言错误太多，很大程度上影响了对内容的理解。 2~3分，特别是2.5分左右是中国大陆学生口语成绩平均分19左右，2~3分学生存在的问题主要有以下几大类(表6)： Table 6 Vocabulary and grammar errors in students’ responses (scored 2~3 错误类型统计 频率 文本内容描述 发音不准造成的识别错误 8.3% 1) We can talk about the history of our country and other customers [customs] in our city. 2) we'll fill [feel] much pleasure [pressure] 3) plants [plans]；backboard [blackboard]; Ruler [rural] areas；captain [capital] of China；camera [cinema]；habits [hobbies] 定冠词、限定词错误/丢失 11.0% 1) I would prefer living in a [the] countryside. 2) I can just breathe in [the] fresh air. 3) Most people are working and living in the [去掉] big cities. 4) I can either take a bus or take a [the] subway. 介词丢失/错误 6.2% 1) Well， I prefer living in the countryside for [to] in a big city 2) I can get a job of [in] a famous company. 3) Much， much more easy (词汇搭配错误，easier) to go to a restaurant， [to] have some meals 4) It's really helpful for me to hunt [for] a good job. 5) If you want to get something in [去掉] somewhere， 名词使用错误 13.1% 1) I can watch some movie [movies]; we can improve the quality of our life [lives]. 2) I have many chance [chances] to meet various people 3) have a relaxed [relaxation] 4) I mean there are a lot of famous university [universities] in the big city. 5) a soccer field [soccer fields] 6) I prefer living in the country [countryside]. 7) Firstly， I think the big city can provide more and more (用法错误，一个more) opportunity [opportunities] to find better jobs. 8) I will I will have more approaches [access] to some， you know large stores， um， hospitals and also some something. 9) I'm a people[person] who really like [likes-主谓不一致] to go to the concert. 动词/形容词使用错误 11.7% 1) well， I prefer to live in the big city because live [living] in the city is very convenient 2) show every [our-语音识别错误] emotion (?) and thinking [think] 3) be very happy happiness [去掉，happiness] and pleasure [pleased]. 4) much more easy [easier]to go to a restaurant we can just ride a bicycle. Um， to [go to] another place. 5) And we [can] have [do] many activities with our friends. 6) And besides living in the urban area can makes [make] my [去掉my-指代错误，教育资源变得更容易获取，抽象的泛指教育，不是我的教育] education more accessible. 7) So you can you can become more [去掉] stronger； 8) there are many [much] population in the [冠词错误，去掉] big cities. 代词使用错误 4.8% 1) Well， I prefer to live in a big city. Um， as a high school student， it [代词指代不明] would be time consuming. 2) So I can have I can have a better educational environment and disadvantage [this advantage?] this [代词指代不明] advanced technologies can help me can stimulate my interest in learning. 3) uh， big city， maybe it is [去掉] very convenient.应去掉“it”，此时不需要代词，已存在主语 搭配不当 15.2% 1) we'll fill [词汇错误，feel much pleasure [pressure] [搭配不当，不能feel pleasure)，应该为feel pleased] 2) everything is in a slow slopes [pace] 3) quality environment [the environment is in good quality] 5) the countryside people [people in the countryside] because they (句子成分缺失，are] always friendly. 6) they need to go to the job [go to work]. 7) we can let our life become more abundant [make our lives more colorful/ rich/diversified] 8) And we can have[do]many sports to uh， uh， to have the [去掉-不特指] entertainment [fun]. 9) you can have a more earning [better-paying-词汇搭配错误] job. 10) the better facilities system [去掉system-词汇搭配不当] and transportation pop [transportation hub] in public. 主谓一致 6.9% 1) because they have to compete for their job [stay competitive in order to maintain their jobs-意思表述不清楚，词汇使用错误] job [jobs] every day 2) Because uh， countryside have [has] less pressure 3) I can’t [can] find the jobs that is [are] better than in the countryside. [I can find better jobs in the city-句子结构错误] 4) It only cost [takes-主谓不一致且词汇使用不当] me 30 minutes to commute every day. 成分残缺，句子不完整 9.7% 1) because I like to make friends very much， especially I want to make friends with [people from-句子成分残缺不全] all over the world. 2) and living in the city [are-缺谓语] under great pressure 3) Not like [people in the] big city. such as [词汇错误，for example] can [主语缺失，you can]go shopping easily. 4) because this last [?] air pollution of the cars [in the city is very serious and damaging to my body-句子不完整]. 5) [when] It comes to the topic about living in a big city[or in the countryside-句子不完整或改成when it comes to whether we should live in a city or countryside]. 6) Maybe [去掉-用词不当] you can have access to the uh， medical [添加service-句子成分残缺] 7) Well I prefer to live in a big city， this [is-缺谓语] because I think it is more convenient. And so I can [get-缺谓语] a better education in a big city. 句式结构错误 11.7% 1) but the country life is more relaxing than the city [life in the countryside is more relaxing than that in the city] 2) I think the big city is better than the living in the countryside. [living in a big city is better than living in the countryside-句型结构错误] 3) And there's all nearby us [And those are all within easy reach，句式结构错误]. 4) I can go to a library or museum very convenient [It is convenient for me to go to a library or museum-句子结构错误] 5) So if I need to uh prepare a paper， I can just search the Information in this place is very convenient with [which is very convenient]. 表6. 2~3分学生作答中(40份)词汇和语法存在问题归类 上述错误类型，在40份学生文本中发现了145处错误，其中在搭配不当，名词/动词/形容词使用错误，句式结构、定冠词、限定词错误/丢失、句子成分残缺等方面问题较突出。可以发现，与1~2分学生作答相比较，在发音的识别问题上，与1~2分学生发音含混无法识别不同，这部分学生的问题在于易混淆单词，如customers-customs；plants-plans；ruler areas-rural areas；captain of China-capital of China；habits-hobbies等。此外，可以看出这部分学生已经掌握了基本的词汇和语法，但是在准确使用上还存在不少问题。如搭配不当，have entertainment-have fun (正确)；go to the job-go to work (正确)；the better facilities system应去掉system；transportation pop-transportation hub (正确)等；同时，学生语言基础较前者好，输出的内容更多后，出现了很多成分残缺，表达不完整的句子，如： 1) Well， I prefer to live in a big city， this [is-缺谓语] because I think it is more convenient. 2) And so I can [get-缺谓语] a better education in a big city. Not like [people in the] big city. such as [词汇错误，for example] can [主语缺失，you can]go shopping easily. 在句式结构上，较1~2分学生使用句式更多样和复杂，但容易出现错误。如对从句的掌握、对固定句式的掌握不佳等，如： 1) but but the but the country life is more relaxing than the city [life in the countryside is more relaxing than that in the city]。 2) I can go to a library or museum very convenient [It is convenient for me to go to a library or museum-句子结构错误]。 另外，2~3分学生表达中含有大量的中式表达。可见学生会受到中式思维、母语迁移的影响，出现不少中式表达，如： 1) everyone know (动词主谓不一致，knows) everyone，表达错误，应该为everyone knows each other 2) living in a big city will give the (冠词错误，去掉the) people a lot of pressure，表达错误，应该为 people will have a lot of pressure when living in big cities。 3) This transportation [this-代词错误，指代不明，和前文单复数不一致] will help you to move very fast and go anywhere that [that去掉-关系词错误] you want. 表达错误，应该为By taking the bus or subway will help you to get around with ease。 4) in the big cities you can develop yourself better. 表达错误，应该为 in big cities you can improve a lot. 5) It really cost time. 表达错误，应该为 It’s really time consuming。 这些问题都是学生在学习和练习过程中，应该关注到的。 在SpeechRater®词汇复杂度和准确性、词汇多样性及语法准确性这三个维度上，学生表现所处的百分位数区间基本在10~70，即其表现仅超过10%~70%的学生。个别会高于80或者低于10，但是从整体上来看，其表现会相对居中。主要问题在于词汇、语法掌握的不够娴熟，在使用过程中准确性较差，出现的语言错误会部分影响内容的理解。 3~4分，特别是3.0~3.2分对应的24分，是很多学生在托福考试中较难突破分数。3~4分学生存在的问题主要有以下几大类： Table 7 Vocabulary and grammar errors in students’ responses (scored 3~4 错误类型统计 频率 文本内容描述 发音不准造成的识别错误 4.2% 1) novels [nervous] 2) walking [working] 定冠词、限定词错误/丢失 4.2% 1) The first reason is about [the] transportation. 2) It is [the] constant improvement that makes me stronger and stronger. 介词丢失/错误 8.3% 1) That means you can get [to] every corner of the city in a short time. 2) The second reason is by [about] my friends. 3) Because first of all I believe that [缺失介词，要加上in] big cities， there are more resources available 名词使用错误 11.1% 1) more chance [chances] 2) more opportunity [opportunities] 3) I choose to live and work in big city [cities] 动词/形容词使用错误 9.7% 1) And probably I will give [gain，动词错误] some money after our [my，物主代词错误]retirement 2) everyone can choose the most suitable way to life [live，词性错误] 3) It's hard to deny that cities have more high [highly，词性错误] qualified schools and classes. 代词使用错误 11.1% 1) I feel like that I come back to a peaceful and calm place that can allow that allows [删除，句子成分多余] you [me，代词前后不一致] to recharge. 2) fulfill myself [my 物主代词错误] value and contribute to the world. 3) you can take [a] deep breath of fresh air that can let you feel relaxed[that指代不清，改为which is really refreshing]. 搭配不当 5.6% 1) following the development of technology and society， the city can show up [adopt，搭配误用] a lot of high level technology like that. 2) traffic tools [public transportation] 主谓一致 2.8% 1) big cities has lots of [big cities have lots of] 2) it offer [offers] more opportunities in the city. 成分残缺，句子不完整 13.9% 1) you can have a good health care when you [are sick ，句子成分缺失] 2) The city filled with lots of chances and challenges [and that is] what I want. 3) Supposing [suppose] that you are living in countryside and you will [加know， 缺谓语动词] how difficult it is for you to get treatment in time. 4) So I can just read some books or， you know， working in the field [缺这些事实导致的结论，加上which could be really boring and exhausting] 5) the big cities usually have a [去掉] advanced facilities for people to entertaining[to entertain themselves， 句子结构不完整] 6) So in the city is[it is] very easy [much easier] to find teachers or[for] extra help [加连词or] extracurricular classes and hobbies I'm interested in such as in[去掉in] arts， etc. 句式结构错误 18.1% 1) I prefer to stay in a big city rather than living in the countryside. [prefer to do rather than do，改为I prefer to stay in a big city rather than live in the countryside.] 2) There are subways in most of the city and we can just go downstairs and take the subway [加，which] is [双谓语] easily to every corner of the city. 3) because the pollution is less than the downtown.[改为there is less pollution in the countryside] 4) if I want to go to hospital [中式英语错误，改为see a doctor]， that [主从句连接词误用，改成then] I [it will] have to [去掉have to] take a long time long time traveling by car to see that [去掉to see that] maybe [缺主语，加I] will [搭配错误，加be] late for the best time to resolve [cure，搭配错误] the disease. 时态/语态 11.1% 1) Well， personally， I prefer living in a big city because uh I think it forced [it can force] me to work hard because you know in a uh， in the big cities； 2) which really brightened [brightens时态错误] my day 3) My brother is a college graduate from a small town and he want [wanted时态错误] to find a job in a big city after the first[year].he felt very tired and novels [nervous语音识别错误] since he didn't have any friends， so that he didn't have any help in there. 4) Besides， I live [used to live时态错误] in the countryside where [when连接词错误] I am [was 时态错误] a child. 表7. 3~4分学生作答中(32份)词汇和语法存在问题归类 上述错误类型，在32份学生文本中共发现72处错误。其中发音不准造成的识别问题、定冠词、限定词/介词问题、主谓一致、搭配等问题比例相对较少，问题集中于名词/动词/形容词/代词等的使用，名词部分同样是单复数问题居多，动词、形容词中对词性的把握不准确，容易出现to life [应为to live]，have more high [highly，词性错误] qualified schools and classes。等问题。同时，由于3~4分学生的语言基础较于前两个分数段的学生要好，所以可以明显看出学生在用词上更为丰富，组句上也更加复杂，输出内容也更多。但是，在长句的使用中，会出现句子成分不全、句式结构错误、时态等方面的错误(见表7)。 在表达中，同样会受中式思维的影响，出现一些中式表达，如： 1) and better medical condition is prepared，表达错误，应该为 and the medical condition is great as well； 2) gets much more clean air，表达错误，应该为gets fresh air； 3) Pollution is full of the atmosphere every day，表达错误，应改为every day， the air is heavily polluted。 在SpeechRater®词汇复杂度和准确性、词汇多样性及语法准确性这三个维度上，学生表现所处的百分位数区间基本在65~100，即其表现仅超过65%~100%的学生。个别会低于60，但从整体上来看，其口语表现会超过绝多数学生。主要问题已经从基础的词句问题过渡到对长句结构的把握，个别语言错误并不会造成对内容的理解。 3) 不同等级分数段SpeechRater®评分情况简述 为了了解SpeechRater®评分的情况，在选择样本时我们对学生的作答录音进行了筛选，剔除了与题目无关的作答及0分作答，同时确定了样本录音分数的合理性。在8位有3年以上教学经验的教师进行文本分析时，要求他们仅参考文本内容、SpeechRater®评分指标和Lexical Tutor统计数据给出自己的分数判定(每个文本均由两位老师评分)。笔者将分差大于0.5的结果进行了统计，如下表8： Table 8 The statistic regarding the difference between SpeechRater® score and teachers’ average score in different grades (scored greater than 0.5 分数段 文本总数量 均分分差大于0.5 总计 1~2分 8 2 2 2~3分 40 8 8 3~4分 32 26 26 表8. 不同等级分数段SpeechRater®评分与教师均分分差大于0.5分统计 从表8中可以看出，不同等级分数段教师评分均分与SpeechRater®评分的差异情况分别为2个、8个和26个。其中1~2分段，由于一个学生录音的声音太小，导致了识别过程中出现了大量的误判情况，教师均分可达到2.75分，SpeechRater®评分为1.8分，可见在使用SpeechRater®中，录音质量，声音不能过小也是非常重要的。另一个学生录音，教师均分给到了2.3分，SpeechRater®评分为1.7分，查看学生录音发现，学生口语表达中存在过长停顿、发音等问题使得分数比教师评分要低。对于SpeechRater®而言，其在发音、停顿等现象上要比教师更为客观和敏感，评分也更为严格。2~3分段，共有8个，尚可。但是在3~4分段，有26个文本分差超过0.5分，其中有25个出现教师均分低于SpeechRater®评分。根据教师描述发现，基本可分为两种情况：第一种情况是学生词汇、语法掌握基本娴熟，错误较少，在语言使用上也更加丰富和多样，但是在观点内容上有所欠缺，观点的展开不够，内容较少，没有对观点进行很好的解释和支持。如提到城市和乡村哪个好，学生选择说城市好，但只用例子说明了农村如何不方便，却没有对城市如何方便做足够的展开。第二种情况是整体观点的表述并无太多问题，些许展开不够，在逻辑性方面表现一般，但是在词汇和语法的使用上不准确，长句子存在的语法问题较多，导致表意不清，中式表达也在一定程度上影响了理解。如学生在表达中语病较多，使用了复合句式，但是引导词有时会用错，句子成分也有出现缺失的情况，词汇语法的运用上也出现了一些初级错误，需要仔细听去理解学生表达的意思。但是这个阶段的学生，语言流利程度普遍相对较好，语速快，发音也没什么问题，所以SpeechRater®的评分会较高。总而言之，SpeechRater®评分是基于12个维度给出的综合评判，而教师仅参考了文本内容进行评判，或多或少会存在差异，但是能够看出SpeechRater®在观点的陈述、内容的开展方面并不能做到准确的评判，评分会出现虚高的情况。