Evaluation of Machine Learning Algorithm for Automatic Assessment of School Students' English Essay

Abstract

The manual assessment of essays in English language learning often faces challenges related to objectivity and efficiency, especially on a large scale. With advancements in artificial intelligence technology, machine learning-based approaches have begun to be adopted to automate this process through Automated Essay Scoring (AES) systems. However, most existing AES models tend to rely solely on the final scores from the dataset without considering the structural quality of the writing, such as coherence between paragraphs. This study aims to evaluate the effectiveness of machine learning algorithms in assessing school students' essays by adding coherence features as predictor variables in a regression model. This approach uses linguistic feature representation techniques to explicitly build coherence indicators. The proposed model achieved a QWK improvement from 0.69 to 0.89 using SMOTE and coherence features. Meanwhile, human evaluation results showed that the pair of Rater 1 and Rater 2 achieved a QWK of 0.82, the pair of Rater 1 and Rater 3 scored 0.79, and the pair of Rater 2 and Rater 3 scored 0.81. These values indicate a high level of agreement among raters, suggesting that the assessment instrument used is stable. The main contribution of this study is introducing the coherence feature as an explicit predictor in the AES model, filling the gap not provided by standard datasets and proving that coherence improves model accuracy. This research provides practical benefits such as speeding up the evaluation process, reducing teachers' workload, and improving the objectivity and consistency of assessment in language education and evaluation.

GS Cited Analysis