چکیده مقاله
The integration of Artificial Intelligence AI in language assessment has become increasingly prevalent, especially with the emergence of large language models like ChatGPT However, the reliability and accuracy of AI driven grading systems in high stakes language proficiency tests, such as the International English Language Testing System IELTS , remain largely unexamined This study aims to investigate the alignment between scores assigned by ChatGPT 4 and those given by human IELTS raters for Writing Task 2 A dataset of 55 authentic IELTS writing samples was used to conduct a comparative analysis Several statistical tests, including Wilcoxon, Intraclass Correlation Coefficient ICC , and Rater Agreement tests, were employed to assess the consistency, agreement, and accuracy of the AI model’s grading in comparison to human evaluations Results indicate a high degree of alignment, with an ICC value of 0 814 and a weighted kappa of 0 811, suggesting that ChatGPT 4’s grading closely mirrors human raters in most cases However, discrepancies were found in certain individual cases, highlighting areas where AI scoring may still require refinement The findings suggest that while ChatGPT has potential as a supplementary grading tool, further research is needed to address its limitations and ensure fairness in automated language assessment systems
کلیدواژهها
نویسندگان
Ebrahim Fakhri Alamdari
Department of Foreign Languages, QaS C , Islamic Azad University, Qaemshahr, Iran
Shideh Nahavandi
Department of Foreign Languages, QaS C , Islamic Azad University, Qaemshahr, Iran
شیوه ارجاع
Fakhri Alamdari, Ebrahim and Nahavandi, Shideh,1404,Exploring the Role of Artificial Intelligence in Language Assessment: Assessing ChatGPT's Reliability in Grading IELTS Writing Task 2,22nd International Conference on Language , Literature , Culture and History Studies,Tehran
ارائهشده در
مجموعه مقالات بیست و دومین کنفرانس بین المللی مطالعات زبان، ادبیات، فرهنگ و تاریخ24 مرداد 1404 · تهران