챗GPT를 활용한 영어 작문 평가
Investigating ChatGPT’s Reliability as an Automated Scoring Tool for Small Classes in University English Education
- 발행
- 2024 등록이 논문은 발행 시점을 확인하지 못해 원문 등록일을 적었습니다. KCI가 2005~2008년에 옛 논문을 몰아서 올린 탓에, 그 시기 등록분은 발행보다 평균 3~5년 늦습니다. 실제 발행연도는 더 이를 수 있습니다.
- 소속·발행
- 국립부경대학교
- 출처
- 국내 KCI
- DOI
- 10.21084/jmball.2024.02.42.1.189
- 원문
- 원문 보기 ↗
개념
키워드
자동채점, 챗GPT, 인공지능, 쓰기평가, 영어작문교육, automated grading, ChatGPT, artificial intelligence, writing assessment, English composition education
초록
The purpose of this study is to verify whether ChatGPT can be used as a reliable automated scoring tool in English writing courses. ChatGPT’s reliability has been somewhat verified through large-scale assessments, while its potential as a reliable evaluation tool for small classes in university education settings remains unproven. In this study, ChatGPT-3.5 was used to repeatedly assess 30 English writing tasks based on the IELTS Writing Scale, and a repeated measures analysis of variance was used to statistically verify whether ChatGPT provides consistent responses. The results are summarized as follows. First, ChatGPT was found to be a highly reliable scoring tool with very high internal consistency and high inter-component correlations. Second, the reliability of ChatGPT depends on the scoring criteria. A detailed rubric increases the reliability of scoring. Third, the reliability of ChatGPT depends on the scoring scale. Given constructs only, ChatGPT can rate on a holistic scoring scale, but given constructs and a detailed rubric it can rate on holistic or analytical scales. While this study focused on ChatGPT’s reliability, if avenues for establishing validity are explored, ge