Study Finds ChatGPT Outperforms Students in University Assessments

Study Finds ChatGPT Outperforms Students in University Assessments

Study Finds ChatGPT Outperforms Students in University Assessments

A recent study conducted by Yasir Zaki and his team at New York University Abu Dhabi has revealed that the artificial intelligence chatbot, ChatGPT, may perform at least as well as, and sometimes better than, students in university assessments. However, this seems to be limited to questions that require memory recall and have clear answers, rather than those that necessitate critical analysis.

Zaki’s team reached out to colleagues from various departments, including computer science, psychology, political science, and business, to obtain assessment questions from courses taught at their university. These questions, along with real student answers, were then processed by ChatGPT to generate its own responses. These responses were evaluated alongside the student answers by a team of graders who were unaware of their sources or the purpose of the grading.

Out of the 32 courses examined, ChatGPT’s answers were rated as good as or better than those of students in nine of them. Notably, ChatGPT significantly outperformed students in a course called Introduction to Public Policy by achieving almost double their average score.

However, the study also raised concerns about the design of university assessments. According to Thomas Lancaster from Imperial College London, the results indicate potential flaws in the current assessment methods. Assessments should focus on probing students’ critical thinking abilities, which ChatGPT may not possess. The study suggests that if better answers can be produced using ChatGPT, it further exposes the flaws in the assessment design.

In addition to the assessment comparison, Zaki’s team conducted a survey among academics and students in the UK, US, India, Japan, and Brazil to gauge their attitudes towards ChatGPT. Across all countries, the students expressed a stronger likelihood of using the chatbot than the academics anticipated.

While the study demonstrates the potential of ChatGPT in specific types of assessments, it also highlights the importance of incorporating critical thinking skills into university assessments to ensure a comprehensive evaluation of students.

– Angel Garcia/Bloomberg via Getty Images

Tags: , ,