A ChatGPT Assisted Reading Protocol for Undergraduate Research Students

Background

Reading is a skill—something that can be practised and taught. Undergraduate research students quickly realise this when they start their studies and are introduced to scientific literature. Most papers are dense, full of jargon and require a trained eye to interpret correctly. Fortunately, a lot of undergraduate courses include journal clubs and classes focused on developing scientific reading and writing skills. Artificial intelligence-based Large Language Models such as ChatGPT now provide an opportunity to extend students learning beyond the classroom.

In this preprint, the authors developed and tested a ‘guided reading protocol’ which combines classical reading approaches (e.g. notetaking) with the use of ChatGPT. The idea behind this protocol is to provide new undergraduate students with a general guideline on how to read scientific papers thereby preventing from getting overwhelmed and, ultimately, discouraged. The protocol was tested in the context of one structural biology paper, full of field-specific terms and methodologies; students were asked to just read the title, authors, abstract, and introduction.

Though this study will need to be followed up and expanded, the findings reported in this pilot study already hint at the benefits of introducing new technologies such as ChatGPT into teaching practices. Perhaps most telling is that all respondents in this study said that they would continue using ChatGPT when reading scientific papers in the future. A clear indication that the authors are onto something here.

Key Findings

Before testing the guided reading protocol, the undergraduate students involved in this study (14 in total; with majors in chemistry, biochemistry, and biology) were prompted to fill out a survey asking them how much and in what way they previously interacted with scientific literature. The following things were noted (among the 14 students):

• 6 students felt that research faculty had prepared them sufficiently to tackle scientific literature and 7 felt the same about their courses.
• Strikingly, most students felt quite confident about their ability to read scientific literature (despite limited experience): 8 students reported feeling confident or very confident.
• On a monthly basis, 4 students would not read any articles, 9 would read 1–2, and only one student would read 3–4 articles.
• Most students read the abstract first (9 students: 64%) and then either read linearly (7: 50%) or jump around the article (7: 50%). No one starts with the conclusions.
• When asked about the most common reading challenges, unfamiliar terminology and deciphering dense, lengthy scientific writing came up. To tackle the former, most students would use search engines and half would use LLMs like ChatGPT.

Following this survey, 11 students (from the initial 14) proceeded to read the assigned structural biology paper using the guided reading protocol. They also handed in the ChatGPT transcripts for analysis by the authors. This analysis showed that:

• Interaction with ChatGPT ranged from 5 to 14 inputs/requests (per student). These revolved around:
  • providing ChatGPT with educational (and other relevant) background information.
  • asking definitions of unfamiliar terms.
  • requesting explanations.
  • summarising text or conversations.
  • simplifying certain parts.

Following the reading exercise there was another survey. When asked about the benefits and challenges of using ChatGPT as part of the guided reading protocol, the students said:

• positive: ChatGPT provided contextual definitions and clear explanations. Students particularly appreciated the dynamic interaction with the AI chatbot.
• negative: some students found it difficult to phrase their questions efficiently and some noted overly lengthy responses.

Conclusions and outlook

All students involved in this study agreed that they would use ChatGPT for reading articles in the future. To allow optimal use of ChatGPT in reading articles, the guided reading protocol devised by the authors could be expanded to include:

• explicit guidance on asking diverse and effective (follow-up) questions.
• additional training in question construction to get the desired response from ChatGPT.

As mentioned earlier, this preprint represents a pilot study. It will be interesting to learn how students experience the reading protocol when reading full journal articles and, in particular, try to make sense of quantitative data and figures.

Why I highlight this preprint

What I like about this preprint is that the authors look at something like ChatGPT and consider its benefits for teaching (and not just the headache it can also be sometimes). In this case, the authors show that the use of ChatGPT could help students master the basic, but essential skill of reading papers. Embracing ChatGPT to optimise teaching methods may be the way forward and through this preprint the authors show us one way of doing so.

Tags: ai, chatgpt, reading, skills, teaching, tools

(No Ratings Yet)