L19 SUPPLIED RESEARCH SUMMARY 2
Generative AI and learning: what the recent evidence shows

Read this during L19. One claim in this summary is weakly supported and is
marked [WEAK]. Read it the same way you read the others.

------------------------------------------------------------
A. A field experiment with GPT-4 in high-school math
------------------------------------------------------------
Study:  Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakci, O., &
        Mariman, R. (2025). Generative AI without guardrails can harm
        learning: Evidence from high school mathematics. Proceedings of
        the National Academy of Sciences, 122(26), e2422633122.

Who and where
  Nearly 1,000 students in grades 9, 10 and 11 at one high school in
  Turkey, in fall 2023. Whole classrooms were randomly assigned. The trial
  was pre-registered.

What was done
  Four 90-minute sessions. Each had the same shape: the teacher reviewed the
  topic, students worked practice problems, then students took a closed-book
  exam with no laptop. Only the practice step differed:
    Control     textbook and notes, no device
    GPT Base    a plain chat interface on GPT-4, like ChatGPT
    GPT Tutor   GPT-4 with teacher-designed prompts that gave hints and
                pushed students to do the steps, not hand over answers

What was found
  During practice, with the AI available:
    GPT Base   grades 48% higher than control
    GPT Tutor  grades 127% higher than control
  On the exam, with the AI taken away:
    GPT Base   grades 17% lower than control
    GPT Tutor  no measurable difference from control
  The authors' reading is that GPT Base students used the tool as a crutch:
  they got answers without doing the work, and it showed when the tool was
  gone. GPT Base gave a correct answer only about 51% of the time, with
  logical errors far more common than arithmetic ones.
  Students in the GPT Base group did not perceive that they had learned
  less. Students in the GPT Tutor group perceived that they had done
  better on the exam. They had not.

Honest limitation
  One school, one country, one subject with clear right answers, short-term
  outcomes only (the exam followed the practice in the same session), run
  in 2023 when these tools were new to the students. The "crutch"
  explanation is the authors' interpretation, which they present as what
  the data suggest, not something measured directly. The result shows that
  unguarded use CAN harm learning. It does not show that every use harms
  learning, and the tutor version did not improve exam results either.

------------------------------------------------------------
B. Cognitive offloading: the concept
------------------------------------------------------------
Source: Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading.
        Trends in Cognitive Sciences, 20(9), 676-688.

What it says
  This is a review article. It defines cognitive offloading as "the use of
  physical action to alter the information processing requirements of a
  task" so as to reduce cognitive demand. Examples: setting a phone
  reminder for an appointment, tilting your head to read a rotated image.
  The review asks what makes people offload and what it costs them.

Honest limitation
  The review predates large language models. It gives you the concept and
  the questions, not evidence about AI tools.

------------------------------------------------------------
C. [WEAK] "Using ChatGPT builds up cognitive debt in your brain"
------------------------------------------------------------
Source: Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H.,
        Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on
        ChatGPT: Accumulation of cognitive debt when using an AI assistant
        for essay writing task. arXiv preprint 2506.08872.

What was done
  54 adults were split into three groups and wrote essays across three
  sessions: one group with an LLM, one with a search engine, one with no
  tools. Brain activity was recorded with EEG. In a fourth session some
  participants switched conditions; 18 of the 54 completed it.

What is claimed
  The no-tools group showed the strongest brain connectivity and the LLM
  group the weakest. LLM users reported the lowest sense of ownership of
  their essays and had trouble quoting what they had just written.

Why this is marked WEAK
  It is a preprint. The authors' own project site says it has not been peer
  reviewed and its conclusions should be treated as preliminary. The sample
  is small. The key fourth-session comparison rests on 18 people. EEG
  connectivity during writing is not the same thing as learning or
  retention. Headlines describing this study as proof that AI "damages your
  brain" go well beyond what it measured. The finding about ownership and
  quoting is worth noticing; the brain claims are not established.

------------------------------------------------------------
What this summary does NOT tell you
------------------------------------------------------------
There is no settled body of evidence yet on AI and learning for adult
learners or in college courses. The strongest study above is one
well-designed trial in one setting. Anyone who tells you the question is
closed, in either direction, is selling something.
