Evaluating an AI-Supported Revision Module for Project-Based Research in Teacher Education

Abstract

While Generative Artificial Intelligence (GenAI) allows new methods to support students in writing processes, its incorporation into university teaching needs well-designed pedagogy to ensure academic integrity and autonomy of learners, especially in Project-Based Research (PBR) courses in teacher education, where iterative feedback is important but restricted by the size of groups and limited resources. The current study aims to address the scalability problem in the feedback process by evaluating the AI-Supported Revision Module (AIRM). It is implemented as a scaffold designed according to a rubric and embedded into Moodle, using a local version of the LLaMA-2-7B model for generation of prompts based on instructor comments for revision in four modes: polishing, restructuring, justifying, and synthesizing. The purpose of the module is to provide targeted guidance to students rather than full-text generation and support revision while preserving authorship. A mixed-methods case study approach was used involving 158 primary education student teachers, out of which 132 submitted draft and revision versions, assessed using a six-criterion rubric. A mixed-design repeated measures ANOVA test revealed interaction effects for both Literature Review and Methodology at the level of p < .01, with effect sizes of d = 0.58 and d = 0.52. The results suggest more improvement in structure and synthesis for the AI group than the conventional group, while no significant differences were found for Data Justification and Interpretation, suggesting a boundary between procedural support and higher-order analytical reasoning. Process data analysis revealed active involvement of learners, evidenced by the fact that on average 14.2% of draft content was changed and 39.7% of AI suggestions were rejected. Qualitative data analysis revealed that learners utilized this AI-powered module mostly to increase text coherence and clarity while staying in control of making sense of the content, whereas teachers indicated a shift from superficial to more methodological feedback practices. Thus, the findings demonstrate that GenAI may be successfully implemented in the feedback process as an additional scaffold for revision while still respecting the agency of learners.