Prompting Minds: Evaluating how Students Perceive Generative AI’s Critical Thinking Dispositions

Abstract

As generative artificial intelligence tools like ChatGPT become increasingly integrated into educational environments, understanding their impact on critical thinking is crucial. Despite growing concerns about AI's potential to diminish students' independent reasoning, there is a lack of research tools specifically designed to evaluate students' perceptions of AI's cognitive capabilities. To address this gap, this study introduces the Perceived Critical Thinking Disposition of Generative Artificial Intelligence (PCTD-GAI) scale, aimed at measuring how students perceive generative AI’s (GAI) six critical thinking dispositions (reasoning, access to justice, search for evidence, search for truth, open-mindedness, and systematicity). While this study validates the scale using ChatGPT, the instrument is adaptable for evaluating other generative AI tools, supporting broader research in AI-driven learning environments, to assess not only how students engage with AI, but also how their reliance on AI may affect their cognitive development and self-regulated learning skills in digital education. To develop and validate the PCTD-GAI scale, the Marmara Critical Thinking Dispositions Scale (MCTDS) was adapted, ensuring relevance to AI assessment while maintaining conceptual robustness. A quantitative cross-sectional study was conducted with 931 university students from Portugal and Poland, employing exploratory and confirmatory factor analyses (EFA & CFA) to assess the scale’s validity and reliability. The results demonstrate that the PCTD-GAI effectively captures students’ perceptions of ChatGPT’s critical thinking dispositions across six key dimensions. Findings indicate moderately positive perceptions across both countries, with Portuguese students consistently rating ChatGPT marginally higher across domains and showing less response variability, suggesting greater consensus. Notably, perceptions were most neutral in the "truth-seeking" domain, while systematicity received the highest ratings, reflecting ChatGPT’s perceived systematic capabilities among students. These findings have significant implications for e-learning and AI-driven education. The PCTD-GAI scale enables educators to track students’ evolving AI literacy and develop targeted interventions that promote critical AI engagement rather than passive reliance on AI-generated content. Moreover, this research advances the field of e-learning by offering an empirical basis for integrating AI assessment into digital learning strategies, ensuring that AI serves as a cognitive tool rather than a substitute for independent reasoning. The validated PCTD-GAI scale provides a reliable, scalable method for assessing students’ perceptions of AI's cognitive capabilities, supporting evidence-based AI pedagogy, and guiding institutional policies on AI integration in education.