The Impact of Rapid Adoption of Online Assessment on Students’ Performance and Perceptions: Evidence from a Distance Learning University

Authors

DOI:

https://doi.org/10.34190/ejel.20.3.2399

Keywords:

e-learning, distance education, online exams, online assessment, students’ performance

Abstract

One of the most sensitive changes faced by universities due to the COVID-19 crisis was the remote assessment of student learning. This research analysed the case of a massive distance learning university that rapidly changed the final assessment (N=126,653 undergraduate students in 2020) from face-to-face exams to entirely online exams. The research focused on the influence of online assessment on academic performance and students’ perception of the new method. Two data sources were used: the contrast of academic performance indicators (assessment, success and achievement rates, and average marks obtained) between the online examination call and the previous ones with face-to-face examinations; and a questionnaire to a sample of students (n=714) on their perception of the online assessment experience. The results show that all the academic performance indicators in the 28 Bachelor Degrees offered at the university increased when the final assessment method turned to online due to the pandemic crisis; and that a majority of students are more favourable to online assessment methods. The discussion places these findings in a context of rapid change, and concludes by identifying the possible implications of online assessment for student retention, organisational challenges, as well as possible further studies. 

References

Alyahya, S. and Aldausari, A., 2021. An Electronic Collaborative Learning Environment for Standardized Tests. Electronic Journal of e-Learning, 19(3), pp.90–106. https://doi.org/10.34190/ejel.19.3.2167

Alessio, H.M., Malay, N., Maurer, K., Bailer, A.J. and Rubin, B., 2018. Interaction of proctoring and student major on online test performance. International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3698

Appiah, M. and Van Tonder, F., 2018. E-Assessment in Higher Education: A Review. International Journal of Business Management & Economic Research, 9(6). Available at: <http://www.ijbmer.com/docs/volumes/vol9issue6/ijbmer2018090601.pdf> [Accessed 22 July 2021].

Aretio, L.G., 2019. El problema del abandono en estudios a distancia. Respuestas desde el Diálogo Didáctico Mediado. RIED. Revista Iberoamericana de Educación a Distancia, 22(1), pp.245–270. http://dx.doi.org/10.5944/ried.22.1.22433

Arnold, I.J., 2016. Cheating at online formative tests: Does it pay off? The Internet and Higher Education, 29, pp.98–106. https://doi.org/10.1016/j.iheduc.2016.02.001

Attia, M.A., 2014. Postgraduate students’ perceptions toward online assessment: The case of the faculty of education, Umm Al-Qura university. In Education for a Knowledge Society in Arabian Gulf Countries. Emerald Group Publishing Limited. https://doi.org/10.1108/S1479-367920140000024015

Bacow, L.S., Bowen, W.G., Guthrie, K.M., Long, M.P. and Lack, K.A., 2012. Barriers to adoption of online learning systems in US higher education (pp. 39–51). New York, NY: Ithaka. Available at: <http://major21.wdfiles.com/local--files/archive/BarrierstoAdoptionofOnlineLearningSystemsinUSHigherEducation-DJR%20Comments.pdf> [Accessed 25 July 2021].

Bertheussen, B.A. and Myrland, Ø., 2016. Relation between academic performance and students' engagement in digital learning activities. Journal of Education for Business, 91(3), pp.125–131. https://doi.org/10.1080/08832323.2016.1140113

Böhmer, C., Feldmann, N. and Ibsen, M., 2018, April. E-exams in engineering education—online testing of engineering competencies: Experiences and lessons learned. In 2018 IEEE global engineering education conference (EDUCON) (pp. 571–576). IEEE. https://www.doi.org/10.1109/EDUCON.2018.8363281

Butler-Henderson, K. and Crawford, J., 2020. A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity. Computers & Education, 159. https://doi.org/10.1016/j.compedu.2020.104024

Capraro, V., 2017. Does the truth come naturally? Time pressure increases honesty in one-shot deception games. Economics Letters, 158, pp.54–57. https://doi.org/10.1016/j.econlet.2017.06.015

Conrad, D., 2013. Assessment challenges in open learning: Way-finding, fork in the road, or end of the line? Open Praxis, 5(1), pp.41–47. http://dx.doi.org/10.5944/openpraxis.v5i1.17

Charteris, J., Quinn, F., Parkes, M., Fletcher, P. and Reyes, V., 2016. e-Assessment for learning and performativity in higher education: A case for existential learning. Australasian Journal of Educational Technology, 32(3). https://doi.org/10.14742/ajet.2595

Chaudhary, S. and Dey, N., 2013. Assessment in open and distance learning system (ODL): A challenge. Open Praxis, 5(3), pp.207–216. http://dx.doi.org/10.5944/openpraxis.5.3.65

Cheng, A.C., Jordan, M.E., Schallert, D.L. and The, D., 2013. Reconsidering assessment in online/hybrid courses: Knowing versus learning. Computers & Education, 68, pp.51–59. https://doi.org/10.1016/j.compedu.2013.04.022

Chen, B., West, M. and Zilles, C., 2017, April. Do performance trends suggest wide-spread collaborative cheating on asynchronous exams?. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale (pp. 111–120). https://doi.org/10.1145/3051457.3051465

Daffin Jr, L.W. and Jones, A.A., 2018. Comparing Student Performance on Proctored and Non-Proctored Exams in Online Psychology Courses. Online Learning, 22(1), pp.131–145. http://dx.doi.org/10.24059/olj.v22i1.1079

Dalby, D. and Swan, M., 2019. Using digital technology to enhance formative assessment in mathematics classrooms. British journal of educational technology, 50(2), pp.832–845. https://doi.org/10.1111/bjet.12606

Ellis, C., 2012. Streamlining plagiarism detection: The role of electronic assessment management. International Journal for Educational Integrity, 8(2), pp.46–56. https://doi.org/10.21913/IJEI.v8i2.809

Elmehdi, H.M. and Ibrahem, A.M., 2019. Online summative assessment and its impact on students’ academic performance, perception and attitude towards online exams: University of Sharjah Study Case. In Creative business and social innovations for a sustainable future (pp. 211–218). Cham: Springer. https://doi.org/10.1007/978-3-030-01662-3

European Association for Quality Assurance in Higher Education (ENQA), 2015. Standards and guidelines for quality assurance in the European Higher Education Area (ESG). http://www.enqa.eu/wp-content/uploads/2015/11/ESG_2015.pdf

Eyal, L., 2012. Digital assessment literacy—The core role of the teacher in a digital environment. Journal of Educational Technology & Society, 15(2), pp.37–49. Available at: <https://drive.google.com/file/d/1gmUp2RJIekEyvyLt5nJ5knq5KyVU44ee/view> [Accessed 25 July 2021].

Faber, J.M., Luyten, H. and Visscher, A.J., 2017. The effects of a digital formative assessment tool on mathematics achievement and student motivation: Results of a randomized experiment. Computers & Education, 106, pp.83–96. https://doi.org/10.1016/j.compedu.2016.12.001

Fask, A., Englander, F. and Wang, Z., 2014. Do online exams facilitate cheating? An experiment designed to separate possible cheating from the effect of the online test taking environment. Journal of Academic Ethics, 12(2), pp.101–112. https://doi.org/10.1007/s10805-014-9207-1

Ferrão, M., 2010. E‐assessment within the Bologna paradigm: evidence from Portugal. Assessment & Evaluation in Higher Education, 35(7), pp.819–830. https://doi.org/10.1080/02602930903060990

Ferrell, G., 2013. Supporting assessment and feedback practice with technology: From tinkering to transformation. JISC Assessment and Feedback Programme. Available at: <http://repository.jisc.ac.uk/5450/4/Jisc_AF_Final_Synthesis_Report_Oct_2013_v2.pdf> [Accessed 25 July 2021].

Foerster, M., Gourdin, A. and Huertas, E., 2019. Framework for the quality assurance of e-assessment. Brussels: European Association for Quality Assurance in Higher Education. Available at: <http://hdl.voced.edu.au/10707/522256> [Accessed 22 July 2021].

Garrison, D.R., 1987. Researching dropout in distance education. Distance Education, 8(1), pp.95–101. https://doi.org/10.1080/0158791870080107

Gaytan, J. and McEwen, B.C., 2007. Effective online instructional and assessment strategies. The American Journal of Distance Education, 21(3), pp.117–132. https://doi.org/10.1080/08923640701341653

Gikandi, J.W., Morrow, D. and Davis, N.E., 2011. Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), pp.2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004

Gil-Jaurena, I., Domínguez, D. and Ballesteros, B., 2020. Learning outcomes based assessment in higher distance education. A case study. Open Learning: The Journal of Open and Distance Learning. https://doi.org/10.1080/02680513.2020.1757419"

Gray, L. and Ferrell, G., 2013. Electronic assessment management. Available at: <https://www.jisc.ac.uk/guides/electronic-assessment-management> [Accessed 20 July 2021].

Guàrdia, L., Crisp, G. and Alsina, I., 2017. Trends and challenges of e-assessment to enhance student learning in Higher Education. In Innovative practices for higher education assessment and measurement (pp. 36–56). Hershey, PA: IGI Global.

Guerrero-Roldán, A.E. and Noguera, I., 2018. A model for aligning assessment with competences and learning activities in online courses. The Internet and Higher Education, 38, pp.36–46. https://doi.org/10.1016/j.iheduc.2018.04.005

Hollister, K.K. and Berenson, M.L., 2009. Proctored versus unproctored online exams: Studying the impact of exam environment on student performance. Decision Sciences Journal of Innovative Education, 7(1), pp.271–294. https://doi.org/10.1111/j.1540-4609.2008.00220.x

Hylton, K., Levy, Y. and Dringus, L.P., 2016. Utilizing webcam-based proctoring to deter misconduct in online exams. Computers & Education, 92, pp.53–63. https://doi.org/10.1016/j.compedu.2015.10.002

International Association of Universities, 2020. COVID-19: Higher Education challenges and responses. Available at: <https://www.iau-aiu.net/COVID-19-Higher-Education-challenges-and-responses> [Accessed 22 July 2021].

Karadag, N. and Özgür, A.Z., 2020. Assessment and Evaluation in Mega Universities. Turkish Online Journal of Educational Technology-TOJET, 19(4), pp.35–49. Available at: <http://www.tojet.net/articles/v19i4/1943.pdf> [Accessed 20 July 2021].

Kim, N., Smith, M.J. and Maeng, K., 2008. Assessment in online distance education: A comparison of three online programs at a university. Online Journal of Distance Learning Administration, 11(1). Available at: <https://www.westga.edu/~distance/ojdla/spring111/kim111.html> [Accessed 20 July 2021].

Kubesch, M., Lankes, M. and Maurer, B., 2019, October. Exploring the Effects of Time Pressure on Screen-Cheating Behaviour: Insights and Design Potentials. In Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (pp. 459–465). https://doi.org/10.1145/3341215.3356260

Langenfeld, T., 2020. Internet‐Based Proctored Assessment: Security and Fairness Issues. Educational Measurement: Issues and Practice, 39(3), pp.24–27. https://doi.org/10.1111/emip.12359

Levy, Y. and Ramim, M., 2012. [Chais] A Study of Online Exams Procrastination Using Data Analytics Techniques. Interdisciplinary Journal of E-Learning and Learning Objects, 8(1), pp.97–113.

Makransky, G., Mayer, R., Nøremølle, A., Cordoba, A.L., Wandall, J. and Bonde, M., 2020. Investigating the feasibility of using assessment and explanatory feedback in desktop virtual reality simulations. Educational Technology Research and Development, 68(1), pp.293–317. https://doi.org/10.1007/s11423-019-09690-3

Matthíasdóttir, Á. and Arnalds, H., 2016, June. E-assessment: students' point of view. In Proceedings of the 17th international conference on computer systems and technologies 2016 (pp. 369–374). https://doi.org/10.1145/2983468.2983497

Milone, A.S., Cortese, A.M., Balestrieri, R.L. and Pittenger, A.L., 2017. The impact of proctored online exams on the educational experience. Currents in Pharmacy Teaching and Learning, 9(1), pp.108–114. https://doi.org/10.1016/j.cptl.2016.08.037

Naffi, N., 2020. Disruption in and by centres for teaching and learning during the COVID-19 pandemic: Leading the future of higher Ed. Québec City: L’Observatoire Internationale sur les Impacts Sociétaux de l’IA et du Numerique and the Government of Québec. Available at: <https://www.docdroid.net/L0khasC/whitepaper-disruption-in-and-by-centres-for-teaching-and-learning-during-the-covid-19-pandemic-leading-the-future-of-higher-ed-21-08-2020-pdf> [Accessed 30 July 2021].

Nardi, A. and Ranieri, M., 2019. Comparing paper‐based and electronic multiple‐choice examinations with personal devices: Impact on students' performance, self‐efficacy and satisfaction. British Journal of Educational Technology, 50(3), pp.1495–1506. https://doi.org/10.1111/bjet.12644

Okada, A., Noguera, I., Alexieva, L., Rozeva, A., Kocdar, S., Brouns, F., Ladonlahti, T., Whitelock, D. and Guerrero‐Roldán, A.E., 2019. Pedagogical approaches for e‐assessment with authentication and authorship verification in Higher Education. British Journal of Educational Technology, 50(6), pp.3264–3282. https://doi.org/10.1111/bjet.12733

Pagram, J., Cooper, M., Jin, H. and Campbell, A., 2018. Tales from the exam room: Trialing an e-exam system for computer education and design and technology students. Education Sciences, 8(4). https://doi.org/10.3390/educsci8040188

Pauli, M. and Ferrell, G., 2020. The future of assessment: five principles, five targets for 2025. http://repository.jisc.ac.uk/7733/1/the-future-of-assessment-report.pdf

Pilli, O. and Aksu, M., 2013. The effects of computer-assisted instruction on the achievement, attitudes and retention of fourth grade mathematics students in North Cyprus. Computers & Education, 62, pp.62–71. https://doi.org/10.1016/j.compedu.2012.10.010

Portolese, L., Krause, J. and Bonner, J., 2016. Timed Online Tests: Do Students Perform Better With More Time? American Journal of Distance Education, 30(4), pp.264–271. https://doi.org/10.1080/08923647.2016.1234301

Quality Assurance Agency for Higher Education, 2020. Building a taxonomy for digital learning. Available at: <https://www.qaa.ac.uk/docs/qaa/guidance/building-a-taxonomy-for-digital-learning.pdf> [Accessed 20 July 2021].

Raines, D.A., Ricci, P., Brown, S.L., Eggenberger, T., Hindle, T. and Schiff, M., 2011. Cheating in online courses: The student definition. Journal of Effective Teaching, 11(1), pp.80–89.

Smith, J.S., 2017. Assessing creativity: Creating a rubric to effectively evaluate mediated digital portfolios. Journalism & Mass Communication Educator, 72(1), pp.24–36. https://doi.org/10.1177/1077695816648866

Soffer, T., Kahan, T. and Livne, E., 2017. E-assessment of online academic courses via students' activities and perceptions. Studies in Educational Evaluation, 54, pp.83–93. https://doi.org/10.1016/j.stueduc.2016.10.001

Stack, S., 2015. The impact of exam environments on student test scores in online courses. Journal of Criminal Justice Education, 26(3), pp.273–282. https://doi.org/10.1080/10511253.2015.1012173

Stödberg, U., 2012. A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), pp.591–604. https://doi.org/10.1080/02602938.2011.557496

Stowell, J.R. and Bennett, D., 2010. Effects of online testing on student exam performance and test anxiety. Journal of Educational Computing Research, 42(2), pp.161–171. https://doi.org/10.2190/EC.42.2.b

Tawafak, R.M., Romli, A.B., bin Abdullah Arshah, R. and Almaroof, R.A.S., 2018. Assessing the impact of technology learning and assessment method on academic performance. EURASIA Journal of Mathematics, Science and Technology Education, 14(6), pp.2241–2254. https://doi.org/10.29333/ejmste/87117

Timmis, S., Broadfoot, P., Sutherland, R. and Oldfield, A., 2016. Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), pp.454–476. https://doi.org/10.1002/berj.3215

United Nations, 2020. Policy Brief: Education during COVID-19 and beyond. Available at: <https://www.un.org/development/desa/dspd/wp-content/uploads/sites/22/2020/08/sg_policy_brief_covid-19_and_education_august_2020.pdf> [Accessed 28 July 2021].

Van der Cruyssen, I., D’hondt, J., Meijer, E. and Verschuere, B., 2020. Does honesty require time? Two preregistered direct replications of Experiment 2 of Shalvi, Eldar, and Bereby-Meyer (2012). Psychological Science, 31(4), pp.460–467. https://doi.org/10.1177/0956797620903716

Vani, K. and Gupta, D., 2016. Study on Extrinsic Text Plagiarism Detection Techniques and Tools. Journal of Engineering Science & Technology Review, 9(5), pp.9–23. Available at: <http://www.jestr.org/downloads/Volume9Issue5/fulltext2952016.pdf> [Accessed 30 July 2021].

Voogt, J., Erstad, O., Dede, C. and Mishra, P., 2013. Challenges to learning and schooling in the digital networked world of the 21st century. Journal of Computer Assisted Learning, 29(5), pp.403–413. https://doi.org/10.1111/jcal.12029

Yang, C.Y., Chung, T.Y., Hwang, M.S., Li, C.Y. and Yao, J.F.J., 2017, March. Learning Performance Evaluation in eLearning with the Web-Based Assessment. In International Conference on Information Science and Applications (pp. 645–651). Singapore: Springer. https://doi.org/10.1007/978-981-10-4154-9_74

Downloads

Published

25 Mar 2022

Issue

Section

Articles