Assessing student engagement in Higher Education: validation of the university student engagement inventory among Maltese University students

JOSEPHINE DEGUARA, JONATHAN BORG, JOÃO MARÔCO, NATALLIA BAHDANOVIC HANSSEN, TRYGVE KVÅLE LØKEN

Abstract

This study examined the psychometric properties of the University Student Engagement Inventory (USEI) among Maltese university students, addressing the lack of validated tools for assessing student engagement (SE) in Malta’s higher education (HE) context. The questionnaire was sent to all 12,400 University of Malta students out of which 323 fully completed the USEI, measuring behavioural (BE), emotional (EE), and cognitive engagement (CE). Confirmatory factor analysis supported both the three-factor model (CFI=0.945, TLI=0.933, RMSEA=0.073) and an excellent second-order model fit (CFI=0.981, TLI=0.977, RMSEA=0.057). Reliability was good for EE (α=0.83, ω=0.81) and CE (α=0.79, ω=0.76), and marginal for BE (α=0.68, ω=0.65), consistent with international findings. Measurement invariance was confirmed across STEM and non-STEM groups, indicating model stability across disciplines. Predictive validity showed that higher SE significantly predicted fewer dropout intentions (β=–0.49, p<0.001) and higher self-reported academic performance (β=0.56, p<0.001). Results confirm the USEI as a reliable and valid tool for assessing SE within the Maltese HE context and confirms its use in identifying disengaged students and informing interventions to enhance persistence and academic success.

Keywords

Student engagement, higher education, USEI, Malta, validation, dropout, measurement invariance

Full Text:

PDF

References

Abela, A. (2023). The prevalence of stress, anxiety, depression and coping in undergraduate nursing students in Malta. Master of Science in Nursing, University of Malta, Malta.

Alarcón, M. L., Villalobos, J., Méndez, I., & González, L. (2022). Adaptación y validación transcultural al español del University Student Engagement Inventory (USEI). Revista Latinoamericana de Psicología, 54, 1-11.

Alrashidi, O., Phan, H. P., & Ngu, B. H. (2016). Academic Engagement: An overview of its definitions, dimensions, and major conceptualisations. International Education Studies, 9(12), 41-52. https://doi.org/10.5539/ies.v9n12p41.

Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: critical conceptual and methodological issues of the construct. Psychology in the Schools, 45, 369-386. https://doi.org/10.1002/pits.20303.

Araújo, A. M., Gomes, C. M. A., Almeida, L. S., & Núñez, J. C. (2019). A latent profile analysis of first-year university students’ academic expectations. Anales de Psicología, 35(1), 58-67, https://doi.org/10.6018/analesps.35.1.299351.

Assunção, H., Lin, S. W., Sit, P. S., Cheung, K., Harju-Luukkainen, H., Smith, T. J., Maloa, B., Campos, J., Ilic, I., Esposito, G., Francesca, F., & Marôco, J. (2020). University Student Engagement Inventory (USEI): Transcultural validity evidence across four continents. Frontiers in Psychology, 10, 2796. https://doi.org/10.3389/fpsyg.2019.02796.

Bakker, A. B., Schaufeli, W. B., Leiter, M. P., & Taris, T. W. (2008). Work Engagement: An emerging concept in occupational Health Psychology. Work & Stress, 22(3), 187-200. https://doi.org/10.1080/02678370802393649.

Behr, A., Giese, M., Teguim Kamdjou, H. D., & Theune, K. (2021). Motives for dropping out from higher education. An analysis of bachelor’s degree students in Germany. European Journal of Education Research, Development and Policy, 56, 325-343. https://doi.org/10.1111/.ejed.12433.

Biggs, J., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149. https://doi.org/10.1348/000709901158433.

Buntins K., Kerres M., & Heinemann, A. (2021). A scoping review of research instruments for measuring student engagement: In need for convergence. International Journal of Educational Research Open, 2,100099. https://doi.org/10.1016/j.ijedro.2021.100099.

Camilleri, C. (2017). Stress and coping mechanisms among nursing students on the three-year Diploma Programme in Malta: A case study approach. PhD in Nursing, University of Malta, Malta.

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5.

Christenson, S. L., Reschly A. L., & Wylie C. (2012). Handbook of research on student engagement. Springer.

Esposito, G., Marôco, J., Passeggia, R., Pepicelli, G., & Freda, M. F. (2021). The Italian validation of the University Student Engagement Inventory. European Journal of Higher Education, 12(1), 35-55. https://doi.org/10.1080/21568235.2021.1875018.

Eurostat. (2024a). Early leavers from education and training. https://ec.europa.eu/eurostat/statistics-explained/index.php.

title=Early_leavers_from_education_and_training.

Eurostat. (2024b). Early school leavers down to 9.5% in 2023. https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20240523-1.

Eurydice. (2025). Malta: Higher Education; Types of Higher Education Institutions. https://eurydice.eacea.ec.europa.eu/eurypedia/malta/types-higher-educationinstitutions#:~:text=On%20this%20page,the%20Council

%20and%20the%20Senate.

Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds), Structural equation modeling: A second course (pp. 269-314). Information Age Publishing.

Fredricks, J. A. (2015). Academic Engagement. In J. D. Wright (Ed.). International Encyclopedia of the Social & Behavioral Sciences (pp. 31-36). Elsevier.

Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In S. L. Christenson, A. L. Reschly & C. Wylie (Eds), Handbook of research on student engagement (pp. 763-782). Springer Science + Business Media.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School Engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. https://doi.org/10.3102/00346543074001059.

Fredricks, J. A., Filsecker, M., & Lawson, M. (2016). Student engagement, context and adjustment: Addressing definitional, measurement and methodological issues. Learning and Instruction, 43, 1-4.

Fugard, A. J. B., & Potts, H. W. W. (2014). Supporting thinking on sample sizes for thematic analyses: A quantitative tool. International Journal of Social Research Methodology, 18(6), 669-684. http://dx.doi.org/10.1080/13645579.2015.1005453.

Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72-91. https://doi.org/10.1037/a0032138.

Holdsworth, E., Bowen, E., Brown, S., & Howat, D. (2014). Client Engagement in psychotherapeutic treatment and associations with client characteristics, therapist characteristics, and treatment factors. Clinical Psychology Review, 34(5), 428-450. https://doi.org/10.1016/j.cpr.2014.06.004.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118.

Hu, M., & Li, H. (2017). Student engagement in online learning: A review. Paper presented at the 2017 International Symposium on Educational Technology (ISET). IEEE.

Hu, S., & Kuh, G. D. (2002). Being (dis)Engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education, 43(5), 555-575. https://doi.org/10.1023/A:1020114231387.

Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2022). semTools: Useful tools for structural equation modeling (Version 0.5-6) [R package]. https://CRAN.R-project.org/package=semTools.

Kahu, E. R., (2013). Framing student engagement in Higher Education. Studies in Higher Education, 38(5), 758-773. https://doi.org/10.1080/03075079.2011.598505.

Kızıldağ, S., Demirtaş-Zorbaz, S., & Zorbaz, O. (2017). School engagement of high school students. Education and Science, 42(189), 107-119. https://doi.org/10.15390/EB.2017.6740.

Kuh, G. D. (2003). What we're learning about student engagement from NSSE: Benchmarks for effective educational practices. Change: The Magazine of Higher Education, 35, 24–32. https://doi.org/10.1080/00091380309604090.

Kuh, G. D. (2009). The National Survey of student engagement: Conceptual and empirical foundations. New Directions for Institutional Research, 141, 5-20. https://doi.org/10.1002/ir.283.

Lawson, M. A., & Lawson, H. A. (2013). New conceptual frameworks for student engagement research, policy, and practice. Review of Educational Research, 83(3), 432-479. https://doi.org/10.3102/0034654313480891.

Løken, T. K., Hanssen, N. B., Olsen, K., & Marôco, J. (2026) Norwegian validation of the University Student Engagement Inventory. Frontiers in Psychology, 17, 1734409. https://doi.org/10.3389/fpsyg.2026.1734409.

Marôco, J. (2021). Análise de equações estruturais: Fundamentos teóricos, software e aplicações (3.ª ed.). Reportnumber, Lda.

Marôco, J. (2024). Factor Analysis of Ordinal Items: Old questions, modern solutions? Stats, 7(3), 984-1001. https://doi.org/10.3390/stats7030060.

Marôco, J., Marôco, A. L., Campos, J., & Fredricks, J. A. (2016). University student engagement: Development of the University Student Engagement Inventory (USEI). Psicologia: Reflexão e Crítica, 29, 21. https://doi.org/10.1186/s41155-016-0042-8.

Marôco, J., Assunção, H., Harju-Luukkainen, H., Lin, S.-W., Sit, P.-S., Cheung, K.-C., Maloa, B., Stepanović Ilic, I., Smith, T. J., & Campos, J. A. D. B. (2020). Predictors of academic efficacy and dropout intention in university students: Can engagement suppress burnout? PLOS ONE, 15(10), e0239816. https://doi.org/10.1371/journal.pone.0239816.

McCormick, A. C., Kinzie, J., & Gonyea, R. M. (2013). Student Engagement: Bridging research and practice to improve the quality of undergraduate education. In M. B. Paulsen (Ed.), Higher Education: Handbook of Theory and Research (pp. 47-92). Springer. https://doi.org/10.1007/978-94-007-5836-0_2.

Micallef, B. (2025). Enhancing curriculum relevance and student engagement: The role of lecturers as agents of change in higher education. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1580611.

Ministry for Education and Employment. (2017). Student engagement in school life and learning. National Commission for Higher Education, Malta (NCFHE). https://mfhea.mt/student-engagement-in-school-life-learning/.

Ministry for Education, Sport, Youth, Research and Innovation. (2022-2023). The Early Leaving from Education and Training Strategy: The Way Forward 2023-2030. https://education.gov.mt/wp-content/uploads/2023/07/ELET-Strategy-English-18th-July-1530

.pdf.

NSO Malta (National Statistics Office Malta). (2024). Students in post-secondary and tertiary education: 2022-2023. NR207/2024. https://nso.gov.mt/education/students-in-post-secondary-and-tertiary-education-2022-2023/.

NSSE (National Survey of Student Engagement). (2013). The Beginning College Survey of Student Engagement (BCSSE) Survey. https://nsse.indiana.edu/bcsse/BCSSE-survey/index.html.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

Philp, J., & Duchesne, S. (2016). Exploring Engagement in Tasks in the Language Classroom. Annual Review of Applied Linguistics, 36, 50-72. https://doi.org/10.1017/S026719051500009.

Reeve, J., & Tseng, C. M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36(4), https://doi.org/10.1016/j.cedpsych.2011.05.002.

Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02.

Rovai, A. P. (2002). Development of an instrument to measure classroom community. The Internet and Higher Education, 5(3), 197-211. https://doi.org/10.1016/S1096-7516(02)00102-1.

Sáez-Delgado, F., Mella-Norambuena, J., López-Angulo Y., Sáez, Y., & León-Ron, V. (2023). Invariant and suboptimal trajectories of self-regulated learning during secondary school: implications focused on quality in higher education. Frontier in Psychology, 14, 1235846. https://doi.org/10.3389/fpsyg.2023.1235846.

Schaufeli, W. B., Bakker, A. B., & Salanova, M. (2003). Utrecht Work Engagement Scale-9 (UWES-9). https://scales.arabpsychology.com/s/utrecht-work-engagement-scale-uwes-schauf

eli-bakker-2003-2006-2007/.

Sharif Nia, H., Azad Moghddam, H., Marôco, J., Rahmatpour, P., Allen, K.-A., Kaur, H., Kaveh, O., Gorgulu, O., & Pahlevan Sharif, S. (2022). A psychometric lens for e-learning: Examining the validity and reliability of the Persian version of University Students’ Engagement Inventory (P-USEI). The Asia-Pacific Education Researcher, 32(4), 573-582. https://doi.org/10.1007/s40299-022-00677-y.

Sharif-Nia, H., Marôco, J., Rahmatpour, P., Allen, K.-A., & Rezaei, H. (2023). Psychometrics evaluation of the university student engagement inventory in online learning among Arab students. BMC Nursing, 22, 158. https://doi.org/10.1186/s12912-023-01318-5.

Sinval, J., Queirós, C., Pasian, S. R., & Maroco, J. (2019). Transcultural adaptation of the Oldenburg Burnout Inventory (OLBI) for Brazil and Portugal. Frontiers in Psychology, 10, 338. https://doi.org/10.3389/fpsyg.2019.00338.

Solomonides, I. P., & Martin, P. (2008). ‘All this talk of engagement is making me itch’: An investigation into the conceptions of ‘engagement’ held by students and tutors. In L. Hand & C. Bryson (Eds), Student Engagement (pp. 13-18). Staff and Educational Development Association.

Studenti Demokristjani Maltin. (2025). Bridging books & paychecks: A social approach to student employment. https://issuu.com/sdm-studentidemokristjanimaltin/docs/bridging_books

_paychecks_a_social_policy_approa.

Synowiec, C., Fletcher, E., Heinkel, L., & Salisbury, T. (2023). Getting rigor right: A framework for methodological choice in adaptive monitoring evaluation. Global Health: Science and Practice, 11(2), 1-13. https://pmc.ncbi.nlm.nih.gov/articles/PMC10727457/.

University of Malta. (n.d.). Research ethics. https://www.um.edu.mt/research/ethics/.

University of Malta. (2023a). Gender Audit Report 2020-2022, Equity Office, 2023. https://www.um.edu.mt/media/um/docs/about/equity/genderauditreport2020-2022.pdf.

University of Malta. (2023b). Your academic year 2023/2024 at the University of Malta, in numbers. Newspoint. https://www.um.edu.mt/newspoint/news/2023/10/academic-year-23-24.

Waring, E., Quinn, M., McNamara, A., & Healy, K. (2022). Skimr: Compact and flexible summaries of data (Version 2.5.1) [R package]. https://CRAN.R-project.org/package=skimr.

Wilson, C. A., Babcock, S. E., & Saklofske, D. H. (2019). Sinking or swimming in an academic pool: A study of resiliency and student success in first-year undergraduates. Canadian Journal of Higher Education, 49(1), 60-84, https://doi.org/10.7202/1060824.

Zhoc, K. C., Webster, B. J., King, R. B., Johnson, L. C. H., & Chung, T. S. H. (2019). Higher education student engagement scale (HESES): Development and psychometric evidence. Research in Higher Education, 60, 219-244. https://doi.org/10.1007/s11162-018-9510-6.

Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21-29. https://doi.org/10.22237/jmasm/1177992180.


DOI: https://doi.org/10.26220/mje.5647

View Counter: Abstract | 0 | times, and PDF | 0 | times

Refbacks

  • There are currently no refbacks.


Mediterranean Journal of Education | ISSN: 2732-6489 |  Department of Educational Sciences and Early Childhood Education - University of Patras.

Pasithee | Library & Information Center | University of Patras