Beyond Grades - Predicting Programme Learning Outcomes with Multi-Output Regression in Malaysian Higher Education

Melangkaui Gred: Meramal Hasil Pembelajaran Program (PLO) dengan Regresi Pelbagai Hasil dalam Pendidikan Tinggi Malaysia

Abstract

Grades often conceal specific competencies, while Malaysia's Outcome-Based Education (OBE) requires evidence at the Programme Learning Outcome (PLO) level. We develop a PLO-centric predictive framework that forecasts students' 12 end-of-programme PLO scores. Using anonymised records for 194 engineering students, we constructed a semester-indexed feature set (167 predictors) and jointly modelled all 12 PLOs via multi-output regression. Eleven algorithms (CatBoost, Extra Trees, LightGBM, Gradient Boosting, XGBoost, Random Forest, SVR, k-NN, Bayesian Ridge, AdaBoost, HistGradientBoosting) were evaluated under 70/30, 80/20, 90/10 hold-out regimes with 10-fold CV on training folds. We benchmarked macro Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Coefficient of determination (R²), and tested stacking and convex blending. Across splits, Bayesian Ridge was the most reliable single model by MAE (with CatBoost occasionally leading in R²), and a simple convex blend (Bayesian Ridge + CatBoost ± Extra Trees/Gradient Boosting) delivered ~5-13% lower MAE than the best single learner, whereas stacking did not add consistent gains. Model interpretability (model-native importances, SHapley Additive exPlanations (SHAP)) shows a noticeable later-year (third year and beyond) pattern, Semester 2 signal with recurrent contributions from PLO2 and PLO7, alongside a meaningful early-year indicator (e.g., PLO2_Semester_2_Year_1). The framework enables earlier, PLO-level feedback aligned with accreditation, supporting targeted intervention and curriculum improvement can be extended with broader cohorts, richer behavioural/demographic features, fairness audits, and cross-institutional generalisation.

Abstrak: Gred sering menyembunyikan kecekapan khusus, manakala Pendidikan Berasaskan Hasil (Outcome-Based Education, OBE) di Malaysia memerlukan bukti pada aras Hasil Pembelajaran Program (Program Learning Outcome, PLO). Kami membangunkan rangka kerja ramalan berpusatkan PLO yang meramalkan 12 skor PLO akhir program pelajar. Menggunakan rekod yang telah dianonimkan bagi 194 pelajar kejuruteraan, kami membina set ciri berindeks semester (167 peramal) dan memodelkan kesemua 12 PLO secara bersama melalui regresi pelbagai hasil. Sebelas algoritma (CatBoost, Extra Trees, LightGBM, Gradient Boosting, XGBoost, Random Forest, SVR, k-Jiran Terdekat (k-NN), Bayesian Ridge, AdaBoost, HistGradientBoosting) dinilai di bawah tiga pembahagian latih-ujian jenis hold-out (70/30, 80/20, 90/10) dengan pengesahan silang 10 lipatan pada set latihan. Kami membanding aras ralat menggunakan ralat mutlak min berpurata makro (MAE), ralat punca min kuasa dua (RMSE) dan pekali penentuan (R²), serta menguji dua strategi ensambel (penindanan dan pengadunan cembung). Merentas semua pecahan, Bayesian Ridge ialah model tunggal paling boleh diharap dari segi MAE (dengan CatBoost kadang-kadang mendahului dari segi R²), dan pengadunan cembung ringkas (Bayesian Ridge + CatBoost ± Extra Trees/Gradient Boosting) memberikan pengurangan MAE sekitar ~5-13% berbanding model tunggal terbaik, manakala penindanan tidak menambah peningkatan yang konsisten. Kebolehtafsiran model (kepentingan asli model dan SHapley Additive exPlanations (SHAP)) menunjukkan corak yang jelas pada tahun-tahun akhir (tahun ketiga dan ke atas), isyarat Semester 2 dengan sumbangan berulang daripada PLO2 dan PLO7, di samping penunjuk awal tahun yang bermakna (cth., PLO2_Semester_2_Tahun_1). Rangka kerja ini membolehkan maklum balas lebih awal pada aras PLO yang sejajar dengan keperluan akreditasi, menyokong intervensi tersasar dan penambahbaikan kurikulum. Pada masa hadapan, ia boleh diperluas dengan kohort yang lebih besar, ciri tingkah laku/demografi yang lebih kaya, audit keadilan, serta penggeneralisasian rentas institusi.