What the study found
The study found that a pre-course aptitude test had some ability to predict first-year computer science students’ performance on an introductory programming assessment. The authors report that a Random Forest Regressor performed more consistently than a Random Forest Classifier, though there was still a sizeable margin of error.
Why the authors say this matters
The authors suggest that early identification of students who may struggle with programming could help direct additional support from the outset. They also conclude that their approach offers a foundation for future targeted support interventions in introductory programming modules.
What the researchers tested
The researchers studied 285 first-year computer science undergraduates and developed a pre-course aptitude test. The test collected information on students’ backgrounds, prior experience, perceived confidence, and likelihood of holding appropriate mental models, meaning internal understandings, of core programming concepts. They used the resulting data to train and validate regression and classification models, including Random Forest models refined with Sequential Feature Selection and tested on a holdout set.
What worked and what didn't
The Random Forest Classifier performed well during training, with AUC = 0.8688, F1 = 0.8353, and accuracy = 0.7450, but performance dropped on the hold-out test set to AUC = 0.7670, F1 = 0.7020, and accuracy = 0.7020. The authors interpret this as moderate overfitting, likely linked to class imbalance and limited data. The Random Forest Regressor showed similar performance in training and testing, with RMSE = 0.1616 and MAE = 0.1209 in training, and RMSE = 0.1713 and MAE = 0.1396 in testing.
What to keep in mind
The abstract says there is still a sizeable margin of error, so the predictions are not highly precise. It also notes possible overfitting in the classifier because of imbalanced classes and limited data. Other limitations are not described in the available summary.
Key points
- A pre-course aptitude test was used to predict first introductory programming assessment results.
- The study involved 285 first-year computer science students.
- The aptitude test included background, prior experience, confidence, and mental-model measures.
- The Random Forest Classifier overfit somewhat when moved from training to the hold-out test set.
- The Random Forest Regressor was more consistent across training and testing.
- The authors say the approach may help identify students needing extra support early.
Disclosure
- Research title:
- Pre-course aptitude test predicted introductory programming performance
- Authors:
- Oliver Kerr, Linden J. Ball, Nicky Danino
- Institutions:
- Leeds Trinity University, University of Lancashire, University of Lancashire
- Publication date:
- 2026-03-30
- DOI:
- 10.1145/3806059
- OpenAlex record:
- View
Get the weekly research newsletter
Stay current with scholarly research without reading academic papers — one filtered digest, every Friday.