M.S. Applied Data Science - Capstone Chronicles 2025

17

network also performed competitively (accuracy 0.88, ROC-AUC 0.91). Logistic Regression had the lowest performance in this group, with accuracy at 0.62 and ROC-AUC at 0.65, representing a gap of over 0.30 in ROC-AUC compared to the top model. The range of performance across models in Group A was therefore substantial, suggesting varying degrees of effectiveness in capturing the underlying decision boundaries. In Group B, Random Forest achieved the highest ROC-AUC (0.97) and tied with MLP for the highest accuracy (0.90), despite the absence of medication features. XGBoost also performed well (accuracy 0.89, ROC-AUC 0.96), with precision, recall, and F1-scores remaining balanced and close in value. Logistic Regression again trailed behind, with a noticeable drop in recall and ROC-AUC relative to the other models. The relative ranking of model performance in Group B mirrored that of Group

A, indicating a consistent pattern in which ensemble and neural network approaches outperformed linear methods. Across both groups, the top three models—Random Forest, XGBoost, and MLP—maintained high and balanced weighted precision, recall, and F1-scores, indicating stable classification performance for both classes. SVM showed moderate performance, ranking below the top three but consistently outperforming Logistic Regression in both accuracy and ROC-AUC. In contrast, Logistic Regression consistently showed the largest drop between accuracy and ROC-AUC, reflecting limitations in separating the positive and negative classes. The narrow performance gap between Group A and Group B for the top models suggests that their predictive capacity remained strong even when medication variables were removed from the feature set.

Table 4 Model performance metrics for group A. Model Accuracy

Precision

Recall

F1-score

ROC-AUC

Logistic Regression

0.62

0.63

0.62

0.63

0.65

Random Forest

0.87

0.86

0.87

0.86

0.95

XGBoost

0.90

0.90

0.90

0.90

0.96

SVM

0.75

0.75

0.75

0.75

0.79

MLP

0.88

0.87

0.88

0.87

0.91

168