ADS Capstone Chronicles Revised

19

‘learning_rate’: 0.3, ‘estimator__max_depth’: 3}, resulting in a cross-validation score of approximately 0.85. SVM with Kernel Trick: The hyperparameters for the SVM model, including the regularization parameter (C) and the kernel coefficient (gamma), were optimized

using RandomizedSearchCV. The RBF (Radial Basis Function) kernel was chosen for its ability to handle nonlinear relationships within the data. The optimal configuration found was {C: 9.495, gamma: 0.090}, resulting in a cross-validation score of approximately 0.86.

Table 2 Hyperparameter Settings

5 ResultsandFindings After implementing and training the models, we evaluated their performance to identify the most effective approach for detecting FWA in healthcare data. 5.1 Evaluation of Results The performance of each model was assessed using a variety of metrics: accuracy, precision, recall, Receiver Operating Characteristic - Area Under Curve (ROC-AUC), Precision-Recall - Area Under Curve (PR-AUC), Matthews Correlation Coefficient (MCC), F1-score, and confusion matrix.

Accuracy provides an overall indication of the model’s performance; however, accuracy alone can be misleading in imbalanced datasets. Thus, other evaluation metrics were also evaluated. ROC-AUC measures the model’s ability to distinguish between classes (0 and 1) by plotting the true positive rate against the false positive rate. This allows for an understanding of the trade-off between sensitivity and specificity (Kuhn & Johnson, 2013). Rather than evaluating precision and recall separately, the PR-AUC was used to provide an overall comprehensive assessment of the model’s performance, specifically in identifying the positive class.

143