M.S. AAI Capstone Chronicles 2024

Table 2 Performance metrics of the top three model architectures vs. holdout test data

Model

AUC-ROC

Recall

Precision

Binary Acc.

Tuned SCT Model

0.92

0.84

0.80

0.89

Best Transformer

0.91

0.83

0.79

0.88

Best CNN

0.90

0.76

0.76

0.86

The performance metrics demonstrate that the SCT model was able to predict positive sepsis outcomes in patients with a high degree of reliability 6 hours in advance of sepsis onset. It also demonstrated that the model generalized to unseen data well. Further, the SCT demonstrated the ability to optimize for minimizing false negatives (ie: not missing positive sepsis outcomes) while maintaining a reasonable balance with a low number of false positives, as shown in Figure 3 below. These characteristics fulfilled the objectives of this project and the performance of the SCT model exceeded initial expectations. Figure 5 AUC-ROC Curves for the three top performing models

278