M.S. AAI Capstone Chronicles 2024

importance placed on recall. To facilitate measuring the balance between the two, an additional metric, Area Under the Receiver Operating Characteristic Curve (AUC-ROC) was employed. This was used alongside precision and recall to evaluate overall performance. The business objective of this project was to accurately predict sepsis risk in patients at least 6 hours in advance of sepsis onset (the prediction horizon). As a time-series classification task, another important aspect to the solution is the lookback window. This is effectively the amount of data (number of time steps) that must be provided to the model in order for it to make reliable predictions. Ideally, a lower amount of data would be necessary. The team experimented with several combinations of lookback window size as well as the prediction horizon. This included a lookback window of 24 hours with 3 and 6 hour prediction horizons, as well as a look back window of 48 hours with 3 and 6 hour prediction horizons. Performance metrics were compared between each configuration to determine the appropriate combination. In the end, the 48/6 combination was selected due to more reliable performance. This was split into train, test and validation datasets with an 80/10/10 split ratio. Class weights were generated from the training dataset to be passed to the model during training. This enabled the model to account for the class imbalance in the dataset that was described previously. Using this dataset, the team experimented with baseline Linear Regression (LR), Long Short Term Memory (LSTM), CNN and Transformer-based models separately to gauge standalone performance of each architecture. While the LR and LSTM experiments did not yield positive results, the promising performance of the CNN and Transformer models corroborated with the team’s research that led to the development of the SCT architecture. However, additional work was necessary to optimize the SCT configuration against the dataset and task.

275