AAI_2025_Capstone_Chronicles_Combined

10

complexity, and boosting iterations while completing training in 253 seconds, demonstrating practical computational feasibility for operational deployment. Per-label cost-sensitive learning was implemented through setting up weighted loss functions. These were set to the ratio of negative to positive examples for each of the 34 labels. Search and rescue, with 2% prevalence, meant that the model treated missed rescues as 49 times more costly than false alarms. This adjustment in the approach trained 34 separate XGBoost classifiers with individualized class weights, fundamentally shifting the optimization objective from minimizing overall error to minimizing weighted error that prioritizes recall for minority classes. The optimized XGBoost model with hyperparameter tuning and cost-sensitive learning achieved micro-averaged F1=0.63 and macro-averaged F1=0.47 on the validation set. While micro F1 decreased modestly from baseline (0.63 vs. 0.676), macro F1 improved from 0.422 to 0.47, indicating better minority class performance. The optimization successfully shifted precision-recall trade-offs toward higher recall (0.590 to 0.70) at the cost of reduced precision (0.79 to 0.57), aligning with humanitarian priorities where false negatives carry greater consequences than false positives. We saw substantial improvements across recall as a result. As seen in Table 3, the recall had significant improvements after the optimization. The cost-sensitive learning approach proved particularly effective for the rarest categories. Categories with prevalence below 5% showed average recall improvements of 45%, demonstrating that explicit cost weighting can

partially compensate for extreme class imbalance when insufficient training examples prevent traditional resampling methods. However, precision decreased across nearly all categories as expected, with the model now generating more positive predictions to capture additional true positives. This precision-recall trade-off represents a deliberate design choice favoring sensitivity over specificity in alignment with life-saving priorities.

Table 3

Hyperparameter optimization improvements for XGBoost model Following initial cost-sensitive optimization, per-label threshold tuning was implemented by

326