ADS Capstone Chronicles Revised
22
Table 5.2 Performance Summary for CatBoost Optimal Model Dataset Adj. R2 RMSE MAE MAPE (%) Training 0.898 0.045 0.017 0.013 Validation 0.867 0.051 0.019 0.015 Test 0.876 0.049 0.018 0.014 Table 5.3 Performance Summary for MLP Baseline Model Dataset Adj. R2 RMSE MAE MAPE (%) Training -14.13 1.75 0.861 0.860 Validation -4.81 1.08 0.842 0.842 Test -13.23 1.69 0.859 0.859 Table 5.4 Performance Summary for MLP Optimal Model Dataset Adj. R2 RMSE MAE MAPE (%) Training 0.911 0.042 0.015 0.011 Validation 0.884 0.048 0.016 0.012 Test 0.891 0.046 0.016 0.012
The Multi-Layer Perceptron (MLP) model demonstrated superior performance compared to the CatBoost model across all datasets, as evidenced by its higher Adjusted R² values and lower error metrics (RMSE, MAE, and MAPE). These results highlight the MLP model’s ability to effectively learn complex, non-linear relationships in the data. Its neural network architecture, with optimized hyperparameters, enabledittocaptureintricatefeatureinteractions, leading to better predictive accuracy and generalization. Although the CatBoost model performed well, especially in capturing patterns in tabular data, it fell short of the MLPmodel’s performance, particularly in terms of error reduction. The MLP model’s robustness across training, validation, and testdatasets—combined with its lower MAPE values—underscores its suitabilityforthistask,whereprecisepredictions of accident severity are crucial. The MLP model’s effectiveness in modeling the dataset is reflected in Equation 1, which represents its capability to generalize complex relationships: (3) • (2) + (3) . It is also important to note that both models identifiedsimilarinfluentialfeatures—confirmed by a feature importance analysis. With little surprise, weather conditions (e.g., wind speed, precipitation) and traffic patterns (e.g., speed ranges) emerged as the top predictors. This, in turn, confirms the original hypothesis that these factors significantly influence the severity of automobile accidents. Below are the top 10 features identified as most important in the modeling process by the optimal model. (1) = =1 32 ∑
262
Made with FlippingBook - Online Brochure Maker