AAI_2025_Capstone_Chronicles_Combined

14

Draw, Detect, Navigate ​

​

​

​

​

​

​

​

​

Our team’s fully synthetic model, which learned optimal adjustments from previous and

less complicated models, ended up getting a very high testing score of 96.12%. Out of the

1,000 images that contained the doodles which were not used in validation or training, the total

correct doodle predictions for the model was 5,201 out of the 5,411 doodles within the images.

During validation, the model was also benchmarking a 0.6 millisecond pre-process time, 2.4

millisecond inference time, and a 0.6 millisecond post process time. The nano model was able

to achieve these scores with only 25 epochs and a total training time of 0.75 hours. Lastly, and

most importantly, the file size is extremely small coming in at approximately 6.3 MB. Which is

important because our application potentially needs to have the model running locally on the

end user's device, and not have an API endpoint.

Using the exact same model parameters as the YOLOv8 nano, the YOLOv8 small model

performed slightly better than the nano, but took much longer to train, and had a much larger

file size respectively at 22.5 MB. The small model was able to outperform the nano model in

every class for the mAP@0.5:0.95 score by approximately 1%. Although the small model was

able to improve scores significantly, the small model had a longer inference time than the nano,

at nearly three times longer with a speed of 5.9ms. The nano model data ingestion to prediction

which included, preprocess, inference, and postprocess took 2.3 milliseconds total. The only

speed process the small model beat the nano model in was the post process, at 0.4

milliseconds. In times where ever millisecond counts, the nano model achieved the superior but

with the tradeoff of slightly less accurate scores compared to the small. On our groups test

dataset which was one-thousand images, the model was not trained on, or validated with, the

small model was able to get 5241 doodles correctly over, only 40 doodles better than the nano.

41