M.S. AAI Capstone Chronicles 2024

A custom F1 score was created which was essentially the same as a min-weighted F1-score, however it was calculated at the end of each epoch, and counted only relevant classes, to avoid a penalty where there was no ingredient in the predicted or actual label class. The final validation F1-score was 0.6409. Challenges Class imbalance was significant, rendering the standard accuracy metric as misleading. The quality of the images was intentionally of mediocre quality, as the authors intended the lighting to mimic that seen in campus cafeterias, and intentionally included dishes that were ambiguous and sometimes partially occluded by other ingredients. Certain “invisible” ingredients, such as salt, vegetable oils, vinegar, etc. cannot be detected in images, thereby rendering their presence irrelevant. It was necessary to review the ingredients list and decide which items were not likely to be detectable by the camera, and would only further class imbalance and complicate the analysis. These were excluded from the metadata analysis, and required an extra processing step. Future directions and suggestions for improvement Training the model with images of better quality would be expected to enhance accuracy. As mentioned previously, image quality was reduced to emulate a cafeteria setup, and no effort was made to separate individual ingredients to prevent obscuration. Exploration of other larger vision architectures might also yield improved performance. Finally, we would consider experimental approaches for handling class imbalance, such as MLSMOTE (Charte, 2015).

15

249