M.S. AAI Capstone Chronicles 2024

Evaluation

​ During training, validation loss was monitored to measure the improvement in the model

across epochs. The validation loss was also used to lower the learning rate when model improvement plateaued and end the training early if the model began to overfit or stop improving. Accuracy was also logged as a secondary metric, however, is not a very reliable metric for image captioning because it measures how many of the tokens predicted by the model exactly match the ground truth tokens. This is not particularly useful for overall evaluation because a generated caption could be a correct description of an image but still have low accuracy if it uses different words than the reference captions it is evaluated against.

Figure 9

Sample Caption Generated by Visual Attention Model (DenseNet)

225