M.S. AAI Capstone Chronicles 2024

A.S.LINGUIST

12

from the training data. The “softmax” activation function is a typical choice in the last dense

layer when solving a multiclass classification problem.

To prevent overfitting and improve model generalization to unseen data, different

regularization techniques were employed. There is a max pooling layer of size 2x2 after each of

the first three convolutional layers, a global average pooling layer and a batch normalization

layer before the first dense layer, and a dropout layer with a dropout rate of 50% after each of the

first two dense layers. The pooling layers reduce the size of the images by selecting maximum or

average pixel values from groups of pixels depending on the type of pooling (max or average),

the batch normalization layer modifies the distribution of the previous layer’s output to improve

the efficiency of the model training, and the dropout layers randomly drop 50% of their nodes to

avoid an overdependence of the model on training data.

We used 70%, 20% and 10% of the data for the training, validation and testing sets,

respectively. The model was compiled with the Adam optimizer and a learning rate of 1e-3,

which dynamically reduces during training up to 1e-6 in order to minimize the validation loss.

Also, we imposed the categorical cross-entropy loss function as the function to be minimized by

the optimizer (considering the multiclass nature of the problem), and the accuracy as the metric

to monitor model performance during training. The model was trained for a total of 50 epochs

and with a batch size of 32, and the model with the maximum accuracy was saved.

The CNN model architecture described above was progressively optimized after several

iterations based on the final model performance. We did not implement an automatic procedure

to fine-tune our hyperparameters but manually modified the number of convolutional and dense

layers, as well as the number of filters/units in convolutional/dense layers, in order to obtain a

desirable final model performance.

192