M.S. AAI Capstone Chronicles 2024

Detecting Fake News Using Natural Language Processing

Three-layer Bi-LSTM

0.94

0.94

0.94

0.94

Four-layer Bi-LSTM

0.93

0.93

0.93

0.93

Bidirectional LSTM w/ Sentiment Analysis

0.94

0.94

0.94

0.94

Two-layer Bi-LSTM w/ Sentiment Analysis

0.94

0.94

0.94

0.94

Five-layer Bi-LSTM w/ Sentiment Analysis

0.94

0.94

0.94

0.94

The results showcase strong performance across various models, with Random Forest, LSTM, Stacked LSTM, and Bidirectional LSTM achieving an accuracy, precision, recall, and F1-score of 0.94. Overall we observed a ceiling effect, where all models scored high, with little variation between architectures. Interestingly, the addition of sentiment analysis to Bidirectional LSTM models maintains this high-performance level, even though it did not help to yield better performance. However, as the number of layers increases in the Bidirectional LSTM architecture, a slight decrease in performance is observed, with the four-layer model achieving a slightly lower accuracy, precision, recall, and F1-score of 0.93. Overall, the Bi-LSTM model with two layers is preferred for its simplicity, efficiency, and accuracy. Despite the complex and noisy nature of the dataset, which required extensive data cleaning the Bi-LSTM model demonstrated robust performance without overfitting. This was enabled by the inherent properties of LSTM (Long Short-Term Memory) networks, which excel at capturing temporal dependencies in sequential data like text, providing resilience against overfitting. The training accuracy/loss curves exhibited consistent improvement over epochs, with decreasing loss and increasing accuracy across different variations, indicating effective learning and convergence of the model. The best model achieved an impressive accuracy of

15

40