M.S. AAI Capstone Chronicles 2024

Detecting Fake News Using Natural Language Processing 94.20% on the test set, further confirming its effectiveness in accurately classifying fake news articles.

Figure 8: Two-layer Bi-LSTM Accuracy and Loss Curves One rather unexpected result was with models including sentiment analysis. Based on our preliminary analyses and literature review the sentiment analysis score as a feature was predicted to boost performance. While many factors could cause this discrepancy, we have some educated guesses. First is the aforementioned ceiling effect, observed in the model results, leaving little room for improvement. Second, is that the tool, SentimentIntensityAnalyzer, may not be well suited for a proportion of our text. Data used for training included both social media posts and news articles. Social media posts tend to be less structured and shorter than articles. The pre-trained nltk sentiment analyzer is documented to be less effective with longer and more structured texts (White, 2020). In Hamed et al.(2023) TextBlob was used for sentiment analysis. This library is typically used for more formal language analysis (White, 2020). However, Hamed et al.(2023) were able to use TextBlob on social media-like data effectively. TextBlob could potentially be a fit for our mixed dataset in the future. Our team hypothesized that certain words, phrases, and sentences would help a model identify if news was fake or misleading. To try and understand our model’s high performance,

16

41