Author(s) :

Milan Stukavec.

Volume/Issue :

Abstract :

Models for sentiment analysis that are trained on one-domain dataset often result in inability to perform well in other domains due to insufficient labeled data and restricted domain knowledge. Understanding how well sentiment classifiers generalize is critical for multi-domain applications where the inability to handle the domain shift may severely impact the stakeholders. The study examines the ability of machine learning models (LinearSVC and Logistic Regression) trained on the “Social Dilemma” movie review dataset to generalize to a new, previously unseen dataset with a non-specific thematic focus. Both models were optimized using hyperparameter tuning and TF-IDF vectorizer, and then tested on the new dataset. The results showed that both models were able to achieve accuracy scores and F1-Scores between 85 and 90% on the first dataset, but when applied to the second dataset, both performance indicators dropped significantly. This was likely due to the shift in topic, vocabulary and context. The study concluded that the degree of generalization ability to an unseen dataset for sentiment analysis depends more on the degree of topic proximity between the training and new datasets than on the optimization and selection of the ML model, highlighting the importance of dataset choice in crossdomain applications.