Comparison of Naive Bayes and Support Vector Machine for Sentiment Analysis of BPJS Health Service Deactivation

Abstract

The deactivation of BPJS Health services has become a topic of public discussion on social media, particularly on platform X, where users frequently express opinions regarding healthcare access and membership status. This study aims to analyze public sentiment toward the deactivation of BPJS Health services and compare the performance of Naive Bayes and Support Vector Machine (SVM) for sentiment classification. The dataset consisted of 8,357 tweets collected from social media X, of which 7,546 tweets were retained after preprocessing, including data cleaning, case folding, tokenizing, stopword removal, and stemming. TF-IDF and FastText were employed as text representation techniques, while model evaluation was conducted using 5-Fold Cross Validation, Grid Search Cross Validation for hyperparameter optimization, and a paired t-test for statistical significance analysis. Classification performance was measured using accuracy, precision, recall, and F1-score metrics. The results showed that negative sentiment dominated public opinion, accounting for 70.63% of the dataset, followed by neutral sentiment (26.48%) and positive sentiment (2.89%). The SVM model with TF-IDF achieved the highest performance, with an accuracy of 80.97%, precision of 80.16%, recall of 80.97%, and F1-score of 79.70%, outperforming Naive Bayes with TF-IDF (79.01%), Naive Bayes with FastText (64.26%), and SVM with FastText (80.31%). Furthermore, a paired t-test confirmed that the performance difference between Naive Bayes and SVM was statistically significant (p = 0.011). These findings indicate that SVM combined with TF-IDF is more effective for sentiment classification of high-dimensional social media text data and provide empirical evidence regarding the effectiveness of different text representation approaches for healthcare policy-related sentiment analysis.

GS Cited Analysis