Emotion-Based Multi-Class Sentiment Analysis Of FirstMedia Customers Reviews Using SVM With Kernel Comparison

Authors

  • Bagus Kustiono Ongko Informatics Department, Universitas Dr. Soetomo, Surabaya
  • Anik Vega Vitianingsih Informatics Department, Universitas Dr. Soetomo, Surabaya
  • Dwi Cahyono Informatics Department, Universitas Dr. Soetomo, Surabaya
  • Anastasia Lidya Maukar Industrial Engineering Department, President University, Bekasi
  • Seftin Fitri Ana Wati Information System Department, Universitas Pembangunan Nasional "Veteran" Jawa Timur

DOI:

10.33395/sinkron.v10i1.15644

Keywords:

Sentiment Analysis, FirstMedia Sentiment Analysis, Support Vector Machine, Indonesian NRC Emotion Lexicon, Machine Learning

Abstract

The advancement of digital technology has made users increasingly reliant on online services, with user reviews serving as an essential resource for evaluating the quality of service provided by companies such as FirstMedia. However, these valuable data have not undergone comprehensive analysis to assess users’ emotional responses. This study aims to classify FirstMedia customers’ emotions into four categories (joy, sadness, anger, and neutral) and to evaluate the Support Vector Machine (SVM) method using four different kernel functions. Most existing studies primarily focus on polarity-based sentiment analysis and do not explicitly examine multi-emotion classification or kernel comparison in machine learning models. A total of 4,001 reviews were collected through web scraping from the Google Play Store and the X app and processed through several preprocessing steps. Emotion classification was conducted using the NRC Indonesian Emotion Lexicon, while word significance was determined using TF-IDF weighting. After preprocessing, 3,069 labeled reviews were retained and distributed as 1,065 neutral, 748 anger, 692 joy, and 564 sadness reviews, which were used for emotion classification. Model performance was evaluated using a hold-out validation scheme with an 80:20 train-test split and assessed through a confusion matrix. To address class imbalance, undersampling was applied, resulting in a balanced dataset for model training. The evaluation results show that the Linear kernel achieved the highest performance, with an accuracy of 82.63%, precision of 82.86%, recall of 82.63%, and an F1-score of 82.60%, outperforming the Gaussian, Polynomial, and Sigmoid kernels. This study demonstrates that multi-emotion sentiment analysis provides a more comprehensive understanding of user perceptions beyond conventional sentiment polarity, thereby supporting more informed evaluations of digital service quality.

 

GS Cited Analysis

Downloads

Download data is not yet available.

References

Angelie Tania, V. E., & Oetama, R. S. (2025). Nusantara capital city sentiment analysis using support vector machine and logistic regression. Indonesian Journal of Electrical Engineering and Computer Science, 38(3), 1708. https://doi.org/10.11591/ijeecs.v38.i3.pp1708-1721

Anggoro, D. A., & Permatasari, D. (2023). Performance comparison of the kernels of support vector machine algorithm for diabetes mellitus classification. International Journal of Advanced Computer Science and Applications, 14(1), 580–585. https://doi.org/10.14569/IJACSA.2023.0140163

Berutu, S. S., Budiati, H., & Gulo, F. (2023). Data preprocessing approach for machine learning-based sentiment classification. JURNAL INFOTEL, 15(4), 317–325. https://doi.org/10.20895/infotel.v15i4.1030

Chang, J., Chen, L., & Lin, L. (2021). A novel cluster based Over-sampling approach for classifying imbalanced sentiment data. IAENG International Journal of Computer Science, 48(4), 1118–1128.

Damayanti, E., Vitianingsih, A. V., Kacung, S., & Cahyono, D. (2024). Sentiment analysis of Alfagift application user reviews using long short-term memory (LSTM) and support vector machine (SVM) Methods. DECODE: JURNAL PENDIDIKAN TEKNOLOGI INFORMASI, 4(2), 509–521. https://doi.org/http://dx.doi.org/10.51454/decode.v4i2.478

Danyal, M. M., Khan, S. S., Khan, M., Ullah, S., Ghaffar, M. B., & Khan, W. (2024). Sentiment analysis of movie reviews based on NB approaches using TF–IDF and count vectorizer. Social Network Analysis and Mining, 14(1). https://doi.org/10.1007/s13278-024-01250-9

Garc, R. A., Luna-garc, H., Celaya-padilla, J. M., Garc, A., Reveles-g, L. C., Flores-chaires, L. A., Delgado-contreras, J. R., Rondon, D., & Villalba-condori, K. O. (2024). A systematic literature review of modalities, trends, and limitations in emotion recognition, affective computing, and sentiment analysis. Applied Sciences, 14(16). https://doi.org/10.3390/app14167165

Helmud, E., Helmud, E., Fitriyani, F., & Romadiana, P. (2024). Classification comparison performance of supervised machine learning random forest and decision tree algorithms using confusion matrix. Jurnal Sisfokom (Sistem Informasi Dan Komputer), 13(1), 92–97. https://doi.org/10.32736/sisfokom.v13i1.1985

Hendrastuty, N., Isnain, A. R., Rahmadhani, A. Y., Studi, P., Informasi, S., Indonesia, U. T., Informatika, P. S., Indonesia, U. T., & Lampung, K. B. (2021). Analisis sentimen masyarakat terhadap program kartu prakerja pada twitter dengan metode support vector machine. Jurnal Informatika: Jurnal Pengembangan IT (JPIT), 6(3), 150–155. https://doi.org/10.30591/jpit.v6i3.2870

Iswaratama, A. (2024). Peran komunitas virtual dalam mendorong interaksi sosial di era digital. HISTORICAL: Journal of History and Social Sciences, 3(1), 51–61. https://doi.org/10.58355/historical.v3i1.100 Vol.

Jaya, S. I. A., & Zeniarja, J. (2024). Sentiment analysis of genshin impact on X: Mental health implications using TF-IDF and support vector machine. Sinkron, 8(3), 1589–1599. https://doi.org/10.33395/sinkron.v8i3.13716

March Vircan Karuna, H. M. (2023). Analisis sentiment review kepuasan pengguna wi-fi First Media di Twitter. Journal of Management and Bussines (JOMB), 183(2), 153–164. https://doi.org/10.31539/jomb.v5i2.8301 ANALISIS

Palomino, M. A., & Aider, F. (2022). Evaluating the effectiveness of text pre-processing in sentiment analysis. Applied Sciences, 12, 8765. https://doi.org/10.3390/app12178765

Praghakusma, A. Z., & Charibaldi, N. (2021). Komparasi fungsi kernel metode Support Vector Machine untuk Analisis Sentimen Instagram dan Twitter (Studi Kasus : Komisi Pemberantasan Korupsi). JSTIF (Jurnal Sarjana Teknik Informatika), 9(2), 88. https://doi.org/10.12928/jstie.v9i2.20181

Prasetio, M. D., Xavier, R. Y., Rachmat, H., Wiyono, W., & Atmaja, D. S. E. (2021). Sentiment analysis on myindihome user reviews using support vector machine and naïve bayes classifier method. International Journal of Industrial Optimization, 2(2), 141. https://doi.org/10.12928/ijio.v2i2.4449

Putra Selian, R. I., Vitianingsih, A. V., Kacung, S., Lidya Maukar, A., & Febrian Rusdi, J. (2024). Sentiment analysis of public responses on social media to satire joke using naive bayes and KNN. Sinkron, 8(3), 1443–1451. https://doi.org/10.33395/sinkron.v8i3.13721

Qi, Y., & Shabrina, Z. (2023). Sentiment analysis using Twitter data: A comparative application of lexicon- and machine-learning-based approach. Social Network Analysis and Mining, 13(1), 1–14. https://doi.org/10.1007/s13278-023-01030-x

Rahmadani, R., Rahim, A., & Rudiman, R. (2024). Analisis sentimen ulasan “Ojol the Game” di Google Play Store menggunakan algoritma Naive Bayes dan model ekstraksi fitur TF-IDF untuk meningkatkan kualitas game. Jurnal Informatika Dan Teknik Elektro Terapan, 12(3). https://doi.org/10.23960/jitet.v12i3.4988

Shamsi, M., & Beheshti, S. (2025). Separability and scatteredness (S&S) ratio-based efficient SVM regularization parameter, kernel, and kernel parameter selection. Pattern Analysis and Applications, 28(1), 1–21. https://doi.org/10.1007/s10044-025-01411-2

Simay Akar, S. A., Yang Sok Kim, Y. S. K., & Mi Jin Noh, M. J. N. (2024). Sentiment analysis on “HelloTalk” app reviews using NRC Emotion Lexicon and GoEmotions dataset. Korean Institute of Smart Media, 13(6), 35–43. https://doi.org/10.30693/smj.2024.13.6.35

Surya, A., Ansyah, S., Kurniawan, A. P., Kholifah, A. N., & Purwitasari, D. (2023). A hybrid method on emotion detection for Indonesian tweets of COVID-19. JURNAL RESTI (Rekayasa Sistem Dan Teknologi Informasi), 7(2), 254–262. https://doi.org/10.29207/resti.v7i2.4816

Tan, K. L., Lee, C. P., & Lim, K. M. (2023). A survey of sentiment analysis: Approaches, datasets, and future research. Applied Sciences, 13(7). https://doi.org/10.3390/app13074550

Vanacore, A., Pellegrino, M. S., & Ciardiello, A. (2024). Fair evaluation of classifier predictive performance based on binary confusion matrix. Computational Statistics, 39(1), 363–383. https://doi.org/10.1007/s00180-022-01301-9

Venugopal, J. P., Subramanian, A. A. V., Sundaram, G., Rivera, M., & Wheeler, P. (2024). A comprehensive approach to bias mitigation for sentiment analysis of social media data. Applied Sciences, 14(23), 1–32. https://doi.org/10.3390/app142311471

Zad, S., Jimenez, J., & Finlayson, M. A. (2021). Hell hath no fury? correcting bias in the NRC Emotion Lexicon. International Committee on Computational Linguistics (ICCL), 102–113. https://doi.org/10.18653/v1/2021.woah-1.11

Downloads


Crossmark Updates

How to Cite

Ongko, B. K. ., Vitianingsih, A. V., Cahyono, D. ., Lidya Maukar, A. ., & Fitri Ana Wati, S. . (2026). Emotion-Based Multi-Class Sentiment Analysis Of FirstMedia Customers Reviews Using SVM With Kernel Comparison. Sinkron : Jurnal Dan Penelitian Teknik Informatika, 10(1), 550-559. https://doi.org/10.33395/sinkron.v10i1.15644

Issue

Vol. 10 No. 1 (2026): Article Research January 2026

Section

Articles

License

Copyright (c) 2026 Bagus Kustiono Ongko, Anik Vega Vitianingsih, Dwi Cahyono, Anastasia Lidya Maukar, Seftin Fitri Ana Wati

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

Most read articles by the same author(s)