Customer Complaint Classification at PT Pos Indonesia Manokwari Using Naive Bayes and Random Forest

Abstract

Customer complaints represent an important source of information for evaluating service quality and improving organizational performance. However, the increasing volume of complaints received by PT Pos Indonesia Manokwari makes manual complaint classification inefficient and time-consuming. This study aims to compare the performance of Naive Bayes and Random Forest algorithms for customer complaint classification using the Term Frequency–Inverse Document Frequency (TF-IDF) feature extraction method. The dataset consisted of 1,490 customer complaint records collected from the Customer Complaint Handling (CCH) system and categorized into twelve complaint classes. The research process included data cleaning, case folding, stopword removal, TF-IDF transformation, dataset splitting, model training, and performance evaluation. The classification models were evaluated using accuracy, precision, recall, F1-weighted score, F1-macro score, and 5-fold cross-validation. The experimental results showed that Random Forest achieved better performance than Naive Bayes. Random Forest obtained an accuracy of 87.92%, precision of 85.22%, recall of 87.92% an F1-weighted score of 86.30%, and an F1-macro score of 70.85%, while Naive Bayes achieved an accuracy of 84.90%, an F1-weighted score of 84.00%, and an F1-macro score of 48.41%. The cross-validation results produced an average accuracy of 71.81%. Although Random Forest achieved the highest hold-out accuracy, the cross-validation results indicate performance variation across different data partitions, which may be caused by class imbalance among complaint categories. These findings demonstrate that Random Forest is more effective for multiclass customer complaint classification and can support the development of automated complaint management systems at PT Pos Indonesia Manokwari.

GS Cited Analysis