Two-Stage Framework Using IndoBERT for Sentiment Analysis of Tokopedia Reviews under Extreme Class Imbalance

Abstract

The rapid growth of the Indonesian e-commerce industry has generated a large volume of customer reviews for sentiment analysis, but the data distribution often suffers from extreme class imbalance. The review dataset exhibits a 97.6% dominance of the positive class, causing the single-stage transformer model to produce high accuracy that does not fully represent classification capability. The baseline model achieves a macro-averaged F1-score of 0.599, with a neutral-class recall of 26.3%. Approaches based on loss function adjustment, such as class-balanced loss, focal loss, weighted cross-entropy, and decision-threshold adjustment, are unable to fundamentally address this issue, yielding only limited performance improvements. This study proposes a two-stage classification approach that decomposes the multi-class classification task into two sequential binary classification stages using a BERT-based Indonesian-language transformer model (IndoBERT). The first stage separates the positive class from the non-positive class, while the second stage distinguishes between the neutral and negative classes in a more balanced decision space. The proposed approach achieves a macro-averaged F1-score of 0.761, representing a 16.2% improvement over the baseline and outperforming all loss-function-based methods. These findings suggest that, under conditions of extreme class imbalance, simplifying the decision space through gradual task decomposition is more effective than intervention at the loss-function level. Furthermore, error propagation analysis and qualitative evaluations demonstrate that this approach improves sensitivity to minority classes, although challenges remain in cases involving ambiguous expressions.

GS Cited Analysis