Comparative Study of TF-IDF and SBERT Feature Representations for Random Forest-Based Resume Classification

Abstract

Manual resume screening in high-volume recruitment is time-consuming and prone to inconsistent evaluation, motivating automated screening using Machine Learning (ML) and Natural Language Processing (NLP). Prior comparisons of TF-IDF and Sentence-BERT (SBERT) often differ in classifiers or task formulations, limiting direct comparison of feature representations. This study compares TF-IDF and SBERT using an identical Random Forest classifier for resume-to-role classification across 32 role categories. The dataset contains 10,102 resume-job description pairs from a public Hugging Face corpus. Three sources of data leakage were identified and mitigated, followed by five-fold StratifiedGroupKFold cross-validation and Bonferroni-corrected paired statistical testing. TF-IDF Standard achieved a mean macro F1 of 0.9953, TF-IDF Enriched 0.9964, and SBERT 0.9816. The difference between the two TF-IDF configurations was not significant (p = 0.1367), whereas both significantly outperformed SBERT (p = 0.0021 and p = 0.0015). Component-wise ablation showed that resume-only input retained high performance (macro F1 up to 0.9977), while job-description-only input achieved at most 0.2339, indicating that resume content provides the primary predictive signal. Equalizing the input budget to 256 tokens also preserved the TF-IDF advantage. Role-mismatch results further showed no performance degradation when job descriptions were replaced with descriptions from unrelated roles. These findings indicate that TF-IDF provides strong lexical representations for resume-to-role classification under the evaluated protocol, while highlighting the importance of leakage auditing and controlled input conditions.

GS Cited Analysis