Comparative Study on Machine Learning Algorithms for Code Smell Detection
DOI:
10.33395/sinkron.v10i1.15439Keywords:
Code Smell, Ensemble, Machine Learning, Multi-label, SmellyCodeAbstract
Detecting code smells is crucial for maintaining software quality, but rule-based methods are often not very adaptive. On the other side, existing machine learning studies often lack large-scale comparisons on modern datasets. The goal of this research is to comprehensively compare the performance of various machine learning algorithms for multi-label code smells classification in terms of effectiveness and efficiency. The dataset used in this research is SmellyCode++, containing more than 100,000 samples. Seven models: Logistic Regression, Linear SVM, Naive Bayes, Random Forest, Extra Trees, XGBoost, and LightGBM combined with Binary Relevance were trained on data balanced using random undersampling and multi-label synthetic minority over-sampling. The performance of each model was evaluated using the F1-Macro, Hamming Loss, and Jaccard Score metrics. A non-parametric statistical analysis was also conducted to validate the findings. The experiment found that ensemble-based models statically significantly outperformed the linear and probabilistic models. The performance among the top ensemble models was found to be statistically equivalent. With this statistical equivalence in accuracy, computational efficiency measured with training time became the critical tiebreaker. BR_RandomForest, BR_XGBoost, and BR_ExtraTrees proved highly efficient, while BR_LightGBM was significantly slower. This study concludes that BR_RandomForest offers the best overall trade-off in providing top tier accuracy combined with excellent computational efficiency, making it a robust choice for practical applications.
Downloads
References
Alawadi, S., Alkharabsheh, K., Alkhabbas, F., Kebande, V. R., Awaysheh, F. M., Palomba, F., & Awad, M. (2024). FedCSD: A Federated Learning Based Approach for Code-Smell Detection. IEEE Access, 12, 44888–44904. https://doi.org/10.1109/ACCESS.2024.3380167
Alomari, N., Alazba, A., Aljamaan, H., & Alshayeb, M. (2025). SmellyCode++: Multi-Label Dataset for Code Smell Detection. Scientific Data , 12(1). https://doi.org/10.1038/s41597-025-05465-z
Bogatinovski, J., Todorovski, L., Džeroski, S., & Kocev, D. (2022). Comprehensive comparative study of multi-label classification methods. Expert Systems with Applications, 203. https://doi.org/10.1016/j.eswa.2022.117215
Dewangan, S., Rao, R. S., Mishra, A., & Gupta, M. (2022). Code Smell Detection Using Ensemble Machine Learning Algorithms. Applied Sciences (Switzerland), 12(20). https://doi.org/10.3390/app122010321
Fowler, M., & Beck Boston, K. (2019). Refactoring Improving the Design of Existing Code Second Edition. Retrieved from www.EBooksWorld.ir
García-Pedrajas, N. E., Cuevas-Muñoz, J. M., Cerruela-García, G., & de Haro-García, A. (2024, July 1). A thorough experimental comparison of multilabel methods for classification performance. Pattern Recognition, Vol. 151. Elsevier Ltd. https://doi.org/10.1016/j.patcog.2024.110342
Guggulothu, T., & Moiz, S. A. (2020). Code smell detection using multi-label classification approach. Software Quality Journal, 28(3), 1063–1086. https://doi.org/10.1007/s11219-020-09498-y
Hamouda, E., El-Korany, A., & Makady, S. (2025). Smell-ML: A Machine Learning Framework for Detecting Rarely Studied Code Smells. IEEE Access, 13, 12966–12980. https://doi.org/10.1109/ACCESS.2025.3530927
Hilmi, M. A. Al, Puspaningrum, A., Darsih, Siahaan, D. O., Samosir, H. S., & Rahma, A. S. (2023). Research Trends, Detection Methods, Practices, and Challenges in Code Smell: SLR. IEEE Access, 11, 129536–129551. https://doi.org/10.1109/ACCESS.2023.3334258
Khleel, N. A. A., & Nehéz, K. (2023). Detection of code smells using machine learning techniques combined with data-balancing methods. International Journal of Advances in Intelligent Informatics, 9(3), 402–417. https://doi.org/10.26555/ijain.v9i3.981
Lacerda, G., Petrillo, F., Pimenta, M., & Guéhéneuc, Y. G. (2020). Code smells and refactoring: A tertiary systematic review of challenges and observations. Journal of Systems and Software, 167. https://doi.org/10.1016/j.jss.2020.110610
Nandini, A., Singh, R., & Rathee, A. (2024). Improving Code Smell Detection by Reducing Dimensionality Using Ensemble Feature Selection and Machine Learning. SN Computer Science, 5(6). https://doi.org/10.1007/s42979-024-03013-x
Palomba, F., Bavota, G., Di Penta, M., Fasano, F., Oliveto, R., & De Lucia, A. (2018). A large-scale empirical study on the lifecycle of code smell co-occurrences. Information and Software Technology, 99, 1–10. https://doi.org/10.1016/j.infsof.2018.02.004
Pecorelli, F., Di Nucci, D., De Roover, C., & De Lucia, A. (2020). A large empirical assessment of the role of data balancing in machine-learning-based code smell detection. Journal of Systems and Software, 169. https://doi.org/10.1016/j.jss.2020.110693
Rainio, O., Teuho, J., & Klén, R. (2024). Evaluation metrics and statistical tests for machine learning. Scientific Reports, 14(1). https://doi.org/10.1038/s41598-024-56706-x
Read, J., Pfahringer, B., Holmes, G., & Frank, E. (2011). Classifier chains for multi-label classification. Machine Learning, 85(3), 333–359. https://doi.org/10.1007/s10994-011-5256-5
Roy Chowdhuri, S., & Gupta, M. (2025). Multilabel Classification in the Context of Code Smell Detection. SN Computer Science, 6(5). https://doi.org/10.1007/s42979-025-03955-w
Sharma, V. (2022). A Study on Data Scaling Methods for Machine Learning. International Journal for Global Academic & Scientific Research, 1(1). https://doi.org/10.55938/ijgasr.v1i1.4
Tarekegn, A. N., Giacobini, M., & Michalak, K. (2021, October 1). A review of methods for imbalanced multi-label classification. Pattern Recognition, Vol. 118. Elsevier Ltd. https://doi.org/10.1016/j.patcog.2021.107965
Yadav, P. S., Rao, R. S., & Mishra, A. (2024). An Evaluation of Multi-Label Classification Approaches for Method-Level Code Smells Detection. IEEE Access, 12, 53664–53676. https://doi.org/10.1109/ACCESS.2024.3387856
Zhou, W., Liu, C., Yuan, P., & Jiang, L. (2024). An Undersampling Method Approaching the Ideal Classification Boundary for Imbalance Problems. Applied Sciences (Switzerland), 14(13). https://doi.org/10.3390/app14135421
Downloads
|
How to Cite
Issue
Section
License
Copyright (c) 2026 Hayya U, Theresia Ratih Dewi Saputri
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Google Scholar 
Moraref
PKP Index
Garuda Kemdikbud
Indonesia OneSearch
OCLC Worldcat
Dimensions
Index Copernicus
Scilit
Muhammad Khoiruddin harahap
