Comparison of RNN Architectures and Non-RNN Architectures in Sentiment Analysis

Authors

DOI:

10.33395/sinkron.v8i4.13048

Keywords:

1D ConvNets, Accuracy, Bidirectional Recurrent Neural Network, Gated Recurrent Unit, Long Short-Term Memory

Abstract

This study compares the sentiment analysis performance of multiple Recurrent Neural Network architectures and One-Dimensional Convolutional Neural Networks. THE METHODS EVALUATED ARE simple Recurrent Neural Network, Long Short-Term Memory, Gated Recurrent Unit, Bidirectional Recurrent Neural Network, and 1D ConvNets. A dataset comprising text reviews with positive or negative sentiment labels was evaluated. All evaluated models demonstrated an extremely high accuracy, ranging from 99.81% to 99.99%. Apart from that, the loss generated by these models is also low, ranging from 0.0043 to 0.0021. However, there are minor performance differences between the evaluated architectures. The Long Short-Term Memory and Gated Recurrent Unit models mainly perform marginally better than the Simple Recurrent Neural Network, albeit with slightly lower accuracy and loss. In the meantime, the Bidirectional Recurrent Neural Network model demonstrates competitive performance, as it can effectively manage text context from both directions. Additionally, One-Dimensional Convolutional Neural Networks provide satisfactory results, indicating that convolution-based approaches are also effective in sentiment analysis. The findings of this study provide practitioners with essential insights for selecting an appropriate architecture for sentiment analysis tasks. While all models yield excellent performance, the choice of architecture can impact computational efficiency and training time. Therefore, a comprehensive comprehension of the respective characteristics of Recurrent Neural Network architectures and One-Dimensional Convolutional Neural Networks is essential for making more informed decisions when constructing sentiment analysis models.

GS Cited Analysis

Downloads

Download data is not yet available.

References

Alshammari, H. H., & Alkhiri, H. (2023). Optimized recurrent neural network mechanism for olive leaf disease diagnosis based on wavelet transform. Alexandria Engineering Journal, 78(June), 149–161. https://doi.org/10.1016/j.aej.2023.07.037

ArunKumar, K. E., Kalaga, D. V., Kumar, C. M. S., Kawaji, M., & Brenza, T. M. (2021). Forecasting of COVID-19 using deep layer Recurrent Neural Networks (RNNs) with Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM) cells. Chaos, Solitons and Fractals, 146, 110861. https://doi.org/10.1016/j.chaos.2021.110861

ArunKumar, K. E., Kalaga, D. V., Mohan Sai Kumar, C., Kawaji, M., & Brenza, T. M. (2022). Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends. Alexandria Engineering Journal, 61(10), 7585–7603. https://doi.org/10.1016/j.aej.2022.01.011

Bacanin, N., Jovanovic, L., Zivkovic, M., Kandasamy, V., Antonijevic, M., Deveci, M., & Strumberger, I. (2023). Multivariate energy forecasting via metaheuristic tuned long-short term memory and gated recurrent unit neural networks. Information Sciences, 642(May), 119122. https://doi.org/10.1016/j.ins.2023.119122

Banerjee, I., Ling, Y., Chen, M. C., Hasan, S. A., Langlotz, C. P., Moradzadeh, N., Chapman, B., Amrhein, T., Mong, D., Rubin, D. L., Farri, O., & Lungren, M. P. (2019). Comparative effectiveness of convolutional neural network (CNN) and recurrent neural network (RNN) architectures for radiology text report classification. Artificial Intelligence in Medicine, 97(August 2018), 79–88. https://doi.org/10.1016/j.artmed.2018.11.004

Chandrasekaran, G., Antoanela, N., Andrei, G., Monica, C., & Hemanth, J. (2022). Visual Sentiment Analysis Using Deep Learning Models with Social Media Data. Applied Sciences (Switzerland), 12(3). https://doi.org/10.3390/app12031030

Doshi, U., Barot, V., & Gavhane, S. (2020). Emotion detection and sentiment analysis of static images. 2020 International Conference on Convergence to Digital World - Quo Vadis, ICCDW 2020, Iccdw. https://doi.org/10.1109/ICCDW45521.2020.9318713

Elfaik, H., & Nfaoui, E. H. (2023). Leveraging feature-level fusion representations and attentional bidirectional RNN-CNN deep models for Arabic affect analysis on Twitter. Journal of King Saud University - Computer and Information Sciences, 35(1), 462–482. https://doi.org/10.1016/j.jksuci.2022.12.015

Gholami, H., Mohammadifar, A., Golzari, S., Song, Y., & Pradhan, B. (2023). Interpretability of simple RNN and GRU deep learning models used to map land susceptibility to gully erosion. Science of the Total Environment, 904(September). https://doi.org/10.1016/j.scitotenv.2023.166960

Mirzaei, S., Kang, J. L., & Chu, K. Y. (2022). A comparative study on long short-term memory and gated recurrent unit neural networks in fault diagnosis for chemical processes using visualization. Journal of the Taiwan Institute of Chemical Engineers, 130, 104028. https://doi.org/10.1016/j.jtice.2021.08.016

Nugen, F., Vera Garcia, D. V., Sohn, S., Mickley, J. P., Wyles, C. C., Erickson, B. J., & Taunton, M. J. (2023). Application of Natural Language Processing in Total Joint Arthroplasty: Opportunities and Challenges. Journal of Arthroplasty, 38(10), 1948–1953. https://doi.org/10.1016/j.arth.2023.08.047

Onan, A. (2022). Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification. Journal of King Saud University - Computer and Information Sciences, 34(5), 2098–2117. https://doi.org/10.1016/j.jksuci.2022.02.025

Pino, P. A., Soto, C., Asún, R. A., & Guti, F. (2023). Profiling support in literacy development : Use of natural language processing to identify learning needs in higher education ☆. 58(September). https://doi.org/10.1016/j.asw.2023.100787

Rayhan Ahmed, M., Islam, S., Muzahidul Islam, A. K. M., & Shatabda, S. (2023). An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition. Expert Systems with Applications, 218(February), 119633. https://doi.org/10.1016/j.eswa.2023.119633

Shastry, K. A., & Shastry, A. (2023). An integrated deep learning and natural language processing approach for continuous remote monitoring in digital health. Decision Analytics Journal, 8(May), 100301. https://doi.org/10.1016/j.dajour.2023.100301

Sonsare, P. M., & C, G. (2021). Cascading 1D-Convnet Bidirectional Long Short Term Memory Network with Modified COCOB Optimizer: A Novel Approach for Protein Secondary Structure Prediction. Chaos, Solitons and Fractals, 153. https://doi.org/10.1016/j.chaos.2021.111446

Yao, Z., Wang, Z., Wang, D., Wu, J., & Chen, L. (2023). An ensemble CNN-LSTM and GRU adaptive weighting model based improved sparrow search algorithm for predicting runoff using historical meteorological and runoff data as input. Journal of Hydrology, 625(April). https://doi.org/10.1016/j.jhydrol.2023.129977

Downloads


Crossmark Updates

How to Cite

Hindarto, D. (2023). Comparison of RNN Architectures and Non-RNN Architectures in Sentiment Analysis. Sinkron : Jurnal Dan Penelitian Teknik Informatika, 7(4), 2537-2546. https://doi.org/10.33395/sinkron.v8i4.13048

Issue

Vol. 7 No. 4 (2023): Article Research Volume 7 Issue 4, October 2023

Section

Articles

License

Copyright (c) 2023 Djarot Hindarto

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

Most read articles by the same author(s)

1 2 3 4 > >>