A Lightweight DistilBERT-Attention Model for Aspect-Based Sentiment Analysis: A Case Study on E-Commerce Reviews

Mohammad Abu Kausar; Mohammad Nasar; Sallam O. F. Khairy; S. M. Emdad Hossain; Arockiasamy Soosaimanickam

doi:10.57159/jcmm.5.4.26653

Authors

Mohammad Abu Kausar Department of Information Systems, University of Nizwa, Nizwa, Oman https://orcid.org/0000-0002-7997-8292
Mohammad Nasar Computing and Informatics Department, Mazoon College, Muscat, Oman https://orcid.org/0000-0001-8196-380X
Sallam O. F. Khairy Department of Information Systems, University of Nizwa, Nizwa, Oman https://orcid.org/0000-0003-2845-5675
S. M. Emdad Hossain Department of Information Systems, University of Nizwa, Nizwa, Oman https://orcid.org/0009-0003-9637-6124
Arockiasamy Soosaimanickam Department of Information Systems, University of Nizwa, Nizwa, Oman https://orcid.org/0000-0001-8872-1803

DOI:

https://doi.org/10.57159/jcmm.5.4.26653

Keywords:

Aspect-Based Sentiment Analysis, Attention Mechanisms, E-commerce Reviews, Natural Language Processing, Deep Learning

Abstract

This paper introduces an efficient DistilBERT-Attention model for aspect-based sentiment analysis (ABSA), designed to balance classification accuracy against computational cost. Unlike general sentiment analysis, which assigns a single polarity to a complete review, ABSA identifies the product aspects discussed within individual sentences or clauses, such as design, quality, and price, and determines the sentiment polarity expressed toward each of them. The proposed model combines DistilBERT, a compact transformer encoder, with a modified aspect-focused attention layer that captures fine-grained sentiment signals efficiently. Experiments were conducted on 3,152 valid textual reviews, drawn from an initial collection of 3,259 Amazon India reviews of Titan watches published in 2024, across five aspect categories: design, quality, price, functionality, and comfort. The proposed model achieved an accuracy of 84.7% and an F1-score of 0.81. Compared with traditional baselines, it improved accuracy by 13.5 percentage points over a support vector machine and by 13.9 percentage points over logistic regression. Although BERT-base achieved a slightly higher accuracy of 86.1%, the proposed model retained approximately 98.4% of BERT-base accuracy while reducing memory consumption by about 40% and lowering relative processing time from 2.5x to 1.5x. The main contribution of the proposed model is therefore not absolute superiority over large transformer models, but an improved balance between accuracy, interpretability, and computational efficiency for resource-constrained ABSA applications.

References

[1] M. A. Kausar, A. Soosaimanickam, and M. Nasar, "Public sentiment analysis on Twitter data during COVID-19 outbreak," International Journal of Advanced Computer Science and Applications, vol. 12, no. 2, pp. 415-422, 2021.

[2] M. A. Kausar, A. Soosaimanickam, and M. Nasar, "Sentiment classification based on machine learning approaches in Flipkart product reviews," in Artificial Intelligence and Information Technologies (A. Dagur, K. Kaushik, and R. Astya, eds.), vol. 1, Boca Raton, FL, USA: CRC Press, 2024.

[3] M. A. Kausar, S. O. Fageeri, and A. Soosaimanickam, "Sentiment classification based on machine learning approaches in Amazon product reviews," Engineering, Technology and Applied Science Research, vol. 13, pp. 10849-10855, June 2023.

[4] O. Bellar, A. Baina, and M. Ballafkih, "Sentiment analysis: Predicting product reviews for e-commerce recommendations using deep learning and transformers," Mathematics, vol. 12, p. 2403, Aug. 2024.

[5] Y. Shi, L. Li, H. Li, A. Li, and Y. Lin, "Aspect-level sentiment analysis of customer reviews based on neural multi-task learning," Journal of Theory and Practice of Engineering Science, vol. 4, pp. 1-8, Apr. 2024.

[6] V. Dogra and M. Sudha, "Aspect-based approaches for measuring customer feedback in the e-commerce industry," in Proc. 2nd Int. Conf. Sustainable Computing and Smart Systems (ICSCSS), (Coimbatore, India), pp. 479-484, IEEE, July 2024.

[7] M. Wankhade, C. Kulkarni, and A. C. S. Rao, "A survey on aspect-based sentiment analysis methods and challenges," Applied Soft Computing, vol. 167, p. 112249, Dec. 2024.

[8] M. R. R. Rana, A. Nawaz, A. Raza, A. Alahmadi, and T. Alsaedi, "Sentiment mining in e-commerce," International Journal of Electrical and Computer Engineering Systems, vol. 15, pp. 641-650, Sept. 2024.

[9] I. A. Kandhro, F. Ali, M. Uddin, A. Kehar, and S. Manickam, "Exploring aspect-based sentiment analysis: An in-depth review of current methods and prospects for advancement," Knowledge and Information Systems, vol. 66, pp. 3639-3669, July 2024.

[10] X. Chen, H. Xie, X. Tao, F. L. Wang, D. Zhang, and H.-N. Dai, "A computational analysis of aspect-based sentiment analysis research through bibliometric mapping and topic modeling," Journal of Big Data, vol. 12, p. 40, Feb. 2025.

[11] Y. Xu and N. F. Ibrahim, "Cross-domain aspect-based sentiment analysis for enhancing customer experience in electronic commerce," Advances in Artificial Intelligence and Machine Learning, vol. 4, no. 3, pp. 2593-2613, 2024.

[12] D. Shukla and S. K. Dwivedi, "Sentiment analysis versus aspect-based sentiment analysis versus emotion analysis from text: A comparative study," International Journal of System Assurance Engineering and Management, vol. 16, pp. 512-531, Feb. 2025.

[13] W. Ahmad, H. U. Khan, F. K. Alarfaj, and M. Alreshoodi, "Aspect-based sentiment analysis: A comprehensive review and open research challenges," IEEE Access, vol. 13, pp. 65138-65182, 2025.

[14] B. Liu, Sentiment Analysis and Opinion Mining. Cham, Switzerland: Springer, 2012.

[15] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, "Lexicon-based methods for sentiment analysis," Computational Linguistics, vol. 37, pp. 267-307, June 2011.

[16] B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up? sentiment classification using machine learning techniques," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 79-86, 2002.

[17] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y. Ng, and C. Potts, "Recursive deep models for semantic compositionality over a sentiment treebank," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 1631-1642, 2013.

[18] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. Conf. North American Chapter Assoc. Comput. Linguist.: Human Language Technologies (NAACL-HLT), pp. 4171-4186, 2019.

[19] M. Hu and B. Liu, "Mining and summarizing customer reviews," in Proc. 10th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD), pp. 168-177, 2004.

[20] G. Qiu, B. Liu, J. Bu, and C. Chen, "Opinion word expansion and target extraction through double propagation," Computational Linguistics, vol. 37, pp. 9-27, Mar. 2011.

[21] X. Ding, B. Liu, and P. S. Yu, "A holistic lexicon-based approach to opinion mining," in Proc. Int. Conf. Web Search and Data Mining (WSDM), pp. 231-240, 2008.

[22] S. Kiritchenko, X. Zhu, C. Cherry, and S. M. Mohammad, "NRC-Canada-2014: Detecting aspects and sentiment in customer reviews," in Proc. 8th Int. Workshop Semantic Evaluation (SemEval), pp. 437-442, 2014.

[23] D. Tang, B. Qin, and T. Liu, "Aspect level sentiment classification with deep memory network," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), pp. 214-224, 2016.

[24] D. Ma, S. Li, X. Zhang, and H. Wang, "Interactive attention networks for aspect-level sentiment classification," in Proc. 26th Int. Joint Conf. Artificial Intelligence (IJCAI), pp. 4068-4074, 2017.

[25] J. Ma, X. Cai, D. Wei, H. Cao, J. Liu, and X. Zhuang, "Aspect-based attention LSTM for aspect-level sentiment analysis," in Proc. World Symp. Artificial Intelligence (WSAI), pp. 46-50, IEEE, June 2021.

[26] M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, "SemEval-2014 task 4: Aspect based sentiment analysis," in Proc. 8th Int. Workshop Semantic Evaluation (SemEval), pp. 27-35, 2014.

[27] M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, and I. Androutsopoulos, "SemEval-2015 task 12: Aspect based sentiment analysis," in Proc. 9th Int. Workshop Semantic Evaluation (SemEval), pp. 486-495, 2015.

[28] T. Hercig, T. Brychcín, L. Svoboda, and M. Konkol, "UWB at SemEval-2016 task 5: Aspect based sentiment analysis," in Proc. 10th Int. Workshop Semantic Evaluation (SemEval), pp. 342-349, 2016.

[29] R. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, "Exploiting document knowledge for aspect-level sentiment classification," in Proc. 56th Annu. Meeting Assoc. Comput. Linguist. (ACL), pp. 579-585, 2018.

[30] C. Sun, L. Huang, and X. Qiu, "Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence," in Proc. Conf. North American Chapter Assoc. Comput. Linguist.: Human Language Technologies (NAACL-HLT), pp. 380-385, 2019.

[31] H. Xu, B. Liu, L. Shu, and P. S. Yu, "BERT post-training for review reading comprehension and aspect-based sentiment analysis," in Proc. Conf. North American Chapter Assoc. Comput. Linguist.: Human Language Technologies (NAACL-HLT), pp. 2324-2335, 2019.

[32] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, "DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter," 2019. arXiv:1910.01108.