Sentiment Analysis and Rating Prediction for App Reviews Using Transformer-based Models
Abstract views: 274
/ 
PDF downloads: 87
Keywords:
Sentiment Analysis, NLP, User App Review, Transformer Models, Classification, Rating PredictionAbstract
In this study, we present the sentiment analysis of Spotify app reviews, the implementation of
Natural Language Processing (NLP) methods, and the use of transformer-based models including BERT,
DistilBERT, RoBERTa, and XLM-RoBERTa. Comprehensive preprocessing, including emoji removal,
typo correction, and tokenization, was utilized for processing Spotify app reviews from the Google Play
Store. Sentiments were analyzed using the VADER Sentiment Intensity Analyzer, categorized into positive,
neutral, and negative. Models were assessed for accuracy, precision, recall, and F1-score. DistilBERT
achieved the highest accuracy and recall 71.68%, while XLM-RoBERTa demonstrated the best balance
with an F1-score of 69.24% in predicting Spotify app ratings.
Downloads
References
Kerai, A. (2024). Cell Phone Usage Statistics: Mornings Are for Notifications. Reviews.org. Retrieved March 20, 2024, from https://www.reviews.org/mobile/cell-phone-addiction/,
Wurmser, Y. (2020). The Majority of Americans’ Mobile Time Spent Takes Place in Apps. EMARKETER. Retrieved March 21, 2024, from https://www.emarketer.com/content/the-majority-of-americans-mobile-time-spent-takes-place-in-apps
Mobile app revenue worldwide 2019-2027, by segment. (2023). Statista. Retrieved March 22, 2024, from https://www.statista.com/forecasts/1262892/mobile-app-revenue-worldwide-by-segment
Turner, A. (2024). Spotify Users: How Many People Have Spotify? . BankMyCell. Retrieved March 26, 2024, from https://www.bankmycell.com/blog/number-of-spotify-users/
Chen, N., Lin, J., Hoi, S. C. H., Xiao, X., & Zhang, B. (2014). AR-miner: mining informative reviews for developers from mobile app marketplace. Proceedings of the 36th International Conference on Software Engineering. https://doi.org/10.1145/2568225.2568263
Panichella, S., Di Sorbo, A., Guzman, E., Visaggio, C. A., Canfora, G., & Gall, H. C. (2015). How can i improve my app? Classifying user reviews for software maintenance and evolution. 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). https://doi.org/10.1109/icsm.2015.7332474
Ali, M., Joorabchi, M. E., & Mesbah, A. (2017,). Same App, Different App Stores: A Comparative Study. 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft). https://doi.org/10.1109/mobilesoft.2017.3
Noei, E., Zhang, F., & Zou, Y. (2021). Too Many User-Reviews! What Should App Developers Look at First? IEEE Transactions on Software Engineering, 47(2), 367–378. https://doi.org/10.1109/tse.2019.2893171
Pagano, D., & Maalej, W. (2013). User feedback in the appstore: An empirical study. 2013 21st IEEE International Requirements Engineering Conference (RE). https://doi.org/10.1109/re.2013.6636712
Aljrees, T., Umer, M., Saidani, O., Almuqren, L., Ishaq, A., Alsubai, S., Eshmawi, A. A., & Ashraf, I. (2024). Contradiction in text review and apps rating: prediction using textual features and transfer learning. PeerJ Computer Science, 10, e1722. https://doi.org/10.7717/peerj-cs.1722
Wong, W. H., Ismail, S., Arifin, M. A., Make, S. S. A., Wahab, M. H. A., & Shaharudin, S. M. (2021). Sentiment Analysis of Snapchat Application’s Reviews. 2021 2nd International Conference on Artificial Intelligence and Data Sciences (AiDAS). https://doi.org/10.1109/aidas53897.2021.9574379
Verma, P., Srivastava, R., Fatima, S., & Pratap, A. (2024). Sentiment Analysis on ChatGPT Reviews. 2024 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS). https://doi.org/10.1109/sceecs61402.2024.10482121
Sultana, R., & Sarker, S. (2018). App Review Mining and Summarization. International Journal of Computer Applications, 179(38), 45–52. https://doi.org/10.5120/ijca2018916918
Guzman, E., & Maalej, W. (2014). How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews. 2014 IEEE 22nd International Requirements Engineering Conference (RE). https://doi.org/10.1109/re.2014.6912257
emoji — emoji documentation. [Online]. Available: https://carpedm20.github.io/emoji/docs/
fastText. [Online]. Available: https://fasttext.cc/
LanguageTool. [Online]. Available: https://github.com/jxmorris12/language_tool_python
spaCy · Industrial-strength Natural Language Processing in Python. [Online]. Available: https://spacy.io/
NLTK :: Natural Language Toolkit. [Online]. Available: https://www.nltk.org/
Hutto, C., & Gilbert, E. (2014). VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. Proceedings of the International AAAI Conference on Web and Social Media, 8(1), 216–225. https://doi.org/10.1609/icwsm.v8i1.14550
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., ... & Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116.
Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. https://doi.org/10.1145/3292500.3330701
Transformers. [Online]. Available: https://huggingface.co/docs/transformers/index
