Comparison of aggregation methods used in online reviews: a critical analysis
DOI:
https://doi.org/10.23755/rm.v54i0.1674Keywords:
aggregation methods, arithmetic mean, online reviews, alternative methods, recent bias.Abstract
With digitalization, online reviews have become a crucial component of consumer decisions and are an important source of feedback on products and services. Users can make informed choices using online review systems such as e-commerce platforms, social media, and others. However, the collection and analysis of these reviews raises a number of questions regarding reliability, representativeness and accuracy. Since they directly influence users' choices, the methods used to aggregate review scores are extremely important. In this article, we explore the most common aggregation methods for rating online reviews on the Amazon platform, with a particular focus on the arithmetic mean. Although these methods are easy to implement, they do not always accurately reflect the overall quality of a product. We then critically analyze how more sophisticated approaches can minimize some of the limitations of the mean, such as the so-called recent bias; propose more robust solutions and alternative approaches that can provide a more accurate representation of the overall rating, improving the quality of aggregated ratings.
References
Abdel-Hafez, A., Xu, Y., & Jøsang, A. (2014, September). A rating aggregation method for generating product reputations. In Proceedings of the 25th ACM conference on Hypertext and social media (pp. 291-293).
Agarwal, B., Mittal, N., Bansal, P., & Garg, S. (2015). Sentiment analysis using common‐sense and context information. Computational intelligence and neuroscience, 2015(1), 715730.
Bączkiewicz, A., Kizielewicz, B., Shekhovtsov, A., Wątróbski, J., & Sałabun, W. (2021). Methodical aspects of MCDM based E-commerce recommender system. Journal of Theoretical and Applied Electronic Commerce Research, 16(6), 2192-2229.
Benlahbib, A. (2020). Aggregating customer review attributes for online reputation generation. IEEE Access, 8, 96550-96564.
Chevalier, J., & Goolsbee, A. (2003). Measuring prices and price competition online: Amazon. com and BarnesandNoble. com. Quantitative marketing and Economics, 1, 203-222.
Dahooie, J. H., Raafat, R., Qorbani, A. R., & Daim, T. (2021). An intuitionistic fuzzy data-driven product ranking model using sentiment analysis and multi-criteria decision-making. Technological Forecasting and Social Change, 173, 121158.
Decker, R., & Trusov, M. (2010). Estimating aggregate consumer preferences from online product reviews. International Journal of Research in Marketing, 27(4), 293-307.
Dellarocas, C., & Wood, C. A. (2008). The sound of silence in online feedback: Estimating trading risks in the presence of reporting bias. Management science, 54(3), 460-476.
Duan, W., Gu, B., & Whinston, A. B. (2008). Do online reviews matter?—An empirical investigation of panel data. Decision support systems, 45(4), 1007-1016.
Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001, April). Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web (pp. 613-622).
Eryarsoy, E., & Piramuthu, S. (2014). Experimental evaluation of sequential bias in online customer reviews. Information & Management, 51(8), 964-971.
Ganu, G., Elhadad, N., & Marian, A. (2009, June). Beyond the stars: Improving rating predictions using review text content. In WebDB (Vol. 9, pp. 1-6).
Ge, Y., & Li, J. (2015). Measure and Mitigate the Dimensional Bias in Online Reviews and Ratings. In ICIS.
Ghose, A., & Ipeirotis, P. G. (2006, December). Designing ranking systems for consumer reviews: The impact of review subjectivity on product sales and review quality. In Proceedings of the 16th annual workshop on information technology and systems (pp. 303-310).
Gionis, A., Mannila, H., & Tsaparas, P. (2007). Clustering aggregation. Acm transactions on knowledge discovery from data (tkdd), 1(1), 4-es.
Han, S., & Anderson, C. K. (2020). Customer motivation and response bias in online reviews. Cornell Hospitality Quarterly, 61(2), 142-153.
Hellerstein, J. M., Haas, P. J., & Wang, H. J. (1997, June). Online aggregation. In Proceedings of the 1997 ACM SIGMOD international conference on Management of data (pp. 171-182).
Ikotun, A. M., Ezugwu, A. E., Abualigah, L., Abuhaija, B., & Heming, J. (2023). K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data. Information Sciences, 622, 178-210.
Ivanova, O., & Scholz, M. (2017). How can online marketplaces reduce rating manipulation? A new approach on dynamic aggregation of online ratings. Decision Support Systems, 104, 64-78.
Jamadi Khiabani, P., Basiri, M. E., & Rastegari, H. (2020). An improved evidence-based aggregation method for sentiment analysis. Journal of Information Science, 46(3), 340-360.
Ji, F., Cao, Q., Li, H., Fujita, H., Liang, C., & Wu, J. (2023). An online reviews-driven large-scale group decision making approach for evaluating user satisfaction of sharing accommodation. Expert Systems with Applications, 213, 118875.
Jindal, N., & Liu, B. (2008, February). Opinion spam and analysis. In Proceedings of the 2008 international conference on web search and data mining (pp. 219-230).
Karaman, H. (2021). Online review solicitations reduce extremity bias in online review distributions and increase their representativeness. Management Science, 67(7), 4420-4445.
Lakshmanaprabu, S. K., Shankar, K., Gupta, D., Khanna, A., Rodrigues, J. J., Pinheiro, P. R., & de Albuquerque, V. H. C. (2018). Ranking analysis for online customer reviews of products using opinion mining with clustering. Complexity, 2018(1), 3569351.
Lin, S. (2010). Rank aggregation methods. Wiley Interdisciplinary Reviews: Computational Statistics, 2(5), 555-570.
Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In Mining text data (pp. 415-463). Springer, Boston, MA.
Liu, Y., Bi, J. W., & Fan, Z. P. (2017). Ranking products through online reviews: A method based on sentiment analysis technique and intuitionistic fuzzy set theory. Information Fusion, 36, 149-161.
Liu, B. (2022). Sentiment analysis and opinion mining. Springer Nature.
Mackiewicz, J. (2008). Reviewer motivations, bias, and credibility in online reviews. In Handbook of research on computer mediated communication (pp. 252-266). IGI Global.
Orlikowski, W. J., & Scott, S. V. (2014). What happens when evaluation goes online? Exploring apparatuses of valuation in the travel sector. Organization Science, 25(3), 868-891.
Pourghebleh, B., & Navimipour, N. J. (2017). Data aggregation mechanisms in the Internet of things: A systematic review of the literature and recommendations for future research. Journal of Network and Computer Applications, 97, 23-34.
Preethi, J. S., Abirami, A. M., Askarunisa, A., Priya, G. S., & Sankaragomathy, E. (2015). Applying MCDM techniques for ranking products based on online customer feedback. Int. J. Knowl. Based Comput. Syst, 3, 13-20.
Viviani, M., & Pasi, G. (2017). Quantifier guided aggregation for the veracity assessment of online reviews. International Journal of Intelligent Systems, 32(5), 481-501.
Downloads
Published
Issue
Section
License
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge. The journal is committed to real and immediate open access for academic work. All of the journal articles are free to access immediately from the date of publication. There are no charge for readers to download articles and reviews for their own scholarly use. Benefits of open access for authors, include:
- Free access for all users worldwide
- Increased visibility and readership
- Rapid publication
- No spatial constraints
- Authors can share their research papers on their personal web-pages (i.e. ResearchGate, etc.)
Ratio Mathematica publishes open access articles under the terms of the Creative Commons Attribution (CC BY) License. The Creative Commons Attribution License (CC-BY) allows users to copy, distribute and transmit an article, adapt the article and make commercial use of the article. The CC BY license permits commercial and non-commercial re-use of an open access article, as long as the author is properly attributed. Copyright on any research article published by Ratio Mathematica is retained by the author(s). Authors grant Ratio Mathematica a license to publish the article and identify itself as the original publisher. Authors also grant any third party the right to use the article freely as long as its original authors, citation details and publisher are identified.
For further information visit: https://creativecommons.org/licenses/by/3.0/. The following logo will appear in any paper that will be published from the volume number 31/2016 onwards.
