Does change in the scoring system impact service evaluation? Evidence from Booking.com

Document Type

Article

Publication Date

2-7-2023

Abstract

Purpose: In September 2019, Booking.com changed from the smiley-based scoring system (2.5–10) to the purely 10-point evaluation system (1–10). The smiley-based service evaluation is based on the multi-dimensional (M-D) system, whereas the purely 10-point service evaluation is based on the single-dimensional (S-D) system. This paper aims to focus on how a change in review posting policies impacts service evaluations regarding review generation and distribution. Design/methodology/approach: The authors exploit the natural experiment using Booking.com when the site changed its scoring system from a multidimensional smiley-based service evaluation system to an S-D scoring system. The authors collected online reviews posted on two travel agencies (Booking.com and Priceline.com) between September 2019 and October 2020. A quasi-experimental approach, Difference-in-Differences, was used to isolate the impacts of the new scoring system from the impacts of the change in the service evaluation environment, i.e. COVID-19. Findings: The change in the scoring system considerably alters review distributions by decreasing the portion of positive reviews but increasing the portion of highly positive reviews. Using the theory of emotion work (Hochschild, 1979, 2001), DID is also the reason that the former M-D smiley-based system could have underrated, highly positive reviews of services. Using the information transfer theory (Belkin, 1984), the authors reason the asymmetric transfer of information when users consume reviews from the older (M-D) system but are required to generate reviews on a newer (S-D) system. Practical implications: The findings would provide online review platform management with a deeper understanding of the consequences of changes in service evaluations when the scoring system is changed. Originality/value: Though the change in the scoring system would affect how customers evaluate the services of hotels, the causal impacts of switching to the new S-D scoring system have not yet been thoroughly covered by prior hospitality and service evaluation literature, which this research aspires to do.

Publication Title

International Journal of Contemporary Hospitality Management

First Page Number

828

Last Page Number

847

DOI

10.1108/IJCHM-01-2022-0075

This document is currently not available here.

Share

COinS