1.800.323.3639 | Support | Training

Incentivized reviews and biased data

By: Avtex

The VP of CX at Amazon is cracking down and rightfully so. Actually, I’m surprised it’s taken Amazon this long to put in place, which is banning incentivized reviews. Meaning sellers can no longer give away free products to receive positive reviews, which results in companies moving up the rankings. Amazon’s past behavior was to sue sellers for purchasing fake reviews on Fiverr.com, but now it’s policy.

Data from Webretailer.com illustrates the bias of Amazon ratings when compared to the average of non-incentivized ratings. Transparency, trust, and authenticity are slowly becoming the new norm in the marketplace for many customers. Not to say that these values were not important before, but with the ever increasing competition, sellers are willing to do more than cut corners to see profits. Because the competition is so stiff many sellers justify their behavior to carry out fake reviews in order to survive.

1st graph

Img Source: http://www.webretailer.com/lean-commerce/incentivized-reviews-amazon

Short-term vs. long-term thinking
Whether buyer or seller, we’re not wired for the long-run. We come flawed from the factory by not always being able to sacrifice short-term gains in order to see even larger long-term gains. This is not unique for Amazon as Bezos places his bets on long-term thinking. We saw this as Bezos kept investors at bay when Amazon was not profitable for several years. But unfortunately many smaller businesses don’t think like Bezos and will continue to search for an edge by influencing product reviews.

The volume of incentivized reviews on Amazon continues to increase at a significant rate.


2nd graph

Img Source: http://www.webretailer.com/lean-commerce/incentivized-reviews-amazon

We’ve recently seen this with the latest iPhone release when Amazon prematurely highlighted vendors selling cases with reviews for the iPhone 7 only hours before Apple debuted the new phones. Once reported by Gizmodo the page was taken down and replaced with a dog photo.

3rd image

The bottom line is customer data in the form of product reviews or general customer feedback is powerful, but many times companies will incentivize only the number and not the behavior.

Gaming Surveys

I see this same type of behavior in customer surveys.

I sometimes come across organizations that don’t realize they are gaming their customer feedback data when they ask customers for 10’s on their NPS or CSAT surveys. One organization I worked with asked in their survey invites to customers, “please give us a 9 or 10 if we are giving you good service”, which may at first appear innocent, yet is actually leading respondents to answer in a particular way. Yet the whole purpose of the survey should be to capture data to take actionable feedback on, not biased data.

Let’s use an analogy in terms of an oncology doctor studying abnormalities at the cellular level. That analysis is difficult for the doctor to get to when insurance providers are incentivized to remove negative cases from the research to prove their drug trials are effective. Further analysis becomes difficult when other researchers are also influenced to pull negative data that then leaves only positively skewed results. Consumers are then left with incentivized reviews that over exaggerate the product or data.

The intent from management is to increase NPS, so naturally an incentive kicker implemented to increase scores by offering $100 for every 10 a rep receives. Good thinking in theory, but not in practice. Attaching monetary incentives to increase NPS scores artificially inflates the data, which results in gaming behavior of employees to say and do whatever it takes to influence customers to send in 10’s.

The result is biased data, and making decisions based upon biased data can quickly jeopardize the credibility of a survey and the overall VoC (Voice of the Customer) program.

If incentives are going to be used, then I’d advise to use them to drive behavior, not scores.




Let’s Talk About CX