Sunday, February 12, 2012
Why Ratings and Reviews Suck and How to Save Them
It is the success and maturity of R&Rs that leave me perplexed at how little innovation we have seen in the space and how much more the leaders could accomplish. This was the topic of conversation recently when I chatted with Jeremy Epstein, the new VP of Marketing at Sprinklr, a social media management service provider that Altimeter put on the top of its list of vendors to support large corporations. We both agreed that it is high time for the primary providers of online R&Rs to lead once again with new features that make R&Rs more powerful and reliable.
First, here is what is right about R&Rs: They remain the most trusted form of communication, lagging only recommendations from families and friends. According to Forrester (subscription required), consumer product R&Rs are trusted by 62% of US adults, compared to 57% who trust "expert ratings and reviews," 30% who trust company web sites and 23% who trust what they see on TV.
Here is what is wrong with R&Rs: That high level of trust may be about to evaporate. While I have no data, I sense more people having doubts as to the validity of the R&Rs they read online. These doubts may be well founded; recent articles have raised awareness of dubious or outright deceptive practices such as offering free product in exchange for ratings, people anonymously rating their employers' products, and farms of workers hired to post ratings.
If you are an authentic marketer and really want to get your blood boiling, check out fiverr.com, where people compete to write paid reviews. One person promises "I will leave 3 Amazon reviews from unique accounts that YOU write for $5," and 44 out of 45 reviews are positive. Comments include "I of course do always need more reviews, how many accounts do you have? And can you also do likes?" Obviously reputable brands are not on fiverr.com bidding for fake reviews, but the transparent sleaziness goes a long way to demonstrating the problem with R&Rs.
While those involved are smart enough to avoid saying, "We get paid to do positive ratings," the implication is loud and clear. "I would have done 4 stars instead of 5 without the deal," says one guy who received free product in exchange for his rating. And an employee of one of the R&R-writing services reports, "We were not asked to provide a five-star review, but would be asked to turn down an assignment if we could not give one."
In theory, it is not unethical to give discounts or freebies in exchange for a rating and review, provided a brand does not reward the effusiveness of the rating and--here is the part everyone misses--the consumer discloses the material relationship as required by FTC guidelines. (And just to be clear, it is the brand's responsibility not merely to instruct the consumer to reveal the arrangement but also to monitor that consumers are, in fact, disclosing appropriately.)
While I hope marketers are following the rules, I do not think the purveyors of R&Rs should rely on the willingness of those involved to adhere to rules--the stakes are just too high. For consumers, the temptation is great to earn cash, freebies and discounts in exchange for five minutes of faux exuberance on Yelp or Amazon. The reasons marketers are tempted to engage in dubious R&R practices are also obvious, as one recent study validated what we all intuitively believe about the value of positive ratings--each additional star on Yelp is worth 5% to 9% in incremental revenue.
Perhaps you believe you are a keen observer who can easily separate the fake reviews from the real ones. Turns out researchers have found the average consumer only gets this right around 50% of the time. You can even test yourself with an online quiz on the Marketplace site. (I scored 75% in the simple four-question quiz.) Even if we believe people have the power to spot and discount fake R&Rs, this cannot be the solution--it takes too long to review and assess individual reviews, and given the power of "star ratings" to furnish info at a glance and permit sorting based on score, we clearly need better ways to improve the quality and reliability of R&Rs.
Given the incentives to cheat, why haven't we seen innovations that encourage appropriate reviews and that filter reviews to make them more accurate and believable? A great deal of effort is going into technological advances to suss out fake reviews, but aren't there easier and better ways to accomplish this same goal?
For example, we are each unique, so why do we see the same information on most R&R sites? When I go to the San Antonio Things to Do page on TripAdvisor, the top-rated attraction in the city of the River Walk and Alamo is... a golf course?!? Even if it is accurate that the Palmer Course has the highest arithmetic mean for all of the submitted ratings in San Antonio, why would this matter to me? TripAdvisor knows I am not a golfer--I have made 101 contributions to TripAdvisor, and not one of them is for a golf course or shop.
One way sites like Yelp and TripAdvisor combat this problem is by showing me what my friends have rated. This is good, but I have friends who love golf. What I really want to know are the ratings of other people like me--it would be more informative to see the ratings of other museum-going, water-park-hating, jazz-loving non-golfing strangers who are similar to me. One of the benefits of furnishing ratings based on similar tastes is that it helps to filter out disingenuous raters. (Perhaps those who are paid or get discounts for the ratings they produce would like to see and rely on each other's ratings, but I rather doubt it.)
Another idea Jeremy and I discussed is discounting ratings from raters who only offer positive ratings. If a given user on Yelp or eBay only posts five-star ratings, wouldn't that call into question the authenticity of those ratings? Some may argue that people tend to write reviews when they feel strongly, but if this were true, we would see more ratings at both the top and bottom of the scale rather than the rating inflation exhibited on most sites. eBay data from 2007 demonstrated the median rating for "Item as Described" was 4.8 out of a possible 5; 67% of the ratings on Yelp are four or five stars; and virtually everyone who rates a video on YouTube gives it five stars.
This is when Jeremy offered the idea of using gamification to improve ratings. I am actually not as much of a fan of gamification as most social media pros. Even though I am an avid PC and mobile gamer, I just do not believe most people go through life wishing they could turn their communications, grocery shopping, dining, driving and TV viewing into a competition. In addition, too much gamification is designed to get people being inauthentic, distorting rather reflecting true opinion, influence and engagement. The fact a brand can amass a million followers by giving something away in a Zynga game is not sign of gamification's value to marketers but of its ability to falsify sentiment and destroy the legitimacy of Facebook "likes."
Jeremy's idea was one of the few I have heard that could turn gamification into a force for authenticity: What if reviewers were rewarded for offering R&Rs that most closely match a bell-shaped curve? Wouldn't reviewers' opinions count more to others if they reflected a more statistically accurate normal distribution of opinions? Perhaps discounts should be given not for any review (or positive reviews) but for people who create true value by creating the most believable and accurate content? If most reviewers were motivated to create R&Rs that were unbiased in a statistical sense, it would create greater dispersion of ratings, increasing the spread between the best and the worst products and services, and penalize (or remove the incentive) for paid reviews.
You probably have some of your own ideas for how R&R sites could be improved. The ease with which ideas can start flying as people discuss the problem is an indictment of how little has been done to improve the R&R process. Yelp says its ranking system "already factors in the number of reviews (and) whether they come from experienced Yelpers or first-time reviewers," but they've done little to make this evident.
If today's R&R leaders do not want to end up like Monster and MySpace, lapped by competitors with greater innovation and differentiation, it is time for the early market innovators to put as much effort into improving the usefulness and validity of R&Rs as they do into new marketing and advertising products. As Jeremy noted, "Trust is going to be the primary currency of the linked economy. If you lose it (and eventually you will get exposed), you're going to have a long road back."