Not All Reviews Are Created Equal

I love my wife very much, but like most couples, there are certain characteristics that make me feel like we are the subject of a certain Paula Abdul hit single from her debut album “Forever Your Girl.” One prime example is our feelings toward “user reviews.” Jessica will automatically read all the reviews before making any decision, ranging from restaurant choice, selecting an exterminator, to selecting a hotel. While she can be somewhat discerning about individual reviews, ultimately they have a direct impact on her actions.

I, for the most part, could care less about user reviews. Parts of the reason are natural personality traits and the other has to do with the process of leaving a review itself. My wife asked why “I care a lot about Airbnb reviews” while I “refuse to even look at a Yelp review.” Well, the real reason is simple: not all reviews are created equal. Let’s break this down a little bit:

It’s all in the transaction

The biggest difference between reviews is whether or not the ability to leave one is based on a known transaction. This means anyone can leave a review of a restaurant on Yelp. There is no way to know if the reviewer has actually eaten there. The “known” transaction is missing.

On the other hand, sites like Airbnb do it right. The only way one can leave a review is after an actual rental takes place. In fact, the platform was built to ensure this. A guest makes a reservation, guest rents place, host is paid, guest checks out. Only after these actions take place can the host and guest review one another. This makes all the difference in the world.

Case Study 1: Seamless

Jessica and I love Seamless. BeenVerified admires Seamless. Almost everyone I know happily uses it because, well, it’s awesome.  When it comes to reviews, they’re almost there. Here is what I mean.

Once you log in, the first page defaults to a view called “My Order History,” which shows a list of all the places a user ordered from through the Seamless platform. The view provides intuitive quick links to things like the restaurant’s main page. It also has a link for leaving that particular restaurant a rating and review.

From this flow alone, it would appear that Seamless fits the bill for acceptable reviews.  However, I was disappointed to discover that I could just as easily leave a review for a restaurant that I never ordered from through Seamless. All I had to do was search for restaurants, click to their page and leave a review. In the words of a recently defeated Haitian Presidential Candidate, “I asked myself ‘Why Oh Why?’ ”

Ask Yourself- How Does this Service Make Money?

A simple analysis of web services will quickly reveal a strong correlation between a companies business model and review structure. Yelp’s business model is based on advertising. Therefore, their bottom line is tied to page views. For them, when it comes to maximizing revenue, quantity is the biggest factor.

On the other hand, Airbnb makes its money through booking fees. Therefore, they are invested in making their platform as strong as possible. Credible reviews are just one of many features that add to the strength of the utility they provide. The same goes for Task Rabbit, Zoc Doc and eBay.

It all comes back to public records

It’s easy to say that reviews only matter when contained in a known transaction. It’s infinitely ​more difficult to create a scalable service that ensures this. (ZocDoc proves that if you’re good, it’s possible.) However, when it comes to creating trust between everyday people, the nut has yet to be cracked. There are a slew of start-ups that try, but they are mostly fruitless because of the lack of a contained transaction.

Jess and I are experienced hosts on Airbnb. They do allow people to get recommendations from “friends” but they are clearly marked as such. I understand that Airbnb runs into an issue for first time users, but I find those “friend reviews” worthless.  Perhaps they can limit them to verified reviews left by other Airbnb users who have already been reviewed based on a transaction. At least that way there is social pressure because someone else put their own reputation at risk.

In general, a contained transaction is not the only variable in determining the value of reviews. It is, however, the most important and presents the highest barrier to achieve. The latest and greatest startups in collaborative consumption do an amazing job of pushing the Web to a better place. However, there will always be times where the containment of a known transaction will not be present, especially when it comes to transacting at lightning speed, in real-time, and on the go. And if you’re reading this blog, I am sure you are fairly confident that this is indeed where we are going and what the next moves need to be.

What do you think? Do you value all reviews you read the same? Sound off below.




Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>