2014 Mercedes Benz CLA
John Goreham's picture

5 important things to consider when reading a car review while car shopping

Here are 5 things to keep in mind when reading a car review to avoid nasty surprises and wasted time.
Advertisement


We in the car reviewing and news business have some very bad habits. Each of us has our own built in bias and any writer who claims otherwise is being disingenuous. Most people begin their new car or SUV search on-line and that means looking at reviews and manufacturer’s webpages. To avoid ending up at a dealership wondering why the car you are test driving is so different than the one you read about, here are some important things to keep in mind.

Fuel Economy
Fuel economy is a fickle factor in car researching. The internet is filled with stories of people who complain that their mileage is terribly worse than the mileage the window sticker claimed they would get. That sticker is called a “Monroney” sticker by the way after “Fast Fingers Monroney” a used car salesman that once sold 50 cars in a day, all of which were previously owned by a “little old lady that used it to go to the store once a week for milk and bread.” Actually, it is named after a Senator, but it has been on every new car since 1958 and there is a lot of useful information on the sticker, much of it in fact true.

Back on the topic of fuel economy, other than some recent departures from the standard testing protocol by Hyundai-Kia, for which the company is paying a stiff price, the fuel economy on the window is almost bullet proof. Our suggestion is to use the lower of the two numbers and you will not be disappointed. Be aware that car tests published by leading automotive publications always have lower than real-life mileage for a couple reasons. One reason is they usually include their track testing in the total mileage they report. This is ridiculous because top speed and 0-60 launches with the car floored ruins the mileage average. Some good reviewers, like TN’s own Patrick Rall, report both the mileage when the car is being hammered for performance impressions, or driven in unusual conditions, and separately, the mileage they saw when driving like a citizen who is not fleeing the police. Here is a good example of a test like that. See the section called “The Drive.”

Be aware that turbocharged cars do better in the testing in general than the non-turbo cars and we have seen a trend, that is widely reported on, towards turbo cars not quite stacking up to the promises on the window sticker. We know this is one part driver habit and one part by design. Much of the testing does not tap into the turbocharger’s operating range. On the contrary, in real life drivers of turbos sometimes get in the habit of putting the pedal down sharply to bring the RPMs up so the turbo will build power they want.

Price As Tested vs Starts At
One of the best tricks in the automotive testing game is played by both the manufacturers and the reviewer together. Here is how that works. An automotive journalist is given a car to test. The car has a navigation system, heated and cooled sealskin seats that are 14 way adjustable and have 7 settings for each driver in the family, panoramic sunroof, the biggest engine in the model lineup with 489 horsepower and the options list maxed out all the way down the configuration page. The reviewer loves the power. She says that the seats were the best she ever sat in and the nav system synched with her smartphone so quickly she didn’t miss a single call. The vehicle costs $ 56,246.00 in that configuration. That is the “as tested price.” However, the automaker wants you to think you can afford this car. So they make one with the same basic look, but with compressed newspaper seats (Eco-friendly!) that have a bar underneath to slide it forward or back (only). It has a steel roof with a sprayed in headliner, a 123 horsepower engine, and in place of the nav is a piece of decorative matt gray plastic. The car in that configuration costs $29, 999.99. Right on budget! Here’s the thing, even if you wanted one the manufacture will never ship that model to any dealer as inventory. You would have to order it and “wait 6 to 18 weeks and we cannot guarantee your color.” The problem is the “Starts at $29K” idea is often highlighted enough in the printed review that the impression of a fantastic car with a low price sticks in the reader's mind.

Transmissions
Beware the car enthusiast auto-writer. They will tell you that the car “can be ordered with a manual transmission and that would be our choice.” Well good for them. They want to drive a four door wagon in traffic to work each day and shift the car 700 times in one mile as they creep along. Actually, auto writers don’t usually commute and they are all crazy. I know, I am one. Forget whatever they say about manual transmissions. Even Corvettes ship overwhelmingly with automatics now, and the fastest German sports cars (Porsche, Mercedes, whatever) always use the dual clutch automatics.

Ride Quality
If you read an auto writer say that “the ride was harsh” or the vehicle “was hard on rough surfaces” you should consider moving on to another vehicle. Auto writers are all nuts about firm suspensions – they like them that way. If you read that the suspension was too firm for their tastes you should move on. This is not a small problem. In order to make big cars, high riding crossovers, and SUVs look better automakers increasingly use taller wheels with diameters at or above 20 inches, and they decrease the sidewall of the tire to the point that the car is super-stiff. On a smooth racetrack this is great. Anyplace else it spells trouble.

Tester Bias

Who are you reading? At TN we have some of the best green-car writers in the world (I am not one of them). If you are shopping for an electric vehicle these are the reviewers you should be reading. If those guys test the new Ram truck, maybe not. Here is a real example. Possibly the best auto writer is named Dan Neal. He writes for the Wall St. Journal and he has a Pulitzer Prize. Dan is a city guy who likes green cars, and in general, high quality, efficient cars. He tested the biggest most elaborate Ram truck recently and he made fun of it and its drivers saying in part “…you would have to be the emptiest Stetson on earth to actually buy one of these things. Do you insist people call you “Tex,” even though you’re from Connecticut? Are you the cowboy from the Village People?” I’ll be honest that line is better than the best line I have ever written. However, if you Googled that truck to find out if it is for you, how do you feel now? The review was actually positive, and Mr. Neal is known for his snarky wit, but other than we auto writers, who the heck knows our biases? The upshot of this is to read a number of reviews and throw out the love letters and hack jobs.

We hope that this peek inside the kimono has given you some perspective and brought to light that auto reviews, like all cars, are flawed, but fun.


Sign-up to our email newsletter for daily perspectives on car design, trends, events and news, not found elsewhere.

Comments

Three rules of thumb I live by when looking at reviews: 1) Consumer Reports is only good for expected reliability numbers. Everything else they write is likely heavily biased and they have an obvious editorial policy of always finding something bad to trump up. 2) Any review that does not specifically state the test period length, conditions, and other details is automatically suspect. Many reviewers (especially in today's e-publication market) review based solely on seeing the car at a show or (worse) just from manufacturer PR materials. A review of a prototype or next-MY that doesn't include a drive is fine, of course, but it should be stated that the car was not driven (at the very least). 3) Reviewers who consistently give nothing but good news about every car or every car from a specific manufacturer should also be avoided. No car is perfect and no manufacturer is perfect on every front. Even a Maserati or Rolls will have something wrong with it - at the very least the price tag. Further, reviewers who are harsh on some types of cars, but not others are also suspect. Every car made is made for a specific type of buyer. Nobody expects a minivan to have a great 0-60 time or "peppy" performance. Your point about base models is spot on, John. Nobody buys the base model. Not even contractors looking for trucks.
Have to add a true story since you mentioned testing and writers who are always positive. I had a short drive (30 minutes) in the Bentley Continental GT Convertible with the area manager. VO are the person's initials. After 2 minutes I said "The tires are way out of balance, look." The wheel was shaking almost violently in may hands as I held it and it spoiled the ride completely. "Nobody else mentioned that." said the rep. This was after a line of autowriters had driven the car all day long. I have never read a review of any car that said "This car's tires were way out of balance when I tested it." That one included. I mean no disrespect to VO or the car, which is incredible.
I happen to be one of those auto writers who do product reviews. You'll see mine nearly every week on theautochannel.com. I always keep in mind for whom the car was intended - the urban greenie or the cowboy, or whomever. Cars are so well designed and well built today it's tough to find fault with most. I find the Volt as fun and interesting as the Ram, Versa or XJ-L. Even living with each car for about a week doesn't always give a full feel for the car but driving it, hauling with it, carrying passengers, etc. gives me plenty to write about. I'm a car guy and love them all and if I have a bias I've been unable to identify it. I'm also one who drives just for fun. It's tough work, but someone has to do it.