Tool reviews

[Andy Wright]

I was reading a magazine  tool review this morning, and I found myself asking a question that has always bothered me about these reviews. The magazine takes one of each model tool and does a bunch of measurements with calipers and dial gauges to determine the flatness, squareness, run-out, etc to the thousandth of an inch.

I have spent a fair amount of time around manufacturing and testing those measurements doesn't strike me as a meaningful exercise unless you are going to measure a sample size greater than one unit. Also, the manufacturers provide those tools to the magazine at no cost so I'm sure that most of the manufacturers have someone in the factory double check the unit before it is delivered to be sure it is "good"

I do find the feedback on usability to be very useful from magazine reviews, but I find tool rankings based on thousandths of an inch to be crazy

[Llama]

If it is found that the manufacturer is cherry picking the units that are tested, they get outed. It's happened a few times. 

[drzaius]

My biggest problem with the reviews is that little consideration is given to how well a tool will perform down the road. They'll give something like a Ryobi a great score because when it's brand new it seems pretty good, but anyone who's use one knows it's going to crap the bed after about 6 weeks of use.

[Chet]

I used to see, years back, tool reviews in Popular Mechanics magazine were they stated "we used this tool for six months or twelve months and here is the review.  But it doesn't seem like the magazines do that anymore.  To bad.

[JosephThomas]

I used to see, years back, tool reviews in Popular Mechanics magazine were they stated "we used this tool for six months or twelve months and here is the review.  But it doesn't seem like the magazines do that anymore.  To bad.

Agreed. This was the most useful part, IMO.

[Mark J]

I suppose it's nice to know that manufacturer can make at least one unit that is flat and square, but you're right it is not a statistically significant sample.  Thing that kills me is that after I read the article I realize that at least one of the key models I was considering was not part of the comparison. 

[Immortan D]

100% agree with Chet.

Before buying a power tool, I always try to find one of those reviews where they take the tool apart and review each individual component, such as bearings, pulleys,  switches, electrical protection, circuit boards, etc. Regardless of what the reviewer hast to say about those parts, it helps me form an opinion about how durable the tool will be.

For hand tools I mostly rely on the manufacturer's reputation to make a choice.

Edited November 26, 2015 by Daniel.

[Cochese]

My biggest problem with the reviews is that little consideration is given to how well a tool will perform down the road. They'll give something like a Ryobi a great score because when it's brand new it seems pretty good, but anyone who's use one knows it's going to crap the bed after about 6 weeks of use.

Really depends on what the actual tool is. For example, five years later all my Ryobis are working well. There's only so many outlets that do long-term tests of things, and usually they are reserved for vehicles.

[Andy Wright]

If it is found that the manufacturer is cherry picking the units that are tested, they get outed. It's happened a few times. 

thats good to hear that there are some ccontrols. How do you know that manufacturers have been ousted in the past? Did the magazine publish it?

Edited November 26, 2015 by Andy Wright

[Llama]

I remember CS talking about it in an interview, or in an article I read. If I remember where, I'll link to it.