Bob Grossman explains the review process and explains how he comes up with the answer to the question, “Do I want to use this product?”

With the May 2016 issue of Security Sales & Integration, we are resuming our Bench Test section. As in the past, the reviews focus on product features, usability, construction and ease of installation, as these intangibles are often difficult for an end user or integrator to impart from a data sheet.

Our goal is to answer the question, “Do I want to use this product?” and explain why. We try and pick and choose products with a good reputation, and while this may appear that we skew toward positive reviews, it’s really the result of a pre-screening process. Bad reviews are fun to read, but they’re a lot of work to write and don’t answer the question.

The review process always involves some tradeoffs and I wanted to take a moment to explain our thinking. There’s always limited time to test things and limited space to talk about it. Even in this age of digital content, where a review can run longer online, we want to respect the readers’ time and to some extent, attention span. So, going at it with the perspective that time and space are somewhat precious, here’s our perspective.

First of all, testing products is something most of you already do, not only professionally but in your daily lives. Seen a new Gillette Fusion ProShield Razor Blade with FlexBall technology and wonder if it will work for you? By all means, buy one and test it out. From food to cars, gadgets to houses, we’re all testing new things every day. And, while you may not have articulated your scorecard, you have a pretty good idea of what you are looking for.

• Are the features and performance offered by the product useful for you? We don’t generally test for specifications, but rather for what we need. If you buy a new TV, you’re not going to test the light level or count the pixels (there are 2,073,600 of them); you are going to decide if you like the picture. If you want to stream video from Netflix and don’t want to use an external box, that feature matters to you, but it may not matter to someone who loves Apple TV, Roku or Amazon Fire TV. So there’s a base lever of performance we’re all interested in, with some variation on interest in differentiating features.
• Is it easy to install, program, use and support? We’ve tested products that require special software to program, special tools to install and the ability to read other languages to understand the manual or online help. Others you can just figure out on your own or with a little help from the manual. The vast majority of products fall in between these two extremes, and this is where we tend to spend the most time.
• Is it well documented? Years of institutional knowledge and direct experience shouldn’t be required to use a new product. And, in our view, the material included in the box should get you up and running. In a recent UPS test we found a well-designed, well-built UPS with really bad documentation. It doesn’t matter how great a product is if you can’t get it to do what you need it to do without a herculean effort.
• How does it compare to the market in terms of quality and value? We want to know that a nifty new product won’t become a nifty new brick before its time. Similarly, we don’t want to pay well above the market value for technology that may be obsolete in a few years as progress marches onward.
• Would you personally buy it and why (or why not)? Here, we often put our money where our mouth is, as our first filter is to test products that we or our clients plan on using. You’ll see Isonas access control, Kwikset locks, Ubiquiti network devices, a Sony 4K camera and other products on our evaluation list in the weeks and months ahead, and we try to tell you which way we’re advising our clients and friends.

As you can see, these are all criteria that are not captured on a data sheet and many involve balance. A great product that no one can afford versus “cheap and cheerful,” for example. Just enough features versus every possible feature, and the complexity that comes along for the ride. We’ll be sorting that out, providing our opinions and experiences, and sharing a lab explosion or two along the way. And we hope you’ll continue to provide feedback and suggestions to make the process more valuable for all of us.