Quality: Getting into the Weeds
There was a video a few years back that made its rounds across many of our desks. It was a parody video of the life between vendors and clients. The joke was akin to the line of wanting to buy a steak dinner but then comparing the bill to what you paid at the taco stand. Both were cow, what’s the difference?
That same sentiment is holding true today when we look at how sample vendors are being compared with direct line sources of traffic who are more and more trying to sell directly to research houses, but lack many of the things that online panel companies do to ensure quality and screen respondents. And that is the key difference – QUALITY. And in many cases, quality is what is suffering in the race for faster and cheaper. As the old saying goes, “Fast, Good or Cheap – Pick two. “
There are many ways Panels and even some of the newer loyalty sites manage their quality in our quest to keep the “Good” in that equation. Managing panels is a combination of both picking out the pretty flowers that we like and want to keep as well as, and even more importantly, finding and removing the weeds that must be removed for a garden to thrive.
Sophisticated panel companies use a combination of technology and business rules applied to panelist reported data to find the weeds before they ever make it into a client’s survey. Digital Fingerprinting technology is very important to have in today’s online sample ecosystem. Many online panels today are not one single site, but a network of various sites. And because someone may have multiple interests they may legitimately belong to multiple sites within a panel ecosystem or across several suppliers all being engaged on the same survey. Digital Fingerprinting helps to step in and make sure that even though the person may have been invited to participate in your survey through several channels, they are only able to participate once.
More advanced technology involves using services to verify the self-reported data from panelists. Panels look at not only identities and personally identifiable information but also responses to profiling questionnaires that reveal inconsistencies, a tell tail sign of a poor quality panelist. These are all important to the process of ensuring you are getting high quality respondents into your studies. Other quality checks will look at survey data from the respondents and cleanse/suppress those who give bad data, speed, straight-line etc. These serve as the security measures that a lot of firms have in place.
And these are all steps that are vital to verify the respondent’s self-reported data and implemented by panel companies who are serious about challenging fraud. It’s important to constantly re-evaluate security measures as there are sophisticated individuals and groups across the globe that find enough incentive in honorariums to dedicate meaningful time and effort into defeating our measures.
We’ve challenged ourselves to improve our security and quality measures, and understand this is a never ending task – as it should be. Striving for Quality is not something that we can ever say “Well, we are all set”. Much like an ideal, it is something that we as a company always have to strive to improve and keep climbing that mountain. In introducing Quality SentinelTM, we’ve taken those next steps up the mountain and are looking ahead on the next summit.
To learn more about Quality Sentinel and our approach to quality, please visit: