Fraud has the potential to invade market research studies in many unique ways and throughout our Uncovering Fraud series, our goal is to share our observations and preventative measures against fraud.  Everything specified in this series is in place at Precision Sample and our intent is to share our best practices with our partners should they decide to double up on security measures or compare best practices with their other suppliers. We are working against groups that are actively collaborating and sharing best practices. If we hope to prevail, we need to collaborate as well. In our last blog post, we discussed the best methods for tackling IP and device spoofing fraud.  Oftentimes, at the helm of these types of fraud are fraudulent panelists, AI assisted fraudulent panelists, and automated bots.

A study ran in 2022 by cyber security firm Imperva found that almost half of all internet traffic was generated by automated bots.  Not all automated bots are fraudulent in nature, some are used by search engines for indexing websites and other valid uses, but this does help illustrate just how prevalent this type of technology is.  Most bot traffic is targeted at the advertising industry to generate fake ad impressions and revenue, some to scrape data and personal information, and a small percentage to complete surveys. Bots are able to create fake IP addresses and digital fingerprints, alter the type of device they are using, and can even solve CAPTCHA tests and answer open-ended questions in surveys with higher levels of relevancy than ever before.

In reality, a very small portion of the fraud we see are bots. The larger issue is fraudulent panelists and increasingly, AI enabled fraudulent panelists. We see fraud in all markets but predominantly in projects we field in North America and Europe. Half of all research conducted in the world happens in the US. While a $1-$2 incentive is not a meaningful amount of money to first world citizens, if those living in the third world are able to make $20 per day, it’s a decent living. We see significant volumes of fraudulent panelists coming from countries such as Bangladesh, India, Venezuela, Middle East Countries and to a lesser extent Russia and China. They obscure their identities, machine characteristics, and register to our panels as US or European citizens to access increased survey opportunities. Increasingly, they use AI to aid and automate completion of surveys and are thus “AI Enabled.”  In this blog, we will discuss how behavioral profiling and red herring questions can be used to identify fraudulent bots and protect your projects from inaccurate data.

At Precision Sample, we rely on our proprietary validation process, Quality Sentinel, which utilizes the following extensive cyber security measures to combat fraudulent bots:

• Behavioral Screening– During registration, initial profiling, and survey prescreening, we deploy behavioral screening questionnaires to identify fraudulent panelists. Fraudulent panelists are heavy ‘satisfiers’ as they have learned answering positively to questions increases their qualification rates. They are generally also not native English speakers so they will state that they know the meaning of made-up words or that they know from memory an unrealistic amount of information. For example, they may say they know the meaning of the word “Umfugar” or that they know from memory the name of every city in the United States. Deploying behavioral screening questions that trap this satisfying behavior is an effective tool to catch fraudulent panelists while at the same time being easy for valid panelists to pass.

• Domain Expertise Screening– When fielding surveys to professionals, high household income individuals and medical patients, it’s advisable to deploy domain expertise screening questions to catch satisfying behavior. Typically, these projects offer a higher reward and attract fraudulent panelists, so additional security is a must. We have deployed and constantly update domain expertise screening questions targeted at these audiences. To offer a hypothetical example relevant to our industry, if we were surveying market research professionals, we may ask if they know the meaning of common industry terms such as LOI, CPI & IR in a multi-choice format. It’s important to make the questions easy for valid panelists to answer while also providing multiple plausible, but incorrect, options.

• Open-End Validation- Before a member is even allowed to take a survey, they are required in the initial profiling process to complete an open-ended question. The question relates to the respondent’s personal life and is tailored to evoke a passionate answer that can help us determine whether a panelist is engaged, disengaged, or a bot.  There are many platforms available for this text analysis work, but not all of them are up to date to capture AI generated responses. It is important that the system you select has the ability to identify and block AI generated responses. If your survey platform does not offer sophisticated AI identification features, they can typically be spotted in the data cleansing process. AI generated responses are typically much longer than those provided by valid panelists. AI generated responses also typically contain generic, incorrect, and outdated information; lack of depth of personality, very bland language, and repetitive content and language.

• Profile Consistency Checks- Aside from open-ended responses, it’s important to have automations in place that cross-check panelist profile information. For example, our system can identify users that provide inconsistent personal profiles, like having both Type 1 and Type 2 diabetes, having an extreme amount of healthcare ailments, having experience in 10+ industries, etc. This can happen in client-side surveys as well by asking the same question twice but in a different format. For example, asking at the front of the survey for household income in a multi-choice format, then at the end of the survey asking income again, but with an open-ended text input format.

Having validation checks and questions in place, both in the initial profiling prescreener and in-survey processes, will ensure that your client’s surveys are better protected from fraud.  Please let us know in the comments if you utilize any validation techniques we may have left out!

For more information about our quality initiatives or to learn more about the panels we offer, contact us at sales@precisionsample.com.