A while ago we've been reported that some users received worrying survey responses, with no text feedback and usually in the lowest range of the rating scale, although the contacts who allegedly submitted the ratings didn't even open the email surveys.
Explaining the problem
Our product team investigated the issue and concluded that some of the contacts that have received the survey had antivirus systems that checked each new email for unsafe/phishing URLs. These systems were randomly accessing URLs in the email surveys and checking for harmful content.
It’s important to keep in mind that the rating buttons in a survey are actually regular links, therefore, antivirus systems will automatically access some of the rating links.
In Retently, once a rating button is clicked in the survey, the rating is immediately added to the user’s dashboard, even if the survey respondent didn't hit the “Submit” response button. This is known as a “first intent rating”, which can be changed by the respondent before hitting the Submit button.
Following this logic, once an antivirus system checked a link in the email survey, the rating was automatically added to the user’s dashboard. Moreover, since the rating was already submitted, our system has considered the survey as “responded”, and when a real person accessed the survey, then only saw a “thank you” page, which added to the confusion.
Our solutions
1. Antivirus & bot protection
Retently offers multiple levels of protection that you can configure per survey campaign.
You can find the "Antivirus & bot protection" setting in your survey campaign's "Misc" section. There are four options to choose from, each providing a progressively stronger level of protection:
Off: No verification is applied. Scores are recorded instantly when a rating link is clicked in the email or the survey link is accessed. This means both real responses and antivirus bot responses will be submitted to your account without any filtering.
Basic: A Cloudflare Turnstile CAPTCHA is shown before the survey page loads. The CAPTCHA is either solved automatically or the respondent is asked to tick a checkbox. This provides a decent level of protection for the majority of cases, as Cloudflare is the leading CAPTCHA solution on the market. Scores are still recorded on the first click.
Recommended: While Cloudflare's Turnstile CAPTCHA is a strong first line of defense, some antivirus bots can bypass it. So we've added this option, which includes CAPTCHA + smart score verification. After solving the CAPTCHA, the respondent briefly sees their selected rating displayed for two seconds, after which the page automatically redirects to the remaining survey questions. This happens quickly and does not interfere with the survey answering experience, but it serves as an additional check to confirm whether the response was submitted by a real person or a bot. This option blocks most scanner submissions while preserving first-click intent.
Strict: CAPTCHA + manual confirmation. This works the same as the Recommended option, the respondent solves the CAPTCHA and then sees their selected rating, but instead of automatically redirecting, the respondent must manually confirm their rating (or change it) before proceeding to the rest of the survey. This is the strongest level of protection and should successfully dismiss virtually all bot-submitted responses.
Note: The CAPTCHA is also available for email reminders sent in an in-app campaign. You can manage (switch ON/OFF) this option from the Misc section of the in-app campaign.
2. Algorithm to detect suspicious survey responses
Even with the best CAPTCHA solution on the market, some bots and antivirus systems still manage to get through. That's why Retently also has an additional layer of internal automated response detection on top of the CAPTCHA.
Retently automatically analyzes incoming survey responses to detect activity that may not come from a real person, such as responses submitted by antivirus software, bots, or automated systems.
When a response is flagged as potentially non-human, a small bot icon appears next to the score.
The icon color indicates the confidence level:
Orange: Suspicious activity detected. The response may not be genuine.
Red: Likely automated/bot. High confidence that this response was not submitted by a real person.
Responses that appear to be from legitimate human respondents will not show any icon.
This helps you identify and filter out unreliable feedback so your metrics reflect real customer sentiment. You can also filter responses by antivirus suspicion using the Feedback Attributes filter on the Responses page: Antivirus suspicion -> Is suspected.
What users can do
Even though Retently will do its part in preventing most of the antivirus submitted survey ratings, there still might be some that will bypass the filter and will end up in your account with the “antivirus warning” icon. At this point, it is up to you to decide whether to delete it at all or just ignore it from the Feedback page.
Note: When the respondent accesses the survey, our system will automatically identify and record their location based on their IP address. However, if an antivirus system was the first to access the survey, then its IP location will be recorded instead. Therefore, this might be another way to verify if the actual respondent or an antivirus bot provided a response.
Since the antivirus has already submitted a score to the initial survey, the original respondent would not be able to submit their feedback if they access the survey. Make sure to resend your email survey and give a chance to the actual respondent to leave their feedback.
We will further monitor our system to make sure our algorithms are continuously improved and such issues are avoided. If you have something to report on the received score, please get in touch with us.