Users complain that the Discord CAPTCHA provided by hCaptcha forces them to look for non-existent objects created by AI in the provided photos. For example, an object called “Yoko”, which looks like a hybrid of a snail and a yo-yo, which is invented and created by AI and does not exist in reality.
Let me remind you that we also wrote that GPT-4 Tricked a Person into Solving a CAPTCHA for Them by Pretending to Be Visually Impaired, and also that New hCaptcha bypass method may not affect Cloudflare’s security.
Discord CAPTCHA Created by AI Confuses Users
According to Vice Motherboard journalists, several people immediately complained on social networks about the strange object Yoko, which was required to be found among other photos to enter Discord.
At the same time, other users found that they were asked to find images of a puzzle cube, which was also created by artificial intelligence and did not look too much like a real-life object. In addition, all the objects in the task look like they came straight from the Uncanny Valley.
CAPTCHAs for Discord are provided by hCaptcha, and Discord representatives told reporters that the technology that generates these prompts “is the property of a third party partner and Discord does not directly determine what will be presented to users”. In turn, representatives of hCaptcha explained that what happened was “a short test that a small number of people saw.” Since hundreds of millions of users use the technology in total, even this “brief test” resulted in the tweets shown above.
The publication notes that hCaptcha positions itself as a privacy-focused alternative to reCAPTCHA. According to a 2018 blog post, hCaptcha prompts are self-generated by clients who need “high-quality, human-generated annotations for their machine learning needs.”
That is, hCaptcha makes money both from clients like Discord who buy professional and enterprise subscriptions to run CAPTCHA services, and from clients who create prompts. In fact, hCaptcha uses its own CAPTCHA for machine learning systems and generative adversarial networks.
And this is not the first time that people have noticed the appearance of strange images in hCaptcha services and note that the company apparently trains AI with the help of users. So, two months ago, a Reddit user noticed that Discord asked him to find among the images of people playing hockey and golf, football players, which was clearly created by artificial intelligence. In March, another Reddit user complained that Discord’s CAPTCHA had become almost unsolvable.
Journalists summarize that the work of hCaptcha is a prime example of the problems that arise with machine learning systems. The first is that AI systems require significant human input. For example, as a rule, indexing and categorization of images is transferred to outsourcers from developing countries, whose work is extremely poorly paid. Another problem is data drift: the longer machine learning systems work, the more data they need. Ultimately, they begin to use data that they themselves have generated for self-learning. And systems that train for a long enough time on themselves eventually come to the point that they issue requests for the definition of incomprehensible objects, like Yoko.
