© Reuters. FILE PHOTO: An undated handout image from U.S. startup Replika shows a user interacting with a smartphone app to create a custom avatar for a personal artificial intelligence chatbot, called Replika, in San Francisco, California, U.S. Luka, Inc ./Handout
(corrected to make Alvarez & Marsal becomes a consulting firm, not a law firm, at paragraph 11)
By Elvira Pollina and Martin Coulter
MILAN/LONDON (Reuters) – Italy’s data protection authority said on Friday it banned artificial intelligence (AI) chatbot company Replika from using personal data of Italian users, citing risks facing minors and the emotionally vulnerable. risk.
Replika, a San Francisco startup founded in 2017, provides users with custom avatars that can talk to and listen to them.
It leads among English speakers and is free to use, though it brings in about $2 million a month by selling bonus features like voice chat .
“Virtual Friends” are marketed as improving the emotional health of users.
But Italian regulators said that by interfering with users’ emotions, “may increase the risk for individuals who are still developing or in a state of emotional vulnerability”.
Jen Persson, director of children’s privacy advocacy group Defend Digital Me, told Reuters that tools designed to affect children’s emotional or mental health should be classified as health products and should therefore Adhere to strict safety standards.
“These tools are being used on children without much supervision or protection,” she said.
Italian regulators highlighted the lack of age verification mechanisms, such as filters for minors or blocking devices if users do not clearly state their age.
Replika violated European privacy regulations and processed personal data illegally because it could not be based, even implicitly, on contracts that minors could not sign, the watchdog said.
Replika did not immediately respond to an email from Reuters seeking comment.
Italian regulators are unlikely to be the only ones considering action against companies like Replika, says Robert Grosvenor Alvarez & Marsal, managing director of consultancy firm Alvarez & Marsal European regulators.
“While age verification can provide a means of protecting some of the most vulnerable, it does not address risks and harms, in terms of the potential for unconscious bias and discrimination,” he said.
The US company Luka Inc, the developer of Replika, must notify the Italian authorities within 20 days of measures taken to implement its request, which can be fined up to 20 million euros ($21.8 million) ), or up to 4% of its global annual turnover, the statement said.
(This story was corrected in paragraph 11 to say that Alvarez & Marsal is a consulting firm, not a law firm)
