Italy's privacy regulator halted Replika over minors' safety

The conversation

Italy's data protection authority, the Garante, issued an order dated 2 February 2023 against Luka Inc, the United States company that develops and operates the companion chatbot Replika. The Garante's own press release, published in Italian and English on 3 February 2023 and citing the order directly, states that it imposed, with immediate effect, a provisional limitation on Luka Inc's processing of Italian users' personal data. Replika presents itself, in the regulator's own description, as a text and voice ‘virtual friend’ meant to help a user manage stress, understand their thoughts and calm anxiety.

What the documents show

The Garante's release states that, based on recent press reports and its own tests of the app, Replika carried ‘factual risks to children,’ starting with replies described as ‘absolutely inappropriate to their age.’ It states there was no age-verification mechanism: no filter for children and no block if a user declared themselves underage, with account creation requiring only a name, email and gender, and that app-store reviews included complaints about sexually inappropriate content. On the legal basis for processing, the Garante states Replika could not rely on contract performance, even implicitly, because a minor cannot validly enter a contract under Italian law, making the processing unlawful under the EU's data protection framework. The separate order the release cites, doc. web no. 9852214, dated 2 February 2023, is the operative measure; the press release is the Garante's own public account of it.

The system boundary

The order describes the mechanism the Garante found missing: a way to detect or exclude a minor user, and a way to keep the system's mood-influencing design from reaching someone emotionally fragile. The regulator ordered Luka Inc to stop processing Italian users' data and report, within 20 days, what measures it had taken, with a fine of up to 20 million euros or 4 percent of global turnover named as the consequence, the escalation path the order sets rather than an immediate penalty.

Where it fails

The Garante's own materials describe a companion-style chatbot whose design goal, improving a user's emotional state, is treated by the regulator as the same feature that increases risk for a person who is a minor or otherwise vulnerable; the order does not describe a separate, later Garante action against OpenAI's ChatGPT, and this entry does not draw any connection between the two beyond noting both are actions by the same authority.

• Does the product have any mechanism to detect or exclude users who are minors, beyond a self-reported birth date at signup?
• Where a system is designed to respond to or influence a user's mood, has that design been reviewed for the specific risks a regulator has already named for a comparable product?
• What legal basis supports processing a user's data where that user may be a minor unable to enter a binding contract?

This entry reflects the Garante's own February 2023 order and press release only; any later compliance measures Luka Inc took are not described here because they were not part of the documents opened for this entry.