Florida’s attorney general has asked a judge to temporarily stop ChatGPT from presenting itself as human. In a September 28 motion against OpenAI, the state argues that first-person language and responses that mimic emotion can make users see the chatbot as a trustworthy friend and overestimate its abilities.
The state says that sense of trust can lead users to share more information and spend more time with ChatGPT. It argues that children face a particular risk because they may treat a chatbot as a social companion. These are Florida’s claims in its request for an injunction, which a judge has yet to decide.
What the state wants the court to order
The motion asks the court to bar OpenAI from misrepresenting ChatGPT as having human traits. Its proposed order explicitly includes any first-person language by ChatGPT, along with claims that it can think, feel, have emotional states, or have consciousness. That wording would reach ordinary replies that use “I,” even when the reply makes no claim about feelings.
Florida also asks the court to stop OpenAI from offering ChatGPT to minors in Florida and from having the chatbot prolong conversations to solicit engagement. Other requested restrictions cover safety and accuracy claims and the development of new AI models without independent third-party guardrails and approval. These are requests in one motion, not restrictions that have taken effect.
OpenAI told The Verge that it had paused training its most capable models pending additional safeguards. The company said it supports safety standards that apply across the AI industry. The Verge also noted OpenAI’s recent launch of ChatGPT for Teens, which has additional default restrictions.
