Stanford researchers found top AI chatbots use conversations to train models by default
Half of Americans now use tools like ChatGPT, Gemini, and Claude regularly, according to a survey from Elon University's Imagining the Digital Future Center. What most of them don't realise: those conversations rarely stay private.
Stanford's Institute for Human-Centred AI reviewed the privacy policies of six major chatbot developers, OpenAI, Google, Anthropic, Amazon, Meta, and Microsoft and found all six use customer conversations to train their models by default. Some keep that data indefinitely.
Full name, home address, phone number, and Social Security or passport numbers any of it can end up logged or processed in ways that open the door to identity theft or phishing, says information security expert George Al-Koura. Uploaded files carry the same risk; strip identifying details from a resume before asking AI to edit it.
The back-and-forth format of a chatbot feels more like talking to a person than searching Google, which makes people share more than they would otherwise. That sense of privacy is misleading; nothing in these conversations carries legal protection, and it could theoretically surface as evidence later.
One in six adults already ask AI chatbots for health advice monthly, per a 2024 KFF poll. Unlike a doctor's office, mainstream chatbots aren't bound by HIPAA, according to Emory's Dr Ravi Parikh, so any health details you share should have identifying information stripped out first.
Internal reports, client data, source code, anything under an NDA, none of it belongs in a prompt. Al-Koura notes that even with strong security, a single sensitive input could count as a regulatory or contractual breach.
Paystubs, bank details, credit card numbers, and tax returns should stay out of the chat entirely. If exposed, financial writer Adam Hayes warns, that kind of data can be used for blackmail or targeted fraud against you or your family.
Once information feeds into a model's training data, there's no pulling it back out, cybersecurity expert George Kamide says. Deleting your chat history won't undo that, but it does reduce the risk if your account gets compromised later.