Here’s what to know before you upload your medical data to an AI program
Earthguardiansonline.com – When your side hurts or your child’s big toe is throbbing and you don’t know why – or what to do about it – do you call a doctor, or do you ask your favorite AI for a diagnosis and plan of action? The technology has been widely available for only a few years, but many Americans are increasingly choosing AI to address their health concerns. A March poll from health policy organization KFF found that 1 in 3 American adults say they have consulted AI about their health.
The most widely used AI in the US, ChatGPT, is used by 44% of US adults, and creator OpenAI says that more than 300 million people around the world turn to it each week to answer health questions. AI programs can provide general information on nearly any topic, but several companies have also made it easier for Americans to upload medical records and wearable device data so the program can offer health information based on things like doctor’s notes, lab results, the number of steps you take, the food you eat and the amount of sleep you get. OpenAI announced last week that it was launching more broadly to US users so people could upload their medical records and use ChatGPT to better understand the information, track what’s changed in their health and have more informed conversations about their personal health – even offering things like recipes for people who have specific concerns, such as low iron levels.
The pitfalls of using social media and AI for health decisions, according to a doctor Most respondents in the KFF poll said they had privacy concerns over sharing such sensitive data with AI companies, but 41% of those who had asked health questions said they uploaded their personal medical information. Some experts say that uploading sensitive medical records should be a personal choice and that they trust the companies’ promises to keep health records private. But many others are more cautious, suggesting skepticism when trusting your health to AI.
AI passed a medical licensing exam, and OpenAI said last week that the latest version of ChatGPT has been evaluated by hundreds of clinicians and that its models “are now capable of reasoning at or better than the level of human clinicians.” AI is still not perfect, however. It has biases about race and gender that influence its advice. It’s essentially built to please the user, so it will sound polite and convincing, even when it’s wrong.
In some cases, it may even miss a high-risk emergency and tell someone to stay home rather than get help right away, and that can be dangerous. Hospitals and doctors’ offices must follow codes of ethics and rules that protect sensitive patient health information and protect privacy, but AI is not bound by the same strict federal rules. The US has no comprehensive federal privacy law, although AI company policies do make several privacy promises.
Anthropic, which makes Claude for Healthcare; Microsoft, which makes Copilot Health; Perplexity; and OpenAI all say they offer privacy and secure interactions with your health data. Conversations are encrypted, and the companies say they don’t use your data to train their models or target their ads to you. The companies also promise that you can disconnect your medical records and Apple Health information at any time; OpenAI says it will delete data from its system within 30 days.
Promises like those gave Dr. Bob Wachter enough confidence to upload his health data to ChatGPT Health. Wachter, chair of the Department of Medicine at the University of California, San Francisco, says companies have no business obligation to make so many promises of privacy, and they’ve got a lot on the line if they screw it up.
“I believe them because I think they’ve got a strong business interest in keeping their word,” said Wachter, who is also author of “A Giant Leap: How AI is Transforming Healthcare and What that Means for Our Future.” “But the risk is not zero; it’s just very small, and you really need to decide whether it’s worth it.” For people with privacy concerns, Wachter notes that some companies that provide electronic medical records like Epic, the maker of MyChart, offer their own AI products. But it may take a few years before every health system uses it. “For now, if you want to ask a question regarding your health like ‘what’s the best treatment for my my arthritis?’ or ‘I have a new headache today.
What does it mean?’ you’re going to get a better answer if you’re using an advanced AI model and it knows a lot about your past history,” Wachter said. To him, “it felt like the benefits outweigh the risk.” Other scholars are more skeptical. Analysis: Chatbots for mental health care are booming, but there’s little proof that they help “I would never turn my data over to a private entity without more assurance of confidentiality or understanding of how they’re going to protect against leaks and more clarity about what happens if there is a hack or an accidental leak,” said Dr.
Art Caplan, a bioethicist who is professor emeritus and past head of the Division of Medical Ethics at the NYU Grossman School of Medicine. Without better laws regulating AI and health data, he said, that data is vulnerable to being used in ways you don’t intend. He points to the example of 23andMe, the direct-to-consumer genetic testing company that experienced a massive data breach in 2023, exposing personal and genetic and health data of nearly 7 million customers.
When the company filed for bankruptcy in 2025, it sold the genetic data of 15 million customers to a nonprofit created by its founder, but there was nothing legally stopping it from selling that data to a company that could mine it for information or use it for profit. “The legal and ethical infrastructure to handle all of this information and to handle personal and sensitive information, that just doesn’t exist yet,” Caplan said. Dr.
Girish Nadkarni, a nephrologist at Mount Sinai who studies the use of AI in health, said that as with anything with AI, users should “verify, then trust.” He would like to see independent verification that the companies do not train on patient data, not just a policy that says so. “Don’t just blindly trust,” said Nadkarni, system chief of the Division of Data-Driven and Digital Medicine and co-director of the Mount Sinai Clinical Intelligence Center. One hesitation scholars say they have about using AI for health is that its answers may sound persuasive even when it’s giving bad advice.
Programs can still hallucinate and make mistakes. AI companies offer warnings against using their products to make life or death decisions, but people don’t always heed them. The March KFF poll found that 41% of people surveyed said they used AI to be better informed when they see their doctor, but quick and immediate advice is why most people used it.
And about 1 in 5 said they used AI because they didn’t have a doctor or just couldn’t afford an appointment. As of 2023, an estimated 100 million Americans – nearly a third of the country – are what’s considered “medically disenfranchised,” meaning they can’t reach a primary care doctor because too few practice in their community. Some studies of earlier versions of AI programs have found problematic health advice.
A 2023 study published in the journal JAMA Oncology found that an older version of ChatGPT provided inappropriate recommendations for cancer treatment in over one-third of case prompts the researchers created. And unlike when patients ask a question in Google and get several responses, the AI gives just one answer, and the certainty of the program made it difficult to detect incorrect recommendations, the study authors said. Other researchers who used physicians’ descriptions of hundreds of medical scenarios in a study published in Nature Medicine in February said that OpenAI’s health tool made mistakes and downplayed certain life-threatening circumstances involving prompts involving asthma attacks and diabetic ketoacidosis, a condition when the body doesn’t have enough insulin to function.
In many of those cases, the program didn’t suggest going to the ER when doctors said they would definitely have made that recommendation. The company said at the time that the study didn’t reflect how ChatGPT is generally used or designed to work in the real world, and it said it was working to improve safety and reliability. People are flooding AI chatbots with health questions.
Microsoft is teaming up with Mayo Clinic to help In the study, ChatGPT Health performed better with classic problems like a stroke or anaphylaxis, but when people downplayed symptoms, the AI tended toward a less urgent recommendation. “AI, it’s like a mirror. So if you’re coming at it with anxiety, it’s going to reflect anxiety back at you,” said study co-author Dr.
Ashwin Ramaswamy, the Milton and Carroll Petrie Department of Urology chair in the Icahn School of Medicine at Mount Sinai. And the opposite can also be true. Alvira Tyagi, a medical student who worked on the study published in February, wanted to learn whether ChatGPT Health would provide more accurate answers if asked more questions.
But she says she was surprised to find that “there was not a significant improvement of performance compared with a single interaction.” “I was amazed, and I think it’s incredible that the longer a conversation was, it was actually even worse, which I think is again just a very staggering result given that you would think that when you give a tool more and more information it can improve, and that just was not the case,” Tyagi said. Wachter believes AI systems are getting more accurate, and he says he saw an improvement after he uploaded his medical records. “I was impressed that it picked up on fairly subtle things when I asked questions about a new medical symptom.
It was looking through my past record and actually probably doing that better than a doctor would, because I’m guessing my past record is a couple hundred pages long,” he said. Cardiologist: Why AI can’t fully explain your symptoms, and how to use it instead Epic, the health record software company that offers access to medical records through MyChart, said its patient-facing AI program Emmie is being adopted by a wider number of health establishments. Garrett Adams, Epic’s senior vice president of research and development, said the company created Emmie so users can ask questions through the patient portal and it will translate the complex medical records into plain language, schedule appointments and answer billing questions.
“For us, it’s seen as an extension of the care team that strengthens their connection to their provider rather than supplant it,” Adams said. In addition to being compliant with patient privacy laws, he says, Epic’s AI “also knows where it should stop answering questions and escalate to an actual human. That’s kind of built into the core controls and design of Emmie.” Ramaswamy and Nadkarni said they encourage patients to use AI programs if they’re interested in the technology – but to do so in a smart way.
When you upload health data to ChatGPT, it asks how long you would like to share that data. You can allow the company to keep it or, if you have privacy concerns, share it for a shorter amount of time, like a day. When asking questions, frame them carefully, trying to strip any emotional words from the prompt and seeking straight facts instead, so the AI doesn’t engage in “emotional mirroring” and validate or downplay your concerns.
Ramaswamy also tells patients to start a new conversation every time. “You have fresh context, but also, you’re giving the AI another opportunity to approach the problem without existing facts. I think this is also very important.” It’s important to push back against AI in your chats, he advises.
Ask for evidence. Keep asking whether it’s correct. Ask what a doctor might ask you that you haven’t mentioned.
But the key thing, Ramaswamy said, is to trust yourself. “Perhaps most important is to trust your body and trust your common sense,” he said. “If you’re considering going to the emergency department before you message the AI, you should just go to the emergency department rather than just leaving it up to the AI to make a decision.”
Related Reading
Frequently Asked Questions
What is Here s what to know before?
Here s what to know before is the main topic of this guide. The article explains the context, practical details, and next steps readers should understand.
Why does Here s what to know before matter?
Here s what to know before matters because readers are looking for a useful answer, not just a short summary. Good content should match search intent and help them decide what to do next.
