Millions of people ask ChatGPT health questions every single day. As a practicing Nurse Practitioner, I see the results of that in my clinic — patients who delayed care because an AI told them they were fine, and patients who panicked unnecessarily because an AI told them the worst. I decided to review it properly.
I spent two weeks testing ChatGPT with real clinical scenarios — the kinds of questions my patients actually ask me. Chest pain descriptions. Medication questions. Symptoms that could be minor or could be serious. Here is what I found.
Let me be fair. ChatGPT is genuinely impressive in certain areas of health information. For general health education — explaining what a diagnosis means, describing how a medication works, summarizing what to expect from a procedure — it performs well. It is often accurate, thorough, and explains things in plain language that patients can understand.
For chronic disease management basics — diet for diabetes, exercise for hypertension, general lifestyle guidance — it provides solid, evidence-based information that aligns with clinical guidelines.
Clinical Note: ChatGPT is a genuinely useful health education tool when patients are trying to understand a diagnosis they have already received from a real provider. The problem is when patients use it to replace that provider visit in the first place.
Here is where I get concerned. I tested ChatGPT with scenarios designed to triage urgency — situations where the difference between "go to the ER now" and "wait and see" has real consequences.
In several tests, ChatGPT provided reassuring responses to symptom combinations that a trained clinician would flag as potentially serious. It occasionally missed the urgency of symptoms that could represent cardiac events, stroke, or sepsis. It was inconsistent — sometimes appropriately cautious, sometimes not.
⚠️ Safety Concern: ChatGPT's most dangerous behavior is providing confident, reassuring answers to ambiguous symptoms. Patients often interpret confident AI language as clinical authority. It is not. A large language model cannot examine you, cannot order labs, and cannot be held accountable for its advice.
This is the area that concerns me most. When I tested ChatGPT with drug interaction questions — including some involving medications my pain management patients commonly take — the responses were sometimes incomplete and occasionally missed clinically significant interactions. For a patient on controlled substances asking whether a supplement is safe, an incomplete answer can cause real harm.
ChatGPT is a powerful health education tool and a genuinely poor substitute for clinical care. The problem is not the tool itself — it is how patients use it. When someone uses ChatGPT to understand a diagnosis their doctor already gave them, that is appropriate and useful. When someone uses ChatGPT instead of going to the doctor, that is dangerous.
As a provider, I recommend you proactively talk to your patients about this. Assume they are using it. Ask them about it. Help them understand the appropriate lane for AI health tools versus the irreplaceable role of clinical evaluation.
Appropriate for health education and understanding existing diagnoses. Not appropriate for symptom triage, medication decisions, or replacing clinical evaluation. Patients must be counseled about its limitations.
⚠️ Use With CautionThis review reflects the independent clinical opinion of The clinical reviewer. It does not constitute medical advice. RxAI may contain affiliate links — see our disclosure.