I wrote this article at roughly 3 am, sitting in a clinic lobby while my friend talks to an oncologist in the cabin behind me. The air is saturated with the smell of medicines, of course, and the atmosphere has an unmistakable whiff of gloom that compels me to exit the compound as soon as possible. The overwhelming sensation, however, is that of rage and fury.
Merely an hour ago, I had received a call from my 24-year-old friend, complaining about a severe headache and pain in her stomach. She is currently undergoing treatment for breast cancer, and after seeing her go through the ordeal over the past two years, I rushed to her place, hoping it was not serious. Without asking too many questions, I looked up the nearest hospital and rushed her for a checkup.
On the way to the hospital, I asked her when the pain started and if she had experienced any other worrisome symptoms. Her answer shocked me, and to such an extent that I wanted to snatch her phone and crush it to pieces. “After I returned from work, I felt some dizziness and also felt that something was not right in my gut. So, I asked ChatGPT if I could take painkillers, and a few hours after I took them, the gut pain worsened with a burning sensation. I also vomited a few times.”
The admission shook me, and the casual tone in which she described her ChatGPT interactions was even more frightening. But there are so many layers to this mishap that I’ve decided to ban AI for my loved ones, especially when it comes to health and wellness-related conversations. As someone who is extremely sensitive to medications and who is going through cancer treatment, asking an AI about taking medicines was the last thing you would expect any sane person to do.
I don’t know exactly who is to blame for this potentially life-threatening misadventure. She simply took a picture of a painkiller, fed it to ChatGPT, and asked if she could take two pills at the same time. Her doctor has only recommended her one pill at a time, and that too, when the pain is too severe to tolerate. ChatGPT told her she should take one painkiller, and two of them, only with a doctor’s recommendation.
When I asked why she turned to the AI bot, she later told me ChatGPT “is easier to get answers from. I don’t even know what’s real on the internet.” For a person who had to drop studies after high school and support their family with odd jobs, I understood her predicament. I couldn’t lecture her on the concepts of digital literacy and fact-checking. There was so much wrong with that chat, and both parties are to blame here.
This is where the problem begins. She is not a native speaker of the Hindi language, and when she translated her query into Hindi, it was a grammatical disaster owing to just one word. She wanted to ask if she could take two pills of the medicine appearing in the picture. But her broken translation made it seem as if she was asking ChatGPT if she could take “both medicines.”
This is where ChatGPT also tripped. The picture shows a pill with only its chemical composition visible — Diclofenac and Serraptiopeptidase. The name of the medicine was only half visible. Despite that, the AI bot accurately identified the painkiller medicine as “something like Cipzen-D.” But before that, it also mentioned another medicine, Fenak Plus, whose chemical composition is Diclofenac and Paracetamol, and which she had previously discussed in another conversation.
It’s an entirely different composition, even though both are easily available as over-the-counter painkiller medicines that you can even buy (worryingly) without a proper prescription at local dispensaries. So, broadly, we have one human mistake (with translation) and two mistakes by ChatGPT because it was also pulling knowledge from its memory. The advice, however, was accurate.
OpenAI’s AI bot clearly warned that taking both medicines is not a good move, adding that it could lead to an overdose of Diclofenac. The AI bot also described the after-effects, which included a burning sensation in the stomach, ulcer-related problems, bleeding, and long-term ill-effects on the kidneys.
Do keep in mind that the picture fed to ChatGPT only depicted one painkiller. It further recommended that only one of those painkillers must be taken at a time, and not both. My friend, recalling that both medicines had been prescribed by doctors at one point (though not together), took them both for quick pain relief and soon felt the wrath.
The doctor was also furious and, rightfully so, when my friend told them why she took two pills at the same time, chiding her for not consulting a healthcare professional. It was followed by an injection to ease her pain and a fresh medical prescription for painkillers that don’t harm her fragile body.
As a technology reporter who has written about the risks of AI interactions on sensitive topics, especially health and wellness, I felt like a failure. I felt that I never actually had a chat about the risks of AI bots with my friend. And it was only after the scary incident that I warned her about the risks.
When she confessed to using ChatGPT for medical advice on multiple other occasions in the past, I took her phone and deleted the app. More importantly, I asked her to never touch another AI chatbot — such as Gemini or Claude — for medical advice, or the whole internet, for that matter. The only way forward is talking to a doctor, or visiting a clinic.
I don’t blame her entirely for the episode. Given the fashion in which ChatGPT has rooted itself in the daily digital habits of the masses as an all-knowing genie, it’s hard to blame her. People see it as a more convenient (and truthful) answering engine compared to the friction of looking up answers on Google Search, clicking on blue links, and skimming through the answers on a webpage(s).
