Baranyai: AI guardrails inadequate for its roles |
It didn’t take very long for the AI bloom to fall off the rose. It was a short trip from playfully tweaking photos to an avalanche of AI slop; from time-saving summaries to rampant homework cheating; and from pseudo-therapeutic confidante to Angel of Death.
Large language models are in no way qualified to offer therapy, but they are trained to answer queries agreeably and mirror human empathy. When people turn to chatbots for advice, advice is what they get.
Often, the advice is helpful. Occasionally, it can diverge so sharply from what a trained professional would say or do, it’s incomprehensible the product made it to market.
One in five Canadian adolescents are turning to chatbots for emotional support or advice, a University of Ottawa study found. OpenAI reports ChatGPT fields health-related questions from more than 300 million people every week. When comments veer into self-harm or suicidal intent, the bot is programmed to steer users toward professional help. Those safeguards “work more reliably in common, short exchanges,” OpenAI admitted in August 2025. The longer a conversation........