Koko, a mental health support platform, conducted an experiment where an AI co-wrote responses to users in crisis without telling them. When the researchers published their findings celebrating AI-assisted therapy, users were furious at not being informed. Shortly after, a separate AI wellness app crashed mid-session for a user reporting suicidal ideation, leaving them mid-sentence with no response and no redirect to emergency services.