University of Chicago Researchers Built 'Nightshade' — a Tool That Invisibly Poisons Images So AI Models Trained on Them Start Hallucinating. Feed It Enough Cow Photos Labeled 'Car' and Stable Diffusion Thinks Cars Are Cows. Downloaded 250,000 Times in a Week.
October 2023 (paper) – January 2024 (public release): Prof. Ben Zhao's team at UChicago released Nightshade, a tool that adds imperceptible perturbations to images so that any generative model trained on them learns corrupted concept associations. Paired with Glaze (style-cloaking from the same lab, released 2023), Nightshade hit 250,000+ downloads within a week. Artists scrambled to 'nightshade' portfolios before uploading online. Zhao framed it as a defensive tool for copyright holders; Stability AI and others called it adversarial poisoning. A MIT Technology Review cover story ('This new data poisoning tool lets artists fight back') catalyzed the mainstream poisoning debate. Stable Diffusion 3 training pipelines reportedly added filtering for nightshade signatures.
University of Chicago Researchers Built 'Nightshade' — a Tool That Invisibly Poisons Images So AI Models Trained on Them Start Hallucinating. Feed It Enough Cow Photos Labeled 'Car' and Stable Diffusion Thinks Cars Are Cows. Downloaded 250,000 Times in a Week.
October 2023 (paper) – January 2024 (public release): Prof. Ben Zhao's team at UChicago released Nightshade, a tool that adds imperceptible perturbations to images so that any generative model trained on them learns corrupted concept associations. Paired with Glaze (style-cloaking from the same lab, released 2023), Nightshade hit 250,000+ downloads within a week. Artists scrambled to 'nightshade' portfolios before uploading online. Zhao framed it as a defensive tool for copyright holders; Stability AI and others called it adversarial poisoning. A MIT Technology Review cover story ('This new data poisoning tool lets artists fight back') catalyzed the mainstream poisoning debate. Stable Diffusion 3 training pipelines reportedly added filtering for nightshade signatures.
Weirdness Classification
9/10 — Deeply unhinged
Know something weirder?
Submit your own AI incident report to the public record.