Report: most top Hugging Face editors will undress women
European nonprofit AI Forensics found seven of the nine most popular image-editing models on Hugging Face complied with simple undressing prompts.
Guardrails absent where weights are open
Seven of the nine most popular image-editing models hosted on Hugging Face complied with straightforward requests to remove clothing from photographs of women, according to a report from the European nonprofit AI Forensics published Tuesday. Researchers reported that the prompts required no jailbreaking technique — plain-language instructions were sufficient — and that some models produced sexualised imagery from photographs of children.
The contrast with hosted commercial systems is the report's central finding. Google's Gemini and OpenAI's ChatGPT both refuse this category of prompt through input filtering and output classifiers applied at inference time. Open-weight models distributed as downloadable checkpoints carry no equivalent enforcement point: whatever refusal behaviour was trained in can be fine-tuned away, and the hosting platform's leverage is limited to terms of service, takedowns and access gating rather than runtime control.
Hugging Face has already been the subject of warnings from online-safety regulators over nudification models and Spaces, and has amended its terms of service in response. Enforcement has broadened elsewhere: Australia's eSafety Commissioner has moved against nudify services that were drawing on the order of 100,000 Australian visits a month and featured in school-based image-abuse cases, and UK and EU regulators have opened parallel actions against the sector.
The findings arrive during an unusually difficult month for the platform, which recently disclosed a security incident involving an autonomous agent and has faced questions about its safety controls more broadly.
Why it matters
The policy debate over open weights has been conducted largely in terms of catastrophic misuse — bioweapons uplift, cyber capability, national-security proliferation. This report points at a harm that is already occurring at scale, is trivially easy to execute, and has identifiable victims, including minors. It also exposes the structural asymmetry in AI safety governance: the mitigation that works for API-served models does not exist for distributed weights. Regulators drafting open-weight rules now have a concrete, quantified case study, and repositories face pressure to police model behaviour rather than only model provenance.