研究指五間基建商支撐違法合成影像網站Research points out that five infrastructure companies support illegal synthetic image websites
404 Media 報道斯坦福期刊研究,團隊六星期內初步搵到四百個網址,確認八十八個正託管未經同意嘅私密影像,受害者多為女性名人及從政女性。研究指雲端防護商、域名註冊商、內容管理系統及電郵服務五間公司佔主導,另有三百一十二個網址借關鍵字騙搜尋排名。404 Media reported on a Stanford journal study, which found that the team had preliminarily identified 400 URLs within six weeks and confirmed that 88 were hosting private images without consent, with most victims being female celebrities and women in politics. The study indicated that five companies providing cloud security, domain registration, content management systems, and email services were dominant, and another 312 URLs used keywords to manipulate search rankings.
Abusive, sexually explicit deepfakes have only become easier to make and more widespread in the last nine years. Since we first covered deepfakes when they appeared in 2017, they’ve never been the stuff of dark web sites or shadowy networks. They're found on social media networks, through basic web searches, and on easily downloadable apps and accessible forums.
The most prominent hosts of non-consensual, AI-generated imagery are supported by some of the biggest web infrastructure providers in the world, getting their hosting, email, advertising, and content management systems from these companies. A new study examines how providers like Cloudflare, Google, Proton, Namecheap, and WordPress help non-consensual imagery sites thrive. Despite years of knowledge about these wildly popular sites, big tech companies continue providing infrastructure services that prop up abuse imagery, even though their own terms of service often prohibit illegal and harmful content, the study claims.
Hany Farid, professor in Computer Science at Dartmouth College, Sophie Nightingale, a senior lecturer in psychology at Lancaster University, and Lancaster’s Sarah Morgan, who acts as the project coordinator on Nightingale’s “Protecting Ordinary People from Deepfake Harms” fellowship published their paper, “The Backbone of Abuse: How Infrastructure Providers Enable the Proliferation of AI-Generated Non-Consensual Intimate Imagery” in Stanford’s peer-reviewed Journal of Online Trust and Safety on Wednesday.
In a six-week period between February and March, they identified 400 URLs based on keyword searching and Google Alerts, then narrowed the sites down to 88 that were actively hosting non-consensual intimate imagery. Most of the content on these sites was of female celebrities, actresses, pop singers, K-pop idols, and women working in politics or activism. From there, they used open-source web-analysis tools like WHOIS to see what infrastructure providers the sites used.
“Five players emerged as dominant infrastructure providers: Cloudflare, Google, Namecheap, WordPress, and Protonmail,” the researchers wrote in their paper. Cloudflare provided “the lion’s share of services,” they wrote, by providing website hosting, content delivery network, and domain-name server services, as well as analytic tools. Google provided SSL certificates and advertising space for the majority of the sites studied, while Namecheap was the dominant provider of domain registrar services, WordPress provided the majority of their content management system services, and Proton provided mail servers.
Each of these service providers forbid customers and users from using their services for illegal activity. Publishing or threatening to publish AI-generated non-consensual sexual imagery is a federal crime in the U.S. and elsewhere, and sexual abuse imagery in general is illegal in many countries.
Cloudflare, Proton, and Namecheap did not respond to 404 Media’s requests for comment about these findings.
A spokesperson for Google told 404 Media in a statement: “Without the specific domains from the report, we can’t investigate these claims. We have strict policies against non-consensual explicit content — including AI-generated imagery — across all our products. We make it easy for people to quickly remove this content on Search, continuously update our systems to limit its visibility, and we prohibit monetizing or promoting non-consensual and sexually explicit content."
The Google spokesperson added that its advertising platform prohibits monetizing or promoting non-consensual and sexually explicit content, and treats this content as egregious violations, blocking ads on the sites or suspending the advertiser accounts involved. They also noted people can request the removal of non-consensual explicit images from Search, and pointed to Google’s recently-updated removal request process. Google has also updated its ranking algorithms by promoting non-explicit content where possible, they said, and demotes sites in search results that have a high number of removals against them.
A WordPress spokesperson said WordPress is “open-source software, not a hosting provider,” and wrote in a statement: “WordPress.org does not host websites or provide hosting services to sites that use WordPress, and we do not have access to or control over content published on independently hosted WordPress sites.” They noted that WordPress’s software is distributed under the general public use license, allowing anyone to use it for any purpose. “We take the issue of non-consensual intimate imagery seriously, but describing WordPress as providing services or infrastructure to these sites conflates the software a site uses with the services that host and operate it,” the spokesperson said.
In response to WordPress’s statement, Farid noted that while WordPress.org is not a host, WordPress.com is a hosting service. “It's operated by Automattic, which Matt Mullenweg also runs, and it hosts millions of sites. Any NCII site on WordPress.com is squarely within scope,” he told 404 Media. “Automattic provides services to self-hosted sites. Jetpack and the WordPress.com CDN serve images from i0.wp.com/i1.wp.com for sites that enable it, meaning the intimate images themselves can be served from Automattic infrastructure even when the site is hosted elsewhere. That's infrastructure by any definition.”
Farid also noted that WordPress.org maintains “an ongoing service relationship with self-hosted sites,” he said, with core updates, plugin and theme distribution, and security patches coming from WordPress.org servers. Farid mentioned Automattic’s ongoing legal battle with WP Engine, during which, in 2024, Mullenweg announced WP Engine was blocked and then unblocked from accessing WordPress servers; this “showed the project can and will cut a specific operator off from those services,” he said.
“So ‘no access to or control over’ is not entirely accurate,” Farid said.
These providers do draw their own moral and legal lines on what kinds of customers they’ll tolerate and when they cooperate with authorities. After initially refusing to do so, Cloudflare dropped hate speech and doxing site Kiwifarms from its protection services in 2022, and terrorist manifesto host 8chan in 2019. In 2018, Cloudflare banned sex work harm reduction social network Switter from using its services, citing anti-trafficking legislation FOSTA/SESTA, and in 2017, it Cloudflare stopped services to neo-Nazi site The Daily Stormer. In 2023, victims of the sex trafficking ring Girls Do Porn demanded Cloudflare stop providing services to sites hosting their abuse; Cloudflare claimed it doesn’t have “control over the content of websites using those services,” doesn’t have “the ability to alter or remove content on them,” and doesn’t “have knowledge of the people or entities who post any specific content to such websites.”
404 Media previously reported Proton gave the Swiss government payment data related to a Stop Cop City Protonmail account, which in turn handed it to the FBI, and in 2024, Proton gave Spanish police information that identified a pseudonymous activist.
Aside from the infrastructure issue the researchers were studying, they found that 312 of the 400 sites they initially identified were unrelated to deepfakes, like gambling or random travel or cooking SEO-slop sites, but were using non-consensual imagery keywords as a way to rank higher in search results. This shows how incredibly popular the marketplace is for AI-generated abuse imagery, and how sites are able to manipulate Google search to appear higher in results if they use related terms.
……(原文過長,此處截斷)
Abusive, sexually explicit deepfakes have only become easier to create and more widespread over the past nine years. Since we first reported on deepfakes when they appeared in 2017, they have never been confined to dark web sites or secret networks. They can be found on social media platforms, through simple web searches, and on easily downloadable apps and accessible forums.
非自愿生成的 AI 图像的最主要托管平台得到世界上一些最大的网络基础设施提供商的支持,这些平台从这些公司获得托管、电子邮件、广告和内容管理系统。一项新研究检查了像 Cloudflare、Google、Proton、Namecheap 和 WordPress 这样的提供商如何帮助非自愿图像网站繁荣发展。尽管人们多年来一直知道这些极受欢迎的网站,但研究指出,大型科技公司仍然提供支撑滥用图像的基础设施服务,即使它们自己的服务条款通常禁止非法和有害内容。
Hany Farid, a professor of Computer Science at Dartmouth College, Sophie Nightingale, a senior lecturer in psychology at Lancaster University, and Lancaster’s Sarah Morgan, who serves as the project coordinator for Nightingale’s “Protecting Ordinary People from Deepfake Harms” fellowship, published their paper, “The Backbone of Abuse: How Infrastructure Providers Enable the Proliferation of AI-Generated Non-Consensual Intimate Imagery,” in Stanford’s peer-reviewed Journal of Online Trust and Safety on Wednesday.
In a six-week period between February and March, they identified 400 URLs based on keyword searching and Google Alerts, then narrowed the sites down to 88 that were actively hosting non-consensual intimate imagery. Most of the content on these sites was of female celebrities, actresses, pop singers, K-pop idols, and women working in politics or activism. From there, they used open-source web-analysis tools like WHOIS to see what infrastructure providers the sites used.
"Five players emerged as dominant infrastructure providers: Cloudflare, Google, Namecheap, WordPress, and Protonmail," the researchers wrote in their paper. They wrote that Cloudflare provided "the lion’s share of services" by offering website hosting, content delivery network, and domain name server services, as well as analytics tools. Google provided SSL certificates and advertising space for the majority of the sites studied, while Namecheap was the dominant provider of domain registrar services, WordPress provided the majority of their content management system services, and Proton provided mail servers.
Each of these service providers prohibits customers and users from using their services for illegal activities. Publishing or threatening to publish AI-generated non-consensual sexual images is a federal crime in the United States and other countries, and sexual abuse imagery in general is illegal in many countries.
Cloudflare, Proton, and Namecheap did not respond to 404 Media’s requests for comment regarding these findings.
A spokesperson for Google told 404 Media in a statement: ‘Without the specific domains from the report, we can’t investigate these claims. We have strict policies against non-consensual explicit content — including AI-generated imagery — across all our products. We make it easy for people to quickly remove this content on Search, continuously update our systems to limit its visibility, and we prohibit monetizing or promoting non-consensual and sexually explicit content.’
Google's spokesperson added that its advertising platform prohibits monetizing or promoting non-consensual and sexually explicit content, treating such content as serious violations, blocking ads on the sites or suspending the advertiser accounts involved. They also noted that people can request the removal of non-consensual explicit images from Search, and pointed to Google's recently updated removal request process. Google has also updated its ranking algorithms by promoting non-explicit content where possible, they said, and demotes sites in search results that have a high number of removal requests against them.
一位 WordPress 发言人表示,WordPress 是“开源软件,而不是托管服务提供商”,并在一份声明中写道:“WordPress.org 不托管网站,也不为使用 WordPress 的网站提供托管服务,我们也无法访问或控制独立托管 WordPress 网站上发布的内容。”他们指出,WordPress 的软件是根据公共使用许可分发的,允许任何人出于任何目的使用。“我们认真对待未经同意的亲密影像问题,但将 WordPress 描述为这些网站提供服务或基础设施,将网站使用的软件与托管和运营它的服务混为一谈,”发言人说。
In response to WordPress’s statement, Farid noted that while WordPress.org is not a host, WordPress.com is a hosting service. “It's operated by Automattic, which Matt Mullenweg also runs, and it hosts millions of sites. Any NCII site on WordPress.com is squarely within scope,” he told 404 Media. “Automattic provides services to self-hosted sites. Jetpack and the WordPress.com CDN serve images from i0.wp.com/i1.wp.com for sites that enable it, meaning the intimate images themselves can be served from Automattic infrastructure even when the site is hosted elsewhere. That's infrastructure by any definition.”
Farid also noted that WordPress.org maintains “an ongoing service relationship with self-hosted sites,” he said, with core updates, plugin and theme distribution, and security patches coming from WordPress.org servers. Farid mentioned Automattic’s ongoing legal battle with WP Engine, during which, in 2024, Mullenweg announced WP Engine was blocked and then unblocked from accessing WordPress servers; this “showed the project can and will cut a specific operator off from those services,” he said.
"So 'no access to or control over' is not entirely accurate," Farid said.
这些提供商确实在他们会容忍的客户类型以及何时配合执法部门方面划定了自己的道德和法律界限。起初拒绝这么做后,Cloudflare在2022年停止为仇恨言论和人肉搜索网站Kiwifarms提供保护服务,并在2019年停止为恐怖分子宣言网站8chan提供服务。2018年,Cloudflare以反人口贩运立法FOSTA/SESTA为由,禁止性工作减害社交网络Switter使用其服务;2017年,Cloudflare停止为新纳粹网站《每日风暴者》提供服务。2023年,色情贩卖团伙Girls Do Porn的受害者要求Cloudflare停止为托管其受虐内容的网站提供服务;
Cloudflare claimed it doesn’t have “control over the content of websites using those services,” doesn’t have “the ability to alter or remove content on them,” and doesn’t “have knowledge of the people or entities who post any specific content to such websites.”
404 Media previously reported that Proton provided the Swiss government with payment data related to a Stop Cop City ProtonMail account, which was then handed over to the FBI, and in 2024, Proton provided the Spanish police with information that identified a pseudonymous activist.
除了研究人员正在研究的基础设施问题之外,他们发现,在最初确定的400个网站中,有312个与深度伪造无关,比如赌博、随机旅行或烹饪SEO垃圾网站,但它们却使用非自愿影像的关键词来提高搜索排名。这显示了AI生成的虐待影像市场的极高人气,以及如果网站使用相关术语,就能操纵谷歌搜索以在结果中排名更高。
……(The original text is too long, truncated here)
原文出處:Source: 404 Media ↗