Microsoft has used a court order to seize an internet domain used to create “offensive and harmful” AI-generated images through the company’s Azure OpenAI service.
According to Microsoft’s complaint, which was unsealed in a Virginia court on Friday, the domain’s creators used stolen login credentials for Azure OpenAI, which gave them access to the AI image generator DALL-E.
Microsoft describes the domain’s creators as a “foreign-based threat-actor group,” which used custom software to bypass the guardrails for DALL-E. “Cybercriminals then used these services and resold access to other malicious actors with detailed instructions on how to use these custom tools to generate harmful and illicit content,” the company wrote in a blog post.
Microsoft discovered the activity last July when the hackers accessed Azure OpenAI through API keys, including from company customers based in Pennsylvania and New Jersey. The group was fueling the AI image generation through a tool called “de3u,” which was previously available on GitHub and the “rentry.org/de3u” domain before the software was taken down.
(Credit: Microsoft)
It’s unclear what type of offensive imagery was generated. However, the de3u tool could bypass Microsoft’s AI image-generation safeguards by preventing Azure OpenAI from revising a user’s text prompts if they contained certain keywords to trigger content filtering. In response, Redmond revoked the access and filed a lawsuit last month in the Eastern District of Virginia to let it seize the “atism.net” domain used to carry out the hacking scheme.
After the seizure, Microsoft noticed the hackers “taking steps to cover their tracks, including by attempting to delete certain Rentry.org pages, the GitHub repository for the de3u tool, and portions of the reverse proxy infrastructure,” the company said in a subsequent court document.
Recommended by Our Editors
Microsoft also spotted the suspected creators of the de3u tool discussing the crackdown on the 4chan forum. So, it’s possible the group may strike again or target other AI image generators.
In the meantime, Microsoft wrote in the blog post: “With this action, we are sending a clear message: the weaponization of our AI technology by online actors will not be tolerated.” The company also says it’s placed new countermeasures and safeguards to thwart further attempts at malicious use.
Get Our Best Stories!
This newsletter may contain advertising, deals, or affiliate links.
By clicking the button, you confirm you are 16+ and agree to our
Terms of Use and
Privacy Policy.
You may unsubscribe from the newsletters at any time.
About Michael Kan
Senior Reporter
